ChatGPT has disrupted the narrative around AI and fired everyone’s imagination. Just like iPhone disrupted the market for mobile phones, Google did for search, Tesla did for cars, and Watson did for question-answering (with Jeopardy!), ChatGPT has people at every level of education spectrum trying it out for applications ranging from scientific articles to real-estate to law and business exams to programming, and much more. But technologies are not accepted by just its perfect performance but also a socio-technical ecosystem. For example, a car must drive properly but the legal, education, and standards framework allow a user to trust the enabling environment and confidently drive their vehicle on the roads. Similarly, conventional or new application domains alike, the adoption of chatbots were already hindered by the lack of a supportive socio-technical environment. With easy access of LLM-based tools like ChatGPT, the risk of harm will only increase unless other pillars are quickly built. To benefit society from the potential of LLM based technologies, the path forward is not to scuttle LLM-based tools but to increase investment and augment necessary other pillars for the technologies’ safe and trusted usage for the society.