ChatGPT: Difference between revisions

{{see also|ChatGPT Guides}}

Developed by the [[AI]] research lab [[OpenAI]], ChatGPT is a [[large language model]] ([[LLM]]) that generates text based on an [[input]] ([[prompt]]) from a user (figure 1). It interacts in a conversational way, answering follow-up questions, admitting to its mistakes, refusing inappropriate requests, and challenging incorrect premises. <ref name="”1”">OpenAI (2022). ChatGPT: Optimizing language models for dialogue. OpenAI. https://openai.com/blog/chatgpt/</ref> <ref name="”2”">Kelly, SM (2022). This AI chatbot is dominating social media with its frighteningly good essays. CNN International. https://edition.cnn.com/2022/12/05/tech/chatgpt-trnd/index.html</ref> <ref name="”3”">Edwards, B (2022). No Linux? No problem. Just get AI to hallucinate it for you. Ars Technica. https://arstechnica.com/information-technology/2022/12/openais-new-chatbot-can-hallucinate-a-linux-shell-or-calling-a-bbs/</ref> The LLM was trained on a massive amount of information <ref name="”2”" /> and it's a derivative of [[InstructGPT]], a program trained to follow natural language instructions in a prompt and give an elaborate response. <ref name="”1”" />

The ChatGPT model was trained using [[Reinforcement Learning]] from Human Feedback (RLHF) following the same methods as InstructGPT (figure 2); only with small differences in the data collection setup. Training for an initial model was made using supervised fine-tuning where human AI trainers had "conversations in which they played both sides—the user and an AI assistant." <ref name="”1”" />

Reinforcement learning uses a reward model for AI training. This was done by collecting comparison data consisting of two or more model responses ranked by quality. According to the official blog of OpenAI, "to collect this data, we took conversations that AI trainers had with the chatbot. We randomly selected a model-written message, sampled several alternative completions, and had AI trainers rank them. Using these reward models, we can fine-tune the model using Proximal Policy Optimization. We performed several iterations of this process." <ref name="”1”" />

OpenAI's ChatGPT has myriad applications, from generating computer code, to writing jokes, explaining scientific concepts in different levels of complexity, to writing college-level essays. Besides its regular text composition skills, it can also provide assistance, by helping programmers spot and fix errors in code or answer questions made to it, analogous to a Google search. <ref name="”4”" /> Indeed, it was denominated as a possible "Google Killer". <ref name="”9”">Ansari, T (2022). These 8 potential use cases of ChatGPT will blow your mind!​​​​​​​. Analytics India Magazine. https://analyticsindiamag.com/these-8-potential-use-cases-of-chatgpt-will-blow-your-mind/</ref> Due to ChatGPT remembering what a user has written before, some have suggested that it could be possible to create personalized therapy bots. <ref name="”4”" />

*Content generation, changing how content is created for thousands of companies.