Learn what is going on behind the scenes when ChatGPT is answering your questions.
I have observed many individuals becoming frustrated or confused about the provided responses from ChatGPT because it is not what they expect it to be. To clear up this misunderstanding, today we will delve into how ChatGPT works and the technology behind this incredibly-powerful AI.
ChatGPT, a trained language model, is developed by OpenAI. Since its release, millions of people have engaged in conversation with it. And, the results it has provided were spectacularly good that it has gained exponential popularity around the globe. However, some individuals have become confused about the answers or received unexpected results. I believe that many of them do not fully comprehend how ChatGPT actually works. Therefore, if you are interested in learning the underlying technology behind ChatGPT, you have come to the right place.
What are Large Language Models (LLMs)
Large language models are a type of artificial intelligence. These models are being trained on vast amounts of text data. At the end of this process, they end up being capable of understanding and generating human language. This way, they can be used for a variety of tasks such as language translation, text summarization, and question answering.
There are lots of large language models on the market today. The most famous one is GPT-3 model, developed by OpenAI. ChatGPT is also built upon GPT-3 model. And OpenAI is getting ready to introduce GPT-4 model in 2023. To learn more about this take a look at our post on the most advanced chatbot.
How ChatGPT Works
There are definitely some rules that ChatGPT has clearly been instructed to follow for its research release. These may pertain to ethical and philosophical concerns. You can even get some hints about them following the Twitter account of Sam Altman, the current CEO of OpenAI, the company that developed ChatGPT. However, our main objective is to understand the base technology of this language model outside of its restrictions.
Murray Patrick Shanahan, a professor of Cognitive Robotics at Imperial College London, wrote a paper on this titled “Talking About Large Language Models” on December 2022.
In his paper, he discussed the subject in depth and gave scientific insights into how large language models (LLMs) like ChatGPT work. You can access the paper directly from the internet.
For a concise explanation of how large language models (LLMs) such as ChatGPT works, we are including the excellent following passage from the paper.
LLMs are generative mathematical models of the statistical distribution of tokens in the vast public corpus of human-generated text, where the tokens in question include words, parts of words, or individual characters including punctuation marks. They are generative because we can sample from them, which means we can ask them questions. But the questions are of the following very specific kind. “Here’s a fragment of text. Tell me how this fragment might go on. According to your model of the statistics of human language, what words are likely to come next?” It is very important to bear in mind that this is what large language models really do. Suppose we give an LLM the prompt “The first person to walk on the Moon was ”, and suppose it responds with “Neil Armstrong”. What are we really asking here? In an important sense, we are not really asking who was the first person to walk on the Moon. What we are really asking the model is the following question: Given the statistical distribution of words in the vast public corpus of (English) text, what words are most likely to follow the sequence “The first person to walk on the Moon was ”? A good reply to this question is “Neil Armstrong”.
Talking About Large Language Models by Murray Patrick Shanahan
For those who may be unsure of what this paragraph even means or prefer a simpler explanation, let me clarify: ChatGPT is not a ‘truth-bot’, but rather a language model.
It generates text by determining the most linguistically correct ending to a sentence based on billions of examples from other sources. When you point out its mistakes it might apologize to you and correct itself, but it’s not doing that because it learned something or ‘accepted’ that it’s wrong… It’s doing that because, in the billions of samples it analyzed, apologies usually came after being told that you’re wrong.
So I asked ChatGPT the mentioned question in the passage and received the same result as the following.
This suggests that when I asked ChatGPT with the prompt “The first person to walk on the Moon was”. It did not possess the answer by truly knowing it. Rather, ChatGPT simply evaluated the question to determine which words were most likely to follow and generated the result based on statistical distributions.
Conclusion
- Because ChatGPT is so advanced, it can be challenging to understand how it works on your own. Additionally, sometimes, its responses could be incorrect or out of context. Therefore people want to understand the underlying mechanism of ChatGPT.
- ChatGPT is a large language model. Large language models (LLMs), a type of artificial intelligence, are being trained on vast amounts of data.
- Murray Patrick Shanahan, a professor at Imperial College London, examined large language models at length in his paper called Talking About Large Language Models.
- As Shanahan stated in his paper, a typical LLM generates responses based on the statistical distribution of words in a large public database of text, rather than having actual knowledge.
This explanation does not diminish its value. However, it does help to explain some strange or wrong answers that we may sometimes receive from ChatGPT. Some of them may even be hilarious. You can see our post featuring humorous conversations with ChatGPT.
Have a good day knowing how ChatGPT works.
Leave a Reply