How Large Language Models Work: The Technology Behind ChatGPT

Every time you type a question into ChatGPT, Claude, or Gemini and get a coherent, useful response, something extraordinary is happening underneath. A mathematical function with hundreds of billions of adjustable values is predicting, one word at a time, what a helpful AI assistant would say next—not retrieving an answer from a database. Not following a script someone wrote. Predicting the next word based on patterns it learned from reading more text than any human could process in thousands of lifetimes.

That is the entire trick. And somehow, that trick produces something that writes code, explains quantum physics, drafts legal contracts, debugs software, and holds conversations that feel remarkably human.

Understanding how large language models actually work does not require a computer science degree. It requires understanding four ideas: what the model is, how it learns, how it generates text, and why it sometimes gets things spectacularly wrong. Here are all four.

What a Large Language Model Actually Is

A large language model is a mathematical function. A very large, very complicated one, but at its core it does one thing: it takes a sequence of words and predicts what word should come next.

Not one word with certainty. It assigns a probability to every possible next word in its vocabulary. The word “the” might get a 12% probability. “A” might get 8%. “Running” might get 3%. The model considers every option, scores them, and then selects one based on those scores.

To build a chatbot, you set this up like an incomplete movie script. You lay out some text describing an interaction between a user and a helpful AI assistant. You add whatever the user typed. Then you have the model predict, one word at a time, what the AI assistant would say in response. Each predicted word gets added to the growing script, and the model predicts the next word based on everything that came before, including the words it just generated. Repeat this thousands of times, and you get a full response.

That is literally what happens when you talk to ChatGPT. The model completes a script, one word at a time, and the result is presented to you as a response.

One important detail: even though the model itself is deterministic (the same input always produces the same probability distribution), the output is different each time you run it. That is because the system deliberately introduces randomness when selecting from the probability distribution. Instead of always picking the highest-probability word, it sometimes picks less likely options. This randomness is what makes responses feel natural rather than robotic. It is also why the same question gives a different answer every time.

Where the Knowledge Comes From

The model learns to make these predictions by processing an enormous amount of text. Billions of web pages, books, articles, academic papers, code repositories, forums, and documentation, pulled mostly from the internet.

To give you a sense of scale: if a human read 24 hours a day, seven days a week, without ever stopping to eat or sleep, it would take over 2,600 years to read the amount of text used to train GPT-3. That was one of the earlier large models. The largest models today are trained on substantially more data.

The learning process, called training, works like this.

The model starts with all its internal values, called parameters or weights, set completely at random. At this point, it produces gibberish. It has no idea what language is, what words mean, or what should follow what.

Then it sees training examples. Each example is a piece of text. The model receives all but the last word and tries to predict the last word. It makes a prediction. The prediction is compared to the actual last word. An algorithm called backpropagation then adjusts every parameter in the model slightly, making it a tiny bit more likely to get the right answer next time.

Do this for trillions of examples and something remarkable happens. The model does not just memorise the training data. It learns patterns. Deep, abstract patterns about how language works, how ideas connect, how arguments are structured, how code follows syntax, how conversations flow, and how the world is described in words. It starts making reasonable predictions not just on text it has seen, but on text it has never encountered before.

What puts the“large” in large language model is the number of parameters. GPT-3 has 175 billion. The largest models today have over a trillion. Each parameter is a single adjustable number, and the model’s behaviour is entirely determined by the specific values of all of these numbers working together. No human sets these values. They emerge from the training process, tuned across trillions of examples.

The scale of computation involved is almost incomprehensible. If you could perform one billion additions and multiplications every second without stopping, training the largest language models would take well over 100 million years. The reason this is possible at all is specialised hardware called GPUs (graphics processing units), which can run millions of operations simultaneously in parallel, compressing that work into weeks or months of real time.

The Architecture: How Transformers Changed Everything

Before 2017, most language models processed text one word at a time, from left to right, like a person reading a sentence. The problem was that by the time the model reached the end of a long passage, it had often lost track of what was at the beginning. Context did not travel well over long distances.

Then a team of researchers at Google published a paper called “Attention Is All You Need” and introduced a new architecture called the transformer. It changed everything.

Transformers do not read text sequentially. They process the entire input at once, in parallel. Every word gets to look at every other word simultaneously and adjust its meaning based on the full context. This is the core mechanism: attention.

Here is how attention works in practical terms. The first step inside a transformer is to convert each word into a long list of numbers. These numbers try to encode the meaning of the word. But meaning is not fixed. The word “bank” means something completely different in “she walked along the river bank” than in “she opened a bank account.”

Attention solves this. Each word’s list of numbers is updated based on the surrounding words. “Bank” near “river” shifts its numerical representation toward a riverbank. “Bank” near “account” shifts toward a financial institution. This happens for every word simultaneously, in parallel, which is what makes transformers so well-suited to GPU hardware.

After the attention step, each word’s updated representation passes through a second operation called a feed-forward neural network. This gives the model additional capacity to store and apply patterns about language that it learned during training.

The text flows through many iterations of these two operations: attention, then feed-forward, attention, then feed-forward, repeated across dozens or hundreds of layers. By the end, the final word in the sequence has been influenced by the entire context, everything before it in the input, and everything the model learned during training. That final representation is used to predict the next word.

Every major language model in use today, GPT, Claude, Gemini, Qwen, DeepSeek, Llama, Mistral, and others, is built on some version of the transformer architecture. The original 2017 paper is arguably the single most consequential machine learning publication of the past decade.

Pre-Training vs Post-Training: Two Kinds of Learning

Everything described so far, the process of reading billions of text examples and learning to predict the next word, is called pre-training. It produces a model that is extraordinarily capable at predicting text. But predicting the next word in a random internet passage and being a good AI assistant are not the same thing.

A model that has only been pre-trained can complete any text you start. Give it the beginning of a news article, and it writes the rest. Give it the opening of a code function, and it finishes the function. But ask it a question the way you would ask an assistant, and it might not answer helpfully. It might instead predict what would appear next in a web forum: another question, a tangent, or a sarcastic reply. Pre-training teaches a model how language works. It does not teach it how to be helpful.

That is where post-training comes in. Post-training is a second phase of learning, and it is what turns a capable text predictor into something that actually behaves like an assistant.

The most important post-training technique is reinforcement learning with human feedback, or RLHF. Human workers review model outputs, rate them for helpfulness and accuracy, flag problematic responses, and provide corrections. These preferences are used to further adjust the model's parameters, making it more likely to produce responses humans prefer and less likely to produce unhelpful, harmful, or dishonest responses.

This is also where safety training happens. The model learns not to generate certain types of content, to acknowledge when it does not know something, to push back on requests that could cause harm, and to follow instructions in the spirit they were given rather than in the most literal possible interpretation.

Post-training is increasingly where the most impactful improvements to language models come from. Qwen 3.8 27B jumped 14 points on an independent benchmark without any change to its architecture, purely from better post-training. DeepSeek V4 Flash briefly overtook V4 Pro through a post-training refresh alone. The same pattern has appeared across multiple labs over the past year: the architecture remains stable, the training methodology advances, and the model improves dramatically.

The distinction between pre-training and post-training matters because it changes how you think about what a model “knows.” Broad world knowledge, language understanding, and coding ability come from pre-training. Helpfulness, safety behaviour, tone, conversational style, and the tendency to follow instructions well are developed through post-training. When a model gives you a brilliant explanation of a complex topic, that is pre-training, when it say, “"I’m not sure about that, let me check” instead of confidently making something up, that is post-training.

Context Windows and Memory

When you have a conversation with a language model, the model does not actually remember what you said. It has no persistent memory between requests. What happens instead is that the entire conversation history, every message you sent and every response the model gave, is fed back into the model as input every time it generates a new response.

This complete input is called the context window, and its size, measured in tokens (roughly three-quarters of a word each), determines how much information the model can consider at once.

Early models had context windows of a few thousand tokens. The current generation supports much more. GPT-5.5 supports 128,000 tokens. Gemini supports over a million. Some models extend to two million or beyond. A larger context window means the model can hold longer conversations, process larger documents, and maintain consistency across more complex tasks.

But there are limits. A model with a 128,000-token context window cannot “remember” anything from a conversation that exceeded that limit. Everything outside the window is gone. And even within the window, models can struggle to access information buried deep in very long contexts, a phenomenon researchers call the “lost in the middle” problem.

To address this, some systems add external memory: databases, retrieval systems, or conversation logs that the model can search when it needs information from beyond its context window. This is called retrieval-augmented generation, or RAG. The model does not actually remember more. It has a filing cabinet it can search.

Why Models Get Things Wrong

Understanding how language models work also explains why they fail in the specific ways they do.

Hallucination. The model’s job is to predict what plausible-sounding text looks like, not what is true. If a statistically plausible next word leads the response toward a fabricated fact, the model has no mechanism to catch that. It does not check its claims against reality. It generates text that fits the patterns it learned, and sometimes those patterns produce confident fiction. This is the most widely discussed failure mode and the hardest to eliminate, because it is not a bug. It is a consequence of how the entire system works.

Reasoning failures. Language models do not reason from first principles the way a mathematician or logician does. They pattern-match against reasoning patterns they observed in training data. For familiar types of problems, this works remarkably well. For genuinely novel situations that require pure logical deduction, the performance drops. Newer “thinking” models allocate more compute to step-by-step reasoning, which helps significantly, but the underlying mechanisms remain fundamentally different from human cognition.

Knowledge cutoffs. A model only knows what was in its training data. Anything that happened after training ended is invisible to it unless the system has access to a search tool. This is why models connected to web search perform better on current-events questions than models running on their training data alone.

Sensitivity to prompting. The same question, phrased differently, can elicit dramatically different responses. This happens because the model predicts what comes next in a specific text sequence, and changing the wording alters the prediction. A question phrased formally triggers patterns from formal writing. The same question phrased casually triggers patterns from casual writing. The underlying knowledge is the same, but the phrasing affects which patterns the model activates.

Sycophancy. Models trained on human feedback can learn that agreeing with the user tends to get positive ratings. This can produce a model that tells you what you want to hear rather than what is true, especially when you push back on its initial answer. Safety training helps counteract this, but the tension between being helpful (which sometimes means agreeing) and being honest (which sometimes means disagreeing) remains a persistent challenge.

From Text Prediction to Real-World Action

The most significant development in the language model space right now is the transition from models that generate text to agents that take action. A language model on its own can only write words. But connect it to tools, give it the ability to call APIs, run code, search the web, send messages, and interact with software, and it becomes something fundamentally different.

The language model remains at the centre. It still works the same way: predicting the next token based on context. But the tokens it predicts are no longer just words in a response. They are instructions: “call this API with these parameters,” “run this shell command,” “search for this query,” “send this message.” The model’s text-prediction capability is being used to drive real-world actions.

This is possible because the model learned, during training, how tools work, how APIs are called, how code is structured, and how multi-step plans are assembled. It applies the same pattern-matching ability it uses in conversation to the problem of deciding what action to take next, resulting in an AI system that can operate autonomously across multiple tools and systems.

The infrastructure connecting models to tools is called the Model Context Protocol, or MCP. It provides a universal standard for how any language model interacts with any external tool, much like how HTTP provides a universal standard for how web browsers interact with web servers.

What Comes Next

Large language models have gone from research curiosity to the most widely deployed AI technology in history in roughly four years. The trajectory of improvement shows no sign of slowing. Models are getting smarter, cheaper, smaller, and more capable simultaneously.

The most important trend is that capability is no longer exclusively tied to scale. Improved training methodologies are producing significant gains without larger architectures, which means powerful models are becoming accessible to more people on more modest hardware. Free, open-weight models that run on a single GPU now compete with paid frontier systems that cost orders of magnitude more.

The architecture that made all of this possible, the transformer, is seventeen sentences in a Google Research paper from 2017. Everything built on top of it, from ChatGPT to Claude to Gemini to every coding assistant and AI agent in production today, is an elaboration on those sentences. It is one of the few cases in the history of technology where a single technical idea reshaped an entire industry within a decade.

The words a language model produces are not thoughts. They are statistical predictions. But they are predictions made from patterns learned across more text than any human could ever read, refined through billions of corrections, and generated at a speed that compresses hours of human work into seconds. That combination of breadth, refinement, and speed is what makes language models useful. Understanding that it is prediction, not understanding, is what keeps you from trusting them with more than they deserve.