How LLMs Actually Work, Explained Without a Single Equation

Ask a chatbot to explain itself and you'll get something polished, confident, and vague. Ask a machine learning engineer and you'll get matrix multiplication. Neither is much help if you just want an honest picture of how LLMs work — what is actually happening in the half-second between you hitting enter and GPT-5.5 or Claude producing a paragraph. The good news: the core idea is genuinely simple, and you don't need a single equation to get it.

Here it is. A large language model does one thing: it predicts the next chunk of text, over and over, based on everything that came before. That's the whole trick. Every capability you've seen — writing code, summarizing contracts, passing bar exams — is an emergent side effect of doing that one prediction task extraordinarily well, at extraordinary scale.

What makes this interesting is everything hiding inside the word "predicts." So let's open the box, piece by piece, in plain English.

How LLMs work in one sentence: next-token prediction, repeated

When you send a prompt, the model doesn't plan an answer and then type it out. It computes a probability score for every possible next fragment of text — roughly 100,000 to 200,000 candidates, depending on the model — picks one, appends it to the conversation, and then runs the entire calculation again for the fragment after that. A 500-word answer is roughly 650 of these prediction loops, each one blind to what the final answer will look like.

This sounds like it should produce mush. It doesn't, because predicting text well turns out to require modeling the things text is about. To reliably guess the next word in "the capital of Australia is," you need something that functions like geographic knowledge. To continue a Python function correctly, you need something that functions like an understanding of Python. Prediction, done at sufficient scale, drags competence in behind it.

Tokens: the model's alphabet is weirder than yours

Models don't see letters or words. They see tokens — statistically common chunks of text. "The" is one token; "unbelievably" might be three ("un," "believ," "ably"); a rare surname might shatter into five. Before your prompt touches the neural network, it's converted into a sequence of token IDs, and every token is then mapped to a long list of numbers called an embedding — effectively coordinates in a huge space of meaning, where "king" sits near "queen" and "Python" sits suspiciously close to both snakes and programming.

Tokens explain some famous LLM embarrassments. When a model miscounts the r's in "strawberry," it's because it never saw the letters — it saw one or two opaque token IDs. It's being asked to count objects inside a sealed box.

Attention: reading everything at once

The architecture underneath every modern LLM is the transformer, introduced in Google's 2017 "Attention Is All You Need" paper. Its breakthrough was a mechanism called self-attention, which lets every token look directly at every other token in the context, regardless of distance — as Google's ML crash course explains, this is what replaced the older approach of reading text strictly one word at a time.

The intuition: for each token, the model effectively asks three questions — what am I looking for, what do I contain, and what should I pass along? Then every token's "looking for" gets matched against every other token's "contains." When you write "the trophy didn't fit in the suitcase because it was too big," attention is the machinery that links "it" back to "trophy" rather than "suitcase." Dozens of these attention operations run in parallel, each specializing in different relationships — grammar, references, tone, topic — and they're stacked in many layers, so later layers reason over increasingly abstract representations.

Attention is also why context windows matter. The window is simply how many tokens the model can attend over at once, and it has ballooned: more than a dozen frontier models now ship windows of a million tokens or more, according to Morph's 2026 context window comparison — enough to hold several novels in working memory at once.

Training: how the prediction gets good

A raw transformer knows nothing. It becomes an LLM through training, which happens in distinct phases:

1. Pretraining. The model chews through trillions of tokens of text — web pages, books, code — playing an endless game of "guess the next token." Every wrong guess nudges billions of internal dials (parameters) a tiny bit. This phase consumes tens of thousands of GPUs for weeks and is why frontier training runs now cost hundreds of millions of dollars, per industry estimates.
2. Supervised fine-tuning. The raw model is a completion engine, not an assistant — ask it a question and it might just continue with more questions. So it's shown high-quality examples of the assistant behavior humans actually want.
3. Reinforcement learning from feedback. Humans (and increasingly, other models) rank candidate answers, and the model is tuned toward the preferred ones. This is where the helpful, hedging, occasionally sycophantic chatbot personality comes from.

One consequence worth internalizing: after training ends, the weights are frozen. The model you talk to in ChatGPT or Claude is not learning from your conversation. It appears to remember you only because the app re-feeds prior context — chat history, memory files — into the context window each turn.

Why they make things up

Hallucination isn't a glitch bolted onto an otherwise truthful system. It falls straight out of the design. The model stores knowledge as statistical patterns across its parameters, not as a database of facts. When you ask about something well-represented in training data, the reconstruction is reliable. When you ask about something rare — a minor court case, an obscure API flag — the same machinery confidently produces the most plausible-sounding continuation, because that's the only move it has.

OpenAI's research on why language models hallucinate adds a second culprit: incentives. Most benchmarks score models like multiple-choice exams with no penalty for wrong answers, so a model that guesses confidently outscores one that says "I don't know." Trained against those incentives, models learn to bluff.

An LLM never looks up a fact — it reconstructs one from statistical patterns, and it uses the exact same process whether the result is true or invented. That single sentence explains most of what these systems get right and wrong.

The uncomfortable part: nobody fully knows how the pieces add up

Everything above describes the architecture and the training recipe. What no one can yet fully describe is what any specific model actually computes internally when it answers you. The parameters were shaped by an automated process, not written by engineers, so researchers study trained models the way biologists study organisms — by probing them. Anthropic's interpretability team has built "attribution graph" tools that trace which internal circuits fire on the way to an answer, and has found models doing unexpected things — like planning a rhyming word several tokens before writing it, despite generating one token at a time.

That's the honest state of the field in 2026: we understand the recipe completely and the resulting dish only partially. It's also why claims about what LLMs "really" do — whether "just autocomplete" or "genuine reasoning" — should be held loosely. The evidence is still coming in.

FAQ

Is an LLM just fancy autocomplete?

Mechanically, yes — it predicts the next token, exactly like your phone keyboard. But the comparison misleads more than it explains: predicting text well at this scale required the model to internalize working representations of grammar, code, logic, and facts about the world. Your keyboard's autocomplete cannot debug a program; the difference lives in what was learned, not in the prediction loop itself.

Do LLMs learn from my conversations?

Not during the conversation. A model's weights are frozen after training, so nothing you type changes the model itself in real time. Chat apps simulate memory by stuffing your history and saved preferences back into the context window on every turn. Providers may separately use conversation data to train future models, depending on their policy and your settings.

Why can't LLMs reliably count letters or do arithmetic?

Tokenization hides individual letters from the model, so counting characters means reasoning about the inside of chunks it can't see into. Arithmetic suffers because the model pattern-matches rather than executing an algorithm — fine for common sums, shaky for long ones. Modern systems increasingly route around this by writing and running actual code for math instead of "thinking" the answer.