Lesson 1 · Easy version

How an LLM writes an answer, and where it gets stuck

An LLM is a model that reads text and writes the text that comes next, one token at a time. It picks each next token by looking at probabilities, and those probabilities come from its parameters and from the text you give it.

The lecture opens with a question: Which office hour this week is least crowded? The course's TAs, senior students who help with teaching, hold office hours twice a week: Tuesday 4pm and Thursday 9am. Office hours are fixed times when any student can drop in and ask questions. Ask an LLM on its own, and it answers with confidence: Try the Wednesday afternoon session - midweek hours are usually quietest.

The course keeps a queue log, a record of how many students were waiting in line at each office hour. Over the last 3 weeks, Tuesday 4pm averaged 14 students and Thursday 9am only 3. So the right answer is Thursday 9am, and there is no office hour on Wednesday at all. The LLM never saw this log, yet it did not hesitate for a moment. This lesson explains why.

The ladder for this lesson

The ladder has 5 steps. Under each step you can see what you will be able to do after it.

  1. Tokens: the model reads pieces of textExplain how 6 words can become 7 tokens.
  2. Next-token prediction: one token per stepBuild an answer token by token in the widget.
  3. Parameters and trainingExplain why the queue log is not inside the model.
  4. The prompt and the context windowWork out how many passages fit in a prompt.
  5. Hallucination: a confident answer without dataExplain the Wednesday answer, and read Stanford's first slide.

What you need first: nothing, school maths is enough. If you already know a step, skip to the next one.

1Tokens: the model reads pieces of text

A token is a small piece of text: sometimes a whole word, sometimes part of a word, sometimes a punctuation mark. An LLM reads text by splitting it into tokens, and it writes its answer in tokens too.

Tokens are needed because a computer can only calculate with numbers. The model has a fixed list of tokens, and each token in it has an ID number. This list is called the vocabulary, and the program that splits text into its tokens is called the tokenizer.

A common word is often a single token, while a rarer word breaks into a few pieces. For example, Office hours are crowded on Tuesday has 6 words but 7 tokens: my split

Office hours are crowded on Tuesday

Here crowded broke into two tokens, crowd and ed. In the same way, if the vocabulary has no entry for a misspelling like Wednesdayy, the tokenizer still splits it into Wednesday and y. The real split depends on each model's own tokenizer.

Since the model reads and writes only tokens, this lecture measures every length in tokens, not in words.

2Next-token prediction: one token per step

While writing an answer, an LLM gives every token a probability at each step, then picks one token and adds it to the text. Then it repeats the same thing on the new text. This loop is called next-token prediction, and the whole answer comes out of it.

A probability is a number between 0 and 1 that gives the chance of something happening: 0 means never, and 1 means certain. For example, if 5 of the last 20 lectures had a quiz, the probability of a quiz in the next one is 5 ÷ 20 = 0.25. The probability of no quiz is 15 ÷ 20 = 0.75, and the probabilities of all the options always add up to 1.

Now the model is given this text: The least crowded office hour this week is. Its probabilities for the next token might look like this: my numbers

Next tokenProbability
" Wednesday"0.34
" Thursday"0.21
" Tuesday"0.18
" Friday"0.15
" Monday"?
Sum1.00

In reality, the thousands of other tokens also get very small probabilities. To keep the arithmetic small, assume there are only these 5 tokens.

Work out the Monday row

The five probabilities must add up to 1. What is Monday's probability?

Show the answer

The other four add up to 0.34 + 0.21 + 0.18 + 0.15 = 0.88. So Monday = 1 − 0.88 = 0.12.

Now suppose the model always takes the token with the highest probability. At the first step it takes Wednesday, and then it looks at the probabilities for the token after that. Press “Pick the highest” three times in the widget below. the later steps use my numbers too

Build an answer yourself

Click any token, or press “Pick the highest” three times.

After three steps the model writes what it wrote at the start of the lesson: Wednesday afternoon. At no step did it look at the queue log. Wednesday got 0.34 because people often write things like “midweek is quiet”. So a probability tells you which token is common after this text, not which one is true.

Real models often choose at random according to the probabilities. Then the 0.34 token comes up about 34% of the time, and the same question can get different answers.

Think first

In the widget you chose Thursday, morning and the full stop, the tokens at 0.21, 0.31 and 0.71. According to the queue log, Thursday 9am really is the least crowded. Did the model know the answer this time?

Show the answer

No. The answer came out right by luck. The model never saw the log, and it gave Thursday a lower probability than Wednesday.

3Parameters and training: where the model's knowledge lives

Parameters are the numbers inside a model. The input tokens are combined with these numbers by a fixed calculation, and that calculation produces the probability of every next token. Another name for parameters is weights, and there are a great many of them. The T5-11B model on Stanford's slides, for example, has about 11 billion parameters. from the paper

The values of these parameters are set by training. At first the parameters are random. Then the model reads a large amount of text, and at every position training checks what probability the model gave to the real next token. Then it nudges the parameters a little so that this probability goes up.

Here is one step. The training text contains the line Office hours are on Thursday. After reading Office hours are on, the model gives the next token Thursday a probability of 0.10. Training changes the parameters slightly, and now the same text gives Thursday 0.12. my example The total stays at 1, so all the other tokens together dropped from 0.90 to 0.88.

So whatever the model “knows” is spread across the values of its parameters, not stored in a separate file or database. Knowledge held this way is called parametric knowledge. Training stops on some date, and after that the parameters do not change. That date is called the knowledge cutoff.

The queue log is this course's own data, so it never went into any model's training. Its 14 and 3 are not in any parameter. This shows the two problems with parametric knowledge:

4The prompt and the context window

The prompt is the text you give the model right now: the question, instructions, and any document that comes with them. At every step, next-token prediction looks at all the text so far, and that includes the prompt. So the prompt's text can change every probability, while the parameters stay the same.

Now put the queue log line in the prompt before the question, the line that says 14 for Tuesday 4pm and 3 for Thursday 9am.

Think first: The parameters are the same, and only the prompt changed. Will Thursday's 0.21 and Wednesday's 0.34 go up or down? Make a guess, then switch on the box below.

Change the prompt, watch the probabilities

The parameters stay the same. Only the prompt changes. my numbers

With the log added, Thursday's probability goes from 0.21 to 0.81, and Wednesday's from 0.34 to 0.05, while the parameters stay the same. Put information that is not in the parameters into the prompt, and the model uses it. The whole lecture is built on this.

The prompt has a limit too. The model can read only a fixed number of tokens at once, and this limit is called the context window. Both the prompt and the answer the model writes count toward it. For example, if the window is 8,000 tokens and the prompt is 7,500, only 500 tokens are left for the answer.

All of a course's documents together can be far larger than this window. So for every question you have to choose which pieces go into the prompt, and lesson 2 starts from that choice.

Think first

The context window is 8,000 tokens. The question and instructions take 300 tokens, and the answer needs 700. Each passage, a piece of a document, is 500 tokens. At most how many passages fit in the prompt?

Show the answer

That leaves 8,000 − 300 − 700 = 7,000 tokens for passages, and 7,000 ÷ 500 = 14 passages.

5Hallucination: a confident answer without data

A hallucination is an answer that reads well and sounds confident, but is wrong or does not rest on any source. The Wednesday afternoon answer is one, and its cause comes from putting the last three steps together.

The queue log was not in the parameters, because it never went into training (step 3). It was not in the prompt either (step 4). Even so, next-token prediction picks some token at every step, and the most common token was Wednesday (step 2).

Next-token prediction has no step that stops when the data is missing. So a wrong answer is written with the same confidence as a right one, and the reader cannot tell them apart.

Stanford slide: Why Retrieval? The answer of an LLM alone and of an LLM with retrieval
Stanford's Why Retrieval slide. On the left is the LLM alone, which wrote Wednesday. On the right the prompt holds the queue log line, and the answer is Thursday 9am. Click the image to open it at full size.

The right half of the slide shows retrieval, which means finding the right text when the question arrives and putting it into the prompt. There the answer's 3 comes straight from the log line, so a reader can check it. Lesson 2 starts from here.

Think first

Which prompt gives the lowest chance of a hallucination? (a) Only the question. (b) The question and the queue log. (c) The question and an old 2024 forum post that says midweek afternoons are usually quiet.

Show the answer

(b). With (c), the model may trust the old post and write Wednesday again, so wrong text in the prompt also does harm. This point comes back in lesson 2.

What the dense version adds

The dense version contains this whole lesson, followed by 4 more steps that take you to the level of the Stanford slides:

  • Step 6: the slide's notation p(y | x, z) and θ, the probability of a whole answer, and greedy decoding versus sampling.
  • Step 7: answers from parameters alone: four sizes of T5, SSM, Exact Match, and the comparison with RAG.
  • Step 8: the papers' experiments: measuring hallucination, and adding new knowledge by swapping the documents.
  • Step 9: Stanford's four slides, which you read yourself.

Open the dense version

3 points from this lesson

  1. An LLM writes one token at a time. At each step it gives every token a probability and picks one.
  2. Its knowledge comes from two places: the parameters built by training, and the prompt given right now. The prompt's space is limited by the context window.
  3. A probability says what is common, not what is true. So even without the data, the model writes a confident answer.

New terms in this lesson

LLM
Large language model: a model that reads text and writes what comes next, one token at a time. For example, the model that wrote Wednesday afternoon.
Token
A small piece of text: a word, part of a word, or punctuation. The model reads and writes these. For example, crowded is two tokens, crowd and ed.
Vocabulary
The model's fixed list of tokens, each with an ID number. For example, it contains Wednesday but not Wednesdayy.
Tokenizer
The program that splits text into the vocabulary's tokens. For example, it splits Wednesdayy into Wednesday and y.
Probability
A number between 0 and 1 that gives the chance of something. The options always add up to 1. For example, a quiz in 5 of 20 lectures gives 5 ÷ 20 = 0.25.
Next-token prediction
Looking at the text so far, giving every token a probability, picking one, adding it, and repeating. For example, Wednesday, then afternoon, then a full stop.
Parameters (weights)
The numbers inside a model that produce the probability of every next token. For example, T5-11B has about 11 billion parameters.
Training
Changing the parameters little by little so that the real next token in the text gets a higher probability. For example, Thursday after Office hours are on goes from 0.10 to 0.12.
Parametric knowledge
Knowledge spread across the parameters. The model cannot name its source, and a new fact needs training again.
Knowledge cutoff
The date on which training stopped. Later events, and your private data, are not in the model. For example, with a March 2026 cutoff, an October 2026 announcement is not in the model.
Prompt
The text you give the model right now: the question, instructions, and accompanying documents. For example, the queue log line that lifted Thursday from 0.21 to 0.81.
Context window
The most tokens a model can read at once. Both the prompt and the answer count. For example, an 8,000-token window with a 7,500-token prompt leaves 500 for the answer.
Hallucination
A confident answer that is wrong or has no source. For example, Wednesday afternoon, written without seeing the log.