CS 329Z · Engineering AI Agents · Lecture 3

RAG + Agents, from first principles

This lecture rests on one question: how do you give an LLM the right information at the right time, and where does that approach break? Here the question is split into 18 lessons, and all of them are ready. Each lesson builds on the earlier ones, and every new term is explained where it first appears.

How to read

I assume you read code and know school maths, but that ML and statistics terms are new to you. If a lesson feels too easy or too heavy, say so.

Map of the lessons

This map shows which lesson rests on which. Each line is a dependency: the lower lesson needs the upper one. Lessons in the same row do not depend on each other, so you can read them in any order.

Click a lesson. The map shows what it needs first, what it opens up, and which lessons are independent of it.

Foundation: everything else rests on these Main path Depth: you can skip these the first time

The lines come from each lesson's “What you need first” line. A dependency that already follows from another path is not drawn again.

18 lessons

Each lesson starts from a gap left by an earlier one.

  1. 1How an LLM writes an answer, and where it gets stuckAsked about office hours, an LLM on its own confidently answered Wednesday without ever seeing the queue log. This lesson shows how an LLM builds an answer token by token, and where it gets stuck when the data is missing.LLM, Token, Vocabulary, Tokenizer, Probability, Next-token prediction, Parameters (weights), Training, Parametric knowledge, Knowledge cutoff, Prompt, Context window, HallucinationEasy · Dense
  2. 2Retrieval: finding before answeringAn LLM can only answer from what is in its prompt, and a prompt holds just 14 of a company's 60,000 passages. This lesson builds the simplest way to pick those 14 for every question.Corpus (collection), Document, Passage, Query, Retrieval, Citation, Non-parametric memory, Retriever, Score, Top-K, Word overlap, Stopwords, Reader, Zero-shot, Recall, Recall ceilingEasy · Dense
  3. 3Turning text into numbers: vectors and similarityThe query “when can I get help from a TA” shares no word with the office hours page that answers it, so word overlap gave every passage 0. Text has to be turned into vectors that can be compared.Vector, Dimension, Dot product (inner product), ∑ (sigma), Length, Cosine similarity, Normalize, Embedding, Embedding model (encoder), Index, Offline and onlineEasy · Dense
  4. 4Probability tools: getting ready to read the RAG formulaA retriever score of 1.56 and an LLM probability of 0.81 cannot simply be combined, yet lesson 5's RAG formula builds the answer's probability from exactly these numbers. This lesson builds the tools you need to run that formula by hand.Conditional probability, p(A|B), Chain rule (for probabilities), ∏ (capital pi), Marginalization, ≈, exp and e, Softmax, ∝, d⊤q, Natural log, ln, Loss, Likelihood, Negative log-likelihood, GradientEasy · Dense
  5. 5The RAG model: retriever and generator togetherAt the end of lesson 4 the retriever gave the old forum post more weight, and the right answer's probability fell from 0.685 to 0.315. The retriever has to be fixed, yet the training data holds only questions and answers.RAG, Latent variable, η and θ, BERT, MIPS, Generator, Encoder-decoder, Concatenate, RAG-Sequence, RAG-Token, End-to-end training, Backpropagation, Exact Match, Closed book and open bookEasy · Dense
  6. 6RAG pipeline: Naive, Advanced and ModularYou cannot train an LLM you rent through an API, so both mistakes from lesson 2 come back. They have to be fixed in the steps before the LLM.Frozen LLM, Pipeline, Naive RAG, Indexing time and query time, Pre-retrieval, Query routing, Query rewriting, Query expansion, Post-retrieval, Rerank and reranker, Summary, Fusion, Advanced RAG, Modular RAG, Pattern, Demonstrations (few-shot), Multi-hop questionEasy · Dense
  7. 7Chunking: cutting documents into pieces for retrievalCut the 40-page syllabus every 50 tokens and the milestone date splits across two chunks. Do not cut it at all, and the milestone's 2 lines dissolve into one vector for the whole syllabus.Word type and token in context, Static word embedding, Contextual embedding, Passage embedding, Multimodal embedding, Graph embedding, Mean pooling, Chunk, Fixed-size chunking, Overlap, Stride, Semantic chunking, Context-enriched chunking, Metadata, AI-driven dynamic chunkingEasy · Dense
  8. 8Giving a chunk back its lost contextAfter chunking, the ACME chunk has lost both the company name and the quarter, so the retriever cannot tell it apart from 49 near-identical chunks. The chunk needs its lost context back.Context loss, Contextual retrieval, Failed retrievals@20, Naive chunking, Late chunking, Long-context encoderEasy · Dense
  9. 9Questions spread across a whole document: RAPTOR and GraphRAGThe answer to “What are all the deadlines?” is spread over three passages, so top-K brings back only one piece of it. The index needs summaries of several passages, so that a question about the whole document finds its answer in one place.Detail question and thematic question, Clustering, Cluster and centroid, Soft clustering, RAPTOR, Tree, node, leaf, root, Entity, Relation, Knowledge graph and edge, Community, GraphRAG, Query-focused summarizationEasy · Dense
  10. 10Fast search over hundreds of millions of vectorsLesson 3's retriever compares the query with every vector in the index, and on 100 million passages each query takes about 7.7 seconds. The goal is to look at far fewer vectors and still find the nearest ones most of the time.Exact search (brute force), Big-O, O(N·d), Nearest neighbors, ANN, Distance, IVF, nprobe, Product quantization (PQ), Codebook, Proximity graph, Greedy search, HNSWEasy · Dense
  11. 11BM25: a score built from rare words, counts and lengthLesson 2's word overlap gave the queue log and the old forum post the same score of 2. Weighting each word by rarity, count and length breaks this tie, and the lesson shows which way it breaks.Term, Document frequency, n(q), IDF, TF-IDF, Term frequency, f(q, D), Saturation and k1, Length normalization, b and avgdl, BM25, Inverted index, Sparse and dense vectorEasy · Dense
  12. 12DPR: a retriever trained on question-passage pairsBM25 ranked the old forum post above the queue log. Two encoders, trained on question-passage pairs, have to bring the right passage to the top.DPR, Bi-encoder, Pretraining, Fine-tuning, Positive and negative passage, Contrastive loss, Batch, In-batch negatives, Hard negative, Top-k retrieval accuracyEasy · Dense
  13. 13Measuring a retriever: Hit@k, MRR, Recall@k and nDCGThe retriever improved and the second question's right passage moved to the top, yet the top-3 count stayed stuck at 0.667. We need numbers that can see that change.Relevance labels, Set, ∩, ∅, |A|, 𝟙[condition], Hit@k, Failed@k, MRR@k, Recall@k, log2, DCG and IDCG, nDCG@kEasy · Dense
  14. 14Hybrid search and rerankingBM25 put E2 at rank 3, and dense retrieval put it at rank 1. The two lists have to become one list, and then the top passages have to be read together with the query to fix their order.Paraphrase, Literal, Hybrid search, Reciprocal rank fusion (RRF), Attention, Cross-encoder, Reranking, ShortlistEasy · Dense
  15. 15ColBERT: a vector for every tokenFor the query error 4012, a one-vector bi-encoder ranked a short, wrong upload note above the long FAQ that has the answer. Keeping one vector per token, and adding up each query token's best match, brings the right passage to the top.Late interaction, MaxSim, ColBERT, Latency, FLOPs, CascadeEasy · Dense
  16. 16Agentic RAG: retrieval as a decisionBM25 ranked the old forum post above the queue log, and the pipeline sent it to the prompt without checking it. Retrieval has to become a decision at every step, and someone has to check what came back.Agent, Action and tool, Observation, Agentic RAG, Self-RAG, Reflection tokens, CRAG, Retrieval evaluator, Knowledge refinement, ThresholdEasy · Dense
  17. 17Search-R1: learning when and what to searchThe training data held only the right answer, not the search steps. The model has to learn when and what to search from the reward on its final answer alone.Reinforcement learning (RL), Policy, Rollout (trajectory), Reward, Search-R1, Advantage, PPO and value model, GRPO, Standard deviation, Reference LLM and KL penaltyEasy · Dense
  18. 18Where RAG breaks: 9 failures in retrieval, reading and the loopThe citation looked right, yet the answer was wrong, because the index still held the old office hours page. From outside every wrong answer looks the same, so the first job is to find where it went wrong: in retrieval, in reading, or in the agent's loop.Stale index, Permission leak, Lost in the middle, Distraction, Evidence conflict, Compounding error, No stopping rule, Prompt injectionEasy · Dense

Slides 2–8 are about the course project, so no lesson covers them.

All terms so far

The new terms from all 18 lessons, A to Z. Each term links to the lesson that explains it.

∑ (sigma)
Keep adding over the range written below it. For example, the sum of aibi for i = 1 to 2 is a1b1 + a2b2.
∏ (capital pi)
Multiply everything over the range written below it. For example, for three tokens, 0.21 × 0.31 × 0.71 = 0.046.
≈
Approximately equal. The RAG formula adds up only the top-K passages, so the sum is only approximately equal.
∝
Is proportional to. At the end, divide everything by one number so that the total is 1. For example, exps 2.718 and 1 become 0.731 and 0.269.
η and θ
The names of the retriever's (eta) and the generator's (theta) parameters. That is why the retriever is written pη(z|x) and the generator pθ(y|x, z).
⌈x⌉ (ceiling)
The smallest whole number that is x or larger. For example, ⌈23.75⌉ = 24, and chunks = ⌈(1,000 − 50) ÷ 40⌉ + 1 = 25.
∂ (partial derivative)
How fast an output changes when one input changes a little and everything else stays the same. For example, ∂𝓛/∂s+ = −0.814: raise the queue log's score by 0.01 and the loss falls by about 0.008.
ℝd
A list of d real numbers, that is, a vector with d dimensions. For example, [1.0, 0.5, 0.5, 1.0] ∈ ℝ4, and DPR's vectors are in ℝ768.
∇ (nabla)
∇θ𝓛: the loss's rate for every parameter in θ, as one list. Training nudges each parameter a little against its rate. For example, on the three scores these rates were −0.814, +0.506 and +0.307.
𝟙[condition]
1 if the condition is true, 0 if not. For example, q3's top 3 holds no right passage, so 𝟙[Rq3 ∩ Top3(q3) ≠ ∅] = 0.
⊆ (subset)
A ⊆ B means everything in A is also in B. For example, Top3(q1) ⊆ Top5(q1), so Recall@5 is never lower than Recall@3.
[CLS] and [SEP]
Special tokens in BERT's input. [CLS] comes first, and in a cross-encoder its last vector carries the whole pair. [SEP] separates the query from the passage, as in [CLS] query [SEP] passage [SEP].
Ablation
Removing or freezing one part of a model and measuring again, to see what that part contributes. For example, freezing the query encoder in RAG-Sequence drops NQ (dev) from 44.0 to 41.2.
Action and tool
What the agent chooses to do, such as searching for OH timings. A tool is the program that does it, such as a search engine.
Advanced RAG
The pipeline you get by adding pre-retrieval and post-retrieval steps to Naive RAG. For example, Rewrite, Retrieve, Rerank, Read.
Advantage
How much better or worse than expected a rollout did. For example, with rewards 0, 0, 0, 1, o4 gets +1.73 and the other three −0.58.
Agent
An LLM that works in a loop: it chooses an action, reads the observation, and decides the next action. For example, when the search for OH timings came back empty, the agent wrote a new query.
Agentic RAG
RAG in which retrieval is an action of the agent, not a fixed step of a pipeline. For example, 0 searches for 17 × 3 and 3 for the Jed Hoyer question.
AI-driven dynamic chunking
Asking an LLM where the topic changes in a text, and cutting there. It needs LLM calls for every document.
ANN
Approximate nearest neighbor search: looking at fewer vectors and finding the right neighbors most of the time. For example, finding 9 of the true top-10 gives recall 0.9.
Attention
Each token takes a weighted average of the other tokens' vectors as its new vector, and the weights add up to 1. For example, if crowded [0.5, 0.5] gives weight 0.6 to averaged [1.0, 0.0], it becomes [0.8, 0.2].
Backpropagation
Working backwards from the loss to get every parameter's gradient, backprop for short. In RAG it reaches the generator and the query encoder, but not the passage encoder.
BART
A model that reads text and writes text, with about 400 million parameters. It is RAG's generator. For example, in the Jeopardy test BART on its own was judged more factual in only 7.1% of pairs.
Batch
A group of questions that training processes together. For example, a batch of 128 questions in DPR.
Beam search
Moving forward while keeping the B unfinished answers with the highest scores at each step. For example, with B = 2 the queue log (2.15) and forum post (1.475) answers go on, and Homework (0.425) is dropped.
BERT
An encoder model Google released in 2018. Both of the RAG retriever's encoders, the query encoder and the passage encoder, are built from it.
Bi-encoder
Two separate encoders for the question and the passage, with the score as their dot product. Passages are encoded in advance. For example, even with 21 million passages the model runs once per question.
BIC
ln(N) · k − 2 ln(L̂): choosing the number of clusters by weighing the fit's likelihood against its parameter count, where the smallest BIC wins. For example, on 6 passages K = 2 gives −23.64 and K = 3 gives −20.11.
Big-O, O(N·d)
How the cost grows as N or d grows. For example, in O(N·d), if N doubles, the cost doubles too.
Bigram
Two words that appear side by side. For example, office hour in the queue log, and least crowded in the forum post.
BM25
Each query term's IDF × its TF part, summed over the terms. For example, 2.749 for the forum post and 2.473 for the queue log.
Candidate generation
Picking candidate passages cheaply, so that the expensive arithmetic runs only on them. For example, in ColBERT each query vector brings back its top 1,000 token vectors, so at most 32,000 passages.
Cascade
A line of retrievers in which each later step does more precise, more expensive work on fewer passages. For example, from tens of millions to 1,000, then 50, then 5.
Chain rule (for probabilities)
p(A and B) = p(A) × p(B|A). With many steps, multiply them all. For example, p(Tuesday) × p(full | Tuesday) = 0.5 × 0.8 = 0.40. The chain rule for rates in lesson 12 is a different rule.
Chain rule (for rates)
Rates that are linked one after another multiply. For example, the rate of ln Z with respect to the queue log's score is 2.72 × (1 ÷ 14.59) = 0.186. Lesson 4's probability chain rule is a different rule.
Chunk
A piece of a document made for retrieval, and the basic unit of retrieval. For example, the Grading part of the 40-page syllabus.
Citation
The source written next to an answer. For example, the [P3] in “Thursday 9am [P3]”, which points to the queue log.
Clipping and ε
Keeping the ratio between 1 − ε and 1 + ε, then taking the smaller term with min. For example, with ε = 0.2 and A = +1.73, the term for a ratio of 1.5 stops at 2.076.
Closed book and open book
Answering from parameters alone, or searching documents first and then answering. For example, on NQ the closed-book T5-11B scores 34.5 and the open-book RAG-Sequence 44.5.
Cluster and centroid
A cluster is a group, and its centroid is the average of its vectors. For example, the centroid of [0.1, 0.9], [0.2, 0.8] and [0.15, 0.85] is [0.15, 0.85].
Clustering
Splitting vectors into groups so that the vectors in a group are close to each other. For example, the syllabus [0.1, 0.9] and the project page [0.2, 0.8] have cosine 0.99, so they share a group.
Codebook
The list of sample vectors (codewords) for a piece. For example, #0 [1, 0], #1 [0, 1], #2 [0.5, 0.5], #3 [0.7, 0.7].
ColBERT
A late-interaction retriever that keeps a vector per token and scores with MaxSim. Built by Khattab and Zaharia in 2020. For example, its Recall@1000 on MS MARCO is 96.8.
Column vector and ⊤
A column vector has its numbers written one above the other, and ⊤ (transpose) turns it into a row. A row times a column is the dot product. For example, the row [0.9, 0.1] times the column [0.8, 0.2] = 0.74.
Community
A tightly linked part of a graph. For example, Office hours, TAs, Room 104 and Zoom.
Compounding error
An early mistake carries into every later step. For example, with each step right at 0.9, 3 steps give 0.729 and 5 steps give 0.59.
Concatenate
Join two texts end to end into one text. For example, question: … followed by passage: ….
Conditional probability, p(A|B)
The probability of A when B is known. Read “|” as “given”. For example, p(full | Tuesday) = 16 ÷ 20 = 0.80.
Context loss
After cutting, a chunk can no longer say which document or topic it belongs to. For example, The company’s revenue grew by 3% does not name the company.
Context window
The most tokens a model can read at once. Both the prompt and the answer count. For example, an 8,000-token window with a 7,500-token prompt leaves 500 for the answer.
Context-enriched chunking
Storing metadata or a summary with every chunk. For example, Source: CS 329Z syllabus, section: Grading above every chunk.
Contextual embedding
A separate vector for each token, shaped by its sentence. For example, BERT. In Late days may not be applied late is [0.1, 0.9], and in late afternoon office hours it is [0.9, 0.1]. my numbers
Contextual retrieval
Indexing each chunk with a 50–100 token context, written by an LLM, placed in front of it. For example, adding From ACME Corp’s Q2 2023 SEC filing.
Contrastive loss
Softmax over the positive and negative scores, then −ln of the positive's share. It looks only at differences between scores. For example, scores 1.0, 2.0, 1.5 give loss 1.68, and 5.0, 2.0, 0.5 give 0.059.
Corpus (collection)
The set of all documents. For example, the company's 2,000 documents, or this course's 6 short texts.
Cosine similarity
The dot product divided by the product of both lengths. It looks only at direction and stays between −1 and 1. For example, 0.74 ÷ (0.82 × 0.91) = 0.99.
CRAG
Corrective RAG: an evaluator calls the retrieval Correct, Incorrect or Ambiguous, and refinement, web search or both follow. For example, the forum post at 0.4 is Ambiguous.
Critic
A model that grades what something else wrote or retrieved. For example, in Self-RAG a critic built from Llama 2 7B adds reflection tokens to the training data, and the CRAG slide calls its evaluator a critic.
Cross-encoder
A reranker that joins the query and the passage, feeds them to one model, and gets a direct score for the pair. It runs in full for every passage, for example 10.7 ms per passage with BERT-base.
DCG and IDCG
Each rank's grade ÷ log2(rank + 1), added up. The IDCG is the DCG of the best order. For example, q1's order queue log (3), forum post (0), office hours page (1) gives DCG 3.5 and IDCG 3.631.
Decomposable similarity
A score that comes from a question vector and a passage vector, where the passage vector is built without seeing the question. Only then can an offline index exist. For example, the dot product, cosine and distance.
Demonstrations (few-shot)
Worked examples placed in the prompt before the question. For example, “Q: When is the milestone due? A: Week 6, Thursday [P1]”.
Detail question and thematic question
The answer is written in one place, or it is built from several passages. For example, “When is the milestone due?” and “What are all the deadlines?”.
Dimension
How many numbers a vector has. For example, [0.9, 0.1] has dimension 2, and BERT-base vectors have 768.
Distance
The length of the difference between two vectors. For example, [0.8, 0.2] and [0.9, 0.1] differ by [0.1, −0.1], so the distance is √(0.01 + 0.01) = 0.14.
Distant supervision
Finding paragraphs automatically for data that has only questions and answers: run the retriever and keep the passages that contain the answer text exactly. For example, the syllabus for the milestone question.
Distraction
A similar but useless passage leads the reader astray. For example, the old 2024 forum post.
Document
One item in the corpus: a web page, a file or an email. For example, the course syllabus.
Document frequency, n(q)
How many documents contain term q. For example, n = 1 for milestone and 2 for office.
Dot product (inner product)
Multiply the numbers in the same position, then add them all. For example, [0.8, 0.2] and [0.9, 0.1]: 0.72 + 0.02 = 0.74.
DPR
Dense Passage Retrieval: a bi-encoder retriever with two BERT encoders, trained with contrastive loss on question-passage pairs. Built by Karpukhin and colleagues in 2020, and the retriever inside lesson 5's RAG.
DrQA
A 2017 retriever-reader system: a fixed TF-IDF retriever returns 5 Wikipedia articles, and a neural reader picks a span from them. For example, 833,500 for the Warsaw question.
DSP
Demonstrate–Search–Predict: a program that passes text between a frozen LM and a frozen retriever, and builds its own demonstrations. Built at Stanford by Khattab and colleagues in 2022. For example, Exact Match 51.4 on HotPotQA.
d⊤q
A way of writing the dot product of vectors d and q. For example, the office hours page [0.9, 0.1] and the question [0.8, 0.2] give 0.74.
ef
The size of the list of nearest nodes found so far in an HNSW search. A larger ef means more distances and higher recall. For example, starting at node 70 with ef = 3 gives {75, 70, 80}.
Embedding
A text's vector, produced by a trained model. Texts saying similar things point in similar directions, such as “help from a TA” and “run by the TAs”.
Embedding model (encoder)
A model that reads a text and produces its fixed-length embedding. For example, turning the query into a vector like [0.8, 0.2].
Encoder-decoder
The encoder turns the input into vectors, and the decoder looks at them while it writes one token at a time. BART is such a model.
End-to-end training
Training all the parts together with one loss. In RAG the query encoder and the generator both learn from the same −ln p(y|x). For example, −ln 0.70 = 0.357.
Entity
Any named thing: a person, a place, a course, a document or a date. For example, Week 6, TAs, Room 104.
Evidence conflict
Two sources, or a source and the model's parameters, say different things. For example, Week 6 and Week 7.
Exact Match
An answer counts as right only if it matches the correct answer exactly. The score is the percentage of such questions. For example, RAG-Sequence's 44.5 on NQ.
Exact search (brute force)
Comparing with every vector to get the guaranteed top-k. For example, with 21 vectors, computing 21 distances.
exp and e
exp(s) = es, where e is about 2.718. Always above 0, and ea+1 = ea × e. For example, exp(1) = 2.718, exp(0) = 1, exp(−1) = 0.368.
Expected value, 𝔼[ · ]
Multiplying each outcome by its probability and adding, that is, the average over many rollouts. For example, 0.6 × 1 + 0.4 × 0 = 0.6.
F1
A score that combines precision and recall, 2PR ÷ (P + R). For example, P = 0.75 and R = 0.6 give F1 = 0.667.
Failed retrievals@20
When each query has one right chunk, the percentage of queries whose right chunk is not in the top 20. Smaller is better. For example, 29 failures in 1,000 queries is 2.9%.
Failed@k
1 − Hit@k, the share of queries with no right passage in the top k. For example, Failed@3 = 1 − 0.667 = 0.333.
FAISS
An open-source library for fast search over vectors. For example, DPR built an HNSW index over 21 million vectors in it, which took about 8.5 hours.
Fine-tuning
Training an already trained model further on data for your own task. For example, DPR trains BERT on question-passage pairs.
Fixed-size chunking
Cutting at a fixed number of tokens, even if a fact breaks in the middle. For example, every 50 tokens.
FLOPs
How many multiplications and additions it took, that is, the computer's work. For example, a dot product of 3 numbers takes 6 FLOPs, and one of 768 numbers takes 1,536.
Frozen LLM
An LLM whose parameters you cannot change. You can only send it prompts. For example, GPT or Gemini used through an API.
Fusion
Merging several ranked lists into one. For example, one list is queue log, forum post, office hours page, and another is office hours page, queue log. The passages that appear in both rise to the top.
GAE
Generalized advantage estimation: at every step δ = reward + next estimate − this estimate, then the sum of all later δ. With γ = λ = 1 the advantage is reward − V, such as 1 − 0.3 = 0.7.
Gaussian mixture model (GMM)
A weighted sum of bell curves: P(x) = ∑ πk 𝒩k(x). It gives every passage a membership for every cluster. For example, [0.6, 0.4] gets 0.904 for office hours and 0.096 for deadlines.
Generator
The model that reads the question and one passage and writes the answer, pθ(y|x, z). In RAG it is BART. For example, 0.9 for Thursday 9am with the queue log.
GiB
230 = 1,073,741,824 bytes, about 7% more than a GB. For example, ColBERT's MS MARCO index is 286 GiB.
Gradient
The calculation of which way, and how much, to change each parameter so that the loss goes down. For example, in lesson 1 training moved the parameters in the direction that raised Thursday's probability from 0.10 to 0.12 and lowered the loss from 2.303 to 2.120.
Graph embedding
A vector for a node or a relation in a network. For example, node2vec. In a course graph: the TA, office hours, and the runs relation between them.
GraphRAG
Answering global questions from a knowledge graph, its communities and their summaries. For example, the all-deadlines answer, built from the Project and Homework 1 communities.
Greedy decoding and sampling
Greedy decoding takes the highest-probability token at every step, and sampling chooses at random according to the probabilities. For example, greedy always writes Wednesday afternoon, while sampling starts with Thursday about 21 times in 100.
Greedy search
Moving at every step to the neighbor nearest to the query, until stuck. For example, moving from 70 to 75 for the query 73.
GRPO
An advantage from the mean and standard deviation of a group's rewards. No value model is needed. For example, for 0, 0, 0, 1 the mean is 0.25 and the standard deviation 0.433.
Hallucination
A confident answer that is wrong or has no source. For example, Wednesday afternoon, written without seeing the log.
Hard negative
A wrong passage that looks similar to the question. DPR takes these from BM25's top passages. For example, the forum post for our question.
Harmonic series
1 + 1/2 + 1/3 + …, the 1 ÷ rank numbers that MRR uses. Their sum grows slowly, about ln n + 0.577. For example, the first 1,000 add up to 7.485.
Hit@k
The share of queries with at least one right passage in the top k. For example, at k = 3, 2 of 3 queries, so 0.667.
HNSW
A layered proximity graph: a greedy walk from the top, dropping one layer when stuck. For example, 4 steps for the query 73.
Hybrid search
Running both a word-based and a dense retriever and merging their lists. For example, making one list from BM25's E1, E3, E2 and dense retrieval's E2, E1, E3.
Hyperparameter
A number in a formula or model that training does not change, picked by trying values on a dev set. For example, BM25's k1 and b, which the DPR paper set to 0.9 and 0.4.
i.i.d. and conditional embeddings
i.i.d.: each chunk’s vector is built completely separately from the other chunks, as in naive chunking. Conditional: each chunk’s vector is built with the whole document in view, as in late chunking, where It gets [0.9, 0.1].
IDF
ln((N − n + 0.5) ÷ (n + 0.5)), a bigger weight for a rarer term. For example, 1.299 for milestone and 0.588 for office.
In-batch negatives
Using the positives of the other questions in the batch as negatives. For example, in a batch of 128 each question gets 127 negatives.
Increasing function
A function where a larger input always gives a larger output, so the order is kept. exp and ln are such functions. For example, scores 1.56 and 0.74 give weights 0.499 and 0.220, in the same order.
Incremental re-indexing
Making embeddings only for new and changed passages, and removing the old passages of a changed document, instead of rebuilding the whole index. For example, 500 + 30 = 530 embeddings instead of 60,000.
Index
The list of passage embeddings built in advance. For example, 60,000 vectors for 60,000 passages.
Index hot-swapping
Replacing the document index without touching the parameters, so that the model's knowledge changes. For example, the 2018 index answered 68% of questions about 2018 leaders correctly, and the 2016 index only 4%.
Indexing time and query time
Steps that run once, and steps that run for every query. For example, the embeddings of 60,000 passages are made once, and the query's embedding every time.
Interaction matrix
A table of the similarity of every query token with every passage token. For example, the FAQ's table holds 6 × 2 = 12 dot products.
Inverted index
For each term, the list of documents that contain it. For example, office → office hours page, queue log.
ITER-RETGEN
Retrieval and generation in turns, where the previous answer joined to the question becomes the next query. Built by Shao and colleagues in 2023. For example, adding Thursday 9am lifts the office hours page from 1 to 3.
Iterative, recursive and adaptive retrieval
Three ways to search more than once: in turns for a fixed count, by breaking the question into smaller ones, and by letting the model decide. For example, ITER-RETGEN is iterative, and the Search-R1 trace is adaptive.
IVF
Inverted file index: splitting the vectors into clusters and looking only at the clusters near the query. For example, 11,000 comparisons for 1 million vectors.
k-means
A way to split vectors into k clusters: give each vector to its nearest centroid, then make each centroid the average, and repeat. For example, on 6 passages it ends after 3 rounds with A = [0.817, 0.2] and B = [0.2, 0.8].
KL divergence
The average of ln(new probability ÷ reference probability), weighted by the new model's probabilities. For example, 1.146 for 0.2 and 0.8 becoming 0.9 and 0.1, and 0.007 for 0.25 and 0.75.
Knowledge cutoff
The date on which training stopped. Later events, and your private data, are not in the model. For example, with a March 2026 cutoff, an October 2026 announcement is not in the model.
Knowledge graph and edge
A network of entities (nodes) and relations (edges). For example, an edge labelled “due” between the milestone and Week 6.
Knowledge refinement
Splitting documents into strips, dropping the useless strips, and joining the rest in order. For example, 2 of the syllabus's 3 strips stay for the milestone question.
Late chunking
Encoding the whole document first, then mean pooling each chunk’s token vectors. Then the vector of It is due Week 6 becomes [0.9, 0.1].
Late interaction
The query's and the passage's token vectors are built separately, and meet only at scoring. For example, ColBERT builds passage vectors offline, and computes the dot products when the query arrives.
Latency
The time from sending a query to getting the answer, in ms. For example, the cross-encoder's latency is 10,700 ms for 1,000 passages, and ColBERT's reranking latency is 61 ms.
Latent variable
A hidden choice that the data does not record. In RAG, the choice of which passage the answer came from. For example, a question-answer pair does not say that the answer came from the queue log.
Leiden and degree
Leiden splits a graph into communities with many edges inside. A node's degree is its number of edges. For example, Project has degree 3.
Length
The length of the arrow: square each number, add the squares, take the square root. For example, [0.8, 0.2] has length √(0.64 + 0.04) = 0.82.
Length normalization, b and avgdl
Lowering the TF part of documents longer than average, by an amount set by b. For example, with avgdl = 16.17 and b = 0.75, a document twice the average length gets 0.710.
Likelihood
The probability the model gave the correct data. For example, if it gave the right answer 0.12, the likelihood is 0.12.
Literal
Text that has to match exactly, such as the error code E-4012 or the order ID A77-310. A dense retriever can miss it, and BM25 catches it as a rare word.
LLM
Large language model: a model that reads text and writes what comes next, one token at a time. For example, the model that wrote Wednesday afternoon.
log2
The power you must raise 2 to in order to get a number. For example, log2 4 = 2, and log2 3 = 1.585.
Long late chunking
Late chunking for a document longer than the encoder’s limit, read in large overlapping windows. For example, 20,000 tokens read in windows of 8,192 tokens, each new window starting 1,000 tokens back.
Long-context encoder
An encoder that can read a very long text in one go. For example, 8,192 tokens, about 10 pages.
Lookup table (ADC)
Computing the score of every query piece with every codeword in advance and keeping them in a table. Each passage's score is then just a sum of table entries. For example, the score of code (0, 1) is 1 + 1 = 2.0.
Loss
A number for how wrong the model is. Training makes it small. For example, if the right answer has probability 0.5, the loss is −ln 0.5 = 0.693.
Lost in the middle
An LLM uses the middle of a long prompt less (position bias). For example, with 20 passages, 75.8% right at first place and 53.8% at 10th.
Map-reduce
Doing the same job on each piece separately (map), then combining the results (reduce). For example, a partial answer and a score from each community, then a final answer in score order.
Marginalization
Each case's weight times the probability within that case, added up over all cases. For example, 0.5 × 0.8 + 0.5 × 0.1 = 0.45.
Matrix and matrix-vector product
A table of numbers in rows and columns, and the dot product of each of its rows with a vector. For example, our 4 × 2 index times the query [0.8, 0.2] = [0.74, 0.26, 0.32, 1.56].
MaxSim
Each query token's largest dot product with the passage's tokens, then the sum of these. For example, 0.9 + 0.95 = 1.85 for the course FAQ.
Mean pooling
Making one vector by averaging several token vectors. For example, the average of [0.9, 0.1], [0.7, 0.3] and [0.8, 0.2] is [0.8, 0.2].
Metadata
Data about data: name, section, date. For example, CS 329Z syllabus, Grading, 30 Sep 2026.
Min-max normalisation
Mapping each list's scores onto 0 to 1 with (x − min) ÷ (max − min). For example, BM25's 7.2, 3.1 and 1.0 become 1, 0.339 and 0. The result depends on the list's min and max.
MIPS
Maximum inner product search: finding the K passages in the index with the largest dot products. For example, the 5 highest scores among 21 million passages.
Modular RAG
Splitting the pipeline into modules and connecting them in different orders. For example, Retrieve, Read, Retrieve, Read.
MRR@k
1 ÷ the rank of the first right passage, averaged over the queries, with 0 if none is in the top k. For example, ranks 1, 2 and none give (1 + 0.5 + 0) ÷ 3 = 0.5.
Multi-hop question
A question that needs several searches, because the parts of the answer sit in different passages. For example, where the least crowded office hour is: first the queue log, then the office hours page.
Multimodal embedding
Vectors for text and for images in the same space. For example, CLIP. A photo of office hours written on a whiteboard and the text office hours schedule can get nearby vectors.
Naive chunking
Cutting first, then encoding each chunk on its own. Then the vector of It is due Week 6 stays at [0.5, 0.5].
Naive RAG
The simplest pipeline: an index, then Retrieve and Read for every query, with no extra step in between.
Natural log, ln
The power you raise e to in order to get x. ln(ex) = x, and ln(a × b) = ln a + ln b. For example, ln(0.1) = −2.303.
nDCG@k
DCG ÷ IDCG, between 0 and 1, which looks at both grade and order. For example, q1's reversed order gives 2.5 ÷ 3.631 = 0.689.
Nearest neighbors
The vectors nearest to the query. For example, among the neighbors of the query 73, 75 comes first.
Negative log-likelihood
−ln p(right answer). The most common loss. For example, at p = 0.12, −ln 0.12 = 2.120.
Negative marginal log-likelihood
−ln of the right answer's probability, when that probability comes from marginalization over passages, added up over all examples. RAG's training loss. For example, over two examples 0.378 + 0.693 = 1.071.
Next-token prediction
Looking at the text so far, giving every token a probability, picking one, adding it, and repeating. For example, Wednesday, then afternoon, then a full stop.
No stopping rule
The agent has no clear condition for stopping. For example, 15 searches on one question.
Non-parametric memory
Knowledge kept outside the model, in the corpus. For example, the queue log: changing it needs no training.
Normalize
Divide every number in a vector by its length, so the length becomes 1 and the direction stays the same. For example, [0.8, 0.2] ÷ 0.82 = [0.97, 0.24].
Normalizer, Z
In softmax, the sum of all the exps. Dividing by it makes the weights add up to 1, and it is the division hidden behind ∝. For example, for exps 2.718 and 1, Z = 3.718.
nprobe
How many clusters IVF looks at. For example, with 1,000 clusters and nprobe = 10, only the vectors of 10 clusters are checked.
Observation
The result of an action, which the agent reads before its next step. For example, the result of the first search: every passage scored 0.
Offline and online
Offline: once, before any query arrives, such as building the passage embeddings. Online: on every query, such as making the query's embedding.
Open-domain QA
A question on any topic that comes without its paragraph, so the system has to find it in the whole corpus. For example, DrQA's Warsaw question over all of Wikipedia.
ORQA
A 2019 dense retriever that trained its retriever with extra pretraining. For example, on NQ the full system's Exact Match is 33.3 with ORQA and 41.5 with DPR.
Overlap
A new chunk starts by repeating the last few tokens of the previous one. To keep a fact of L tokens whole, it must be at least L − 1. For example, 6 for a 7-word date.
Parameters (weights)
The numbers inside a model that produce the probability of every next token. For example, T5-11B has about 11 billion parameters.
Parametric knowledge
Knowledge spread across the parameters. The model cannot name its source, and a new fact needs training again.
Paraphrase
The same statement in other words. For example, upload failed and upload keeps failing. BM25 misses it, because it counts shared words.
Passage
A small piece of a document, which the retriever returns. For example, the syllabus paragraph that holds the milestone date.
Passage embedding
One vector for a whole chunk, the kind the retrieval index stores. For example, the vectors from Sentence-BERT, DPR and E5.
Pattern
A fixed order of modules, such as Retrieve, Read.
Permission leak
The retriever brings a passage this user is not allowed to see. For example, other students' grades for a student.
Pipeline
A line of steps in which the output of each step is the input of the next. For example, Retrieve outputs the top-K passages, and those are the input of Read.
Points (absolute difference)
The plain difference between two percentages. For example, 52.2 to 54.0 is 1.8 points, which is an improvement of about 3.45% relative to 52.2.
Policy
Whatever makes the decision at each step. Here the LLM, which picks the next token every time: think, <search> or <answer>.
Positive and negative passage
The question's correct passage (pi+), and wrong passages (pi,j−). For example, the least-crowded question's positive is the queue log, and its negatives are the forum post and the Homework announcement.
Post-retrieval
Steps that improve the returned list after retrieval: rerank, summary and fusion.
Posterior and prior
The prior pη(z|x) is a passage's weight before seeing the answer, and the posterior p(z|x, y) is its weight after, that passage's share of the total probability. For example, the queue log's prior is 0.602 and its posterior 0.773.
PPO and value model
A common RL method that trains a separate model (the value model) to estimate the coming reward. For example, an estimate of 0.25 and a reward of 1 give an advantage of +0.75.
Pre-filter and post-filter
A pre-filter removes forbidden passages before ranking, and a post-filter after taking the top K. For example, with 5 forbidden passages in the top 14, a post-filter leaves 9.
Pre-retrieval
Steps that improve the query before retrieval: routing, rewriting and expansion.
Precision
The share of the returned passages that are right. For example, at K = 2 the queue log is right and the forum post is wrong, so precision = 1 ÷ 2 = 0.5.
Pretraining
A model's first, large training on lots of general text. For example, BERT is first trained on general text.
Primacy and recency bias
An LLM's habit of using the start (primacy) and the end (recency) of a prompt more. For example, with 20 passages, 75.8% at the first position, 63.2% at the last and 53.8% at the 10th.
Probability
A number between 0 and 1 that gives the chance of something. The options always add up to 1. For example, a quiz in 5 of 20 lectures gives 5 ÷ 20 = 0.25.
Probability ratio
πθ(token) ÷ πold(token), how many times larger a token's probability is than under the model that made the rollout. For example, 0.55 ÷ 0.5 = 1.1.
Product quantization (PQ)
Cutting a vector into pieces and storing, for each piece, the number of its nearest codeword. For example, [0.9, 0.1, 0.2, 0.8] becomes the code (0, 1).
Prompt
The text you give the model right now: the question, instructions, and accompanying documents. For example, the queue log line that lifted Thursday from 0.21 to 0.81.
Prompt caching
The LLM service keeps a repeated part of the prompt after the first call, such as the whole document, so that part is cheap in later calls. According to the blog, contextual retrieval then costs $1.02 per million document tokens.
Prompt injection
Retrieved text becomes a command for the agent. For example, a hidden Ignore previous instructions in a post.
Proximity graph
A graph in which every vector is linked to its nearby vectors. For example, 70 is linked to its neighbors 65 and 75.
QRels
A file of relevance labels: the IDs of the right documents for each query. For example, the late chunking paper computed nDCG@10 from such a file.
Query
The question you search for. For example, “which office hour is least crowded”.
Query augmentation
Filling the query with [mask] tokens up to 32 tokens. BERT builds vectors for these too, and they join MaxSim like extra query tokens.
Query expansion
Adding related words or other forms to the query. For example, adding “office hours” to “help ta” raises the office hours page from 0 to 2.
Query rewriting
Having an LLM turn the query into a clean search query. For example, “OH timings?” becomes “CS 329Z office hours schedule”.
Query routing
Deciding which index or source to search. For example, “when is the project milestone due” goes to the syllabus index.
Query-focused summarization
Writing a summary with the question in mind. For example, asking for “all the deadlines” makes the Project community's summary give only the dates.
RAG
Retrieval-Augmented Generation: joining a retriever and a generator into one model and training them together with one loss. For example, bringing the queue log and writing Thursday 9am from it.
RAG-Sequence
The whole answer with each passage, then the weighted sum. For example, 0.6 × 0.18 + 0.4 × 0.45 = 0.288.
RAG-Token
A weighted sum at every token, then the product of the tokens. For example, 0.74 × 0.48 = 0.355.
RAPTOR
Building a tree bottom up from summaries of clusters, and searching every level. For example, the all-deadlines question matches summary A.
Reader
The part that reads the query and the top-K passages and writes the answer. Today usually an LLM. For example, reading the queue log and writing “Thursday 9am”.
Reading comprehension
Answering a question about a text you are given. This is the reader's job after the retriever. For example, reading the Ottoman empire paragraph and answering 32.
Recall
How often the retriever finds the passage that is needed. For example, 60 out of 100 questions gives a recall of 60%.
Recall ceiling
The retriever's recall becomes the upper limit of the whole system. With a recall of 60%, no reader can get more than 60 of 100 answers from the passages.
Recall@k
Right passages found in the top k ÷ all right passages, averaged over the queries. For example, at k = 1 q1 found 1 of 2, so 0.5.
Reciprocal rank fusion (RRF)
1 ÷ (k + rank) from a passage's rank in each list, summed over all lists, with k = 60. For example, E1: 1 ÷ 61 + 1 ÷ 62 = 0.03252.
Recursive chunking
Cutting along a ladder of separators: paragraph first, then line, then sentence, then space. Only a piece still larger than the chunk size gets cut at the next separator. For example, at size 20 the Grading section becomes two chunks of 19 and 7 words.
Reference LLM and KL penalty
A frozen copy of the model from before training, and a penalty that stops the new model from moving far from it. For example, if <search> goes from probability 0.2 to 0.9, the penalty is large.
Reflection tokens
Special tokens added to the vocabulary, which the model writes like any other text. For example, Retrieve, Relevant or Irrelevant, and Supported or Partially.
Reinforcement learning (RL)
Learning by making attempts and using each attempt's reward. For example, if 2 of 4 rollouts have reward 1, the probability of the tokens in those two goes up.
Rejection sampling
Making several rollouts per question and fine-tuning only on those with the right answer. For example, 5 rollouts per question in the paper, with an average of 0.348 over 7 datasets.
Relation
A link between two entities. For example, (Project milestone, due, Week 6).
Relative gain
The difference divided by the old number, written as a percentage. For example, (51.4 − 36.9) ÷ 36.9 = 0.393, about 39%. The plain difference, 14.5, is in percentage points.
Relevance labels
The right passages fixed in advance for each query, sometimes with grades. For example, for q1 the queue log (grade 3) and the office hours page (grade 1).
Rerank and reranker
A slower but more accurate model that sorts the returned passages again. For example, it puts the forum post below the queue log when both tie at 2.
Reranking
Scoring and sorting a fast retriever's shortlist again with an accurate model. For example, on MS MARCO, MRR@10 goes from 16.7 for BM25 alone to 34.7 after reranking with BERT-base.
Residual
What is left after subtracting a vector's cluster centroid from it. IVFADC stores the code of this difference. For example, queue log [0.85, 0.2] − [0.817, 0.2] = [0.033, 0].
Retrieval
Finding useful passages for a query and putting them into the prompt. For example, the queue log going into the prompt with the question.
Retrieval collapse
The retriever starts bringing the same passages for every question during training, after which the generator learns to ignore them. For example, on the paper's story-writing task.
Retrieval evaluator
A small model that reads the question and one document and returns a confidence. For example, 0.4 for the forum post at K = 1.
Retrieved token masking
Computing training terms only on tokens the model wrote, leaving out the tokens inside <information>. For example, 114 of the 430 words in the slide's trace. Without the mask the average falls from 0.431 to 0.343.
Retriever
The part that scores every passage and returns the top K. For example, this lesson's word-overlap retriever.
Reward
The number at the end of a rollout. For example, 1 if the answer matches exactly, 0 otherwise.
Rollout (trajectory)
One complete attempt at a question, from the start to the final answer. For example, o1 on the Hoyer question: it searched both dates, then answered.
Rule for choosing neighbors (heuristic)
A candidate is linked only if it is nearer to the new node than to every node already linked. For example, X links to A and E2, and B and C are skipped.
Salient span masking (SSM)
An extra round of pretraining in which the model fills in blanked-out names and dates in Wikipedia sentences. For example, it lifted T5-11B's TriviaQA score from 50.1 to 60.5.
Saturation and k1
As the count grows, the gain fills up slowly towards a ceiling of k1 + 1. For example, with k1 = 1.2 the TF part for f = 1, 2, 10 is 1.000, 1.375, 1.964.
Score
A number for how useful a passage is for the query. For example, the queue log's word-overlap score of 2.
Search-R1
A method that teaches an LLM search decisions with RL, using think, search, information and answer tags. The reward is given only on the final answer.
Segment score (Self-RAG)
The weighted sum of a segment's critique shares: S = 1.0 × sIsRel + 1.0 × sIsSup + 0.5 × sIsUse. For example, the queue log segment's S = 0.9 + 0.875 + 0.5 × 0.75 = 2.15.
Self-RAG
The generator itself writes reflection tokens to decide when to retrieve and which segment to keep. For example, Relevant and Partially on the forum post segment.
Semantic chunking
Cutting at logical boundaries such as sentences or sections. For example, cutting the syllabus at its Grading, Office hours and Project headings.
Set, ∩, ∅, |A|
A group of things, the things two sets share, the empty set, and how many things a set holds. For example, |{queue log, office hours page}| = 2, and for q3, Rq3 ∩ Top3(q3) = ∅.
Shortlist
The retriever's top few passages, on which the reranker runs. For example, the top 100 of 60,000 passages, which the cross-encoder handles in 1,070 ms.
Soft clustering
Putting one passage into more than one cluster. For example, the FAQ page, whose cosine is high with both centroids (0.86 and 0.78).
Softmax
The exp of each score, divided by the sum of all the exps. It turns scores into probabilities. For example, 2.0, 1.0, 0.1 become 0.659, 0.242, 0.099.
Sparse and dense vector
A long vector that is mostly zeros, and a short one that is almost full. For example, 55 of the 69 slots in the forum post's vector are 0.
SQuAD
Stanford Question Answering Dataset: questions people wrote about Wikipedia paragraphs, with the answer always a span of the paragraph. For example, 87,599 training questions.
Stale index
The document changed, but the index keeps its old passage and embedding. For example, Tuesday 4pm after the move to Wednesday 2pm.
Standard deviation
A measure of how spread out numbers are: the average of the squared differences from the mean, then the square root. For example, for 0, 0, 0, 1 the mean is 0.25, the squares are 0.5625, 0.0625, 0.0625, 0.0625, their average is 0.1875, and the square root is 0.433.
Static word embedding
One vector per word type, whatever the context. For example, word2vec and GloVe. The vector for late stays [0.5, 0.5] in both sentences. my numbers
Stopwords
Very common words such as is, the and which, left out of the count. From “which office hour is least crowded”, which and is are removed.
Stride
The distance between the starts of two neighbouring chunks: chunk size − overlap. For example, 9 − 6 = 3.
Summary
Shortening passages before the prompt. For example, if a 500-token passage becomes 100 tokens, 14 passages take 1,400 tokens instead of 7,000. The risk: a detail such as Room 104 can drop out.
Term
One word of the query in search. For example, office and hour in which office hour.
Term frequency, f(q, D)
How many times term q appears in document D. For example, office has f = 2 in the queue log.
TF-IDF
The term's count × ln(N ÷ n), an older method than BM25. For example, N = 6, n = 2, f = 2 gives 2 × ln 3 = 2.20.
Thorough and Fast decoding
Two ways for RAG-Sequence to write an answer. Each passage produces one candidate. Thorough checks every candidate with every passage, Fast only with the passage that produced it. For example, with 50 passages on NQ, Thorough needs up to about 2,500 generator runs.
Threshold
A limit set in advance, where being above or below it triggers a different action. For example, in CRAG above 0.6 means Correct, and below 0.2 means Incorrect.
Token
A small piece of text: a word, part of a word, or punctuation. The model reads and writes these. For example, crowded is two tokens, crowd and ed.
Tokenizer
The program that splits text into the vocabulary's tokens. For example, it splits Wednesdayy into Wednesday and y.
Top-K
The K passages with the highest scores. For example, the queue log and the old forum post at K = 2.
Top-k retrieval accuracy
The percentage of questions with an answer passage in the top k. For example, DPR's 78.4% on NQ at top-20.
Training
Changing the parameters little by little so that the real next token in the text gets a higher probability. For example, Thursday after Office hours are on goes from 0.10 to 0.12.
Tree traversal and collapsed tree
Picking the top k on each level from the top down, or sorting all nodes in one list and taking them up to a token limit. For example, for the all-deadlines question the collapsed tree takes 5 nodes within 100 tokens, A first.
Tree, node, leaf, root
A structure with levels: a node is a box, a leaf is a node at the bottom, and the root is the node at the top. For example, 2 summaries above 6 chunks (the leaves), and 1 root above them.
UMAP
Turning vectors into ones with fewer numbers while keeping nearby points nearby. For example, RAPTOR's code makes every vector 10 numbers long.
Vector
A list of numbers with a fixed length, such as [0.9, 0.1]. With 2 numbers, it is an arrow on a plane.
Vocabulary
The model's fixed list of tokens, each with an ID number. For example, it contains Wednesday but not Wednesdayy.
Win rate
The percentage of judgments that favour one of two systems, where 50% means a tie. For example, about 450 of 625 judgments, or 72%.
Word overlap
How many of the query's words appear in the passage. The simplest score. For example, the queue log has office and hour, so its score is 2.
Word type and token in context
A word as listed in the dictionary, and one use of it in a sentence. For example, late is a word type, and its use in Late days may not be applied is a token in context.
Zero-shot
Doing a task from an instruction alone, without training on that task. For example, the prompt line “Answer the question using the passage below”.