In agentic RAG, retrieval is no longer a fixed step in a pipeline. At every step an LLM decides whether to retrieve, what to retrieve, and whether what came back is good enough. An LLM working this way is called an agent, and this lesson shows two ways of making these decisions: Self-RAG and CRAG.
The need for it showed up in lesson 11. For the question which office hour is least crowded, BM25 gave the old forum post 2.749 and the queue log 2.473. At K = 1 only the forum post reaches the prompt, and the LLM writes midweek afternoons [P4], where [P4] is that post's ID. This is the wrong answer from lesson 1, now with a citation. The opposite mistake happens too: even for what is 17 × 3? the pipeline searches 60,000 passages.
The ladder for this lesson
The ladder has 9 steps. The first 5 are the same as in the easy version, and steps 6 to 9 take you to the level of the Stanford slides. Under each step you can see what you will be able to do after it.
What you need first: Lesson 1 (next-token prediction, vocabulary), Lesson 6 (pipeline, query rewriting, ITER-RETGEN) and Lesson 14 (reranker). From step 6 on, Lesson 4 (probability, softmax, ln, loss) and Lesson 13 (points) also help. If you already know a step, skip to the next one. A word with a dotted underline and ▾ opens its definition when you click it.
1Agents: action, observation and the loop
An agent is an LLM that works in a loop. It chooses the next thing to do, that thing gets done, and then it reads the result and chooses again. What the agent chooses to do is called an action, such as searching or writing the answer. A program actually carries out the action, a search engine for example, and that program is called a tool. What the tool returns, and what the agent reads before its next step, is called the observation.
The need for a loop showed up in lesson 6. There, searched twice, and a person had fixed that count in advance. But on the Jed Hoyer question the model made three searches, and 17 × 3 needs none. No fixed count suits both, so the count should come from whoever is reading the result of each search.
Here is a small loop. The user asks in shorthand, OH timings? The retriever is the word overlap from lesson 6, which counts how many query words also appear in a passage. my example loop
Step
The agent's action
Observation
1
Search: OH timings
Every passage scores 0
2
Search: CS 329Z office hours schedule
Office hours page 2, queue log 1
3
Answer: Tuesday 4pm, Room 104, or Thursday 9am, Zoom [P2]
Loop ends
The first query's words, oh and timings, appear in no passage. In the second query, office and hours both appear in the office hours page, so it scores 2. The queue log has office, but it says hour instead of hours, so it scores 1. The [P2] in the answer is the office hours page's ID.
The second step came from the first step's observation: nothing came back, so the agent rewrote the query. This is the from lesson 6. Lesson 6's Advanced RAG pipeline rewrites every query, which costs an extra LLM call on every query. Each step of the agent also costs an LLM call, but here the rewriting step happened only because the search came back empty.
Think first
Suppose the user had typed office hour timings in the first place. What would each passage score, and would the agent need to rewrite the query?
Show the answer
The queue log has both office and hour, so it scores 2. The office hours page has only office, so it scores 1. The search did not come back empty, so there is no reason to change the query. But the queue log has the times and not the place, so the agent now has to decide whether what it found is enough. That is the third decision in the next step.
2Three decisions, and where a pipeline falls short
At every step, an agent that retrieves faces three decisions. First, retrieve or answer directly. Second, which query to send. Third, whether what came back is good enough. All three appeared in the loop of step 1: the agent chose to search, then wrote a new query, then judged the result good enough and answered.
A pipeline makes none of these decisions. It runs the same steps on every question. Below is one question per decision where the pipeline falls short:
Decision
Question
What the pipeline does
What an agent can do
1. Retrieve?
What is 17 × 3?
Searches 60,000 passagesand puts about 2,500 tokens into the prompt
0 searches, answers 51 directly
2. Which query?
OH timings?
Sends that query as is, gets 0 passages
Changes the query when the search comes back empty
3. Good enough?
Least crowded, K = 1
Sends the forum post alone
The post is a 2024 opinion, so it searches again
When retrieval becomes an action of the agent, and these three decisions are made at every step, the result is called agentic RAG. Stanford's slide says exactly this:
Stanford's slide: retrieval inside the loop. The top line says that nothing so far decides whether to retrieve, what to ask, or when to stop. Below are the same three decisions. Click the image to open it at full size.
The when to stop at the top and the third decision at the bottom are the same thing. If the result is good enough, the agent stops and answers, and if it is not, the agent searches again.
The question now is who makes these decisions. This lesson gives two answers. In Self-RAG the model that writes the answer makes them itself, and in CRAG a separate, small model checks the result.
3Self-RAG: decisions as tokens
The most direct option is to let the LLM that writes the answer make the decisions, because it is reading both the question and the passages. The difficulty is that an LLM only writes the . It does not announce decisions separately.
Self-RAG solves this by turning the decisions into tokens as well. It adds some special tokens to the model's , called reflection tokens. The model gets special training, and then it writes these tokens just like any other text. Self-RAG is a 2024 method by Asai and colleagues, and its main tokens are these:
Token
Meaning
Decision
Retrieve
Search now
Decision 1
Relevant / Irrelevant
Whether the retrieved passage is useful for the question
Decision 3
Supported / Partially
Whether the passage fully or only partly backs up what the model wrote
A second check for decision 3
The paper has a few more tokens, such as No support. A fourth kind gives the whole answer a score from 1 to 5, and it is covered in the dense version.
Now take our question: Which office hour this week is least crowded? Suppose the model wrote Retrieve, and the retriever brought the queue log, the old forum post and the Homework 1 announcement. The model writes the next segment separately with each passage, and adds reflection tokens to each segment. my example and tokens
Passage
Reflection tokens
Segment the model wrote
Queue log
Relevant, Supported
Thursday 9am, about 3 waiting.
Old forum post
Relevant, Partially
Wednesday afternoon is quietest.
Homework 1 announcement
?
Homework 1 will be released on Oct 5.
The forum post segment is Partially, because the post says midweek, not Wednesday, and it also says usually. So the Wednesday claim is only half in the passage. The queue log claim is written out in full in the queue log, so that segment is Supported.
Work out the third row
Which token goes on the Homework segment? What is the order of the three segments, and which one joins the answer?
Show the answer
Irrelevant. The Homework date is true, and the announcement even backs it up, but the question is about crowding at office hours. The order is the queue log, then the forum post, then Homework. The queue log segment is Relevant and Supported, so that is the one that joins the answer: Thursday 9am.
So Self-RAG checks each segment on two counts: whether it is useful for the question, and whether the passage backs it up. Stanford's slide shows the paper's own example, and you can now read it:
Stanford's slide: the three steps of Self-RAG. Green boxes are the good tokens, the orange box is Partially, and the red box is Irrelevant.
In step 1 the model writes a first sentence, then Retrieve, and three passages arrive. In step 2 it writes one segment with each passage, and in step 3 the tokens give the order 1 > 3 > 2. The Texas claim is true, but its passage is Irrelevant, so it comes last. Then the model writes Retrieve again and repeats the process.
Self-RAG has a cost. The generator needs special training, so a from an API does not work here. It has to write a separate segment with each passage, which means K generations, like the model of lesson 5. And Supported only says that the claim is in the passage. It does not say that the passage is still true.
4CRAG: a separate evaluator and two thresholds
Most systems are built on a frozen LLM, so decision 3 has to be made by something outside the generator. A first idea is the from lesson 14. But a reranker only changes the order. Suppose the retriever brought only the Homework 1 announcement and the project page, both useless for the crowding question. The reranker still puts one of them on top, and that one goes into the prompt. my example
In CRAG, a small model gives every document a confidence: how right this document is for this question. This model is called the retrieval evaluator. CRAG stands for Corrective RAG, a 2024 method by Yan and colleagues. The confidence has a meaning of its own, so you can set a limit on it in advance, and such a limit is called a threshold.
CRAG keeps two thresholds, an upper one and a lower one. Here the confidence runs from 0 to 1, and the thresholds are 0.6 and 0.2. my scale and numbers The paper's own numbers are in the dense version. Start with the case where the retriever brought a single document, K = 1:
Single document
Confidence
Verdict
Queue log
0.85
Correctabove 0.6
Old forum post
0.4
Ambiguousbetween 0.2 and 0.6
Homework 1 announcement
0.05
Incorrectbelow 0.2
With several documents, one document above 0.6 is enough for Correct, while Incorrect needs every document below 0.2. In the reranker example Homework is at 0.05 and the project page at 0.1, so the verdict is Incorrect. my 0.1 for the project page Each verdict triggers a different action:
Verdict
Action
What the generator gets
Correct
Knowledge refinement: split the documents into strips, drop the useless strips, join the rest
Question + cleaned documents
Incorrect
Drop the documents, rewrite the query, search the web
Question + web results
Ambiguous
Both
Question + cleaned documents + web results
CRAG's verdict at K = 1
The retriever brought one document. Change the evaluator's confidence and see what CRAG does. The thresholds are 0.6 and 0.2. my thresholds and confidences
Even on the Correct path the document does not go in whole. CRAG splits it into small pieces called strips. It then checks each strip with the evaluator, drops the useless strips, and joins the rest in order. This is called knowledge refinement. For the question When is the milestone due?, the syllabus's three sentences become three strips: my example
Syllabus strip
Keep?
The project milestone is due Week 6, Thursday 11:59pm.
Yes, this is the answer
Late days may not be applied to the milestone.
Yes, it is about the milestone deadline
The final report is due Week 10.
No, this is about the final report
The generator gets 2 of the 3 strips. The Week 10 line never reaches the prompt, so the generator cannot mistake it for the milestone date.
On the Incorrect path CRAG drops its documents. An LLM rewrites the question as keywords suited to a search engine, and CRAG searches the web. On the slide the question is Who was the screenwriter for Death of a Batman?, and the new query has three parts: Death of a Batman, screenwriter and Wikipedia.
Stanford's slide: CRAG. Correct runs Knowledge Refinement, Incorrect runs Knowledge Searching, and Ambiguous runs both. The bottom row shows what the generator gets for each verdict.
Watch out: Not every text from the web can be trusted. It can contain false claims, or even instructions that steer the agent off course, which come up in lesson 18. That is why the slide has a Select step after the web search too.
Think first
At K = 1 the forum post arrives alone, and the evaluator gives it 0.4. What does CRAG do, and what does the generator get?
Show the answer
0.4 is between 0.2 and 0.6, so the verdict is Ambiguous. CRAG refines the forum post and keeps it, and it also rewrites the query and searches the web. The generator gets the question, the cleaned part of the forum post and the web results.
5Self-RAG and CRAG side by side, and their limits
Now go back to the two questions from the start. On 17 × 3, a well-trained Self-RAG does not write Retrieve, and writes 51 directly. In the paper's own example, too, the model does not search for a prompt that asks for an essay. from the paper CRAG would search here as well, because it searches on every question and only checks the result.
If the forum post arrives alone at K = 1, Self-RAG's Wednesday segment is Partially, and the model can write Retrieve again. CRAG's evaluator gives the forum post 0.4, so CRAG also searches the web. Neither of them passes the forum post on unchecked any more.
Self-RAG
CRAG
Who decides
The generator itself, with reflection tokens
A separate evaluator, with two thresholds
Does the generator change?
Yes, special training
No, any LLM
Decision 1: retrieve?
With the Retrieve token
Always retrieves
Decision 2: which query?
Does not write a separate queryfrom the paper
Rewrites the query for the web
Decision 3: good enough?
With the Relevant and Supported tokens
With the evaluator's confidence
Both have limits. A person sets CRAG's thresholds, and the evaluator can make mistakes too. Self-RAG's reflection tokens are also written by the model, so they can be wrong as well. And neither of them learned these decisions by trying things out.
In lesson 17 the model learns these decisions by trying and seeing the result: when to search, which query to write, and when to stop. In a loop, one wrong decision also affects the later steps, which is the subject of lesson 18.
From here: the Stanford level
The 5 steps above are the same as in the easy version. The 4 steps below take you to the level of the Stanford slides: first the maths of Self-RAG's tokens, then its training and results, then CRAG's real evaluator and its results. Finally you read the slides yourself. Every piece of maths is explained where it first appears.
6The maths of Self-RAG: from token probabilities to a segment score
Step 3 had three kinds of tokens, and I gave the order of the segments by looking at the tokens. The paper has four kinds of tokens, and the order comes from a number. Each kind has a few values of its own: from the paper
Kind
Values
What it checks
Retrieve
yes, no, continue
Search now, do not search, or keep using the passage retrieved earlier
IsRel
relevant, irrelevant
Whether the passage is useful for the question
IsSup
fully supported, partially supported, no support
How far the passage backs up the segment
IsUse
5, 4, 3, 2, 1
How useful the answer is for the question
The paper calls the last three kinds critique tokens, because they judge what was written. In each kind one value is the best, shown in bold in the table. The Supported of step 3 is fully supported here, and Partially is partially supported.
From a token's probability to a share
At every step the model gives each token in the whole vocabulary a , and together they add up to 1. Self-RAG picks out the tokens of one kind and shares their probabilities among themselves. Suppose that after the queue log segment, relevant has probability 0.54, irrelevant 0.06, and all other tokens together 0.40. my numbers
Add the two IsRel tokens: 0.54 + 0.06 = 0.60.
Divide relevant by that sum: 0.54 ÷ 0.60 = 0.9.
This 0.9 is relevant's share among the IsRel tokens only. The other 0.40 plays no part, so the share always stays between 0 and 1. The paper writes it like this:
Here p(relevant) is the probability of the relevant token, 0.54, and the bottom is the sum of both, 0.60. In lesson 4's the share was found the same way: divide one number by the sum of its group.
The retrieval decision: the threshold δ
Decision 1 uses the same kind of share. Self-RAG retrieves when yes's share, between yes and no, is above a threshold. The paper writes this threshold as δ, read "delta":
retrieve if p(yes) ÷ ( p(yes) + p(no) ) > δ
Look at two questions, where p(yes) and p(no) are the probabilities of the Retrieve tokens: my numbers
Question
p(yes), p(no)
yes's share
What is 17 × 3?
0.03, 0.87
0.03 ÷ 0.90 = 0.033
The least-crowded question
0.56, 0.24
0.56 ÷ 0.80 = 0.70
Now change δ and see which question retrieves:
δ
17 × 3 (0.033)
Least crowded (0.70)
0
Retrieves
Retrieves
0.2
Does not retrieve
Retrieves
0.5
Does not retrieve
Retrieves
0.8
Does not retrieve
Does not retrieve
So the larger δ is, the less the model retrieves. The paper sets δ = 0.2 for most tasks, to make the model retrieve often. On a task called ALCE, where every claim needs a citation, it sets δ = 0. from the paper
Each segment's score: a weighted sum
Now the shares of the three critique kinds combine into one number. First each kind's share, by the paper's rules: from the paper
IsRel: relevant's share, as computed above.
IsSup: fully's share, plus half of partially's share. So partially supported gets half weight.
IsUse: the five values get weights. 5 gets 1, 4 gets 0.5, 3 gets 0, 2 gets −0.5, and 1 gets −1. Each value's share is multiplied by its weight, and the products are added.
The IsUse sum is a kind of average. Suppose 5 has share 0.6, 4 has 0.3 and 3 has 0.1. Then sIsUse = 1 × 0.6 + 0.5 × 0.3 + 0 × 0.1 = 0.75. If all the share were on 5 it would be 1, and if all of it were on 1 it would be −1. So sIsUse stays between −1 and 1.
For our three segments, the model gave these shares. Within each group the shares add up to 1: my numbers
In step 4 the three shares were added with weights. The paper's default weights are 1.0 for IsRel and IsSup, and 0.5 for IsUse. from the paper The formula is exactly these four steps:
S = wIsRel sIsRel + wIsSup sIsSup + wIsUse sIsUse
Here each w is a weight (1.0, 1.0 and 0.5), and each s is that kind's share (0.9, 0.875 and 0.75). The same arithmetic on all three segments:
Segment
sIsRel, sIsSup, sIsUse
S
Queue log
0.9, 0.875, 0.75
2.15
Old forum post
0.7, 0.55, 0.45
1.475
Homework 1 announcement
0.05, 0.75, −0.75
0.425
The order is the same as in step 3: the queue log, then the forum post, then Homework. Homework's sIsSup is 0.75, so its claim is well backed by the announcement. But sIsRel of 0.05 and sIsUse of −0.75 push it to the bottom.
Think first
Suppose only IsSup counted, so wIsRel = 0, wIsUse = 0 and wIsSup = 1. What would the order of the three segments be?
Show the answer
Then S is just sIsSup: queue log 0.875, Homework 0.75 and forum post 0.55. Homework moves above the forum post, because its claim is backed by its own passage, even though it has nothing to do with the question. That is why IsRel is needed: Supported only says that the claim is in the passage.
The paper's full score also adds the segment's own probability to S:
f(yt, d) = p(yt | x, d, y<t) + S
Here x is the question, d the passage, yt the t-th segment, and y<t all the segments written before it. The paper treats each segment as one sentence. p(yt | x, d, y<t) says how likely the model finds this sentence, given the question, the passage and the earlier segments. from the paper
Changing the weights changes the model's behaviour without any new training. Suppose wIsSup = 2. Then the scores are 3.025, 2.025 and 1.175, so the order stays the same, but being backed up counts for more. In the paper, raising the IsSup weight on the ASQA task raised citation precision, meaning more citations really backed up their claims. At the same time a fluency score called MAUVE went down. from the paper, trend read from a graph
Beam search: two unfinished answers at once
The paper does not move forward with just the single best segment at each step. It keeps the B unfinished answers with the highest scores, and extends each of them to the next step. This is called beam search, and B is the width of the beam. In the paper B = 2. from the paper
On our three segments, B = 2 means that the unfinished answers with the queue log segment (2.15) and the forum post segment (1.475) both go on to the next step. The Homework one (0.425) is dropped there. At the end the best complete answer is returned. With B = 1 this would be like the of lesson 1, only with a whole sentence instead of a token.
7Self-RAG's training, and the paper's results
A model can write reflection tokens only if the training data has these tokens in the right places. Adding tokens to every sentence by hand is expensive, so the paper has another model do it. A model that grades what something else wrote is called a critic. Self-RAG's training takes three steps: from the paper
GPT-4 was given an instruction and a few examples for each kind, and asked to write the tokens. This gave 4,000 to 20,000 examples per kind, such as 19,317 for IsRel, 11,181 for IsSup and 3,831 for IsUse.
A critic was trained from Llama 2 7B on these examples, so that from then on the critic writes tokens instead of GPT-4.
The critic added reflection tokens and retrieved passages to 145,619 training examples. Then the generator, Llama 2 7B or 13B, was trained on these examples.
The critic exists only to build training data. Once trained, the generator writes the tokens itself, so no critic is needed when answering.
The generator's training uses the same as lesson 4: −ln of the probability of each next token, with reflection tokens included. For example, if a correct token's probability goes from 0.5 to 0.9, its −ln falls from 0.693 to 0.105. The passage text gets no loss, because the model did not write it. The retriever supplied it. The paper writes the same thing like this:
maxM 𝔼 [ log pM(y, r | x) ]
Here M is the generator, x the question, y the answer, and r its reflection tokens. 𝔼 means the average over all training examples. Making log as large as possible and making −log, the loss, as small as possible are the same thing.
The paper measured how trustworthy the critic is by comparing it with GPT-4's tokens: from the paper
Kind
Agreement of critic with GPT-4 (%)
Retrieve
93.8
IsSup
93.5
IsRel
80.2
IsUse
73.5
IsUse is the weakest, because the critic often confuses 5 and 4, and people disagree on these two as well. Human annotators also checked 50 PopQA answers. The model's IsRel tokens matched their judgement 95.0% of the time, and its IsSup tokens 90.0% of the time.
When answering, the paper took the top 5 passages from a dense retriever called Contriever. The weights were 1.0, 1.0 and 0.5, δ = 0.2, and the beam width 2. from the paper
The results, read in points
Lesson 13 introduced : the plain difference between two percentages. For example, 50.8% to 54.9% is 4.1 points. The paper's four tasks are these. PopQA has 1,399 questions about rare people and things, PubHealth has health claims to judge true or false, ARC-Challenge has multiple-choice science questions, and Bio asks for biographies. from the paper
Task
Self-RAG 7B
ChatGPT + retrieval
ChatGPT
PopQAaccuracy
54.9
50.8
29.3
PubHealthaccuracy
72.4
54.7
70.1
ARC-Challengeaccuracy
67.3
75.3
75.3
BioFactScore, the % of small claims that are true
81.2
–
71.8
On PopQA, Self-RAG 7B is 54.9 − 50.8 = 4.1 points ahead of ChatGPT with retrieval.
On ARC, ChatGPT is 75.3 − 67.3 = 8.0 points ahead of Self-RAG 7B. The paper's 13B model also stays at 73.1, so Self-RAG does not win on every task.
The paper also measured what happens when one part is removed, which is called an . This was done on a 7B model trained on 50,000 examples, which is why the first row is lower than in the table above: from the paper
What changed
PopQA
PubHealth
Full Self-RAG
45.5
73.5
Never retrieves when answering
24.7
73.0
Always uses only the top 1 passage
41.8
73.1
Instead of δ, retrieves only when the model itself writes Retrieve = yes
28.3
72.6
No critic tokens in the training data
42.6
72.0
On PopQA, never retrieving drops accuracy from 45.5 to 24.7, a fall of 20.8 points. On PubHealth the same fall is only 0.5 points. So retrieval is not equally useful on every task, and that is the reason for decision 1. On PopQA the δ = 0.2 rule is 45.5 − 28.3 = 17.2 points better than retrieving only on Retrieve = yes, because it retrieves more often.
8CRAG's real evaluator, thresholds and results
The first figure of the CRAG paper shows the problem clearly. The question is Who was the screenwriter for Death of a Batman?, and the retriever brings a document about Batman (1989 film). The name Hamm appears in it, and the generator answers Hamm, which is wrong. from the paper
CRAG's evaluator is a model called T5-large, with 0.77 billion parameters. Self-RAG's critic is Llama 2 7B, about 9 times larger. The evaluator reads the question and one document together and returns a score between −1 and 1. In training, a correct document had the label 1, and a wrong one −1. from the paper
PopQA has about 14,000 questions, and the paper trained the evaluator on all of them except the 1,399 test questions. The same evaluator was then used on the other three tasks as well. Each question usually brings 10 documents, and the evaluator scores each one separately. from the paper
The −1 to 1 scale
In the easy version I put the confidence on a 0 to 1 scale. The paper's scale runs from −1 to 1, and one simple rule links the two: c = (s + 1) ÷ 2. Here s is the paper's score and c the 0 to 1 confidence. For example, s = −1 gives c = 0, s = 0 gives c = 0.5, and s = 1 gives c = 1. The rule adds 1 to every score and divides by 2, so the order is the same on both scales.
The paper set the thresholds by trying values on each dataset, so they are : from the paper, last column my arithmetic
Dataset
Upper
Lower
On 0 to 1
PopQA
0.59
−0.99
0.795, 0.005
PubHealth, ARC
0.5
−0.91
0.75, 0.045
Biography
0.95
−0.91
0.975, 0.045
The last column comes from c = (s + 1) ÷ 2. For PubHealth's lower threshold: (−0.91 + 1) ÷ 2 = 0.09 ÷ 2 = 0.045. The lower thresholds sit almost at −1, so Incorrect happens only when every document is almost completely useless.
Think first
In the easy version, the Homework 1 announcement alone at 0.05 was Incorrect. With PubHealth's thresholds, on the 0 to 1 scale, what is its verdict?
Show the answer
PubHealth's thresholds on the 0 to 1 scale are 0.75 and 0.045. 0.05 is neither above 0.75 nor below 0.045, so the verdict is Ambiguous. With the paper's thresholds, CRAG rarely throws documents away. It keeps them and also searches the web.
Verdicts on a set of documents
The paper's rule applies to the whole set: from the paper
Correct: at least one document scores above the upper threshold.
Incorrect: every document scores below the lower threshold.
Ambiguous: every other case.
Now take my scores on the paper's scale: queue log 0.70, forum post −0.20, Homework −0.95 and project page −0.97. my scores
What the retriever brought
PopQA0.59, −0.99
PubHealth0.5, −0.91
Biography0.95, −0.91
Queue log, forum post, Homework
Correct
Correct
Ambiguous
Forum post alone
Ambiguous
Ambiguous
Ambiguous
Homework, project page
Ambiguous
Incorrect
Incorrect
In the first row the queue log's 0.70 is above PopQA's 0.59, so the whole set is Correct, even though Homework sits at −0.95 in the same set. For Biography the upper threshold is 0.95, which the queue log does not reach.
In the third row PopQA's lower threshold is −0.99. Homework's −0.95 is above it, so the verdict is Ambiguous. For PubHealth, −0.95 and −0.97 are both below −0.91, so the verdict is Incorrect.
Why Ambiguous exists came out of an early experiment in the paper. With only Correct and Incorrect, CRAG took a completely wrong path on every mistake of the evaluator. Ambiguous is the middle path, where the documents stay and web results come in too. from the paper
The numbers for strips and web search
Refinement uses the same evaluator. A long document is split into strips of a few sentences, the evaluator scores each strip, and of the strips above −0.5 the top 5 stay. from the paper For the milestone question, suppose the syllabus strips got 0.8, 0.1 and −0.7. my scores The first two are above −0.5 and stay, and the Week 10 strip at −0.7 is dropped.
For the web, GPT-3.5 Turbo gets a prompt asking for at most three keywords from the question, separated by commas. The paper's example turns What is Henry Feilden's occupation? into the query Henry Feilden, occupation. The Google Search API returns the top 5 pages, and CRAG puts Wikipedia pages first. The same evaluator then runs on the paragraphs of these pages. from the paper
For our question, such a query would look something like office hour, least crowded, this week. my example But this course's queue log is not on the web, so a web search brings back nothing useful here. CRAG still searches, because it acts only on the evaluator's verdict.
The results: points and percent
The paper added CRAG on top of plain RAG, with SelfRAG-LLaMA2-7b as the generator, the same model that Self-RAG trained: from the paper
Task
RAG
CRAG
Difference
PopQAaccuracy
52.8
59.8
+7.0
BiographyFactScore
59.2
74.1
+14.9
PubHealthaccuracy
39.0
75.6
+36.6
ARC-Challengeaccuracy
53.2
68.6
+15.4
The paper writes these differences as 7.0% accuracy and so on. They are points, and the percentage gain is a different number:
Points: 59.8 − 52.8 = 7.0.
Percentage gain: 7.0 ÷ 52.8 = 0.133, that is 13.3%.
On PubHealth, the 36.6-point gain is 93.8% relative to 39.0, so the accuracy almost doubles. Check: 75.6 ÷ 39.0 = 1.94.
The paper also added CRAG on top of Self-RAG, and calls this Self-CRAG. On PopQA it went from Self-RAG's 54.9 to 61.8, 6.9 points ahead. On ARC it went from 67.3 to 67.2, 0.1 point lower. from the paper
When the generator was plain LLaMA2-hf-7b, with no training for reflection tokens, the Self-RAG method fell to 29.0 on PopQA. With the same generator, plain RAG scored 50.5 and CRAG 54.9. This is the point from step 5: Self-RAG needs its own trained generator, and CRAG works with any generator.
How well the evaluator judges was measured on PopQA. The T5 evaluator was right 84.3% of the time. ChatGPT doing the same job reached 58.0%, 62.4% with a step-by-step prompt, and 64.7% with examples. from the paper
In the ablation, removing refinement brought PopQA from 59.8 to 54.2, removing query rewriting to 56.2, and removing the selection among web results to 58.6. Adding web results every time to plain RAG raised accuracy only from 52.8 to 53.8. So the web alone gives 1.0 point, and CRAG's checking adds 6.0 points on top. from the paper
This checking is not free. In the generation phase, RAG took about 0.363 seconds per question and CRAG 0.512, about 1.41 times as long. The paper says this is a rough estimate that leaves out the time for retrieval and web search. from the paper
9Now read Stanford's slides yourself
Below are Stanford's three agentic RAG slides, unchanged, together with enlarged parts. Under each slide are a few questions. Think of the answer first, then open it. If something stops you, go back to the step named in the answer. If a slide looks small on a phone, click the image to open it at full size.
Stanford's slide: agentic RAG, retrieval inside the loop.
Read the agentic RAG slide
The first line says Everything so far answers one question well. Which parts of this lecture does that cover, and what do they lack?
Show the answer
Chunking, BM25, DPR, hybrid search and reranking. All of them improve a single retrieval for a single question. None of them decides whether to retrieve, what to ask, or when to stop. (Step 2)
In Retrieval should be an action the agent takes, what is the action, what is the tool, and what would the observation be?
Show the answer
Searching is the action, the retriever or search engine is the tool, and the passages that come back are the observation. The agent reads them and chooses the next action. (Step 1)
The top line mentions when to stop, but the word stop is missing from the three decisions below. Which decision holds it?
Show the answer
The third one, whether the result is good enough. If the result is good enough, the agent stops and answers. (Step 2)
Stanford's Self-RAG slide, and below it steps 1 and 2, then step 3, enlarged.
Read the Self-RAG slide
Step 1 is called Retrieve on demand. Which decision does on demand refer to, and when does the model retrieve in the paper?
Show the answer
Decision 1. The model writes the Retrieve token, and in the paper the system retrieves when yes's share, between yes and no, is above δ = 0.2. (Steps 3 and 6)
In step 3, segment 1 has two green boxes, segment 3 has a green and an orange one, and segment 2 has a red one. Which tokens are these, and how does the score give 1 > 3 > 2?
Show the answer
Segment 1 has Relevant and Supported, segment 3 has Relevant and Partially, and segment 2 has Irrelevant. In IsSup, partially gets half weight, so segment 3 scores below segment 1. Segment 2 has a small IsRel share, so it comes last. In our example the same order came from 2.15, 1.475 and 0.425. (Step 6)
The final answer cites passages 1 and 4, although only three passages arrived in step 1. Where did 4 come from? And what do retrieve, critique and generate on the right correspond to?
Show the answer
The model wrote Retrieve again, the Repeat part, and the Utah claim came from passage 4 of the next round. The Retrieve token is the retrieval decision, IsRel, IsSup and IsUse are the critique tokens, and generate is the writing of each segment. (Steps 3 and 6)
Stanford's CRAG slide, and below it the evaluator, both boxes and the generation row, enlarged.
Read the CRAG slide
The evaluator's thought bubble says If retrieved documents are correct to x? What is x here, and in what form does the evaluator answer?
Show the answer
x is the question. The evaluator gives each question-document pair a score from −1 to 1, and two thresholds turn the scores into Correct, Ambiguous or Incorrect. The blue box on the left says the same: judge relevance, and pick an action from the confidence. (Steps 4 and 8)
In the Knowledge Refinement box, what are d1, d2, the strips and kin? Why is strip2 a different colour?
Show the answer
d1 and d2 are the documents the retriever brought. Decompose splits them into strips, Filter carries the evaluator's icon, and Recompose joins the remaining strips into kin, the internal knowledge. strip2 no longer appears after Filter, so it is the strip that was dropped, like the Week 10 strip of our syllabus. (Steps 4 and 8)
How does the query q in the Knowledge Searching box differ from the question x, and what is kex? Then read the generation row at the bottom.
Show the answer
q holds only keywords: Death of a Batman, screenwriter and Wikipedia. The paper's prompt asks for at most three keywords, and CRAG puts Wikipedia pages first. The web search returns k1 to kn, and after Select they become kex, the external knowledge.
In the generation row, Correct gives the generator x + kin, Ambiguous gives x + kin + kex, and Incorrect gives x + kex. (Steps 4 and 8)
3 points from this lesson
An agent takes an action in a loop, reads the observation, and decides the next action. Every step has three decisions: retrieve or not, which query to send, and whether the result is good enough.
Self-RAG turns the probabilities of each critique kind into shares within that kind, and the weighted sum of the shares is each segment's score. The retrieval decision comes from yes's share and δ = 0.2.
CRAG's T5 evaluator scores from −1 to 1. One document above the upper threshold gives Correct, and all documents below the lower one give Incorrect. On PopQA it lifts RAG from 52.8 to 59.8, which is 7.0 points.
New terms in this lesson
Agent
An LLM that works in a loop: it chooses an action, reads the observation, and decides the next action. For example, when the search for OH timings came back empty, the agent wrote a new query.
Action and tool
What the agent chooses to do, such as searching for OH timings. A tool is the program that does it, such as a search engine.
Observation
The result of an action, which the agent reads before its next step. For example, the result of the first search: every passage scored 0.
Agentic RAG
RAG in which retrieval is an action of the agent, not a fixed step of a pipeline. For example, 0 searches for 17 × 3 and 3 for the Jed Hoyer question.
Self-RAG
The generator itself writes reflection tokens to decide when to retrieve and which segment to keep. For example, Relevant and Partially on the forum post segment.
Reflection tokens
Special tokens added to the vocabulary, which the model writes like any other text. For example, Retrieve, Relevant or Irrelevant, and Supported or Partially.
CRAG
Corrective RAG: an evaluator calls the retrieval Correct, Incorrect or Ambiguous, and refinement, web search or both follow. For example, the forum post at 0.4 is Ambiguous.
Retrieval evaluator
A small model that reads the question and one document and returns a confidence. For example, 0.4 for the forum post at K = 1.
Knowledge refinement
Splitting documents into strips, dropping the useless strips, and joining the rest in order. For example, 2 of the syllabus's 3 strips stay for the milestone question.
Threshold
A limit set in advance, where being above or below it triggers a different action. For example, in CRAG above 0.6 means Correct, and below 0.2 means Incorrect.
Critic
A model that grades what something else wrote or retrieved. For example, in Self-RAG a critic built from Llama 2 7B adds reflection tokens to the training data, and the CRAG slide calls its evaluator a critic.
Segment score (Self-RAG)
The weighted sum of a segment's critique shares: S = 1.0 × sIsRel + 1.0 × sIsSup + 0.5 × sIsUse. For example, the queue log segment's S = 0.9 + 0.875 + 0.5 × 0.75 = 2.15.
Beam search
Moving forward while keeping the B unfinished answers with the highest scores at each step. For example, with B = 2 the queue log (2.15) and forum post (1.475) answers go on, and Homework (0.425) is dropped.