Hybrid search merges the ranked lists of two retrievers into one list. Reranking then sorts the top passages of that list again, using a slower but more accurate model.
BM25 put E2 at rank 3, and dense retrieval put it at rank 1. The query was E-4012 upload failed, and it has two correct passages. E1 contains this error code exactly, and E2 says the same thing in other words. Lesson 12 ended on this point: the strengths and weaknesses of are the opposite of a dense retriever's.
The ladder for this lesson
The ladder has 5 steps. Under each step you can see what you will be able to do after it.
Reranking a shortlistWork out the time reranking takes, and explain why a reranker cannot raise recall.
What you need first: Lesson 3 (embedding, cosine), Lesson 6 (fusion, rerank), Lesson 11 (BM25), Lesson 12 (bi-encoder, DPR) and Lesson 13 (MRR). If you already know a step, skip to the next one. A word with a dotted underline and ▾ opens its definition when you click it.
1BM25 and dense retrieval miss different passages
BM25 and a dense retriever fail on different kinds of text, so on the same query they leave out different correct passages. BM25 counts the words a query and a passage share, while a dense retriever such as compares their .
BM25 fails when a passage says the same thing in other words, which is called a paraphrase. For example, the query says upload failed and the passage says upload keeps failing. To BM25, failing and failed are different words, so the only match it finds is upload.
A dense retriever fails when something has to match exactly, such as an error code or an order ID. Text like this is called a literal. Lesson 3 showed that passages with E-4012 and E-4021 can get almost the same embedding. To BM25, 4012 is a rare word with a high IDF, so it finds the right passage.
Now look at how both retrievers rank three passages: my example
Passage
Correct?
BM25
Dense
E1, the one with the error codeError E-4012: file larger than 50 MB. Compress the file and upload again.
Yes
Rank 14012 matches exactly
Rank 2
E2, the one in other wordsIf your upload keeps failing on Gradescope, make the file smaller before submitting.
Yes
Rank 3only upload matches
Rank 1closest vector
E3, the one with another errorError E-4021: submission locked after the deadline.
No
Rank 2an E- code, short passage
Rank 3
At top-1, BM25 brings only E1 and dense retrieval brings only E2. Each misses a different correct passage, so merging their lists can bring both correct passages to the top.
Real data shows the same thing. In lesson 12, top-20 accuracy on SQuAD was 68.8% for BM25 and 63.2% for DPR. On the same Stanford slide, a retriever that combines the two reaches 71.5%, ahead of both.
Think first
The query is refund policy for order #A77-310. Which part will BM25 catch, and which part will the dense retriever catch?
Show the answer
BM25 catches the rare literal A77-310 exactly. The dense retriever can match refund policy to a passage about money back rules, which shares no word with it.
2Hybrid search, and simply adding the scores
Hybrid search runs both retrievers and merges their two ranked lists into one list.Lesson 6 called merging lists . The simplest way is to add each passage's two scores: the BM25 score and the dense retriever's . my scores
Think first
The BM25 scores of E1, E3 and E2 are 7.2, 3.1 and 1.0, and their cosines are 0.79, 0.75 and 0.82. If you add the two, where does E2, the dense retriever's first passage, end up?
Show the answer
At rank 3, the bottom. The table below has the arithmetic.
Passage
BM25 score
Cosine
Plain sum
E1
7.2
0.79
7.99
E3 (wrong)
3.1
0.75
3.85
E2
1.0
0.82
1.82
After adding, the order is E1, E3, E2, exactly BM25's order. The reason is the scale of the two scores. The cosines spread from 0.75 to 0.82, only 0.07, while BM25 spreads from 1.0 to 7.2, which is 6.2.
BM25's spread is about 90 times the cosine's, so in the sum the cosine has almost no effect. A cosine always stays between −1 and 1, while BM25 has no upper limit. You could divide BM25 by some number, but that number would differ for every corpus, and you would have to find it each time.
3Reciprocal rank fusion: merging ranks instead of scores
The scores have different scales, but ranks have the same scale in every retriever: first, second, third. Reciprocal rank fusion, or RRF, turns a passage's rank in each list into 1 ÷ (60 + rank), and adds these numbers over all lists. The passages are then ordered by this sum.
E1 is at rank 1 in BM25's list, so it gets 1 ÷ (60 + 1) = 1 ÷ 61 = 0.01639, about 0.0164. In the dense list it is at rank 2, so it gets 1 ÷ 62 = 0.01613. E1's RRF is the sum: 0.01639 + 0.01613 = 0.03252.
Passage
From BM25
From dense
Sum
E1
1 ÷ (60 + 1) = 0.01639
1 ÷ (60 + 2) = 0.01613
0.03252
E2
1 ÷ (60 + 3) = 0.01587
1 ÷ (60 + 1) = 0.01639
0.03227
E3 (wrong)
1 ÷ (60 + 2) = 0.01613
1 ÷ (60 + 3) = 0.01587
0.03200
The new order is E1, E2, E3. Both correct passages moved to the top, and E3 dropped to the bottom, because it was not first in any list.
The three sums sit close together, and the reason is how a fraction behaves. When the bottom of 1 ÷ (60 + rank) grows, the fraction shrinks. But going from 61 to 62 makes the bottom only 1.6% bigger, so the fraction also shrinks by only about 1.6%. That is why RRF numbers stay close together, and only their order matters.
Stanford's slide writes the same arithmetic as a formula:
RRF(d) = ∑r ∈ R1k + rankr(d), k = 60
d
A passage, such as E1.
R
The set of retrievers, here {BM25, dense}. The r ∈ R under the means: add over every retriever r in R.
rankr(d)
The rank of d in retriever r's list. For E1, 1 in BM25 and 2 in dense. A list that does not contain d adds nothing.
k
A fixed number, 60. It is not the k of top-k.
Stanford's slide: Hybrid Search and Reciprocal Rank Fusion. The first line is step 1, the second is step 2, and at the bottom is the RRF formula. The small line says RRF needs no per-corpus tuning and works with any number of retrievers. Click the image to open it at full size.
What k = 60 does
k = 60 shrinks the gap between rank 1 and rank 2, so rank 1 in a single list cannot decide the result on its own. With k = 0, rank 1 would get 1 and rank 2 would get 0.5, half as much. With k = 60, they get 0.01639 and 0.01613.
Suppose passage A is only in BM25's list, at rank 1, and passage B is at rank 3 in both lists: my example
With k = 0, A gets 1 ÷ 1 = 1, and B gets 1 ÷ 3 + 1 ÷ 3 = 0.667. A wins.
With k = 60, A gets 1 ÷ 61 = 0.0164, and B gets 1 ÷ 63 + 1 ÷ 63 = 0.0317. B wins.
So with k = 60, the winner is the passage that both retrievers place near the top.
Change k yourself
A is at rank 1 in one list, and B is at rank r in both lists. Change k and r and see how long B stays ahead.
Passage
Arithmetic
RRF
Think first
In lesson 6, the original query's list was: queue log, forum post, office hours page. The rewritten query's list was: office hours page, queue log. What order does RRF give?
Show the answer
Queue log: 1 ÷ 61 + 1 ÷ 62 = 0.03252. Office hours page: 1 ÷ 63 + 1 ÷ 61 = 0.03227. The forum post is only in the first list: 1 ÷ 62 = 0.01613. The order is queue log, office hours page, forum post.
The price of RRF is that it forgets how strong a score was. Whether E1's BM25 score is 7.2 or 70, rank 1 gives 1 ÷ 61 both times.
4Cross-encoder: reading the query and the passage together
E1 and E2 are at the top of the RRF list only because the two retrievers agreed on them. Neither retriever read the query and the passage together. In a , a passage's vector is made before any question arrives, and the two meet only at the final dot product.
A cross-encoder joins the query and the passage into one text, feeds it to one model, and gets a single score for that pair. The model is usually an encoder such as .
Attention: each token updates its vector from the other tokens
An encoder makes a vector for every and then updates it many times. In attention, each token takes a weighted average of the other tokens' vectors and uses it as its new vector. In a weighted average you multiply each vector by its weight and add them up, and the weights add up to 1.
In a cross-encoder, the query's tokens can also look at the passage's tokens, which Stanford's slide calls full attention across both. Below, every vector has 2 numbers: the first says how much the token leans toward counted data, and the second how much it leans toward opinion. my numbers
The query token crowded starts as [0.5, 0.5]. When the query is joined with the queue log, crowded gives weight 0.6 to the queue log's averaged [1.0, 0.0] and 0.4 to itself:
With the forum post, the same 0.6 goes to usually [0.0, 1.0], and crowded becomes [0.2, 0.8]. So the same query token gets a different vector with every passage. In a bi-encoder, attention runs only inside the query, so crowded keeps one vector for every passage.
At the end, the model gathers the whole pair into one vector and turns it into a score. The reranker in lesson 6 gave the queue log 0.92, the office hours page 0.41 and the forum post 0.35 on the least-crowded question. The retriever had given the queue log and the forum post the same 2. my scores from lesson 6
Stanford's slide: Cross-Encoders and Reranking. On the left, in the bi-encoder, the query (Inx) and the passage (Iny) go through separate encoders. On the right, in the cross-encoder, both go through one Encoder together.
The price: nothing can be built in advance
Every vector in a cross-encoder depends on the query, so nothing about a passage can be computed before the question arrives. On every query it has to run the whole model once for every passage, while a bi-encoder ran the model once per query.
According to Stanford's slide, a cross-encoder built on BERT-base takes 10,700 ms for 1,000 passages, which is 10.7 ms per passage. BERT-base is the smaller version of BERT. For the 60,000 passages of the company in lesson 2, one query would take 60,000 × 10.7 ms = 642 seconds, about 11 minutes.
5Reranking a shortlist
A retriever is fast but looks at passages roughly, and a cross-encoder is accurate but slow. So a fast retriever first picks the top few passages, such as the top 100, and then the cross-encoder scores and sorts only those again. This short list is called the shortlist, and the second step is reranking.
Step
On what
Time
1. BM25 and dense, then RRF
The whole corpus
BM25 62 msdense tens of ms, from Stanford's slide
2. Reranking with the cross-encoder
Top-100 shortlist
100 × 10.7 = 1,070 ms
3. Into the LLM's prompt
Top 5 after reranking
–
The whole corpus would take 642 seconds, and the shortlist takes about 1 second. A Stanford table shows the gain in accuracy. Its data is MS MARCO: real Bing search questions and about 8.8 million passages. MS MARCO description from the paper
There, BM25 alone gets an of 16.7. In lesson 13 this was one over the rank of the first correct passage, such as 0.5 at rank 2. The table writes it multiplied by 100. Rerank BM25's top 1,000 with a BERT-base cross-encoder, and MRR@10 becomes 34.7. So reranking roughly doubles MRR@10: 34.7 ÷ 16.7 = 2.08.BM25's top 1,000 from the paper
A reranker has a limit: it only changes the order of the shortlist, and it cannot bring in a passage that is not there. This is the from lesson 2. If the shortlist holds the right passage for 60 out of 100 questions, the reranker cannot go above 60 either.
So the first step needs high recall, and hybrid search provides it: both E1 and E2 reach the shortlist.
Think first
Each query has a budget of 500 ms. The retriever takes 50 ms, and the cross-encoder takes 10.7 ms per passage. At most how many passages can the shortlist hold?
Show the answer
That leaves 500 − 50 = 450 ms for reranking. 450 ÷ 10.7 = 42.06, so 42 passages.
What the dense version adds
The dense version contains this whole lesson, followed by 5 steps at the Stanford level:
Step 6: the maths of RRF: 1 ÷ (k + rank) for different k, and the RRF paper's experiment on k.
Step 7: min-max normalisation, its weakness, and the DPR paper's BM25 + DPR combination.
Step 8: the cross-encoder from inside: attention weights, score to probability, and the training loss.
Step 9: the cost in full: running the model N times against once, FLOPs, and the real recall ceiling.
Step 10: five Stanford slides, which you read yourself.
BM25 misses paraphrases, and a dense retriever misses rare literals. Hybrid search runs both and merges their lists.
The scores have different scales, so RRF merges only ranks: 1 ÷ (60 + rank) from each list, then a sum. With k = 60, the passage that both lists place high wins.
A cross-encoder reads the query and the passage together, so it is accurate, but it runs in full for every passage. That is why it reranks only a shortlist.
New terms in this lesson
Paraphrase
The same statement in other words. For example, upload failed and upload keeps failing. BM25 misses it, because it counts shared words.
Literal
Text that has to match exactly, such as the error code E-4012 or the order ID A77-310. A dense retriever can miss it, and BM25 catches it as a rare word.
Hybrid search
Running both a word-based and a dense retriever and merging their lists. For example, making one list from BM25's E1, E3, E2 and dense retrieval's E2, E1, E3.
Reciprocal rank fusion (RRF)
1 ÷ (k + rank) from a passage's rank in each list, summed over all lists, with k = 60. For example, E1: 1 ÷ 61 + 1 ÷ 62 = 0.03252.
Attention
Each token takes a weighted average of the other tokens' vectors as its new vector, and the weights add up to 1. For example, if crowded [0.5, 0.5] gives weight 0.6 to averaged [1.0, 0.0], it becomes [0.8, 0.2].
Cross-encoder
A reranker that joins the query and the passage, feeds them to one model, and gets a direct score for the pair. It runs in full for every passage, for example 10.7 ms per passage with BERT-base.
Reranking
Scoring and sorting a fast retriever's shortlist again with an accurate model. For example, on MS MARCO, MRR@10 goes from 16.7 for BM25 alone to 34.7 after reranking with BERT-base.
Shortlist
The retriever's top few passages, on which the reranker runs. For example, the top 100 of 60,000 passages, which the cross-encoder handles in 1,070 ms.