DPR: a retriever trained on question-passage pairs
DPR is a retriever that turns the question and every passage into vectors. It then picks the passages whose vectors have the largest dot product with the question's vector. Two encoders produce these vectors, and both encoders are trained on pairs of a question and its correct passage. DPR stands for Dense Passage Retrieval, and it is the retriever inside the RAG model from lesson 5.
The need for it showed up in lesson 11. For the question which office hour is least crowded, BM25 gave an old 2024 forum post 2.749 and ranked it first. The answer is in the queue log, which came second with 2.473, because it does not contain the words least and crowded. DPR's encoders are trained so that this question's vector moves close to the queue log's vector.
The ladder for this lesson
The ladder has 5 steps. Under each step you can see what you will be able to do after it.
What you need first: Lesson 3 (vector, dot product, encoder), Lesson 4 (softmax, ln, loss) and Lesson 11 (BM25). If you already know a step, skip to the next one. A word with a dotted underline and ▾ opens its definition when you click it.
1Two encoders and a dot product
DPR has two separate encoders: the question encoder EQ makes the question's vector, and the passage encoder EP makes a vector for every passage. An is a model that reads text and returns a vector of fixed length. Both DPR encoders start from , and DPR them on question-passage pairs.
The score is the of the two vectors:
s(q, p) = EQ(q)⊤ EP(p)
The here is only a way of writing the dot product, and Stanford's slide writes it this way.
Now look at a small example. A real vector has 768 numbers. Here each has only 4, so you can do the arithmetic by hand. my vectors
Text, and the vector the encoder makes for it
Dot product with the question
Question: least crowded office hour[1.0, 0.5, 0.5, 1.0]
–
Queue log, which has the answer[0.2, 0.4, 0.4, 0.4]
1.0
Old forum post[0.8, 1.0, 0.4, 0.5]
2.0
Homework 1 announcement[0.5, 0.4, 0.6, 0.5]
1.5
The queue log's score comes from 1.0 × 0.2 + 0.5 × 0.4 + 0.5 × 0.4 + 1.0 × 0.4 = 1.0. The encoders are not trained yet, so the wrong forum post is on top with 2.0. Training in step 2 changes this order.
DPR reads the question and the passage separately, and the benefit is speed. Its work splits into two parts:
, before any question arrives: the passage encoder makes each passage's vector once and stores them all in an .
Online, when a question arrives: the question encoder runs once. Then the dot products with the index vectors are computed, and the passages with the k largest scores come back.
If one model read the question and the passage together, it would have to run once for every passage on every question. The Wikipedia from lesson 5 had about 21 million passages, so the model would run 21 million times per question. In DPR the model runs only once per question. This two-encoder design is called a bi-encoder.
Stanford's slide: the offline and online parts of DPR. The top half is offline, and the bottom half is online. You can now read every box on it. Click the image to open it at full size.
Think first
Double the question's vector, so it becomes [2.0, 1.0, 1.0, 2.0]. What are the three scores now, and does the order change?
Show the answer
The scores become 2.0, 4.0 and 3.0. In a dot product, every multiplication takes one number from the question's vector. That number doubled, so every product doubled, and so did their sum: for the queue log, 2.0 × 0.2 + 1.0 × 0.4 + 1.0 × 0.4 + 2.0 × 0.4 = 2.0. Every score doubled, but the order stayed the same, and the forum post is still on top. This matters in step 2.
2Contrastive loss: putting the right passage ahead of the wrong ones
Contrastive loss compares the score of the right passage with the scores of the wrong passages. The further ahead the right passage is, the smaller the loss. Both DPR encoders are trained to make this loss small.
For this comparison, every training question needs two kinds of passages:
Positive
The question's correct passage. The training data lists it with every question. For our question it is the queue log.
Negative
A wrong passage for the same question. In our example, the forum post and the Homework 1 announcement.
The comparison matters because a retriever passes on only the top passages, so the order is what counts. In the question at the end of step 1 you saw that a longer question vector raises every score but keeps the order. If training only asked for a higher positive score, the encoder could take exactly that easy route.
Now compute the loss on the scores from step 1: queue log 1.0, forum post 2.0 and Homework 1.5. The loss takes three steps:
Take for each score: 2.72, 7.39 and 4.48. e is a fixed number, about 2.718, so e1.0 = 2.72 and e2.0 = 2.718 × 2.718 = 7.39. The three add up to 14.59.
Find the positive's share: 2.72 ÷ 14.59 = 0.186.
Take −ln of that share: −ln 0.186 = 1.68. This is the loss. ln tells you the power you must raise e to in order to get a number. Check: e−1.68 = 1 ÷ e1.68 = 1 ÷ 5.37 = 0.186.
Steps 1 and 2 together are , and step 3 is the from lesson 4. The formula holds the same three steps:
𝓛 = − ln es+es+ + ∑ es−
On top is the positive's escore (2.72), and below is the sum of escore for the positive and all negatives (14.59).
Think first
Add 1 to all three scores: 2.0, 3.0 and 2.5. What is the loss now?
Show the answer
Still 1.68. Adding 1 to a score makes its escore 2.72 times bigger, because ea+1 = ea × e. For example, e2.0 = e1.0 × 2.72 = 7.39. The top and the bottom are both multiplied by 2.72, and the division cancels it. So the queue log's share stays at 0.186.
The table below has four situations. The last two rows are two stages of training, and step 3 uses the middle stage. my scores
Scores (queue log, forum post, Homework)
Queue log's share
Loss
Before training: 1.0, 2.0, 1.5
0.186
1.68
Question vector doubled: 2.0, 4.0, 3.0
0.090
2.41
Partway through training: 3.0, 2.0, 0.5
0.690
0.372
After training: 5.0, 2.0, 0.5
0.943
0.059
Doubling the question vector raises the loss from 1.68 to 2.41, because the forum post moves further ahead. So the easy route of a longer vector does not work here.
As training goes on, the queue log moves ahead, and the loss falls from 1.68 to 0.372 and then to 0.059.
So the encoders have only one way to lower the loss: give the queue log a higher score than the other two.
Run the contrastive loss yourself
Change the three scores. The loss is small when the queue log is well above the other two. Move all three by the same amount, and the loss does not change.
3Negatives: from the batch and from BM25
Contrastive loss needs negatives for every question, and the more negatives there are, the stricter the comparison. But every negative has to go through the passage encoder, and that is expensive. DPR takes negatives from two places where this cost is low.
The positives of the other questions in the batch
In-batch negatives means treating the positives of the other questions in a batch as negatives for this question. Training runs questions in small groups, and each group is called a batch. The positive of every question in the batch is already encoded in that same step, so these negatives need no extra encoding.
Below is a batch of 3 questions, partway through training. In each row the bold number is that question's positive, and the other numbers in the row are its negatives. my scores
Question ↓ / passage →
Queue log
Syllabus
Office hours
Loss
Least crowded office hour?
3.0
1.0
0.5
0.197
When is the milestone due?
0.8
3.0
1.2
0.244
Where are office hours held?
0.3
2.5
2.0
?
Work out the third row
The positive is the office hours page, with score 2.0. The negatives are the queue log (0.3) and the syllabus (2.5). What is the loss, and why is this row different from the other two?
Show the answer
e0.3 = 1.35, e2.5 = 12.18 and e2.0 = 7.39, which sum to 20.92. The office hours page's share is 7.39 ÷ 20.92 = 0.353, and the loss is −ln 0.353 = 1.041. In this row the wrong syllabus (2.5) is above the right page (2.0), so its loss is the largest.
In the paper a batch has 128 questions, so each question gets 127 in-batch negatives. from the paper
Hard negatives from BM25
A hard negative is a wrong passage that looks similar to the question but does not contain the answer. For our question, the forum post is exactly such a passage. It has both words, least and crowded, but no data about how crowded anything is.
In-batch negatives are mostly easy, because they are passages for other questions. Hard negatives produce a large loss, and the larger the loss, the more training changes the encoders. Partway through training the queue log is at 3.0, and the two negatives compare like this: my scores
Negative
Loss with only this negative
What the encoder learns
Homework announcementeasy negative, score 0.5
0.079queue log's share 0.924
Almost nothing, because the gap is already clear
Forum posthard negative, score 2.0
0.313, about 4 times as muchqueue log's share 0.731
The question's words are not enough. The passage needs crowd data.
The first row works out like this: e3.0 = 20.09 and e0.5 = 1.65, which add up to 21.73. The queue log's share is 20.09 ÷ 21.73 = 0.924, and the loss is −ln 0.924 = 0.079.
To find such passages, DPR uses BM25. For each training question it looks at BM25's top passages and takes one that does not contain the answer. This works because BM25 makes exactly the mistake the encoders need to unlearn. On our question, too, BM25 put the forum post on top.
Watch out: Check the labels before you take hard negatives. Sometimes a top BM25 passage really does contain the answer, and only its label is missing. Then the encoder learns to push a correct passage down.
4How far ahead of BM25 DPR is
The DPR paper measures a retriever by top-k retrieval accuracy: the percentage of questions whose answer passage appears in the top k. The NQ dataset holds real questions people typed into Google. On it, BM25's top-20 accuracy is 59.1%, and DPR's is 78.4%.
Out of 1,000 questions, that is 591 against 784. So with DPR, the answer passage reaches the reader for 193 more questions.
DPR is ahead on 4 of the 5 datasets. On SQuAD it is behind, 63.2 against 68.8, because there people wrote the questions after reading the passage. Those questions therefore reuse the passage's own words, and matching exact words is what BM25 does. this reason is in the paper
Numbers for all five datasets
Dataset
BM25
DPR
NQ
59.1
78.4
TriviaQA
66.9
79.4
WQ
55.0
73.2
TREC
70.9
79.8
SQuAD
68.8
63.2
Top-20 accuracy (%) on the test sets. Here DPR was trained separately on each dataset.
DPR needs labeled pairs, and the paper also measured how many. A DPR trained on only 1,000 pairs from NQ already beats BM25, and accuracy keeps rising with more pairs.
5Where DPR is weak
DPR wins at semantic matching across a whole corpus, but it has three weaknesses:
If a code like error 4012 appears in only one or two passages, BM25 treats it as rare and ranks those passages high. DPR's vector sums up the whole passage, and a single code like that is easily lost in it. Lesson 15 starts from exactly this case.
A passage's vector is made without seeing the question, so the question and the passage meet only at the final dot product. The reranker in lesson 14 works on this gap.
Training needs labeled question-passage pairs, even if only 1,000 of them.
BM25's strengths and weaknesses are the opposite of these. That is why lesson 14 combines the lists from BM25 and DPR.
What the dense version adds
The dense version contains this whole lesson, followed by 4 more steps that take you to the level of the Stanford slides:
Step 6: the slide's symbols (qi, ℝd, θ, ∇), and the maths of the gradient: what a rate is, what ∂ means, the chain rule for rates, and one gradient step on the question's vector.
Step 7: matrices, the transpose and the whole batch as one product S = QP⊤, and the paper's experiments on which negatives help how much.
Step 8: the paper's real scale and full results: 21 million passages, 768-number vectors, 255 negatives, speed, and the comparison with ORQA and REALM.
Step 9: Stanford's four DPR slides, which you read yourself.
DPR is a bi-encoder: two BERT encoders, with the score as the dot product of their vectors. Passage vectors are built offline into an index, so the question encoder runs only once per question.
Contrastive loss compares the positive's score with the negatives: softmax, then −ln. Raising every score by the same amount does not lower this loss.
Negatives come free from the batch, and the hard ones come from BM25. DPR beats BM25 on 4 of 5 datasets, but BM25 can still win on questions that hinge on exact words.
New terms in this lesson
DPR
Dense Passage Retrieval: a bi-encoder retriever with two BERT encoders, trained with contrastive loss on question-passage pairs. Built by Karpukhin and colleagues in 2020, and the retriever inside lesson 5's RAG.
Bi-encoder
Two separate encoders for the question and the passage, with the score as their dot product. Passages are encoded in advance. For example, even with 21 million passages the model runs once per question.
Pretraining
A model's first, large training on lots of general text. For example, BERT is first trained on general text.
Fine-tuning
Training an already trained model further on data for your own task. For example, DPR trains BERT on question-passage pairs.
Positive and negative passage
The question's correct passage (pi+), and wrong passages (pi,j−). For example, the least-crowded question's positive is the queue log, and its negatives are the forum post and the Homework announcement.
Contrastive loss
Softmax over the positive and negative scores, then −ln of the positive's share. It looks only at differences between scores. For example, scores 1.0, 2.0, 1.5 give loss 1.68, and 5.0, 2.0, 0.5 give 0.059.
Batch
A group of questions that training processes together. For example, a batch of 128 questions in DPR.
In-batch negatives
Using the positives of the other questions in the batch as negatives. For example, in a batch of 128 each question gets 127 negatives.
Hard negative
A wrong passage that looks similar to the question. DPR takes these from BM25's top passages. For example, the forum post for our question.
Top-k retrieval accuracy
The percentage of questions with an answer passage in the top k. For example, DPR's 78.4% on NQ at top-20.