Lesson 12 · Easy version

DPR: a retriever trained on question-passage pairs

DPR is a retriever that turns the question and every passage into vectors. It then picks the passages whose vectors have the largest dot product with the question's vector. Two encoders produce these vectors, and both encoders are trained on pairs of a question and its correct passage. DPR stands for Dense Passage Retrieval, and it is the retriever inside the RAG model from lesson 5.

The need for it showed up in lesson 11. For the question which office hour is least crowded, BM25 gave an old 2024 forum post 2.749 and ranked it first. The answer is in the queue log, which came second with 2.473, because it does not contain the words least and crowded. DPR's encoders are trained so that this question's vector moves close to the queue log's vector.

The ladder for this lesson

The ladder has 5 steps. Under each step you can see what you will be able to do after it.

  1. Two encoders and a dot productCompute scores by hand, and read Stanford's first DPR slide.
  2. Contrastive lossCompute the loss by hand, and explain why raising every score does not lower it.
  3. Negatives: from the batch and from BM25Show with numbers how in-batch negatives and hard negatives differ.
  4. How far ahead of BM25 DPR isRead the paper's main result, and say where DPR falls behind.
  5. Where DPR is weakSee why lessons 14 and 15 are needed.

What you need first: Lesson 3 (vector, dot product, encoder), Lesson 4 (softmax, ln, loss) and Lesson 11 (BM25). If you already know a step, skip to the next one. A word with a dotted underline and ▾ opens its definition when you click it.

1Two encoders and a dot product

DPR has two separate encoders: the question encoder EQ makes the question's vector, and the passage encoder EP makes a vector for every passage. An is a model that reads text and returns a vector of fixed length. Both DPR encoders start from , and DPR them on question-passage pairs.

The score is the of the two vectors:

s(q, p) = EQ(q)⊤ EP(p)

The here is only a way of writing the dot product, and Stanford's slide writes it this way.

Now look at a small example. A real vector has 768 numbers. Here each has only 4, so you can do the arithmetic by hand. my vectors

Text, and the vector the encoder makes for itDot product with the question
Question: least crowded office hour[1.0, 0.5, 0.5, 1.0]–
Queue log, which has the answer[0.2, 0.4, 0.4, 0.4]1.0
Old forum post[0.8, 1.0, 0.4, 0.5]2.0
Homework 1 announcement[0.5, 0.4, 0.6, 0.5]1.5

The queue log's score comes from 1.0 × 0.2 + 0.5 × 0.4 + 0.5 × 0.4 + 1.0 × 0.4 = 1.0. The encoders are not trained yet, so the wrong forum post is on top with 2.0. Training in step 2 changes this order.

DPR reads the question and the passage separately, and the benefit is speed. Its work splits into two parts:

  1. , before any question arrives: the passage encoder makes each passage's vector once and stores them all in an .
  2. Online, when a question arrives: the question encoder runs once. Then the dot products with the index vectors are computed, and the passages with the k largest scores come back.

If one model read the question and the passage together, it would have to run once for every passage on every question. The Wikipedia from lesson 5 had about 21 million passages, so the model would run 21 million times per question. In DPR the model runs only once per question. This two-encoder design is called a bi-encoder.

Stanford slide: the offline and online parts of DPR
Stanford's slide: the offline and online parts of DPR. The top half is offline, and the bottom half is online. You can now read every box on it. Click the image to open it at full size.

Think first

Double the question's vector, so it becomes [2.0, 1.0, 1.0, 2.0]. What are the three scores now, and does the order change?

Show the answer

The scores become 2.0, 4.0 and 3.0. In a dot product, every multiplication takes one number from the question's vector. That number doubled, so every product doubled, and so did their sum: for the queue log, 2.0 × 0.2 + 1.0 × 0.4 + 1.0 × 0.4 + 2.0 × 0.4 = 2.0. Every score doubled, but the order stayed the same, and the forum post is still on top. This matters in step 2.

2Contrastive loss: putting the right passage ahead of the wrong ones

Contrastive loss compares the score of the right passage with the scores of the wrong passages. The further ahead the right passage is, the smaller the loss. Both DPR encoders are trained to make this loss small.

For this comparison, every training question needs two kinds of passages:

Positive
The question's correct passage. The training data lists it with every question. For our question it is the queue log.
Negative
A wrong passage for the same question. In our example, the forum post and the Homework 1 announcement.

The comparison matters because a retriever passes on only the top passages, so the order is what counts. In the question at the end of step 1 you saw that a longer question vector raises every score but keeps the order. If training only asked for a higher positive score, the encoder could take exactly that easy route.

Now compute the loss on the scores from step 1: queue log 1.0, forum post 2.0 and Homework 1.5. The loss takes three steps:

  1. Take for each score: 2.72, 7.39 and 4.48. e is a fixed number, about 2.718, so e1.0 = 2.72 and e2.0 = 2.718 × 2.718 = 7.39. The three add up to 14.59.
  2. Find the positive's share: 2.72 ÷ 14.59 = 0.186.
  3. Take −ln of that share: −ln 0.186 = 1.68. This is the loss. ln tells you the power you must raise e to in order to get a number. Check: e−1.68 = 1 ÷ e1.68 = 1 ÷ 5.37 = 0.186.

Steps 1 and 2 together are , and step 3 is the from lesson 4. The formula holds the same three steps:

𝓛 = − ln es+es+ + ∑ es−

On top is the positive's escore (2.72), and below is the sum of escore for the positive and all negatives (14.59).

Think first

Add 1 to all three scores: 2.0, 3.0 and 2.5. What is the loss now?

Show the answer

Still 1.68. Adding 1 to a score makes its escore 2.72 times bigger, because ea+1 = ea × e. For example, e2.0 = e1.0 × 2.72 = 7.39. The top and the bottom are both multiplied by 2.72, and the division cancels it. So the queue log's share stays at 0.186.

The table below has four situations. The last two rows are two stages of training, and step 3 uses the middle stage. my scores

Scores (queue log, forum post, Homework)Queue log's shareLoss
Before training: 1.0, 2.0, 1.50.1861.68
Question vector doubled: 2.0, 4.0, 3.00.0902.41
Partway through training: 3.0, 2.0, 0.50.6900.372
After training: 5.0, 2.0, 0.50.9430.059

So the encoders have only one way to lower the loss: give the queue log a higher score than the other two.

Run the contrastive loss yourself

Change the three scores. The loss is small when the queue log is well above the other two. Move all three by the same amount, and the loss does not change.

1.0 2.0 1.5

3Negatives: from the batch and from BM25

Contrastive loss needs negatives for every question, and the more negatives there are, the stricter the comparison. But every negative has to go through the passage encoder, and that is expensive. DPR takes negatives from two places where this cost is low.

The positives of the other questions in the batch

In-batch negatives means treating the positives of the other questions in a batch as negatives for this question. Training runs questions in small groups, and each group is called a batch. The positive of every question in the batch is already encoded in that same step, so these negatives need no extra encoding.

Below is a batch of 3 questions, partway through training. In each row the bold number is that question's positive, and the other numbers in the row are its negatives. my scores

Question ↓ / passage →Queue logSyllabusOffice hoursLoss
Least crowded office hour?3.01.00.50.197
When is the milestone due?0.83.01.20.244
Where are office hours held?0.32.52.0?

Work out the third row

The positive is the office hours page, with score 2.0. The negatives are the queue log (0.3) and the syllabus (2.5). What is the loss, and why is this row different from the other two?

Show the answer

e0.3 = 1.35, e2.5 = 12.18 and e2.0 = 7.39, which sum to 20.92. The office hours page's share is 7.39 ÷ 20.92 = 0.353, and the loss is −ln 0.353 = 1.041. In this row the wrong syllabus (2.5) is above the right page (2.0), so its loss is the largest.

In the paper a batch has 128 questions, so each question gets 127 in-batch negatives. from the paper

Hard negatives from BM25

A hard negative is a wrong passage that looks similar to the question but does not contain the answer. For our question, the forum post is exactly such a passage. It has both words, least and crowded, but no data about how crowded anything is.

In-batch negatives are mostly easy, because they are passages for other questions. Hard negatives produce a large loss, and the larger the loss, the more training changes the encoders. Partway through training the queue log is at 3.0, and the two negatives compare like this: my scores

NegativeLoss with only this negativeWhat the encoder learns
Homework announcementeasy negative, score 0.50.079queue log's share 0.924Almost nothing, because the gap is already clear
Forum posthard negative, score 2.00.313, about 4 times as muchqueue log's share 0.731The question's words are not enough. The passage needs crowd data.

The first row works out like this: e3.0 = 20.09 and e0.5 = 1.65, which add up to 21.73. The queue log's share is 20.09 ÷ 21.73 = 0.924, and the loss is −ln 0.924 = 0.079.

To find such passages, DPR uses BM25. For each training question it looks at BM25's top passages and takes one that does not contain the answer. This works because BM25 makes exactly the mistake the encoders need to unlearn. On our question, too, BM25 put the forum post on top.

Watch out: Check the labels before you take hard negatives. Sometimes a top BM25 passage really does contain the answer, and only its label is missing. Then the encoder learns to push a correct passage down.

4How far ahead of BM25 DPR is

The DPR paper measures a retriever by top-k retrieval accuracy: the percentage of questions whose answer passage appears in the top k. The NQ dataset holds real questions people typed into Google. On it, BM25's top-20 accuracy is 59.1%, and DPR's is 78.4%.

Out of 1,000 questions, that is 591 against 784. So with DPR, the answer passage reaches the reader for 193 more questions.

DPR is ahead on 4 of the 5 datasets. On SQuAD it is behind, 63.2 against 68.8, because there people wrote the questions after reading the passage. Those questions therefore reuse the passage's own words, and matching exact words is what BM25 does. this reason is in the paper

Numbers for all five datasets
DatasetBM25DPR
NQ59.178.4
TriviaQA66.979.4
WQ55.073.2
TREC70.979.8
SQuAD68.863.2

Top-20 accuracy (%) on the test sets. Here DPR was trained separately on each dataset.

DPR needs labeled pairs, and the paper also measured how many. A DPR trained on only 1,000 pairs from NQ already beats BM25, and accuracy keeps rising with more pairs.

5Where DPR is weak

DPR wins at semantic matching across a whole corpus, but it has three weaknesses:

BM25's strengths and weaknesses are the opposite of these. That is why lesson 14 combines the lists from BM25 and DPR.

What the dense version adds

The dense version contains this whole lesson, followed by 4 more steps that take you to the level of the Stanford slides:

  • Step 6: the slide's symbols (qi, ℝd, θ, ∇), and the maths of the gradient: what a rate is, what ∂ means, the chain rule for rates, and one gradient step on the question's vector.
  • Step 7: matrices, the transpose and the whole batch as one product S = QP⊤, and the paper's experiments on which negatives help how much.
  • Step 8: the paper's real scale and full results: 21 million passages, 768-number vectors, 255 negatives, speed, and the comparison with ORQA and REALM.
  • Step 9: Stanford's four DPR slides, which you read yourself.

Open the dense version

3 points from this lesson

  1. DPR is a bi-encoder: two BERT encoders, with the score as the dot product of their vectors. Passage vectors are built offline into an index, so the question encoder runs only once per question.
  2. Contrastive loss compares the positive's score with the negatives: softmax, then −ln. Raising every score by the same amount does not lower this loss.
  3. Negatives come free from the batch, and the hard ones come from BM25. DPR beats BM25 on 4 of 5 datasets, but BM25 can still win on questions that hinge on exact words.

New terms in this lesson

DPR
Dense Passage Retrieval: a bi-encoder retriever with two BERT encoders, trained with contrastive loss on question-passage pairs. Built by Karpukhin and colleagues in 2020, and the retriever inside lesson 5's RAG.
Bi-encoder
Two separate encoders for the question and the passage, with the score as their dot product. Passages are encoded in advance. For example, even with 21 million passages the model runs once per question.
Pretraining
A model's first, large training on lots of general text. For example, BERT is first trained on general text.
Fine-tuning
Training an already trained model further on data for your own task. For example, DPR trains BERT on question-passage pairs.
Positive and negative passage
The question's correct passage (pi+), and wrong passages (pi,j−). For example, the least-crowded question's positive is the queue log, and its negatives are the forum post and the Homework announcement.
Contrastive loss
Softmax over the positive and negative scores, then −ln of the positive's share. It looks only at differences between scores. For example, scores 1.0, 2.0, 1.5 give loss 1.68, and 5.0, 2.0, 0.5 give 0.059.
Batch
A group of questions that training processes together. For example, a batch of 128 questions in DPR.
In-batch negatives
Using the positives of the other questions in the batch as negatives. For example, in a batch of 128 each question gets 127 negatives.
Hard negative
A wrong passage that looks similar to the question. DPR takes these from BM25's top passages. For example, the forum post for our question.
Top-k retrieval accuracy
The percentage of questions with an answer passage in the top k. For example, DPR's 78.4% on NQ at top-20.