Exam-style
An evaluation run shows the correct passage sits inside the top 25 retrieved chunks for 90 percent of questions, but inside the top 3 passed to the model for only 40 percent. What change lifts answer quality most directly?
Reveal the answer
B. Retrieve 25 candidates and rerank them with the Bedrock Rerank API, passing the top 3 reranked chunks on
Recall is already good, since the right passage is in the top 25 nine times out of ten; the problem is ordering, and that is what reranking fixes. A cross-encoder reads the query and each candidate together rather than comparing two precomputed vectors, so it scores relevance far more precisely than the first-stage embedding. Stuffing all 25 chunks into the prompt dilutes attention and costs more tokens, and the other options attack recall or chunk shape, neither of which is failing here.
Q. The right chunk is retrieved but ranks twelfth, below the cutoff. What promotes it?
A. A cross-encoder reranker (the Bedrock Rerank API, e.g. Cohere or Amazon Rerank) re-scores the top-N candidates by reading query and document together, far more precisely than the first-stage embedding score. Retrieve wide, rerank narrow.
Why? Reranking fixes ordering, not recall: if the chunk is not in the top-N at all, fix retrieval first.