RRF: the tiebreaker that fuses two rank lists

You have a vector search that scores 0–1 and a BM25 search that scores 0–30. Adding them is meaningless. Reciprocal Rank Fusion sidesteps the whole scale problem by using ranks, not scores. This lesson walks through the algorithm, the code, and the three numbers you have to know: k=60, VECTOR_WEIGHT=0.7, BM25_WEIGHT=0.3.

1. Recap: the score-scale problem

In Lesson 0003 we ran the same query through both methods and got two different universes of scores:

cosine :  [0.10 .. 0.67]      (bounded 0..1)
BM25   :  [7.49 .. 29.98]     (unbounded positive)

They are not on the same axis. You cannot add them. You cannot take a weighted average. You cannot min-max them into the same range without losing information about confidence.

The brute-force idea: normalize. Min-max into [0,1]. Take a weighted average. Done. We did that exercise in §2 of Lesson 0003. It works — barely — but it has three nasty failure modes. Let me show you, because the pain of normalization is the whole reason RRF exists.

2. The naive solution: normalize

Three ways to normalize, and why each one breaks something:

3. The RRF insight: just use ranks

The Cormack, Clarke & Buettcher paper from 2009 had a simple, beautiful idea: when you sort a list of candidates by score, the actual score values don't matter — only the position does. A candidate that ranked 1st by cosine is more similar to the query than one that ranked 5th, regardless of whether the score was 0.95 or 0.7.

So instead of mixing scores, we mix ranks. Each ranker (vector, BM25) produces an ordered list. RRF walks every list, and for each document that appears in any list, it sums the per-list contribution:

RRF(d) = Σᵢ  wᵢ / (k + rankᵢ(d))

Where i indexes the ranker (vector, BM25, …), rankᵢ(d) is the 1-indexed position of d in ranker i's list (or "did not appear" — handled below), k is a smoothing constant (we use 60), and wᵢ is the weight for that ranker (we use 0.7 for vector, 0.3 for BM25).

4. The formula, slowly

Take a document d and two rankers, vector and BM25. Compute the RRF score step by step:

RRF(d) =  w_vec / (k + rank_vec(d))   +   w_bm25 / (k + rank_bm25(d))

The contributions are inverse-rank: a rank of 1 gives 1/(60+1) = 0.01639, a rank of 2 gives 1/62 = 0.01613, a rank of 100 gives 1/160 = 0.00625. The first few ranks dominate; after rank 30 or so, additional rank positions barely matter.

What if a document is missing from one list? Two valid choices:

• Treat it as having infinite rank (contribution ≈ 0). This is what we do.
• Treat it as a tie at the bottom. Less common.

What if a document is missing from both lists? It never enters the RRF computation at all. Trivially correct.

Worked example, by hand

Say we have a document d that ranked 1st by cosine and 4th by BM25. With k=60, w_vec=0.7, w_bm25=0.3:

RRF(d) = 0.7 / (60 + 1)   +   0.3 / (60 + 4)
       = 0.7 / 61          +   0.3 / 64
       = 0.01148           +   0.00469
       = 0.01617

Now another document d' that ranked 3rd by cosine and 1st by BM25:

RRF(d') = 0.7 / (60 + 3)  +   0.3 / (60 + 1)
       = 0.7 / 63          +   0.3 / 61
       = 0.01111           +   0.00492
       = 0.01603

d beats d' by 0.00014 — a hair. RRF correctly says "both lists like both documents, but vector likes d a bit more."

5. Walkthrough: HybridSearch.search

Here's the actual code from rag_pipeline.py:

class HybridSearch:
    def search(self, query, top_k=5):
        t0 = time.time()
        # Step 1: get vector hits (top_k * 2 for a fudge factor)
        query_vec = self.embedder.embed_one(query)
        vec_results = self.vector_store.vector_search(query_vec, top_k=top_k * 2)

        # Step 2: get BM25 hits (same fudge factor)
        bm25_results = self.bm25.search(query, top_k=top_k * 2)

        # Step 3: build a dict of {chunk_id: result_with_scores}
        scores: dict[str, dict] = {}

        # Step 4: walk the vector rank list, add RRF contribution
        for rank, r in enumerate(vec_results):
            cid = r["id"]
            scores[cid] = {
                **r,
                "rrf_score":  self.vector_weight / (self.rrf_k + rank + 1),
                "vector_rank": rank + 1,
                "bm25_rank": None,
                "vector_score": float(1 - r.get("score", 0)),
            }

        # Step 5: walk the BM25 rank list, add RRF contribution
        for rank, r in enumerate(bm25_results):
            cid = r["doc_id"]
            boost = self.bm25_weight / (self.rrf_k + rank + 1)
            if cid in scores:
                # already in dict — just add BM25's contribution
                scores[cid]["rrf_score"] += boost
                scores[cid]["bm25_rank"] = rank + 1
            else:
                # not in vector list at all — start a new entry
                scores[cid] = hit_from_bm25_only(
                    cid, r["content"], self.bm25.chunk_meta.get(cid, {}),
                    rank + 1, boost,
                )

        # Step 6: sort by rrf_score, return top-k
        results = sorted(scores.values(), key=lambda x: x["rrf_score"], reverse=True)[:top_k]
        return results, timing

Reading the code, line by line

Step 1-2: retrieve top 2k from each method. The 2k is a fudge factor — we over-fetch because RRF can rescue a chunk that was rank 7 in cosine but rank 1 in BM25. If we only fetched k from each, that chunk would be excluded from both lists.

Step 3: a dict keyed by chunk id. This is the union operation. The dict will grow to be the union of all chunks that appear in either list.

Step 4: walk the vector list. For each hit at rank r (0-indexed), compute the per-rank contribution w_vec / (k + r + 1). Note the + 1 — the code uses 0-indexed ranks, so we shift to 1-indexed. Rank 0 in code → rank 1 in formula → contribution 0.7 / 61.

Step 5: walk the BM25 list. If the chunk is already in the dict (it appeared in the vector list too), we add the BM25 contribution to its existing RRF score. If not, we create a new entry from scratch with only the BM25 contribution. This is how chunks "missing" from one list get a partial score (just the other list's contribution).

Step 6: sort the union dict by RRF score, take the top k. Done.

6. Why k=60

From the original Cormack et al. paper: k=60 was chosen empirically as a robust default that performs well across many document collections. The intuition:

• Too small (k → 0): RRF degenerates into "first rank wins everything." A document at rank 1 contributes 1.0, a document at rank 100 contributes 0.01 — the gap is huge, and you lose any signal from secondary rankers.
• Too large (k → ∞): all ranks become equal. 1/(k+1) ≈ 1/(k+100). You lose all positional information; RRF degenerates into "count how many lists contain this document."
• k=60: a balance. The first few ranks still dominate, but the tail contributes meaningfully.

Concretely, here's how the contribution 1/(k + rank) decays:

The top 5 ranks contribute almost identically; the top 20 contribute within 76% of the leader. After rank 50, contributions are still meaningful. That's the property we want: rank 1 vs rank 2 vs rank 5 vs rank 20 all matter, but the gap between them is graceful, not cliff-like.

7. Why the weights are 0.7 and 0.3

Our config.py:

VECTOR_WEIGHT = 0.7
BM25_WEIGHT   = 0.3

The reasoning is empirical, not theoretical. For a well-tuned dense encoder like all-MiniLM-L6-v2 on a domain corpus, vector search has higher top-1 precision than BM25 on most queries. But BM25 still catches:

• queries that contain rare, distinctive terms (e.g. product names, error codes)
• queries where the embedding model is confused by polysemy
• queries where the user's wording is lexically close to a chunk's wording

A 70/30 split says: "trust the semantic ranking first, but let the lexical ranking break ties." It's the same intuition as a tiebreaker in sports — the primary score decides, the secondary score settles close calls.

The exact ratio depends on your domain. If your chunks are long, well-written prose, you might want 50/50 or even 30/70 (BM25 has more terms to work with). If your chunks are short and contain lots of unique identifiers, the same. For a TIL-style corpus like ours, where every article is short and the vocabulary is technical, 70/30 is a reasonable starting point.

8. Exercise: playground_rrf.py

The playground runs the three queries from the paraphrase robustness test through the real HybridSearch class and shows the fused ranking. You'll see exactly how RRF reorders the BM25-only hits and the cosine-only hits into a final list.

1. Open playground_rrf.py in your editor.
2. Run it: uv run python playground_rrf.py
3. For each query, look at the Fusion trace table: the BM25 rank, the cosine rank, the per-list RRF contribution, and the final RRF score. The TARGET chunk (the Neovim-folding article) is highlighted.
4. Predict before each one: will the TARGET be in the final top-3?

Pay special attention to the third query, "collapse code blocks in my editor":

• BM25 ranked TARGET at 16 (basically invisible).
• Cosine ranked TARGET at 3 (a soft hit).
• Watch RRF: the cosine contribution rescues the chunk, the BM25 contribution is ≈ 0 (missing from top-10), and the chunk climbs to a useful final position.

This is the RRF "rescue" pattern — the most important behavior in the whole lesson.

9. Check your understanding

1. The scale problem. Vector search returned scores in [0, 1] and BM25 in [0, 30]. Without RRF, can you just min-max both to [0, 1] and add them with weights? What's the failure mode?
2. The formula. For a chunk at cosine rank 1 and BM25 rank 1, with our weights, what is the RRF score? (Do the math by hand before running the playground.)
3. The smoothing constant. If we set k=0, what would the formula become? Why is that wrong? If we set k=10⁶, what would it become? Why is that also wrong?
4. The weights. If you built a RAG system over a code corpus where users typically search for exact function names or error strings, which weight would you increase: vector or BM25? Why?
5. The rescue pattern. In the playground, the third query has TARGET at cosine #3 but BM25 #16. How much of the final RRF score for TARGET comes from cosine vs BM25? Why is the BM25 contribution so small?

Try to answer these in your head, then run the playground and check. When you're satisfied, tell me and we'll move on to Lesson 0005: agentic RAG (the agentic.py layer that uses this hybrid search as a primitive).