Dense, Sparse, and Hybrid Retrieval

In the last post we treated "retrieval" as a vector-search problem: embed the query, look up the nearest neighbors, done. That's the picture almost every RAG diagram draws. If you've worked on search at all, you probably know it's half the story: a twenty-five-year-old bag-of-words algorithm called BM25 still wins on a lot of real queries, and production systems almost never pick just one. They run both, and merge.

This post is about why: why pure dense retrieval misses things any keyword index would catch, why pure BM25 misses things any embedding model would catch, and how a dumb fusion trick called reciprocal rank fusion ends up beating either method alone on almost every benchmark run against it.

I want you to come away with a sticky mental model of three things: what BM25 computes, what a dense biencoder computes, and why the gap between them is exactly why you want both.

The problem

You have a corpus of documents and a stream of queries, and for each query you need to return a ranked list of the documents most likely to answer it. The task is ranking: you don't care about "relevant or not" as a binary, you care about getting the good stuff into the top-10 and leaving the bad stuff out.

The catch is that "most likely to answer it" depends entirely on what the query looks like. If I hand you the query error code XRN-9042, what I want is documents that literally contain that token. If I hand you the query how do I stop hair loss after pregnancy, what I want is documents about postpartum shedding, which may not share a single word with my query. A system tuned for one of those queries is going to be bad at the other.

For most of the history of IR, the answer was keyword search with clever weighting. For most of the last six years, the answer has been dense embedding search. For basically every production system in 2026, the answer is both, with a fusion step.

Before we talk about fusion, we have to understand what each side is doing.

BM25: still the baseline

BM25 (Best Matching 25) is a term-weighting function from the mid-1990s, descended from Robertson and Spärck Jones's probabilistic retrieval framework. It scores each (query, document) pair as a sum over the query terms, and that sum boils down to three ideas, explained here in plain English before the math.

The first idea is inverse document frequency: a word that appears in every document tells you nothing, a word that appears in only a few tells you a lot. BM25 weights rare words heavily and common words ("the", "of", "and") almost not at all.

The second idea is term frequency saturation. If a document mentions "quantum" once, great. If it mentions "quantum" ten times, that's more signal but not ten times more. BM25 takes term frequency and bends it through a saturating curve so the first occurrence matters most and the tenth barely moves the score.

The third idea is length normalization. A short document where the query term appears once is a stronger match than a long one where it appears once, because the short document is more about that term. BM25 downweights long-document matches and boosts short-document ones, with a tunable knob b controlling how aggressive that is.

Put them together and you get the BM25 score for a term t in a document d:

The constants are usually $k_1 \approx 1.5$ and $b \approx 0.75$, and basically nobody tunes them past that. Sum over query terms and you have the document's score.

No training, no embeddings, no neural anything. You build an inverted index, look up the postings lists for each query term, sum the weighted contributions, and return the top-k. On a modern machine you can do that for hundreds of millions of documents in milliseconds.

Play with the term picker for a minute: common words contribute almost nothing even when they match, which is why a BM25 score is non-zero for only a tiny handful of documents.

The payoff: BM25 is fast, interpretable, and needs no training data. The cost: anything outside exact token match gets nothing back. Which brings us to the other side.

Dense retrieval: cosine similarity in a trained space

By 2019, it was clear that word-level matching had a ceiling: the postpartum-shedding example from a minute ago is exactly the case it can't handle. You need something that understands the query and the document live in the same semantic space, so the query can be about the document even when the surface strings disagree.

Dense Passage Retrieval (DPR), from Karpukhin et al. at Facebook in 2020, made this work at scale with a biencoder: one encoder for the query, one for the document, both trained to produce a fixed-size vector in the same space, scored by cosine similarity (or dot product, same thing up to normalization).

Training uses the same contrastive setup, InfoNCE with in-batch negatives, from the embeddings post: pull the query and its positive passage together, push the negatives apart, repeat over millions of hard-negative-mined triples.

At inference, the document side is precomputed and stored in a vector index (the ANN structures from the last post); the query side is encoded once, then a k-nearest-neighbor lookup gives the top candidates.

The core capability dense retrieval adds is that the query and the document don't have to share any tokens: search for "cheap place to sleep in tokyo" and get back a document titled "budget hostels in shinjuku", because the embedding model has learned those two strings are geometrically nearby. You can't do that with BM25, no matter how clever your tokenizer is.

There's a cost: training the encoders needs labeled query-passage data, which nobody outside a few large labs has for their own domain, and you inherit the encoder's blind spots. A product SKU, part number, or internal project codename the encoder has never seen gets mapped to noise, and a keyword query that should trivially match silently fails.

The failure modes, side by side

The clean way to see the gap is to put both methods on a query that exposes one of them.

These aren't edge cases. They're the two modes every retrieval system alternates between, usually in the same hour and sometimes in the same query. The lexical case covers product searches, error codes, entity lookups, code search, anything involving names and identifiers. The paraphrase case covers natural-language questions, conversational search, and most of what people do when they talk to an LLM.

Learned sparse: SPLADE

Learned sparse retrieval keeps the inverted-index machinery of BM25 but learns the term weights from data, and (this is the clever part) can add terms to a document that aren't literally in its text.

SPLADE, from Formal et al., runs a BERT-style model over each document, but instead of a fixed-size dense vector it produces a sparse vector over the whole vocabulary: most entries zero, the non-zero ones weighted terms that include synonyms and related concepts the model thinks are implied by the document. Store those in an inverted index, and at query time you match on both original and expanded terms with BM25-style efficiency.

SPLADE occupies a genuinely different point on the tradeoff curve: it trains on labeled data like a dense model, but retrieves over an inverted index like BM25, handling lexical matches out of the box (real tokens) and some paraphrase through learned expansion. On BEIR, it often beats both a clean BM25 baseline and many dense models without hybrid fusion.

ColBERT, from Khattab and Zaharia, keeps a vector per token instead of compressing each passage to one, so scoring becomes a sum of maxes over per-token similarities, more expressive than a single-vector biencoder and more expensive to index. It gets its own post later in this arc, in rerankers and two-stage pipelines.

The mental picture: BM25 and dense retrieval sit at opposite corners of a design space, and SPLADE and ColBERT each occupy clever interior points.

Hybrid: reciprocal rank fusion

Combining two ranked lists into one is an old, well-studied problem. The method almost every production system uses in 2026 is simple enough to write in a couple of lines: reciprocal rank fusion, proposed by Cormack, Clarke, and Büttcher in 2009, before dense retrieval was even a thing.

Each retriever produces a ranked list. A document's RRF score is the sum, across lists, of one over its rank plus a constant k (usually 60). Higher score wins.

Reciprocal rank is monotone in rank but discounts heavily after the first few positions: a document ranked #1 contributes $1/61 \approx 0.0164$, ranked #50 only $1/110 \approx 0.0091$. So a document has to be high in at least one list to matter, and being high in both is a real boost. Noise in the tails gets washed out.

The other reason it works: it's scale-free. BM25 scores and cosine similarities live on totally different scales, so merging them with a weighted sum of raw scores would mean calibrating weights per query, per corpus, and ideally per mood of the machine. RRF only cares about ranks. Nothing to calibrate.

The narrative lives in those rank flips: docs both retrievers agree on climb, docs only one retriever liked get discounted, and none of it needed tuning.

The other common fusion strategy, a weighted sum of min-max-normalized BM25 and dense scores tuned on a validation set, can beat RRF if you tune it well, and loses if you don't. Most production systems just use RRF because it needs no tuning.

The worked example

Let's grind through the full example by hand on a 10-document corpus so the numbers in all those vizzes stop feeling like magic.

Our corpus is ten short "documents":

1. the king of england visited france
2. queen elizabeth ruled england for decades
3. the royal leader addressed the nation
4. a chess king can move one square
5. monarchs of europe met in paris
6. france and england share a border
7. pawns protect the king in chess
8. the president of france spoke today
9. elizabeth crowned king edward the second
10. royal family visits the paris monument

Total tokens: 61. Average document length: 6.1. We'll run the query king of england through both BM25 and a pretend dense encoder, then fuse.

For BM25 we need three idf values. With $N=10$ and Lucene-style idf $\ln(1 + (N - n + 0.5) / (n + 0.5))$:

• "king" appears in 4 docs. $\text{idf} = \ln(1 + 6.5/4.5) = 0.89$
• "of" appears in 3 docs. $\text{idf} = \ln(1 + 7.5/3.5) = 1.15$
• "england" appears in 3 docs. $\text{idf} = \ln(1 + 7.5/3.5) = 1.15$

For document 1, "the king of england visited france", all three query terms match with $\text{tf} = 1$. The length normalization term is $1 - 0.75 + 0.75 \cdot (6/6.1) = 0.988$, so the saturation factor is $2.5 / (1 + 1.5 \cdot 0.988) \approx 1.007$. The three contributions are $0.89 \cdot 1.007 \approx 0.90$, $1.15 \cdot 1.007 \approx 1.16$, and $1.16$ again. Sum: about $3.22$.

Document 9, "elizabeth crowned king edward the second", only has "king". Its score is about $0.90$. Documents 2, 5, 6, 8 each have exactly one of the query terms and end up at about $1.16$ (or $0.90$ if the matching term was "king"). Document 3, "the royal leader addressed the nation", contains zero query terms. Its score is zero.

Ranked by BM25: d1, d2, d5, d6, d8, d9, d7, d4, with d3 and d10 tied at zero. Take the top 5: [d1, d2, d5, d6, d8].

Now pretend we have a trained biencoder that loves d1 (exact semantic match), d9 (crowning a king of england), d3 ("royal leader" ≈ "king of england"), d2 (queen of england), and d5 (monarchs of europe), but not d6 (a border) or d8 (the french president). Dense top 5: [d1, d9, d3, d2, d5].

Now RRF with $k = 60$. A document's score is $\sum_r 1 / (k + \text{rank}_r)$, where the rank is counted across the full ranked lists (not just top-5). Computing that out:

• d1: $1/61 + 1/61 = 0.03279$ (rank 1 in both)
• d2: $1/62 + 1/64 = 0.03176$ (BM25 rank 2, dense rank 4)
• d5: $1/63 + 1/65 = 0.03126$ (BM25 rank 3, dense rank 5)
• d9: $1/66 + 1/62 \approx 0.03128$ (BM25 ~rank 6, dense rank 2)
• d3: $1/69 + 1/63 \approx 0.03037$ (BM25 near bottom, dense rank 3)

Fused top 5: [d1, d2, d5, d9, d3].

When each wins

That heatmap is the same split from the problem statement and the failure-modes viz above, just measured: BM25 leads on keyword-heavy and entity queries, dense leads on paraphrase and multi-hop QA, and SPLADE splits the difference as the strongest single-method baseline in most BEIR tasks. Hybrid sits at or near the top of every column, because it never has to guess which kind of query is coming before it retrieves.

The implementation sketch

Here's a minimal end-to-end version in Python: BM25, a toy dense retriever, and RRF fusion, on the same corpus we worked through above. The BM25 numbers match what we computed by hand.

import math
from collections import Counter
 
corpus = [
    "the king of england visited france",
    "queen elizabeth ruled england for decades",
    "the royal leader addressed the nation",
    "a chess king can move one square",
    "monarchs of europe met in paris",
    "france and england share a border",
    "pawns protect the king in chess",
    "the president of france spoke today",
    "elizabeth crowned king edward the second",
    "royal family visits the paris monument",
]
docs = [d.lower().split() for d in corpus]
N = len(docs)
avgdl = sum(len(d) for d in docs) / N
 
def bm25(query, docs, k1=1.5, b=0.75):
    # Document frequency for each query term
    df = {t: sum(1 for d in docs if t in d) for t in query}
    scores = []
    for d in docs:
        tf = Counter(d)
        dl = len(d)
        s = 0.0
        for t in query:
            if tf[t] == 0 or df[t] == 0:
                continue
            idf = math.log(1 + (N - df[t] + 0.5) / (df[t] + 0.5))
            sat = (tf[t] * (k1 + 1)) / (tf[t] + k1 * (1 - b + b * dl / avgdl))
            s += idf * sat
        scores.append(s)
    return scores
 
bm25_scores = bm25(["king", "of", "england"], docs)
bm25_rank = sorted(range(N), key=lambda i: -bm25_scores[i])
print("BM25 top 5:", [i + 1 for i in bm25_rank[:5]])
# [1, 2, 5, 6, 8]

# --- Toy "dense" retriever ---
# In real life this would be cosine sim between trained embeddings.
dense_scores = [0.92, 0.78, 0.81, 0.55, 0.74, 0.48, 0.52, 0.46, 0.86, 0.61]
dense_rank = sorted(range(N), key=lambda i: -dense_scores[i])
print("Dense top 5:", [i + 1 for i in dense_rank[:5]])
# [1, 9, 3, 2, 5]
 
# --- Reciprocal rank fusion ---
def rrf(rankings, k=60):
    scores = [0.0] * N
    for r in rankings:
        for pos, doc_id in enumerate(r):
            scores[doc_id] += 1.0 / (k + pos + 1)
    return scores
 
fused = rrf([bm25_rank, dense_rank])
fused_rank = sorted(range(N), key=lambda i: -fused[i])
print("Fused top 5:", [i + 1 for i in fused_rank[:5]])
# [1, 2, 5, 9, 3]

No fancy libraries, no training, no vector database. Just a bag-of-words BM25, a handcrafted dense score, and twelve lines of RRF.

Misconceptions

"Dense retrieval obsoleted BM25." This comes up a lot, and it's wrong: BM25 is still the single strongest method on a surprising fraction of queries, especially anything keyword-driven. BEIR showed that across 18 datasets, BM25 beats or ties the best dense retriever on about half of them out of the box. Dense unlocked a capability BM25 never had; BM25 kept doing the job it was already doing.

"Hybrid retrieval needs a trained fusion model." It doesn't, and you probably shouldn't start with one. RRF with $k=60$ and no tuning beats most learned fusion models on most benchmarks, and the ones that do win by margins small enough to vanish when the corpus shifts. Learned fusion is a great way to improve a mature system by a few points of Recall@10, and a bad way to start a new one: begin with RRF, measure, and graduate only when you can see it limiting you.

"Embeddings are a replacement for the inverted index." Vector indexes and inverted indexes have different failure modes and strengths, and the production pattern is to run both with matching doc_id schemas so fusion is trivial: less "pick a backend," more "hang two indexes off the same corpus."

"Bigger embeddings always beat BM25." Scaling a biencoder helps on paraphrase-heavy queries. It doesn't help on rare-identifier queries, because no amount of representation capacity fixes an identifier that's rare in the training data. That's why production teams keep BM25 in the stack even alongside enormous dense models: a functional hedge, not a cost optimization.

What's next

This post handed you the two halves of retrieval, lexical and semantic, and the fusion trick that glues them together. Next is chunking: before you can retrieve anything, you have to decide what your "documents" even are. A naive character-count split versus a layout-aware strategy turns out to matter almost as much as the retriever choice.

The next post covers Chunking Strategies, the part of the RAG pipeline that gets very little research attention but quietly decides whether your whole system works.

Additional reading (and watching)

• Robertson, S. & Zaragoza, H. (2009). The Probabilistic Relevance Framework: BM25 and Beyond. Foundations and Trends in Information Retrieval. The canonical reference for BM25's derivation and parameter choices.

• Karpukhin, V., et al. (2020). Dense Passage Retrieval for Open-Domain Question Answering. EMNLP 2020. The biencoder-plus-contrastive recipe that made dense retrieval practical at scale.

• Formal, T., et al. (2021). SPLADE: Sparse Lexical and Expansion Model for First Stage Ranking. SIGIR 2021. Learned sparse retrieval that keeps inverted-index machinery while adding expansion terms.

• Thakur, N., et al. (2021). BEIR: A Heterogeneous Benchmark for Zero-Shot Evaluation of Information Retrieval Models. NeurIPS 2021. The benchmark that made clear BM25 isn't obsolete: dense retrievers lose to it on many out-of-distribution tasks.

• Khattab, O. & Zaharia, M. (2020). ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT. SIGIR 2020. The late-interaction architecture we'll revisit in the rerankers post.

• Cormack, G., Clarke, C., & Büttcher, S. (2009). Reciprocal Rank Fusion Outperforms Condorcet and Individual Rank Learning Methods. SIGIR 2009. The original RRF paper, still the simplest and most robust fusion method in production.

• Lin, J., Nogueira, R., & Yates, A. (2021). Pretrained Transformers for Text Ranking: BERT and Beyond. Synthesis Lectures on Human Language Technologies. Book-length survey of the neural IR landscape, mapping where BM25, DPR, SPLADE, and ColBERT sit relative to each other.

• Supabase. (2024). Hybrid search with Postgres, pgvector, and tsvector. The small-scale Postgres version of the same pattern, retrieval inside the application database.