Vector Databases & Hybrid Search (Dense + BM25 with RRF)
Implement enterprise-grade search combining dense semantic embeddings with sparse keyword inverted indices using Reciprocal Rank Fusion (RRF).
Key Takeaways
- Dense vector search excels at conceptual matching but frequently fails on exact serial numbers, product SKUs, and acronyms
- Sparse keyword search (BM25) guarantees exact token matching but misses synonyms and paraphrase nuances
- Hybrid search combines Dense + BM25 candidate lists using Reciprocal Rank Fusion (RRF) to produce superior retrieval rankings
- Vector index structures (HNSW vs IVFFlat) balance query latency (QPS), recall accuracy, and memory consumption
The Diagnostic Context
Pure vector search is often insufficient for production enterprise RAG. When a user searches for "Error code ERR-8042 in Kafka connector", semantic embeddings might match general networking articles while missing the exact error documentation. Hybrid search solves this permanently.
The Core Technique
Reciprocal Rank Fusion (RRF) Algorithm in Python
RRF combines two ranked lists without requiring score normalization:
def reciprocal_rank_fusion(
dense_results: list[dict],
bm25_results: list[dict],
k: int = 60
) -> list[dict]:
"""
RRF Score = sum(1 / (k + rank_i)) for each retriever list.
k=60 is the standard constant established in information retrieval research.
"""
rrf_scores: dict[str, float] = {}
doc_lookup: dict[str, dict] = {}
# Score dense ranks
for rank, doc in enumerate(dense_results, start=1):
doc_id = doc["id"]
doc_lookup[doc_id] = doc
rrf_scores[doc_id] = rrf_scores.get(doc_id, 0.0) + (1.0 / (k + rank))
# Score BM25 ranks
for rank, doc in enumerate(bm25_results, start=1):
doc_id = doc["id"]
doc_lookup[doc_id] = doc
rrf_scores[doc_id] = rrf_scores.get(doc_id, 0.0) + (1.0 / (k + rank))
# Sort combined results descending by RRF score
sorted_doc_ids = sorted(rrf_scores.keys(), key=lambda did: rrf_scores[did], reverse=True)
return [
{**doc_lookup[did], "rrf_score": round(rrf_scores[did], 5)}
for did in sorted_doc_ids
]
HNSW vs IVFFlat Indexing
- HNSW (Hierarchical Navigable Small World): Graph-based index offering ultra-low search latency and ~98%+ recall, at the cost of higher RAM usage.
- IVFFlat (Inverted File Flat): Cluster-based partition index with lower memory footprint, suitable for multi-million vector datasets with fast build times.
Try This Right Now
Simulate 5 search results from a dense vector model and 5 results from BM25 with 2 overlapping documents. Run them through the RRF function with k=60 to see how overlapping documents rise to the top!
Tip: Knowledge only becomes capability once you run the prompt yourself.