Hybrid Retrieval & Reranking

Combining keyword + semantic search and reranking for precision.

8 min readAI for SRE WorkflowsMaster

Why Hybrid?

Semantic search finds meaning matches but misses exact terms. Keyword search finds exact matches but misses synonyms. Hybrid retrieval combines both to get the best of each approach.

Consider a query like OOMKill on checkout-service. Semantic search might find runbooks about memory exhaustion but miss the exact error message. Keyword search would find the exact OOMKill logs but miss runbooks about memory limits.

BM25 Keyword Search

BM25 is the industry-standard ranking function for keyword search. It scores documents based on term frequency and inverse document frequency, giving higher scores to documents that contain rare, relevant terms.

BM25 with rank_bm25


from rank_bm25 import BM25Okapi
import jieba

# Tokenize documents
corpus = [
    "Redis connection timeout in production cluster",
    "OOMKill on checkout-service pods",
    "DNS resolution failures in us-east-1",
]

tokenized = [list(jieba.cut(doc)) for doc in corpus]
bm25 = BM25Okapi(tokenized)

# Search
query = "Redis timeout production"
query_tokens = list(jieba.cut(query))
scores = bm25.get_scores(query_tokens)

# Rank results
ranked = sorted(enumerate(scores), key=lambda x: x[1], reverse=True)
for idx, score in ranked:
    print(f"[{score:.4f}] {corpus[idx]}")

BM25 excels at exact term matching. It handles technical terms, error codes, and specific service names better than semantic search.

Combining Results

Combine BM25 and semantic search results using a fusion algorithm like Reciprocal Rank Fusion (RRF). RRF merges ranked lists by considering the rank position of each document in both result sets.

Reciprocal Rank Fusion


def reciprocal_rank_fusion(result_lists, k=60):
    """Combine multiple ranked lists using RRF."""
    scores = {}
    for result_list in result_lists:
        for rank, (doc_id, _) in enumerate(result_list):
            if doc_id not in scores:
                scores[doc_id] = 0
            scores[doc_id] += 1 / (k + rank + 1)
    return sorted(scores.items(), key=lambda x: x[1], reverse=True)

# BM25 results: [(doc_id, bm25_score), ...]
bm25_results = [(0, 8.5), (2, 6.2), (1, 3.1)]

# Semantic results: [(doc_id, similarity_score), ...]
semantic_results = [(1, 0.92), (0, 0.78), (2, 0.45)]

# Fuse results
fused = reciprocal_rank_fusion([bm25_results, semantic_results])
for doc_id, score in fused:
    print(f"[{score:.4f}] {corpus[doc_id]}")

RRF is parameter-free and robust. It does not require normalizing scores across different search methods.

Reranking with Cross-Encoders

After retrieving candidate documents, a reranker re-scores them using a more accurate but slower model. Cross-encoders process the query and document together, producing a more precise relevance score than separate embeddings.

Reranking with Cohere or a cross-encoder


import cohere

co = cohere.Client("your-api-key")

# Rerank retrieved chunks
results = co.rerank(
    model="rerank-english-v3.0",
    query="Redis is down, how do I failover?",
    documents=[
        "Runbook: Redis failover for production",
        "Incident: DNS failures in us-east-1",
        "How to scale Kubernetes pods on CPU",
        "Redis cluster failover procedure and replica promotion",
    ],
    top_n=2,
    rank_fields=["text"]
)

for result in results.results:
    print(f"[{result.relevance_score:.4f}] {result.index}: {result.text}")

Reranking is expensive but highly effective. It takes the top 20 candidates from retrieval and re-scores them, returning the top 5 for the LLM context.

Use reranking for high-stakes queries

For alert triage and incident response, the accuracy of retrieval matters more than speed. Use reranking to improve precision on these critical queries.

Implementation

Full hybrid retrieval pipeline


class HybridRetriever:
    def __init__(self, vector_db, bm25_index, reranker):
        self.vector_db = vector_db
        self.bm25 = bm25_index
        self.reranker = reranker

    def retrieve(self, query, top_k=5, rerank_top=20):
        # Step 1: Semantic search
        semantic_results = self.vector_db.search(query, top_k=rerank_top)

        # Step 2: BM25 search
        bm25_results = self.bm25.search(query, top_k=rerank_top)

        # Step 3: Fuse results
        fused = reciprocal_rank_fusion([semantic_results, bm25_results])
        candidates = [doc_id for doc_id, _ in fused[:rerank_top]]

        # Step 4: Rerank
        reranked = self.reranker.rerank(query, candidates, top_k=top_k)

        return reranked

# Usage
retriever = HybridRetriever(vector_db, bm25_index, reranker)
results = retriever.retrieve("Redis connection pool exhausted")
for chunk in results:
    print(f"[{chunk.score:.4f}] {chunk.source}: {chunk.text[:100]}")

This pipeline retrieves broadly with two methods, fuses the results, and reranks for precision. It is the gold standard for operational RAG.

Mark this lesson complete to store local progress and unlock a cleaner resume path the next time you visit.

Stage 7 · Master

RAG Over Your Ops Knowledge