Design a Search Service

Requirements

A search service provides full-text search across millions of documents. It must support queries in < 200ms, index new documents within 60 seconds, handle 50K QPS, and provide relevant results ranked by quality. Search is read-heavy with a 100:1 read-to-write ratio.

Indexing Pipeline

Document Created/Updated
  │
  ▼
CDC / Kafka ──► Indexing Pipeline
                 │
                 ├── 1. Parse document (extract text, metadata)
                 ├── 2. Tokenize text (split into terms)
                 ├── 3. Apply analyzers (lowercase, stem, remove stop words)
                 ├── 4. Generate embeddings (for semantic search)
                 ├── 5. Write to Elasticsearch bulk API
                 └── 6. Update search cache

Elasticsearch Architecture

Cluster: search-production
  │
  ├── Master Nodes (3)
  │   └── Cluster state management, shard allocation
  │
  ├── Data Nodes (10)
  │   ├── Index: products (5 shards, 1 replica)
  │   ├── Index: orders (3 shards, 1 replica)
  │   └── Index: users (2 shards, 1 replica)
  │
  ├── Coordinating Nodes (3)
  │   └── Query routing, result aggregation
  │
  └── Ingest Nodes (2)
      └── Pipeline processing, enrichment

Freshness Targets

Relevance Ranking

• TF-IDF — Term frequency-inverse document frequency. Classic relevance algorithm.
• BM25 — Improved TF-IDF with field-length normalization. Elasticsearch default.
• Learning to Rank — ML model trained on click data for custom ranking.
• Semantic search — Vector embeddings for meaning-based search.
• Hybrid — Combine BM25 + semantic for best of both.

Search Optimization

• Shard routing — Use routing to co-locate related documents.
• Index caching — Cache frequently accessed field data.
• Query caching — Cache frequent query results.
• Prefiltering — Use filters (cached) before queries (uncached).
• Denormalization — Pre-compute joins to avoid runtime joins.