Fuse lexical and vector rankings
Your task
Design a hybrid retrieval system that combines lexical (BM25 inverted index) and dense vector (ANN) search and returns a single fused ranked list. The two retrievers disagree often: lexical nails exact-term/code/SKU queries, vectors nail paraphrase/semantic queries, and their scores are on totally different scales (BM25 unbounded, cosine in [-1,1]). Corpus: 200M documents, 8k QPS, p99 < 200ms. How do you run both, and, the crux, how do you fuse two incomparable score distributions into one ranking?
It covers
- Hybrid retrieval
- Lexical vector fusion
- Reciprocal rank fusion