Vector search with metadata filter

DesignMedium45 minOpenAI style

Your task

Design a vector-search service backing a semantic product/document retrieval system: 200M embeddings (768-dim), 10k QPS of top-50 nearest-neighbor queries with a 50ms p99 budget, frequent inserts/deletes as the corpus changes, and a requirement to filter by structured metadata (category, language, in-stock) at query time. Cover the index choice, sharding, how you handle updates and metadata filtering, and the recall/latency trade-off.

It covers

  • Ann
  • Vector index
  • Hybrid search