ANN index updates and deletes

DesignHard50 minOpenAI style

Your task

Design the operational lifecycle of an ANN (approximate nearest neighbor) index for a semantic-retrieval service holding 500M item embeddings, serving 20k QPS at p99 < 30ms, where ~5M items are added/changed and ~2M removed every day. The hard part isn't the query, HNSW handles that, it's keeping the index correct and fresh: HNSW graphs don't support efficient deletes, full rebuilds take hours, and a stale index silently returns retired/deleted items. Design the update and rebuild strategy.

It covers

  • Embedding index refresh
  • Ann index
  • Incremental updates