Code search indexing
Your task
Design the indexing subsystem for a code-search engine over 100M repositories (~2B files, 50TB of source). It must support exact substring/regex-ish matches plus symbol search, ingest pushes within minutes of a commit, and handle that 30% of files change weekly. Query p95 under 500ms across the corpus.
It covers
- Segment merge
- Real time indexing
- Deletes