LLM retrieval-augmented QA

DesignHard55 minMicrosoft and Anthropic style

Your task

Design a retrieval-augmented LLM system that answers questions over a company's 10M-document knowledge base for 50k employees, with a 3s p95 end-to-end budget, answers grounded in (and citing) source documents, freshness within minutes of a document changing, and strict access control so users only see content they're authorized for. Cover ingestion/indexing, the retrieval + generation path, how you enforce permissions and freshness, and how you evaluate answer quality.

It covers

  • Retrieval augmented
  • Caching
  • Guardrails
  • Evaluation