Suggested replies for a help desk

DesignEasy25 min

Your task

Design the suggested-reply feature for a help desk used by a 40-person support team. When a customer email arrives, the agent who opens the ticket should see a reply drafted by a hosted LLM from the ticket text and the customer's last few tickets. The agent edits and sends it; nothing reaches the customer until a person presses send. Volume is about 2,000 new tickets a day, mostly during business hours, and one model call takes 3 to 8 seconds. The provider sometimes times out or rate-limits. Describe when the draft is generated, where it is stored, and what the agent sees when the model is slow or down.

It covers

  • Llm drafts
  • Async jobs
  • Fallback