Suggested replies for a help desk
Your task
Design the suggested-reply feature for a help desk used by a 40-person support team. When a customer email arrives, the agent who opens the ticket should see a reply drafted by a hosted LLM from the ticket text and the customer's last few tickets. The agent edits and sends it; nothing reaches the customer until a person presses send. Volume is about 2,000 new tickets a day, mostly during business hours, and one model call takes 3 to 8 seconds. The provider sometimes times out or rate-limits. Describe when the draft is generated, where it is stored, and what the agent sees when the model is slow or down.
It covers
- Llm drafts
- Async jobs
- Fallback