Prompt change checks for a support email tagger

Your support tool tags each incoming email with one of 15 topics, such as refund, delivery or login. The tag decides which team's queue the email lands in. A hosted model does the tagging, using a prompt your team writes.

Engineers change the prompt about twice a week, usually to fix one topic. Last month a change meant for delivery emails sent most refund emails to the wrong queue. Nobody noticed for 3 days.

You get about 8,000 emails a day, and about 10% are refunds. When an agent moves an email to another queue, the tool records the topic it moved to. You have 18 months of these records. A tagging call costs about $0.001, and your provider allows 50 calls a second.

What runs before a prompt change goes live, and how would you spot a drop in tagging after it ships?