Pick which mistake you can afford, then set the threshold to match it.
An agent that classifies, triages, or decides gets two kinds of things wrong, and they are not interchangeable. It can act when it should not, and it can fail to act when it should have. Which one hurts more depends entirely on the job, and every other evaluation decision follows from that answer.
The tool is a golden set: real cases, with the right answer recorded by the people who currently do the work. Score the agent against it, then move the confidence threshold and watch precision and recall move in opposite directions. There is no setting where both are perfect, and choosing between them is the actual work.
Then one more move that most teams skip. Let the agent abstain. A band of cases around the threshold, where it genuinely does not know, routed to a human, buys a large drop in errors for a small amount of extra human work.
The sweep does not run itself because your system asks for reduced motion. Raise it and Lower it still work.
Four counts, two ratios, one cost. Everything an evaluation report should open with fits in this much code.
def score(cases, threshold, band=0):
tp = fp = tn = fn = abstained = 0
for c in cases:
if abs(c.confidence - threshold) <= band:
abstained += 1 # routed to a human, counted separately
continue
acted = c.confidence > threshold
if acted and c.truth: tp += 1
elif acted and not c.truth: fp += 1 # false alarm
elif not acted and c.truth: fn += 1 # miss
else: tn += 1
precision = tp / (tp + fp) if tp + fp else None # of what it acted on, how much was right
recall = tp / (tp + fn) if tp + fn else None # of what deserved action, how much it caught
return precision, recall, abstained
def cost(fp, fn, abstained, fp_cost, fn_cost, review_cost=1):
"""The number that actually decides the threshold. Ratios do not."""
return fp * fp_cost + fn * fn_cost + abstained * review_cost
Precision and recall are diagnostics. The cost function is the decision, because it is the only one of the three that knows what your business loses when each mistake happens.
| Move | Precision | Recall | Use when |
|---|---|---|---|
| Raise the threshold | Up | Down | A false alarm is expensive: an unwanted refund, a wrongly closed ticket |
| Lower the threshold | Down | Up | A miss is expensive: a security finding, a safety issue, a churn signal |
| Open an abstain band | Up | Roughly flat | Human review is available and cheaper than either error |
| Grow the golden set | Unchanged | Unchanged | Always. It narrows the error bars on all of the above |
Abstention is the one move that is not a pure trade. It converts the agent's least confident decisions, which are also its least accurate, into a small amount of human work.
An agent triages security findings. A miss is a vulnerability that ships. A false alarm is twenty minutes of an engineer's time.
Those are not close in cost, so recall wins and the threshold goes low. The panel above shows what that does: nearly every real finding is caught, and the queue fills with false alarms that somebody has to clear. That is the deal you signed when you said a miss is expensive, and it is the right deal.
Now open the abstain band. The cases nearest the threshold are where the agent is least reliable, so routing them to a human removes a disproportionate share of both error types. Precision climbs, recall barely moves, and the price is a review queue you can count. Compare that with the same improvement bought by moving the threshold, which always costs you the other metric.
Then switch the cost model to "a false alarm costs more" without moving anything else, and watch the recommended threshold move to the other end of the axis. The agent did not change. What changed is what you told it you could afford.
Your agent flags suspicious transactions. It catches 98% of fraud, and 1 in 3 of the transactions it flags turns out to be fine. What should you say about it?