Autonomy level from eval results and blast radius
Your task
Before an agent task type may act on its own, the platform team sets its autonomy level from two facts: how it did on the offline eval suite (passes out of runs) and its blast radius ("low", "medium" or "high": how much damage one bad action can do).
Write autonomy_level(passes, runs, blast_radius). It returns one of four levels, from most to least freedom:
"auto": the agent acts on its own."approve": the agent acts only after a human approves each action."suggest": the agent drafts the action and a human carries it out."off": the task type does not run.
Step 1. The pass rate passes / runs picks a level from this table. Thresholds are inclusive: a pass rate exactly on a threshold earns that level. Below the suggest threshold the level is "off".
| Blast radius | auto | approve | suggest |
|---|---|---|---|
| low | 95% | 85% | 60% |
| medium | 98% | 90% | 70% |
| high | never | 97% | 80% |
Step 2. A small eval proves little, so the run count caps the level. With fewer than 100 runs the level is at most "approve"; with fewer than 20 runs it is at most "suggest". A cap only ever lowers a level, never raises it.
Fail closed. Zero runs, or a blast radius that is not exactly one of the three strings (case matters), gives "off".
You can rely on 0 <= passes <= runs. Compare without floating-point rounding, for example passes * 100 >= runs * 95.
Examples:
autonomy_level(196, 200, "low")returns"auto": 98% clears 95%, and 200 runs is enough evidence.autonomy_level(196, 200, "high")returns"approve": a high blast radius never earns auto.autonomy_level(19, 19, "low")returns"suggest": 100% would be auto, but 19 runs caps it at suggest.
Examples
autonomy_level(passes: int, runs: int, blast_radius: str) -> strautonomy_level(196, 200, "low")"auto"autonomy_level(196, 200, "high")"approve"autonomy_level(19, 19, "low")"suggest"