The release gate blocked a version over one task
A release gate agent compares each candidate version with the one in production on the nightly eval. Last night it held v18 back over a single task, paged the release owner, and kept the block after a rerun.
Your task
Name the span where the gate first went wrong, then fix gate() so the replay passes.
- root_cause_span() returns the FIRST span whose plan contradicted what the agent already knew
- gate(old, new) returns "ship", "rerun", "block", or "escalate"; old and new are one task's runs, oldest first, "P" for a pass and "F" for a failure
- no old runs, no new runs, or any letter other than P and F escalates before anything else; otherwise a new version that passes every run ships
- a task that passed every old run is stable, and any failure on the new version blocks it
- a task that failed at least once in its old runs is flaky: with fewer than 3 new runs, a failure earns a rerun; with 3 or more, it blocks when every new run failed and ships otherwise
- leave call_tool() and check() alone; the eval store is not the bug
The replay runs your gate() across task histories at submit, and your named span is checked against the key.