Find the flaky tests an agent may quarantine
Your task
People have stopped reading red builds: they rerun them and hope. An agent hunts for flaky tests, ones that pass and fail on the same code, and proposes them for quarantine: out of the blocking suite until fixed.
Write find_quarantine_candidates(run_records): the rule your agent follows to pick the tests it proposes.
run_records(list of str): one CI result per entry, writtentest,commit,result, for example"login_flow,c1,pass".- Return the names of the tests to propose, each once, sorted in ascending order.
Rules:
- Split each record on
,. A record that does not split into exactly 3 fields is skipped, so an empty string is skipped too. Fields are used exactly as written, with no trimming. - Only the exact result
"pass"is a pass. Any other result ("fail","error","PASS", anything else) is a failure. - A test is a candidate only if, at one single commit, it has at least one pass and at least one failure. A pass at one commit and a failure at another does not count: that may be a new bug, not a flake.
Examples
find_quarantine_candidates(run_records: list[str]) -> list[str]Example 1
Input
find_quarantine_candidates(["login_flow,c1,pass", "login_flow,c1,fail", "checkout_sum,c1,fail"])Output
["login_flow"]Example 2
Input
find_quarantine_candidates(["auth_token,c1,pass", "auth_token,c2,fail"])Output
[]Example 3
Input
find_quarantine_candidates(["cache_evict,c9,fail", "cache_evict,c9,fail", "cache_evict,c9,pass"])Output
["cache_evict"]