Next

Find the flaky tests an agent may quarantine

Agent buildEasy10 minGoogle and Microsoft style

Your task

People have stopped reading red builds: they rerun them and hope. An agent hunts for flaky tests, ones that pass and fail on the same code, and proposes them for quarantine: out of the blocking suite until fixed.

Write find_quarantine_candidates(run_records): the rule your agent follows to pick the tests it proposes.

  • run_records (list of str): one CI result per entry, written test,commit,result, for example "login_flow,c1,pass".
  • Return the names of the tests to propose, each once, sorted in ascending order.

Rules:

  1. Split each record on ,. A record that does not split into exactly 3 fields is skipped, so an empty string is skipped too. Fields are used exactly as written, with no trimming.
  2. Only the exact result "pass" is a pass. Any other result ("fail", "error", "PASS", anything else) is a failure.
  3. A test is a candidate only if, at one single commit, it has at least one pass and at least one failure. A pass at one commit and a failure at another does not count: that may be a new bug, not a flake.

Examples

find_quarantine_candidates(run_records: list[str]) -> list[str]
Example 1
Inputfind_quarantine_candidates(["login_flow,c1,pass", "login_flow,c1,fail", "checkout_sum,c1,fail"])
Output["login_flow"]
Example 2
Inputfind_quarantine_candidates(["auth_token,c1,pass", "auth_token,c2,fail"])
Output[]
Example 3
Inputfind_quarantine_candidates(["cache_evict,c9,fail", "cache_evict,c9,fail", "cache_evict,c9,pass"])
Output["cache_evict"]