Evaluate test quality without reading
An AI agent generated a 200-line test suite for a date-range overlap library and reports 97% line coverage, all green. You don't have time to read all 200 lines, and you've learned that AI tests can be high-coverage but vacuous. Without trusting the coverage number, how do you efficiently determine whether this suite would actually catch bugs — and what's your decision rule for accepting it?
Implement
surviving_mutants(cases: list[list[int]]) → list[str]Examples
in
[[]]out["always_false","always_true","off_by_one","strict_left","strict_right","swap_and_or"]in
[[[1,5,3,8]]]out["always_true","off_by_one","strict_left","strict_right","swap_and_or"]in
[[[1,5,3,8],[1,2,5,6]]]out["off_by_one","strict_left","strict_right"]What a strong answer looks like
Treat the AI’s output as a draft to verify, not an answer to trust. Name the specific flaw and the input that triggers it, say how you’d catch it (tests, edge cases, reading critically), and how you’d re-prompt or decompose to get it right.
0:00 of about 20 min
Vibe & agentic: describe the solution in plain language (or narrate it) and the coach grades your approach.
Which questions mattered is sealed until you submit. Telling you now would just be handing over the edge cases.
Run or narrate your approach, then ask the coach.