Fix the vacuous test finder for a webhook suite

Review the AI's codeEasy12 min

Your task

An agent wrote tests for your webhook payload parser. Coverage is high and every test passes. But one test only checks that the result is defined, and another asserts inside a catch block, which never runs if the parser fails to throw. Your team now runs a check that flags tests like these.

An assistant wrote flag_vacuous_tests(test_names, assertions). It is in the editor. It has a bug. Find it and fix it.

Input and output:

  • test_names: the names of the tests.
  • assertions: for the test at the same position, its matchers separated by ;. A matcher written with a catch: prefix, like catch:toEqual, sits inside a catch block.
  • Return the names of the flagged tests, in input order.

Rules the tests check:

  • If the two lists differ in length, use only the positions both have.
  • Split on ; and trim each item: remove every character whose code point is 32 or below from both ends. Ignore items that are then empty.
  • An item that starts with catch: never counts, whatever matcher follows. It only runs if the code throws.
  • toBeDefined, toBeTruthy, toBeFalsy, toBeUndefined and toBeNull never count, because they do not pin a value. These names compare exactly, case included.
  • Every other item counts as a real assertion, including names not listed here.
  • Flag a test that has no real assertion.

Examples

flag_vacuous_tests(test_names: list[str], assertions: list[str]) -> list[str]
Example 1
Inputflag_vacuous_tests(["parses valid payload", "rejects invalid payload"], ["toBeDefined", "catch:toBeTruthy"])
Output["parses valid payload", "rejects invalid payload"]
Example 2
Inputflag_vacuous_tests(["parses valid payload"], ["toEqual;toBe"])
Output[]
Example 3
Inputflag_vacuous_tests(["rejects invalid payload"], ["rejects.toThrow"])
Output[]