AI-assisted engineering/Working with an assistant

Fix the coaching rule for a stuck agent loop

Review the AI's codeEasy10 minAnthropic and OpenAI style

Your task

An engineer has asked an agent several times to fix a flaky integration test. Each round brings a different plausible change, and the test still flakes. You coach the team with a simple rule for the next move, based on two facts about each attempt so far: did the engineer give the agent new evidence (logs, a failing seed, a trace), and did the failure change afterwards.

An assistant wrote coach_next_move(attempt_added_evidence, attempt_changed_failure). It is in the editor. It has a bug. Find it and fix it.

Input and output:

attempt_added_evidence, attempt_changed_failure: two lists of booleans of the same length, one entry per attempt, oldest first.
Return "gather_evidence", "re_prompt_with_evidence" or "take_over".

Rules the tests check, in this order:

No attempts yet: "gather_evidence".
The latest attempt added no new evidence: "gather_evidence".
The latest attempt changed the failure: "re_prompt_with_evidence".
Count the stalled attempts: those that added new evidence and did not change the failure. Attempts without new evidence never count. If there are 3 or more: "take_over".
Otherwise: "re_prompt_with_evidence".

Examples

coach_next_move(attempt_added_evidence: list[bool], attempt_changed_failure: list[bool]) -> str

Example 1

Inputcoach_next_move([], [])

Output"gather_evidence"

Example 2

Inputcoach_next_move([False, False, False], [False, False, False])

Output"gather_evidence"

Example 3

Inputcoach_next_move([True], [False])

Output"re_prompt_with_evidence"