Recognizing when AI chases symptoms
You ask an AI agent to fix a flaky Go test that fails ~5% of the time. It diagnoses a 'race condition' and adds a `sync.Mutex` around the shared map; still flaky. You re-prompt, it adds a `time.Sleep(50 * time.Millisecond)` 'to let the goroutine settle'; still flaky, now slower. Third try it bumps the sleep to 200ms. You're watching it chase the symptom. How do you recognize the model is stuck and reset the approach?
detect_magic_number_loop(patches: list[str]) → str[["mutex=1","sleep_ms=50","sleep_ms=200"]]out"continue"[["sleep_ms=50","sleep_ms=200","sleep_ms=500"]]out"reset:sleep_ms"[["timeout_ms=1000","timeout_ms=1000","timeout_ms=1000"]]out"continue"Treat the AI’s output as a draft to verify, not an answer to trust. Name the specific flaw and the input that triggers it, say how you’d catch it (tests, edge cases, reading critically), and how you’d re-prompt or decompose to get it right.
Vibe & agentic: describe the solution in plain language (or narrate it) and the coach grades your approach.