Detecting silent API contract changes
You asked an AI agent to "clean up and tighten types" in the request-handling module of a public REST API written in TypeScript. The PR looks tidy. Your concern: did it silently change the external contract (response shape, status codes, nullability, error envelope) that thousands of customers depend on? You don't want to read every line. What's your verification strategy to prove the public contract is unchanged?
diff_api_contract(baseline_fields: list[str], candidate_fields: list[str]) → list[str][["id|string|required","email|string|optional"],["id|string|required","email|string|optional"]]out[][["id|string|required","email|string|required","legacy|number|optional"],["id|number|required","email|string|optional","added|string|optional"]]out["breaking:now_optional:email","breaking:removed:legacy","breaking:type_changed:id","compatible:added:added"]Treat the AI’s output as a draft to verify, not an answer to trust. Name the specific flaw and the input that triggers it, say how you’d catch it (tests, edge cases, reading critically), and how you’d re-prompt or decompose to get it right.
Vibe & agentic: describe the solution in plain language (or narrate it) and the coach grades your approach.