Question
Your VP loved a vendor slide claiming '55% faster coding with AI' and wants you to prove the same for your 30-engineer org so they can justify the seat spend. You suspect the real picture is messier. Design the measurement: what do you actually track to know whether AI is helping or just moving work around, what vanity metrics will you refuse to report, and how do you avoid a measurement that makes the tool look good while the system gets worse?
Treat the AI’s output as a draft to verify, not an answer to trust. Name the specific flaw and the input that triggers it, say how you’d catch it — tests, edge cases, reading critically — and how you’d re-prompt or decompose to get it right.
Vibe coding: describe the solution in plain language (or narrate it) and the coach grades your approach. Generating runnable code from your description is coming next.