Monthly active users query
You ask an AI to write a Postgres query for 'monthly active users' and it returns a clean `COUNT(DISTINCT user_id) ... GROUP BY date_trunc('month', event_time)`. The numbers look right in a quick check, so you ship it — then finance flags that March is double-counting versus their dashboard. You re-prompt 'fix the double counting' and it adds `DISTINCT` in three more places, changing nothing. What assumption did the model bake in, and how do you re-steer to actually fix it?
monthly_active_users(event_rows: list[str], qualifying_types: list[str], tz_offset_hours: int) → list[str][["u1|2026-03-01T02|login","u1|2026-03-01T14|login","u2|2026-03-15T09|purchase","u2|2026-03-16T09|heartbeat"],["login","purchase"],-5]out["2026-02=1","2026-03=2"][["u1|2026-04-01T12|login","u1|2026-04-02T12|login","u1|2026-04-03T12|purchase"],["login","purchase"],0]out["2026-04=1"][["u3|2026-02-28T20|login"],["login"],5]out["2026-03=1"]Treat the AI’s output as a draft to verify, not an answer to trust. Name the specific flaw and the input that triggers it, say how you’d catch it (tests, edge cases, reading critically), and how you’d re-prompt or decompose to get it right.
Vibe & agentic: describe the solution in plain language (or narrate it) and the coach grades your approach.