Reverse-engineered design doc accuracy
You hand an AI your existing codebase and ask it to reverse-engineer an architecture design doc 'describing how the system actually works' for a new team onboarding. It produces confident sections on the request lifecycle, the data model, and the failure-handling strategy. The failure-handling section in particular reads cleanly. How do you verify a reverse-engineered design doc reflects the real system and isn't describing how the AI assumes a system like this works?
find_unverified_claims(claim_lines: list[str], code_index: list[str]) → list[str][["request lifecycle|handler parses then dispatches|src/server.rs:dispatch","failure-handling|we retry with exponential backoff|src/http.rs:retry_with_backoff","data model|orders hold line items|src/db/schema.rs:Order"],["src/server.rs:dispatch","src/db/schema.rs:Order","src/http.rs:send_once"]]out["failure-handling: we retry with exponential backoff"][["failure-handling|circuit breaker opens after 5 failures|"],["src/http.rs:send_once"]]out["failure-handling: circuit breaker opens after 5 failures"][["data model|orders hold line items|src/db/schema.rs:Order"],["src/db/schema.rs:Order"]]out[]Treat the AI’s output as a draft to verify, not an answer to trust. Name the specific flaw and the input that triggers it, say how you’d catch it (tests, edge cases, reading critically), and how you’d re-prompt or decompose to get it right.
Vibe & agentic: describe the solution in plain language (or narrate it) and the coach grades your approach.