PreviousNext
AI-assisted engineering/Reviewing AI code

Fix the README claim checker for a CLI tool

Review the AI's codeEasy10 minAnthropic and OpenAI style

Your task

An AI agent wrote the README for your team's Python CLI tool. It documented a --config flag, a DEBUG_LEVEL environment variable and pip install yourtool, and the repo has none of them. The agent filled the gaps with what CLI tools usually have. Before generated docs are merged, your team now checks every concrete claim against the code.

An assistant wrote unverified_claims(claims, real_flags, real_env_vars, published_packages). It is in the editor. It has a bug. Find it and fix it.

Input and output:

  • claims: the README's claims, each written kind=value, like flag=--config.
  • real_flags: the flags the CLI really defines.
  • real_env_vars: the environment variables the source really reads.
  • published_packages: the package names that are really published.
  • Return the claims that nothing supports, exactly as written, without repeats, sorted by plain character order.

Rules the tests check:

  • Split a claim at its first = into a kind and a value.
  • A flag claim is supported when its value is in real_flags. An env claim is supported when its value is in real_env_vars. A package claim is supported when its value is in published_packages.
  • Kinds and values are matched exactly: case matters and nothing is trimmed.
  • A claim with any other kind, or with no = at all, is unsupported. If you cannot check a claim, you do not vouch for it.

Examples

unverified_claims(claims: list[str], real_flags: list[str], real_env_vars: list[str], published_packages: list[str]) -> list[str]
Example 1
Inputunverified_claims(["flag=--config", "env=DEBUG_LEVEL", "package=yourtool"], ["--verbose", "--input"], ["TOOL_HOME"], ["realtool"])
Output["env=DEBUG_LEVEL", "flag=--config", "package=yourtool"]
Example 2
Inputunverified_claims(["flag=--verbose"], ["--verbose", "--input"], [], [])
Output[]
Example 3
Inputunverified_claims(["env=TOOL_HOME", "package=realtool"], [], ["TOOL_HOME"], ["realtool"])
Output[]