PreviousNext
AI-assisted engineering/Working with an assistant

Fix the grounding check for invented crate methods

Review the AI's codeEasy10 minAnthropic and OpenAI style

Your task

You asked an AI to integrate a Rust crate, and twice it called methods the crate does not have, each time inventing a new name. Before a fourth prompt, you check its calls against the crate's real API, copied from the docs, and list every method it made up.

An assistant wrote find_ungrounded_calls(proposed_calls, api_surface). It is in the editor. It has a bug. Find it and fix it.

Input and output:

  • proposed_calls: the calls the model wrote, such as conn.query_one(sql).
  • api_surface: the method names the crate really has.
  • Return the invented method names.

Rules the tests check:

  • A call's method name is found in two steps. First cut the call at its first (, if it has one. Then take what follows the last ., or the whole remaining text if there is no ..
  • Skip a call whose method name is empty.
  • A name is invented when it is not in api_surface. The comparison is exact: case and underscores matter, so streamRows is not stream_rows.
  • Return each invented name once, sorted in ascending order by plain string comparison (uppercase letters sort before lowercase).

Examples

find_ungrounded_calls(proposed_calls: list[str], api_surface: list[str]) -> list[str]
Example 1
Inputfind_ungrounded_calls(["conn.execute_batch(sql)", "conn.query_one(sql)", "conn.fetch_all()"], ["execute_batch", "query_one", "close"])
Output["fetch_all"]
Example 2
Inputfind_ungrounded_calls(["pool.acquire()", "pool.acquire()"], ["acquire"])
Output[]
Example 3
Inputfind_ungrounded_calls(["client.stream_rows(q)", "client.streamRows(q)"], ["stream_rows"])
Output["streamRows"]