Catch API calls the agent made up
Your task
Your team keeps adding third-party API integrations, and an agent now scaffolds the client code for each one. Language models happily invent endpoints that do not exist, so every generated call is checked against the provider's spec before it reaches the codebase.
Write find_ungrounded_calls(spec_endpoints, generated_calls): the check your agent's output passes through.
spec_endpoints(list of str): the endpoints the provider's spec declares, each writtenMETHOD /path, for example"GET /v1/users".generated_calls(list of str): the calls the agent's code makes, in the same form.- Return the normalized form of every generated call the spec does not declare, each once, sorted in ascending order.
Normalize every entry in both lists the same way:
- Trim whitespace from both ends.
- Split at the first space into method and path. With no space, the whole entry is the method and the path is empty.
- Uppercase the method. Trim the path, but do not change its case:
/v1/Usersand/v1/usersare different routes. - If the path is longer than 1 character and ends in
/, remove that one/. A bare/stays. - The normalized form is
METHOD path, or justMETHODwhen the path is empty. So an empty string normalizes to"".
A generated call is ungrounded when its normalized form is not the normalized form of any spec endpoint.
Examples
find_ungrounded_calls(spec_endpoints: list[str], generated_calls: list[str]) -> list[str]Example 1
Input
find_ungrounded_calls(["GET /v1/users", "POST /v1/users", "GET /v1/orders/{id}"], ["GET /v1/users", "GET /v1/invoices"])Output
["GET /v1/invoices"]Example 2
Input
find_ungrounded_calls(["get /v1/users"], ["GET /v1/users/"])Output
[]Example 3
Input
find_ungrounded_calls([], ["GET /health"])Output
["GET /health"]