First divergence of a shadow run with reordered tool calls

Agent buildHard40 min

Your task

Before a new version of an agent is promoted, it runs in shadow mode on the same tickets as the current version, and the two tool-call traces are compared. Exact equality is too strict, because the new version is free to issue independent calls in a different order. Two calls are independent when they touch no common resource, and swapping two adjacent independent calls never changes the outcome. A trace is equivalent to another if it can be turned into it by a series of such swaps.

Write first_divergence(current, candidate). Each trace is a list of calls [name, resources]. name identifies the call (its tool and arguments): the same name always touches the same resources, and two calls with the same name are interchangeable. resources lists the distinct resources the call reads or writes, possibly none. Names and resources can be any strings, even "constructor" or "__proto__", and can contain any character. Each trace has at most 2,000 calls, and each call touches at most 4 resources.

Return:

  • -1 if candidate is equivalent to current;
  • otherwise the smallest i such that candidate[:i + 1] is not how any trace equivalent to current begins;
  • or, if every prefix of candidate is how some equivalent trace begins but candidate stops before making all the calls, len(candidate).

Examples, all with current = [["read a", ["a"]], ["read b", ["b"]], ["write a", ["a"]]]:

  • candidate = [["read b", ["b"]], ["read a", ["a"]], ["write a", ["a"]]] returns -1: read b and read a touch different resources.
  • candidate = [["write a", ["a"]], ["read a", ["a"]], ["read b", ["b"]]] returns 0: write a can never move ahead of read a, since both touch a.
  • candidate = [["read b", ["b"]], ["read a", ["a"]]] returns 2.

Examples

first_divergence(current: list[list], candidate: list[list]) -> int
Example 1
Inputfirst_divergence([["read a", ["a"]], ["read b", ["b"]], ["write a", ["a"]]], [["read b", ["b"]], ["read a", ["a"]], ["write a", ["a"]]])
Output-1
Example 2
Inputfirst_divergence([["read a", ["a"]], ["read b", ["b"]], ["write a", ["a"]]], [["write a", ["a"]], ["read a", ["a"]], ["read b", ["b"]]])
Output0
Example 3
Inputfirst_divergence([["read a", ["a"]], ["read b", ["b"]], ["write a", ["a"]]], [["read b", ["b"]], ["read a", ["a"]]])
Output2