Find the chance an agent's run ends resolved
Your task
Your eval models a support agent as a state machine fitted from logged runs. For each state, the logs say how many times the agent moved from it to each next state. The logged runs started in many different states, so the share of them that ended well is not the chance for a run that starts at the beginning.
Write resolve_chance(moves) that returns the chance a run starting in the state "start" ends in the state "resolved".
Input and output:
moves: up to 2,000 entries[from_state, to_state, count]: the agent moved fromfrom_statetoto_statethat many times. A state may move to itself.- States are non-empty strings, compared exactly, with at most 50 different states. Counts are whole numbers from 1 to 1,000. The same pair of states may appear more than once: add up its counts.
- Return a float. The tests accept any value within 0.000001 of the exact one.
Rules:
- From a state, the agent moves to each next state with a chance equal to that pair's count divided by the total count of all moves out of the state.
"resolved"and"failed"end a run, and moves out of them are ignored. A state with no moves out also ends the run, and that run is not resolved.- A run moves until it ends. Return the chance that it ever reaches
"resolved". A run that never ends is not resolved.
Examples
resolve_chance(moves: list[list]) -> floatExample 1
Input
resolve_chance([["start", "search", 10], ["search", "resolved", 6], ["search", "failed", 4]])Output
0.6Example 2
Input
resolve_chance([["start", "tool", 4], ["tool", "tool", 3], ["tool", "resolved", 1]])Output
1Example 3
Input
resolve_chance([["start", "plan", 1], ["plan", "act", 1], ["act", "plan", 1], ["start", "resolved", 1]])Output
0.5