Rank agent versions against the fewest judge verdicts
Your task
An LLM judge compares agent versions in pairs on the same tasks and picks a winner each time. Its verdicts are not always consistent: A can beat B, B beat C, and C beat A. The release review still needs one ranking.
Write rank_versions(versions, verdicts) that returns the versions in ranked order, best first.
versions: the version names, at most 10, all different. Names use only letters, digits,.and-.verdicts: a list of[winner, loser]pairs, one per judge comparison. The same pair can appear many times, in either order.- Return a list that holds every version once.
Rules:
- A ranking goes against a verdict when it puts the loser above the winner.
- Return a ranking that goes against as few verdicts as possible.
- If several rankings tie, return the first one. Compare rankings name by name from the top, in plain string order: by character code, so
Bsorts beforea. - Ignore a verdict that names a version not in
versions, or names the same version twice.
Examples
rank_versions(versions: list[str], verdicts: list[list[str]]) -> list[str]Example 1
Input
rank_versions(["v1", "v2", "v3"], [["v1", "v2"], ["v2", "v3"], ["v1", "v3"]])Output
["v1", "v2", "v3"]Example 2
Input
rank_versions(["a", "b", "c"], [["a", "b"], ["b", "c"], ["c", "a"]])Output
["a", "b", "c"]Example 3
Input
rank_versions(["x", "y"], [["y", "x"], ["x", "y"], ["y", "x"]])Output
["y", "x"]