Pick the tests to run for a diff
Your task
Your full regression suite takes 4 hours, so an agent picks the tests each diff needs. A missed regression in production is the nightmare, so whenever the agent is unsure, it runs everything.
Write select_tests_for_diff(test_records, changed_files): the rule your agent follows for each diff.
test_records(list of str): the coverage map, one test each, writtenname|tag|files, wherefilesis a list of the files the test covers separated by;. For example"test_auth|unit|src/auth.py;src/util.py".changed_files(list of str): the files the diff touches.- Return the names of the tests to run, each once, sorted in ascending order.
Rules:
- Split each record on
|into the name, the tag and the files. A missing tag or file list counts as empty. Split the file list on;and drop empty entries. - Tests tagged exactly
smokeorcriticalalways run. - If no files changed: run only the always-run tests.
- If any changed file is not covered by any test: run every test.
- Otherwise: run the always-run tests plus every test that covers at least one changed file.
Examples
select_tests_for_diff(test_records: list[str], changed_files: list[str]) -> list[str]Example 1
Input
select_tests_for_diff(["test_auth|unit|src/auth.py;src/util.py", "test_billing|unit|src/billing.py", "test_smoke_login|smoke|src/app.py"], ["src/billing.py"])Output
["test_billing", "test_smoke_login"]Example 2
Input
select_tests_for_diff(["test_auth|unit|src/auth.py;src/util.py", "test_billing|unit|src/billing.py", "test_smoke_login|smoke|src/app.py"], ["src/new_feature.py"])Output
["test_auth", "test_billing", "test_smoke_login"]Example 3
Input
select_tests_for_diff(["test_auth|unit|src/auth.py;src/util.py", "test_billing|unit|src/billing.py", "test_smoke_login|smoke|src/app.py"], [])Output
["test_smoke_login"]