Decide what a pipeline triage agent may do
Your task
A nightly data pipeline fails a few times a month, and someone has to decide whether to rerun it, backfill it, and who to warn. An agent makes the first call on each failure, and it may never start a backfill on its own.
Write triage_pipeline_failure(failure_class, is_idempotent, prior_retries, downstream_consumers): the rule your agent follows for each failure.
failure_class(str): the diagnosed cause.is_idempotent(bool): the failed task is safe to run twice, because it overwrites rather than appends.prior_retries(int): how many times the task was already retried.downstream_consumers(int): how many downstream consumers read the stale tables.
It returns one of the values in the rules below. Apply them in order. The first one that matches decides:
- Trim and lowercase
failure_class. If it is notinfra,upstream_late,schema_driftorcode:"escalate_to_human". - The class is
infraorupstream_late, the task is idempotent, andprior_retriesis 0 or less:"retry_once". downstream_consumersis above 0:"notify_consumers_and_await_approval".- Otherwise:
"await_backfill_approval".
Examples
triage_pipeline_failure(failure_class: str, is_idempotent: bool, prior_retries: int, downstream_consumers: int) -> strExample 1
Input
triage_pipeline_failure("infra", True, 0, 3)Output
"retry_once"Example 2
Input
triage_pipeline_failure("schema_drift", True, 0, 2)Output
"notify_consumers_and_await_approval"Example 3
Input
triage_pipeline_failure("code", False, 1, 0)Output
"await_backfill_approval"