Decide what a pipeline triage agent may do

Agent buildEasy10 minDatabricks and Snowflake style

Your task

A nightly data pipeline fails a few times a month, and someone has to decide whether to rerun it, backfill it, and who to warn. An agent makes the first call on each failure, and it may never start a backfill on its own.

Write triage_pipeline_failure(failure_class, is_idempotent, prior_retries, downstream_consumers): the rule your agent follows for each failure.

  • failure_class (str): the diagnosed cause.
  • is_idempotent (bool): the failed task is safe to run twice, because it overwrites rather than appends.
  • prior_retries (int): how many times the task was already retried.
  • downstream_consumers (int): how many downstream consumers read the stale tables.

It returns one of the values in the rules below. Apply them in order. The first one that matches decides:

  1. Trim and lowercase failure_class. If it is not infra, upstream_late, schema_drift or code: "escalate_to_human".
  2. The class is infra or upstream_late, the task is idempotent, and prior_retries is 0 or less: "retry_once".
  3. downstream_consumers is above 0: "notify_consumers_and_await_approval".
  4. Otherwise: "await_backfill_approval".

Examples

triage_pipeline_failure(failure_class: str, is_idempotent: bool, prior_retries: int, downstream_consumers: int) -> str
Example 1
Inputtriage_pipeline_failure("infra", True, 0, 3)
Output"retry_once"
Example 2
Inputtriage_pipeline_failure("schema_drift", True, 0, 2)
Output"notify_consumers_and_await_approval"
Example 3
Inputtriage_pipeline_failure("code", False, 1, 0)
Output"await_backfill_approval"