Next

Decide which log anomalies page the on-call

Agent buildEasy15 minGoogle style

Your task

Your monitoring flags about 5,000 log anomalies a day, and the on-call has learned to ignore most pages. An agent sits between the detector and the pager and routes each anomaly.

Write route_anomalies(anomaly_lines, always_page_signatures, page_score_threshold, ticket_score_threshold): the rule your agent follows for one batch of anomalies.

  • anomaly_lines (list of str): one anomaly per line, written signature|slo|security|score, for example "checkout-latency|0|0|91". The signature names the alert. slo is 1 if the anomaly breaches a service level objective (an agreed reliability target). security is 1 for a security signal. score is the detector's score.
  • always_page_signatures (list of str): signatures that must always page.
  • page_score_threshold, ticket_score_threshold (int): the score bars for paging and for filing a ticket.
  • Return one decision per line, in input order.

Go through the lines in order and remember which signatures you have routed in this batch. For each line:

  1. If it does not split on | into exactly 4 fields, or its score is not one or more digits (0 to 9 only): "ticket". A malformed line is not remembered. Fields are used exactly as written, with no trimming.
  2. If slo is 1, security is 1, or the signature is exactly one of always_page_signatures: "page-mandatory", even if the signature was seen before. Remember the signature.
  3. If the signature was already routed in this batch: "suppress-duplicate".
  4. Remember the signature. Then return "page-score" if the score is at least page_score_threshold, "ticket" if it is at least ticket_score_threshold, and "suppress-low" otherwise.

Signatures are compared exactly, and an empty signature is still a signature.

Examples

route_anomalies(anomaly_lines: list[str], always_page_signatures: list[str], page_score_threshold: int, ticket_score_threshold: int) -> list[str]
Example 1
Inputroute_anomalies(["checkout-latency|0|0|91", "log-noise|0|0|12", "db-timeouts|1|0|40"], ["auth-failures"], 80, 30)
Output["page-score", "suppress-low", "page-mandatory"]
Example 2
Inputroute_anomalies(["cache-miss-spike|0|0|55", "cache-miss-spike|0|0|55", "auth-failures|0|0|5", "auth-failures|0|0|5"], ["auth-failures"], 80, 30)
Output["ticket", "suppress-duplicate", "page-mandatory", "page-mandatory"]
Example 3
Inputroute_anomalies([], ["auth-failures"], 80, 30)
Output[]