Fix the secret classifier for outbound model calls

Review the AI's codeMedium15 minAnthropic and OpenAI style

Your task

Your app sends text to a third-party AI model, and an assistant once wrote the redactor that guards it. It matched two key formats and called everything else "safe to send", so a secret in any third format went straight out. The replacement sorts every outbound token into three groups: redact it, send it to a person for review, or pass it.

An assistant wrote classify_outbound_tokens(tokens). It is in the editor. It has a bug. Find it and fix it.

Input and output:

  • tokens: the strings about to be sent.
  • Return one label per token, in order: redact, review or pass.

Rules the tests check. Apply them in order and use the first that fits:

  1. redact: the token starts with sk-, and the rest is 20 or more characters, all ASCII letters or digits.
  2. redact: the token starts with AKIA, and the rest is exactly 16 characters, each an uppercase ASCII letter or a digit.
  3. review: the first = in the token is neither its first nor its last character, and the text before that = ends with password, secret, token or key, ignoring the case of ASCII letters.
  4. review: split on ., the token has exactly 3 parts, and each part is at least 8 characters, using only ASCII letters, digits, - and _. This is the shape of a JWT, a signed web token.
  5. review: the token is at least 20 characters long, and it contains a digit or contains both an uppercase and a lowercase ASCII letter.
  6. Otherwise: pass.

A token that starts with sk- or AKIA but fails rule 1 or 2 goes on to the next rules.

Examples

classify_outbound_tokens(tokens: list[str]) -> list[str]
Example 1
Inputclassify_outbound_tokens(["sk-ABCDEFGHIJKLMNOPQRSTU", "hello", "AKIAIOSFODNN7EXAMPLE"])
Output["redact", "pass", "redact"]
Example 2
Inputclassify_outbound_tokens(["wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY", "sk-short"])
Output["review", "pass"]