Trace untrusted reads into an agent's risky tool calls

Agent buildMedium30 min

Your task

Your email agent's planner writes a short program before anything runs: read some data, compute values, call tools. Text from outside can carry injected instructions, so a guard finds every untrusted read whose data could reach a tool that acts on the outside world. A person checks those reads before the plan runs.

Write reads_to_review(plan, trusted, sensitive) that returns the indexes of those reads.

  • plan: a list of steps, run in order. Each step is one of these:
    • ["read", var, source]: store data from source in the variable var.
    • ["set", var, inputs]: store in var a value computed from the variables listed in inputs, a list that may be empty.
    • ["call", var, tool, inputs]: call tool with the variables in inputs and store its result in var.
    • ["if", var]: start a block of steps that runs only when var holds a true value.
    • ["end"]: close the most recent open block. Blocks can nest, and every block is closed.
  • trusted: the sources whose data is safe.
  • sensitive: the tools that act on the outside world.
  • Return the indexes in plan of the reads to check, in increasing order, each once.

Rules:

  1. A read from a source that is not in trusted is untrusted, and its data carries a tag: the read's index. A trusted read carries no tag.
  2. A read stores its own tag, if it has one. A set or a call stores every tag its inputs hold when the step runs. Storing into a variable replaces what it held. A variable that nothing has stored into holds no tags.
  3. A block may or may not run. After its end, each variable holds the tags it held just before the block's if, together with every tag it holds at the end of the block.
  4. Inside a block, every step that stores into a variable also stores the tags the block's if variable held at that if. In nested blocks this applies for every block the step is in.
  5. A read must be checked when one of its tags reaches a call to a sensitive tool, through the call's inputs or through rule 4.

Examples

reads_to_review(plan: list[list], trusted: list[str], sensitive: list[str]) -> list[int]
Example 1
Inputreads_to_review([["read", "page", "web"], ["read", "me", "contacts"], ["set", "note", ["page"]], ["call", "r", "send_email", ["me", "note"]]], ["contacts"], ["send_email"])
Output[0]
Example 2
Inputreads_to_review([["read", "x", "web"], ["set", "x", []], ["call", "r", "post", ["x"]]], [], ["post"])
Output[]
Example 3
Inputreads_to_review([["read", "x", "web"], ["read", "c", "calendar"], ["if", "c"], ["set", "x", []], ["end"], ["call", "r", "post", ["x"]]], ["calendar"], ["post"])
Output[0]