Tenant spend and rate caps during an agent rollout

Agent buildMedium30 min

Your task

A billing-assistant agent is being rolled out to a few enrolled tenants. Every task it runs costs model and tool spend, so during the rollout each tenant gets three caps: a budget in cents for the whole rollout, at most per_minute tasks started in any 60-second window, and at most in_flight tasks running at once. A gateway decides, for every task the agent wants to start, whether it may.

Write gate_requests(caps, events). caps maps each enrolled tenant to {"budget": int, "per_minute": int, "in_flight": int} (the last two at least 1). events is a list in time order (whole seconds, never decreasing), of two kinds:

  • ["start", t, tenant, task_id, estimate]: the agent asks to start a task for tenant, with an estimated cost of estimate cents (at least 1). Task ids are unique across all start events.
  • ["finish", t, task_id, actual]: a task finished, and its actual cost was actual cents (zero or more).

For each "start", return the first of these that applies:

  1. "not_enrolled": the tenant is not in caps.
  2. "busy": the tenant already has in_flight admitted tasks that have not finished.
  3. "rate_limited": the tenant had per_minute or more tasks admitted in the last 60 seconds, that is, admitted at a second s with t - 60 < s <= t.
  4. "over_budget": spent + reserved + estimate > budget, where spent is the sum of the actual costs of the tenant's finished tasks and reserved is the sum of the estimates of its admitted tasks still running.
  5. "admit": the task starts. Its estimate is reserved, it counts as running, and it counts toward the rate window from second t.

A refused request leaves no trace: it counts toward no cap and reserves nothing.

For a "finish" of an admitted task that is still running: release its reservation, add actual to the tenant's spend, and stop counting it as running. It still counts toward the rate window from the second it was admitted. A finish for anything else (a refused task, an unknown id, a task already finished) is ignored. actual may exceed the estimate, so spend can go over the budget; every later start for that tenant is then over budget.

Events in the same second take effect in list order.

Example, with caps = {"acme": {"budget": 100, "per_minute": 2, "in_flight": 2}} and these events:

["start", 0, "acme", "t1", 30]     admit         (reserved 30)
["start", 1, "acme", "t2", 30]     admit         (reserved 60)
["start", 2, "acme", "t3", 30]     busy          (two running)
["finish", 10, "t1", 50]                         (spent 50, reserved 30)
["start", 20, "acme", "t4", 10]    rate_limited  (admitted at 0 and 1)
["start", 61, "acme", "t5", 25]    over_budget   (50 + 30 + 25 > 100)
["start", 61, "acme", "t6", 20]    admit         (50 + 30 + 20 = 100)
["start", 62, "zeta", "t7", 1]     not_enrolled

gate_requests(caps, events) returns ["admit", "admit", "busy", "rate_limited", "over_budget", "admit", "not_enrolled"]. At second 61 the admissions at seconds 0 and 1 have left the window, because 61 - 60 < s excludes s = 1.

Examples

gate_requests(caps: dict, events: list[list]) -> list[str]
Example 1
Inputgate_requests({"acme": {"budget": 100, "per_minute": 2, "in_flight": 2}}, [["start", 0, "acme", "t1", 30], ["start", 1, "acme", "t2", 30], ["start", 2, "acme", "t3", 30], ["finish", 10, "t1", 50], ["start", 20, "acme", "t4", 10], ["start", 61, "acme", "t5", 25], ["start", 61, "acme", "t6", 20], ["start", 62, "zeta", "t7", 1]])
Output["admit", "admit", "busy", "rate_limited", "over_budget", "admit", "not_enrolled"]
Example 2
Inputgate_requests({"a": {"budget": 10, "per_minute": 5, "in_flight": 5}}, [["start", 0, "a", "x", 8], ["finish", 1, "x", 2], ["start", 2, "a", "y", 8]])
Output["admit", "admit"]