Centralized egress proxy

DesignMedium40 min

Your task

Design a centralized egress (forward) proxy that all outbound HTTP from a 5,000-service platform must route through, handling 60k outbound requests/sec to arbitrary third-party and customer-specified URLs. It must enforce an allow/deny policy (block internal IP ranges and metadata endpoints to prevent SSRF), apply per-destination rate limits and circuit breakers, reuse upstream connections for efficiency, and add under 5ms p99. Walk through the proxy architecture, the policy-enforcement model, and the central trade-off.

It covers

  • Forward proxy
  • Ssrf protection
  • Connection reuse