MODULE 01 · EDGE & TRAFFIC CONTROL · VENDOR-AGNOSTIC

Distributed Rate Limiter & API Gateway

Protect downstream services from traffic spikes, noisy neighbors, and DDoS bursts with sub-2ms atomic quota enforcement, boundary-safe sliding windows, and resilient fail-open degradation.

100K Peak QPS < 2ms P99 Overhead 3 Interactive Flows 4 Chaos Drills
Act I · Scope & Back-of-the-Envelope Scale Math

Traffic Protection Guarantees & Capacity Envelope

Step 1 of 5
Functional & Non-Functional Requirements
Edge SLAs
  • Multi-Dimensional Quotas: Enforce configurable limits per-tenant, per-IP, and per-route (e.g. 100 req/min on public reads, 10 req/min on login/payments) with variable request cost.
  • Standardized 429 Signals: Return HTTP 429 Too Many Requests with Retry-After and X-RateLimit-* headers so well-behaved clients back off cleanly.
  • Distributed Consistency Across Stateless Gateways: 50+ edge gateways share a single atomic quota truth with zero read-modify-write drift under concurrency.
  • Sub-2ms Overhead & 99.99% Availability: Circuit-break degraded state stores past a 50ms budget, degrading gracefully via policy-driven Fail-Open (Local L1) or Fail-Closed (503).
Back-of-the-Envelope Scale Math
100K QPS Target
Peak Throughput
100,000 QPS

100K QPS / 50 stateless gateway nodes = 2,000 QPS/node. Horizontally scalable.

State Footprint
~120 MB RAM

1M active keys × ~120 B (hash + TTL + metadata) = 120 MB hot RAM in 1 sharded cluster.

Latency Budget
< 2.0 ms P99

0.4ms L7 parse + 1.1ms single-RTT atomic script + 0.5ms serialize = 2.0ms.

Cluster Bandwidth
~25 MB/s

100K × 250 B RPC = 25 MB/s internal east-west traffic over multiplexed pools.

Act II · Wire Protocol, HTTP Headers & Atomic Storage Contract

429 Edge Headers, Internal gRPC & Atomic Key Schema

Step 2 of 5
HTTP 429 Headers & Internal gRPC Contract
Atomic In-Memory State Layout & Script Contract
Single-RTT Atomicity: Executing refill + decrement inside a single server-side atomic script (or CAS loop) eliminates the classic GET -> compute -> SET race condition across 50 concurrent gateways.
Act III · Interactive 5-Node Cluster Topology & Chaos Switchboard

Token Bucket, Sliding Window & Fail-Open Circuit Breaker

CLUSTER HEALTHY
Step 1 / 6
5-Node Cluster Topology · Token Bucket (Burst & Refill)
BURST ARRIVAL
devices Client Fleet Tenants & Edge IPs COST=1 · BURST router API Gateway Fleet 50 Stateless Nodes POLICY · 1.4ms electric_bolt Local L1 & Breaker Quota / N_nodes CLOSED · STANDBY database State Cluster Sharded · Atomic Script SYNCED · 1.1ms dns Upstream Origin Protected Core API PROTECTED · 200 OK
BURST ARRIVAL

Loading simulator...

Invariant active
Live Event Console
Live Telemetry HUD
Real-Time
Gateway P99
1.4 ms
Budget < 2.0 ms
Bucket / Window
74 / 100
Refill: 10 tok/s
Traffic Verdict
200 OK
Forwarded to Origin
Cluster & Breaker
CLOSED · SYNCED
Policy: Fail-Open L1
Chaos Switchboard
0 Active
Act IV · Zero-Dependency Go Reference Blueprints

Idiomatic Go Contracts & Production Stack Mapping

Auto-Synced to Simulator
Production Stack Mapping Strip (Vendor-Agnostic → Real Systems) 6 Core Primitives
Vendor-Agnostic Primitive Production Stack Equivalents Operational Role
L7 API Gateway Envoy / NGINX / Kong / AWS API GW Terminates TLS, parses tenant/route keys, enforces 429 headers
Distributed Atomic State Redis Cluster / Valkey / Dragonfly Sub-millisecond sharded in-memory quota counters with TTL
Atomic Execution Engine Lua Scripts / Redis Functions / CAS Single-RTT read-refill-decrement eliminating race conditions
Local L1 Fallback In-Process Token Bucket / Go x/time/rate Protects origin during state-cluster partitions (quota / N)
Config & Policy Control Plane etcd / Consul / Envoy xDS Hot-reloads tenant tiers and fail-open/closed rules in <1s
Telemetry & Alerting Prometheus / OpenTelemetry / Grafana Tracks P99 overhead, 429 ratios, hot-key skew, and breaker state
Act V · CAP / PACELC Tradeoffs & Production SLOs

Distributed Design Tradeoffs & Reliability Targets

Step 5 of 5
Tradeoff 01 Algorithm Choice

Token Bucket vs. Leaky Bucket vs. Sliding Window

Token Bucket / Sliding Window

Token Bucket allows healthy short bursts up to capacity; Sliding Window Counter prevents 2x edge spikes at minute boundaries with only 2 keys.

Leaky Bucket / Sliding Log

Leaky Bucket smooths egress strictly (good for downstream queues) but penalizes bursty UX; Sliding Log is exact but consumes O(requests) RAM.

Verdict Token Bucket for burst-friendly public APIs; Sliding Window Counter for strict billing/quota boundaries.
Tradeoff 02 PACELC: PC/EC vs PA/EL

Centralized Atomic Store vs. Local Synced Counters

Centralized Atomic Store

Strong global quota accuracy across all 50 stateless gateways at the cost of ~1.1ms internal east-west RTT per check.

Local Async Gossip

Zero-RTT checks (<0.1ms) and immune to store bottlenecks, but permits temporary over-admission during sync intervals.

Verdict Centralized Atomic Store by default; use local quota leasing (reserve 50 tokens/batch) for ultra-hot 50K+ QPS keys.
Tradeoff 03 Failure Mode

Fail-Open vs. Fail-Closed Policy

Fail-Open + Local L1

Preserves availability for public catalog/read APIs during state-cluster partitions while local L1 caps prevent origin meltdown.

Strict Fail-Closed (503)

Immediately rejects requests when quota truth is unreachable—mandatory for auth/OTP, payment capture, and expensive LLM inference.

Verdict Configure policy per route in the Control Plane—never apply one global failure mode across read and financial endpoints.
Tradeoff 04 Concurrency

Race Conditions: GET/SET vs. Single-RTT Atomic Script

Single-RTT Atomic Script

Executes read, time-refill, check, decrement, and TTL update inside one serialized server command—zero race window and half the RTT.

Naive GET then SET

Under 50 concurrent gateways, 10 threads read remaining = 1 simultaneously and all admit traffic—leaking 10x quota.

Verdict Always co-locate quota math with state via Lua scripts / CAS; never split check and decrement across two network hops.
Production Observability SLOs
Edge Telemetry Targets
Gateway Overhead P99 < 2.0 ms Single-RTT atomic script
False-429 Rate < 0.01% Accurate refill & clock sync
Failover Recovery < 500 ms Breaker trip → L1 active
Limiter Availability 99.99% Multi-AZ sharded cluster