MODULE 01 · EDGE & TRAFFIC CONTROL · VENDOR-AGNOSTIC
Distributed Rate Limiter & API Gateway
Protect downstream services from traffic spikes, noisy neighbors, and DDoS bursts with sub-2ms atomic quota enforcement, boundary-safe sliding windows, and resilient fail-open degradation.
speed
100K Peak QPS
timer
< 2ms P99 Overhead
hub
3 Interactive Flows
electric_bolt
4 Chaos Drills
architecture
Act I · Scope & Back-of-the-Envelope Scale Math
Traffic Protection Guarantees & Capacity Envelope
Step 1 of 5
tune
Multi-Dimensional Quotas: Enforce configurable limits per-tenant, per-IP, and per-route (e.g. 100 req/min on public reads, 10 req/min on login/payments) with variable request cost.
http
Standardized 429 Signals: Return HTTP 429 Too Many Requests with Retry-After and X-RateLimit-* headers so well-behaved clients back off cleanly.
dns
Distributed Consistency Across Stateless Gateways: 50+ edge gateways share a single atomic quota truth with zero read-modify-write drift under concurrency.
shield
Sub-2ms Overhead & 99.99% Availability: Circuit-break degraded state stores past a 50ms budget, degrading gracefully via policy-driven Fail-Open (Local L1) or Fail-Closed (503).
100,000 QPS
100K QPS / 50 stateless gateway nodes = 2,000 QPS/node . Horizontally scalable.
~120 MB RAM
1M active keys × ~120 B (hash + TTL + metadata) = 120 MB hot RAM in 1 sharded cluster.
< 2.0 ms P99
0.4ms L7 parse + 1.1ms single-RTT atomic script + 0.5ms serialize = 2.0ms .
~25 MB/s
100K × 250 B RPC = 25 MB/s internal east-west traffic over multiplexed pools.
data_object
Act II · Wire Protocol, HTTP Headers & Atomic Storage Contract
429 Edge Headers, Internal gRPC & Atomic Key Schema
Step 2 of 5
bolt
Single-RTT Atomicity: Executing refill + decrement inside a single server-side atomic script (or CAS loop) eliminates the classic GET -> compute -> SET race condition across 50 concurrent gateways.
hub
Act III · Interactive 5-Node Cluster Topology & Chaos Switchboard
Token Bucket, Sliding Window & Fail-Open Circuit Breaker
CLUSTER HEALTHY
devices
Client Fleet
Tenants & Edge IPs
COST=1 · BURST
router
API Gateway Fleet
50 Stateless Nodes
POLICY · 1.4ms
electric_bolt
Local L1 & Breaker
Quota / N_nodes
CLOSED · STANDBY
database
State Cluster
Sharded · Atomic Script
SYNCED · 1.1ms
dns
Upstream Origin
Protected Core API
PROTECTED · 200 OK
water_drop
BURST ARRIVAL
Loading simulator...
verified
Invariant active
Gateway P99
timer
1.4 ms
Budget < 2.0 ms
Bucket / Window
water_drop
74 / 100
Refill: 10 tok/s
Traffic Verdict
gavel
200 OK
Forwarded to Origin
Cluster & Breaker
electric_bolt
CLOSED · SYNCED
Policy: Fail-Open L1
Hot-Key / DDoS Burst (50K QPS)
OFF
State Cluster Latency (+180ms)
OFF
Strict Fail-Closed Mode
OFF
crisis_alert
Trigger State Cluster Partition (Failover Drill)
code
Act IV · Zero-Dependency Go Reference Blueprints
Idiomatic Go Contracts & Production Stack Mapping
Auto-Synced to Simulator
layers
Production Stack Mapping Strip (Vendor-Agnostic → Real Systems)
6 Core Primitives
Vendor-Agnostic Primitive
Production Stack Equivalents
Operational Role
L7 API Gateway
Envoy / NGINX / Kong / AWS API GW
Terminates TLS, parses tenant/route keys, enforces 429 headers
Distributed Atomic State
Redis Cluster / Valkey / Dragonfly
Sub-millisecond sharded in-memory quota counters with TTL
Atomic Execution Engine
Lua Scripts / Redis Functions / CAS
Single-RTT read-refill-decrement eliminating race conditions
Local L1 Fallback
In-Process Token Bucket / Go x/time/rate
Protects origin during state-cluster partitions (quota / N)
Config & Policy Control Plane
etcd / Consul / Envoy xDS
Hot-reloads tenant tiers and fail-open/closed rules in <1s
Telemetry & Alerting
Prometheus / OpenTelemetry / Grafana
Tracks P99 overhead, 429 ratios, hot-key skew, and breaker state
balance
Act V · CAP / PACELC Tradeoffs & Production SLOs
Distributed Design Tradeoffs & Reliability Targets
Step 5 of 5
water_drop
Tradeoff 01
Algorithm Choice
Token Bucket vs. Leaky Bucket vs. Sliding Window
check_circle Token Bucket / Sliding Window
Token Bucket allows healthy short bursts up to capacity ; Sliding Window Counter prevents 2x edge spikes at minute boundaries with only 2 keys.
tune Leaky Bucket / Sliding Log
Leaky Bucket smooths egress strictly (good for downstream queues) but penalizes bursty UX; Sliding Log is exact but consumes O(requests) RAM.
Verdict
Token Bucket for burst-friendly public APIs; Sliding Window Counter for strict billing/quota boundaries.
hub
Tradeoff 02
PACELC: PC/EC vs PA/EL
Centralized Atomic Store vs. Local Synced Counters
check_circle Centralized Atomic Store
Strong global quota accuracy across all 50 stateless gateways at the cost of ~1.1ms internal east-west RTT per check.
tune Local Async Gossip
Zero-RTT checks (<0.1ms) and immune to store bottlenecks, but permits temporary over-admission during sync intervals.
Verdict
Centralized Atomic Store by default; use local quota leasing (reserve 50 tokens/batch) for ultra-hot 50K+ QPS keys.
shield
Tradeoff 03
Failure Mode
Fail-Open vs. Fail-Closed Policy
check_circle Fail-Open + Local L1
Preserves availability for public catalog/read APIs during state-cluster partitions while local L1 caps prevent origin meltdown.
tune Strict Fail-Closed (503)
Immediately rejects requests when quota truth is unreachable—mandatory for auth/OTP, payment capture, and expensive LLM inference.
Verdict
Configure policy per route in the Control Plane—never apply one global failure mode across read and financial endpoints.
bolt
Tradeoff 04
Concurrency
Race Conditions: GET/SET vs. Single-RTT Atomic Script
check_circle Single-RTT Atomic Script
Executes read, time-refill, check, decrement, and TTL update inside one serialized server command—zero race window and half the RTT.
tune Naive GET then SET
Under 50 concurrent gateways, 10 threads read remaining = 1 simultaneously and all admit traffic—leaking 10x quota.
Verdict
Always co-locate quota math with state via Lua scripts / CAS ; never split check and decrement across two network hops.
Gateway Overhead P99
< 2.0 ms
Single-RTT atomic script
False-429 Rate
< 0.01%
Accurate refill & clock sync
Failover Recovery
< 500 ms
Breaker trip → L1 active
Limiter Availability
99.99%
Multi-AZ sharded cluster