Choose rate limits with evidence instead of guesswork.
ratelimit-simulator replays a traffic profile — synthetic or recorded from production —
through seven rate limiting algorithms and reports exactly what each one does: how many
requests it accepts, how big a burst it lets through, which tenant it starves, how much
state it costs, and how far a multi-node deployment overshoots the limit you configured.
It is a simulator, not a limiter. Nothing here goes in your request path. The point is to answer "what happens if I set this to 100/minute?" before your users answer it for you.
Python 3.9+, standard library only, no dependencies at all.
Rate limits are usually chosen by picking a round number and waiting to see who complains. That works until one of the standard surprises arrives:
- Your fixed window limit of 100/second admits 200 requests in 200 milliseconds, because a client discovered the boundary. Your backend did not survive it.
- You moved from one gateway node to three and your effective limit silently tripled, because each node caches the shared counter.
- Your "fair" per-tenant limit is fair per request, and one tenant sends 50% of the requests, so everyone else is fine and that tenant is at 28% acceptance and furious.
- You switched from a token bucket to a leaky bucket for smoother traffic and added 400ms of p99 latency you did not budget for.
Every one of those is measurable in advance. This tool measures them.
Not published to any package index. Clone the repository and run it from the clone:
cd ratelimit-simulator
python3 -m ratelimit_simulator --help
That is the whole installation. There is nothing to build and nothing to install.
usage: python3 -m ratelimit_simulator [-h] COMMAND ...
Simulate rate limiting algorithms against synthetic or recorded traffic and report exactly how each one behaves.
positional arguments:
COMMAND
run run one algorithm over a trace and report it in detail
compare run several algorithms over the same trace, side by side
sweep find the tightest limit that meets a target reject rate
profile describe or emit a trace without simulating anything
explain explain an algorithm: how it works, when to use it, how it fails
options:
-h, --help show this help message and exit
exit codes:
0 the command ran and any requested target was met
1 the command ran but the target was not met (sweep found no qualifying value,
or --max-reject-rate was exceeded)
2 usage error: bad arguments, unknown algorithm, unreadable trace file
Run 'explain <algorithm>' for how each algorithm works, when to use it, and how it
fails, with pointers to further reading on request-routing.com.
Run the tests with python3 -m unittest discover -s tests -t ..
All output below is real, produced by the command shown above it.
examples/boundary-burst.csv is a recorded trace of a client that has worked out where the
window boundary is: ten requests in the last 100ms of one second, ten more in the first
100ms of the next, repeated five times. The limit is 10 per second.
$ python3 -m ratelimit_simulator compare \
-a fixed-window,sliding-log,sliding-counter,token-bucket \
--trace examples/boundary-burst.csv --limit 10 --window 1
Trace: profile=file seed=None requests=100 span=8.19s offered=12.21 req/s
Keys (1): tenant-a 100.0%
algorithm accept reject acc/s max/1s max/10s 1st rej Jain Gini cells
--------------- ------ ------ ----- ------ ------- ------- ------ ------ -----
fixed-window 100.0% 0 12.21 20 100 - 1.0000 0.0000 2
sliding-log 50.0% 50 6.11 10 50 0.10s 1.0000 0.0000 10
sliding-counter 50.0% 50 6.11 10 50 0.10s 1.0000 0.0000 3
token-bucket 55.0% 45 6.72 11 55 0.11s 1.0000 0.0000 2
Accepted-request timelines (60 bins of 0.14s)
fixed-window █▃ ▃█▁ ▆▆ ▁█▃ ▃█
sliding-log █ ▄▄ █ ▁▇ ▅▃
sliding-counter █ ▄▄ █ ▁▇ ▅▃
token-bucket █ ▄▄ ▇▁ ▁▇ ▄▄
Interpretation
fixed-window boundary artefact: a single key got 10 accepted in one window, and up to 20 in a rolling 1s span -- the classic double burst.
sliding-log exact reference: no boundary artefact and no approximation, at the cost of 10 state cells (it stores a timestamp per accepted request).
sliding-counter approximation: identical accept count to the exact log on this trace, using 3 state cells instead of 10.
token-bucket burst-tolerant: allows up to the bucket size at once (peak 11 accepted in 1s) then settles to the sustained rate.
The max/1s column is the whole story. Fixed window accepted every request and passed
20 in a rolling second against a limit of 10. The exact log and the counter approximation
both held the line at 10. This is not a pathological trace; any client with a retry loop
and a clock finds this behaviour by accident.
$ python3 -m ratelimit_simulator compare --profile bursty \
--duration 60 --rate 50 --keys 4 --limit 20 --seed 7
Trace: profile=bursty seed=7 requests=2964 span=52.49s offered=56.47 req/s
Keys (4): tenant-1 25.6%, tenant-2 24.4%, tenant-3 25.7%, tenant-4 24.3%
algorithm accept reject acc/s max/1s max/10s 1st rej Jain Gini cells
--------------- ------ ------ ----- ------ ------- ------- ------ ------ -----
fixed-window 48.5% 1525 27.42 103 254 0.25s 0.9993 0.0138 8
sliding-log 45.4% 1619 25.62 80 241 0.25s 0.9999 0.0058 80
sliding-counter 39.4% 1796 22.25 84 203 0.25s 0.9994 0.0132 12
token-bucket 55.6% 1315 31.42 155 284 0.34s 0.9994 0.0127 8
gcra 29.5% 2091 16.63 62 151 0.01s 0.9999 0.0051 4
Accepted-request timelines (60 bins of 0.87s)
fixed-window ███ █▆▇▃ ▃▅██ ██▇▁ ▅▄▆█ ███
sliding-log ██▇ █▆▅▄ ▃▅█▇ █▇▅▂ ▅▄▇▇ █▇▇
sliding-counter █▆▆ █▃▇▂ ▃▅▇▅ █▄▇▁ ▅▄▇▄ █▅▇
token-bucket █▄▃ ▆▄▄▁ ▂▇▄▃ ▆▅▄ ▃▆▄▂ ▆▅▄
gcra █▇▆ ▄▇▇▂ ▁▇▇▅ ▅▇█▁ ▂█▇▄ ▆▇▇
Interpretation
fixed-window boundary artefact: a single key got 20 accepted in one window, and up to 103 in a rolling 1s span -- the classic double burst.
sliding-log exact reference: no boundary artefact and no approximation, at the cost of 80 state cells (it stores a timestamp per accepted request).
sliding-counter approximation: -177 accepted vs the exact log (13.16% fewer), using 12 state cells instead of 80.
token-bucket burst-tolerant: allows up to the bucket size at once (peak 155 accepted in 1s) then settles to the sustained rate.
gcra shaped: paces arrivals to one every 0.050s, rejecting anything early rather than letting a burst through.
Read the timelines. The token bucket and GCRA rows show traffic spread across the whole burst window; the counter algorithms show it clumped at the front and then flatlined until the window rolls. Same trace, same nominal limit of 20/second, a 26 percentage point spread in acceptance and a 2.5x spread in the worst-case second the backend has to absorb.
Note also that sliding-counter accepted 13% fewer requests than the exact log here. The
approximation is not uniformly permissive — for front-loaded bursts it over-counts the
previous window and rejects traffic it should have allowed.
Five gateway nodes, a shared counter, a limit of 100/second, varying the sync interval:
$ python3 -m ratelimit_simulator run -a distributed --profile poisson \
--duration 60 --rate 200 --keys 1 --limit 100 --nodes 5 \
--sync-interval 0.2 --seed 12 --no-keys
--sync-interval |
accepted | peak accepted in one window | accept rate |
|---|---|---|---|
0 (synchronous) |
6000 | 100 | 50.1% |
0.05 |
6499 | 114 | 54.2% |
0.2 |
7924 | 143 | 66.1% |
1.0 |
11973 | 243 | 99.9% |
A one-second local cache on a one-second window means the limit does essentially nothing: 243 requests admitted against a configured limit of 100. The synchronous baseline is exact, and costs a Redis round trip on every single request. Everything in between is the trade you are actually making, whether or not you have measured it. See dynamic rate limiting with Redis backends for how gateways implement the shared-counter side of this.
Real tenant traffic is not uniform. The zipf profile gives one tenant half the volume:
$ python3 -m ratelimit_simulator run -a token-bucket --profile zipf \
--duration 30 --rate 60 --keys 5 --limit 8 --seed 4 --zipf-exponent 1.2
Trace: profile=zipf seed=4 requests=1733 span=29.98s offered=57.80 req/s
Keys (5): tenant-1 50.5%, tenant-2 21.4%, tenant-3 13.3%, tenant-4 9.5%, tenant-5 5.3%
Algorithm: token-bucket(burst=8, rate=8)
requests 1733 offered, 974 accepted (56.2%), 759 rejected (43.8%)
cost 1733.0 offered, 974.0 accepted
throughput 32.49 accepted req/s over 29.98s
first reject 0.147s
burst max 46 accepted in any 1s, 346 in any 10s
peak/window 15.0 accepted cost in the busiest 1s window for a single key
state 10 cells at peak
retry-after p50 0.048s p90 0.093s p99 0.115s
fairness Jain 0.8849 Gini 0.1813 worst key tenant-1 at 28.2%
The timeline and per-window histogram follow; further down, the part that matters here:
per key
key offered accepted rejected accept
-------- ------- -------- -------- ------
tenant-1 875 247 628 28.2%
tenant-2 371 244 127 65.8%
tenant-3 231 227 4 98.3%
tenant-4 164 164 0 100.0%
tenant-5 92 92 0 100.0%
A per-key limit is doing precisely what it was asked to do, and the aggregate 56% acceptance rate hides the fact that your largest customer is being rejected three times out of four while three others notice nothing. Jain's index of 0.885 is the number to put in the ticket. Fairness here is computed over per-key accept ratios, not raw counts, so a small tenant getting everything it asked for is not counted as disadvantaged.
sweep answers the actual planning question: what is the tightest limit that keeps rejects
under my error budget?
$ python3 -m ratelimit_simulator sweep -a sliding-counter --profile poisson \
--duration 60 --rate 40 --keys 4 --min 8 --max 24 --step 4 \
--target-reject 0.01 --seed 5
Trace: profile=poisson seed=5 requests=2431 span=59.97s offered=40.54 req/s
Keys (4): tenant-1 24.7%, tenant-2 24.5%, tenant-3 26.1%, tenant-4 24.7%
Sweeping limit over sliding-counter
limit accepted rejected reject rate acc/s Jain meets target
----- -------- -------- ----------- ----- ------ ------------
8 1559 872 35.9% 26.00 0.9996 no
12 2137 294 12.1% 35.63 0.9999 no
16 2375 56 2.3% 39.60 1.0000 no
20 2425 6 0.2% 40.44 1.0000 yes
24 2431 0 0.0% 40.54 1.0000 yes
Recommended limit = 20: rejects 0.2% of requests (target 1.0%), admitting 40.44 req/s.
If nothing in the range qualifies, the command says so and exits 1, which makes it usable as a CI gate against a recorded trace. Pairing that with a documented error budget is the same exercise as defining SLOs for gateway latency and error budgets.
$ python3 -m ratelimit_simulator profile --profile herd \
--duration 40 --rate 25 --keys 3 --seed 11
Trace: profile=herd seed=11 requests=410 span=39.84s offered=10.29 req/s
Keys (3): tenant-1 31.7%, tenant-2 33.7%, tenant-3 34.6%
Quiet baseline, then a synchronised retry storm with decaying echoes. The thundering herd.
arrivals ▁▁▁ ▁▁ ▁ ▁ ▁▁▁▁ ▁ █▇ ▁▁▇▁ ▁▁▁▁ ▁▁ ▁▄ ▁▁ ▁▁ ▁ ▁ ▁ ▁ peak 60/bin (0.66s bins)
cost ▁▁▁ ▁▁ ▁ ▁ ▁▁▁▁ ▁ █▇ ▁▁▇▁ ▁▁▁▁ ▁▁ ▁▄ ▁▁ ▁▁ ▁ ▁ ▁ ▁ peak 60.0/bin
per key
tenant-1 130 ███████████████████████████
tenant-2 138 █████████████████████████████
tenant-3 142 ██████████████████████████████
Add --emit csv or --emit jsonl to write the trace itself to stdout, so the exact same
arrivals can be replayed later or fed into another tool.
explain <algorithm> prints a longer version of each of these in the terminal.
One counter per key per wall-clock bucket. Accept while the counter is below the limit; reset on the boundary.
- Use it for coarse quotas where the instantaneous rate does not matter — daily billing caps, abuse ceilings — or as an outer backstop above a finer limiter.
- Failure mode: the double burst shown above. A client spending its allowance either side of a boundary passes 2x the limit within one window-length span. It also releases every blocked client simultaneously at the reset, which is a rate limiter that manufactures thundering herds.
- State: 2 cells per key.
Store a timestamp per accepted request, drop anything older than the window, count what remains. Exact at every instant.
- Use it for small limits on expensive operations, and as the reference to measure your
cheap limiter against — which is exactly what
comparedoes. - Failure mode: memory and CPU proportional to
limit x active keys. A 10,000/minute limit across 100,000 keys is a billion timestamps, and the trim work is worst under a flood, when you can least afford it. - State: up to
limitcells per key.
Keep the current and previous fixed-window counters, and estimate the trailing window as
previous * (1 - elapsed_fraction) + current.
- Use it for almost everything. It removes nearly all of the boundary artefact at fixed-window cost, and it is what most gateways ship by default. Compare the specifics in NGINX vs Kong for rate limiting.
- Failure mode: it assumes the previous window's traffic was spread evenly. Back-loaded
traffic gets under-counted (extra requests slip through), front-loaded traffic gets
over-counted (legitimate requests rejected).
compareprints the signed error against the exact log so you can see the direction and size for your traffic. - State: 3 cells per key.
A bucket of burst capacity refills at rate tokens per second; each request spends its
cost in tokens.
- Use it for the common case where clients are legitimately bursty — mobile sync, batch jobs, page loads that fan out. It gives you a sustained rate and a burst allowance, which is the vocabulary your API docs already use.
- Failure mode: the burst is real, and every idle client is holding a full bucket. With N tenants at burst B, the worst case your backend must absorb is N x B at once — a capacity planning question, not a rate limiting one.
- State: 2 cells per key.
A leaky bucket expressed as a single theoretical arrival time per key. With emission
interval T = 1/rate and burst tolerance tau = (burst - 1) * T, a request at time t
conforms when t >= TAT - tau, and acceptance sets TAT = max(TAT, t) + T. With
--max-delay above zero it queues instead of rejecting, and the report's wait percentiles
are those hold times.
- Use it for smooth, paced output rather than merely bounded output: protecting a downstream with no queue of its own, or a metered third-party API. Cheapest exact algorithm there is.
- Failure mode: strictness. At the default burst tolerance it rejects two back-to-back requests from a client that has been silent for an hour, which looks broken to well-behaved clients. The queueing variant converts that into latency — check the p99 wait before shipping it, because that latency is now yours.
- State: 1 cell per key.
Cap requests that have been admitted but not completed. Uses each request's service time.
- Use it for protecting a fixed pool — DB connections, worker threads, a bounded upstream queue. It is the only limiter here that reacts to the backend getting slower, because 100 req/s is harmless at 10ms and fatal at 2s.
- Failure mode: says nothing about rate, so it is useless as an abuse control or a billing quota, and without per-key caps one tenant's slow requests starve everyone else.
- State: up to
limitcells per key.
Models N gateway nodes sharing one counter, each admitting against cached_global + local_delta and flushing every --sync-interval seconds.
- Use it for putting a number on the overshoot you already have the moment you run two
gateway pods.
--sync-interval 0gives the synchronous baseline to compare against. - Failure mode: overshoot, roughly
limit + nodes x (per-node traffic per sync interval). Shortening the interval converts the error into Redis load and per-request latency; at zero you have moved your availability story onto Redis. Note this models eventual consistency, not partition — if the shared store is unreachable you must choose between failing open and failing closed, and that deserves a runbook entry. - State: 3 cells per key per node, plus the shared counter.
| Profile | Shape |
|---|---|
constant |
Perfectly even arrivals. The control case: any rejection is the limiter, not the traffic. |
poisson |
Memoryless arrivals at a mean rate. What independent clients actually look like. |
diurnal |
Sine wave over the trace, peaking mid-run. A daily cycle compressed in time. |
bursty |
On/off square wave: idle stretches punctuated by hard bursts at several times the mean rate. |
herd |
Quiet baseline, then a synchronised retry storm with decaying echoes. |
zipf |
Poisson arrivals whose key choice follows a long tail: one tenant dominates. |
All are seeded and deterministic: the same --seed always produces byte-identical arrivals,
so a difference between two algorithms is never a difference in the traffic. Non-constant
rates are generated by thinning, which is exact for any bounded intensity function.
Per-key shares default to uniform (or Zipf for the zipf profile) and can be set explicitly
with --weights 9,3,1. Request cost defaults to 1.0 and can be varied with --cost and
--cost-spread.
| Command | Purpose |
|---|---|
run |
Run one algorithm and print a detailed report. |
compare |
Run several algorithms over one identical trace, side by side, with interpretation. |
sweep |
Vary the primary parameter to find the tightest value meeting a target reject rate. |
profile |
Describe a trace, or emit it as CSV/JSONL. |
explain |
Print a long-form note on one algorithm. Omit the name to list them. |
Accepted by run, compare, sweep and profile.
| Flag | Default | Meaning |
|---|---|---|
--profile |
poisson |
Synthetic traffic shape. |
--trace PATH |
— | Replay a recorded CSV/JSONL trace instead of generating one. |
--duration SECONDS |
60 |
Length of the generated trace. |
--rate RPS |
20 |
Mean offered arrival rate across all keys. |
--keys N |
4 |
Number of distinct keys/tenants. |
--weights W1,W2,... |
uniform | Per-key traffic weights. |
--seed N |
1 |
RNG seed. |
--cost |
1.0 |
Mean cost per request. |
--cost-spread FRACTION |
0 |
Relative spread of cost; 0 is constant. |
--service-time SECONDS |
0 |
Mean service time; required by concurrency. |
--zipf-exponent |
1.0 |
Skew of the zipf key distribution. |
--burst-factor |
4.0 |
Peak-to-mean ratio for bursty and herd. |
--duty-cycle |
0.25 |
Fraction of each period that is "on" for bursty. |
--period SECONDS |
10 |
Cycle length for diurnal and bursty. |
| Flag | Default | Meaning |
|---|---|---|
--limit |
10 |
Requests (or cost) per window. For rate-shaped algorithms the per-second rate is limit/window. |
--window SECONDS |
1 |
Window length for counter algorithms. |
--burst |
limit / 1 |
Bucket capacity for token-bucket, burst tolerance for gcra. |
--max-delay SECONDS |
0 |
gcra only: queue an early request this long instead of rejecting it. |
--nodes N |
3 |
distributed only: number of gateway nodes. |
--sync-interval SECONDS |
0.1 |
distributed only: shared-counter sync / local cache TTL. 0 is synchronous. |
--duration-default SECONDS |
0.1 |
concurrency only: service time assumed when the trace has none. |
A single --limit is deliberately made to mean a comparable thing across all seven
algorithms, so compare is an apples-to-apples run.
--output text|json|csv and --ascii (plain ASCII charts instead of block characters).
JSON and CSV are stable, machine-readable and safe to pipe.
run:--algorithm/-a,--max-reject-rate FRACTION(exit 1 if exceeded),--no-keys.compare:--algorithms/-aas a comma-separated list.sweep:--algorithm/-a,--min,--max,--step,--target-reject FRACTION.profile:--emit csv|jsonl.
| Code | Meaning |
|---|---|
0 |
The command ran and any requested target was met. |
1 |
The command ran, but the target was not met: sweep found no qualifying value, or run --max-reject-rate was exceeded. |
2 |
Usage error: bad arguments, unknown algorithm, unreadable or malformed trace file. |
User errors produce a one-line message on stderr, never a traceback.
--trace accepts CSV (header row required) or JSONL (one object per line; .jsonl,
.ndjson or .json extension).
| Field | Required | Meaning |
|---|---|---|
timestamp |
yes | Seconds. Relative or absolute epoch. |
key |
yes | Tenant / API key / IP. |
cost |
no | Request weight; defaults to --cost. |
duration |
no | Service time, used by concurrency. |
timestamp,key,cost
0.90,tenant-a,1
0.91,tenant-a,1
1.00,tenant-a,1{"timestamp": 0.90, "key": "tenant-a", "cost": 1}Rows are sorted by timestamp but not shifted to t=0. Fixed-window and distributed limiters align their windows to the epoch, so moving a trace in time changes the answer; keeping your original timestamps is what makes a replay faithful. Reported times are relative to the first request.
To get a trace out of production, log the arrival time and the rate limit key at the gateway. Any structured access log already has both.
The CLI is a thin wrapper around a small public API.
from ratelimit_simulator import (
Decision, Simulator, TokenBucketLimiter, generate, load_trace,
)
# Deterministic synthetic traffic, or load_trace("prod.csv") for the real thing.
trace = generate(profile="bursty", duration=60, rate=50, keys=4, seed=7)
result = Simulator(trace).run(TokenBucketLimiter(rate=20, burst=40))
print(result.accept_rate) # 0.7182860998650472
print(result.max_accepted_1s) # worst second the backend must absorb
print(result.jain, result.worst_key)
print(result.to_dict()) # JSON-ready
# A custom limiter is anything with .name, .check(), .reset(), .describe(),
# .params() and .memory_cells(). Here: a limiter that gets stricter for keys
# that have recently been rejected, which no built-in algorithm models.
class PenaltyBoxLimiter:
name = "penalty-box"
def __init__(self, rate, penalty_seconds=5.0):
self.rate = rate
self.penalty_seconds = penalty_seconds
self.reset()
def reset(self):
self._next_allowed = {}
self._penalised_until = {}
def check(self, request):
now = request.timestamp
interval = 1.0 / self.rate
if now < self._penalised_until.get(request.key, 0.0):
interval *= 4.0 # four times slower while in the box
if now >= self._next_allowed.get(request.key, 0.0):
self._next_allowed[request.key] = now + interval
return Decision(True)
self._penalised_until[request.key] = now + self.penalty_seconds
return Decision(False, retry_after=self._next_allowed[request.key] - now)
def describe(self):
return "penalty-box(rate=%g, penalty=%gs)" % (self.rate, self.penalty_seconds)
def params(self):
return {"rate": self.rate, "penalty_seconds": self.penalty_seconds}
def memory_cells(self):
return 2 * len(self._next_allowed)
custom = Simulator(trace).run(PenaltyBoxLimiter(rate=20))
print(custom.algorithm, custom.accept_rate, custom.jain)Public API surface:
Simulator(trace, window_size=1.0, bins=60)—.run(limiter),.run_many(limiters),.decide(limiter)for raw per-request decisions.sweep(trace, factory, values, target_reject_rate).- Limiters:
FixedWindowLimiter,SlidingWindowLogLimiter,SlidingWindowCounterLimiter,TokenBucketLimiter,GCRALimiter,ConcurrencyLimiter,DistributedLimiter, plusbuild(name, ...)andREGISTRY. - Traffic:
generate(...),load_trace(path),write_trace(trace, stream, fmt),zipf_weights(count, exponent),PROFILES. - Model:
Request,Decision,Event,Trace,SimulatorError. - Metrics:
Result,KeyStats,analyse(...),jains_index(...),gini(...),percentile(...).
- accept / reject rate — overall and per key.
- max accepted in any 1s / 10s — the burst the backend actually has to survive, measured on a rolling window rather than the limiter's own aligned window. This is the number that exposes the fixed-window artefact.
- time to first rejection — how long a client gets before it sees a 429.
- Jain's fairness index — 1.0 is perfectly fair, 1/n is one key taking everything. Computed over per-key accept ratios.
- Gini coefficient — 0.0 is perfect equality; higher means acceptance is concentrated.
- worst starved key — the key with the lowest accept ratio, and that ratio.
- accepted-per-window histogram — the boundary artefact, visible as tall bars.
- wait distribution (p50/p90/p99/max) — for queueing algorithms only.
- retry-after distribution — what you would be putting in the header.
- state cells — peak count of numbers or log entries held, as a proxy for memory cost.
- It is a decision simulator, not a queueing simulator. Accepted requests are assumed
to be served; there is no backend model, no response latency, no failure. The concurrency
limiter uses the trace's
durationfield as given, and does not slow down under load. - Rejected requests do not retry. A real client that gets a 429 retries, which changes
the arrival process. The
herdprofile bakes a retry storm into the trace, but there is no closed-loop feedback between rejections and subsequent arrivals. - The distributed model is one specific model — round-robin request distribution, periodic delta flush, per-node local cache. It captures the overshoot mechanism honestly but does not model Redis latency, failure, partition, or Lua-script atomicity variants.
- Fairness is measured over accept ratios, which is one defensible definition among several. If you care about absolute throughput fairness, read the per-key table directly.
- No cost model for the limiter itself. State cells are counted; CPU, lock contention and network round trips are not.
Which algorithm should I use?
Sliding window counter for a general per-tenant API limit; token bucket if your clients are
legitimately bursty and you have sized the backend for the burst; concurrency limiting to
protect a fixed pool. Run compare on your own trace before committing — the gap between
algorithms on your traffic is usually larger than the gap in the abstract.
Why does --limit mean different things per algorithm?
It deliberately means a comparable thing: a per-window quota for counter algorithms, and
limit/window per second for the rate-shaped ones. That makes compare an
apples-to-apples run. Construct the limiter classes directly if you want independent
control of rate and burst.
Why is the sliding window counter sometimes stricter than exact?
Because it assumes the previous window's traffic was evenly spread. Front-loaded traffic is
over-counted and gets rejected; back-loaded traffic is under-counted and slips through. The
compare output prints the signed difference for your trace.
Can I use this to test my actual gateway config? Not directly — it does not read Kong, Envoy or NGINX configuration. Use it to choose the numbers, then apply them, then verify with a load test against the real thing.
How big should --duration be?
Long enough for the slowest cycle you care about. At least ten --window lengths for
counter algorithms, and at least a few --period lengths for diurnal and bursty.
The results changed between runs.
They do not, for a fixed --seed and identical flags. If they moved, a flag moved.
The algorithms here are the ones API gateways actually implement. For how they are configured and where they behave differently in practice:
- Rate limiting and throttling strategies — the algorithms as gateways implement them.
- Dynamic rate limiting with Redis backends — the shared-counter problem modelled by the
distributedwrapper. - NGINX vs Kong for rate limiting — two production implementations of the same ideas.
- Tuning retry budgets to prevent thundering herd — the feedback loop behind the
herdprofile. - Scaling limits and capacity planning — sizing the limit against what the backend survives.
- Load testing an API gateway with k6 — verifying the chosen limit against the real gateway.
MIT. See LICENSE.
Built alongside request-routing.com, a technical reference on API gateway architecture and request routing.