The sliding window counter is the right default for most production APIs: it's the best trade-off between memory cost and accuracy, and production data at scale shows it scales to real traffic. Switch to a token bucket only when clients need to burst for valid reasons. The real trade-off in each case is memory and precision vs. how much burst behaviour you're willing to tolerate. Redis, Lua, and gateway enforcement implementation details follow below.
TL;DR:
The sliding window counter is preferred for most production APIs because it's more memory efficient than others and provides near-accurate request counting.
Implement a token bucket algorithm instead of a fixed rate when clients have burst capacity, for example: webhook retry logic or a mobile app sync process.
- With Redis it can be implemented using atomic lua scripts and lazy refill to avoid race condition and high overhead on request checks.
Sliding window logs provide accurate tracking but have high memory costs, making them best for privacy-sensitive or low-volume endpoints. Fixed windows can be used for internal, low-precision limits.
- Adopt multi-tiered, hybrid techniques of combining edge token buckets for short-term burst control with a sliding window counter at a central point for long-term throughput quotas and advertise limits using common headers for improved client-side self-throttling.
Build APIs That Hold Under Load
PODTECH develops scalable enterprise software, automation, and systems to power critical and high-demand infrastructure, including custom applications and smart digital platforms.
LEARN MORETable of Contents
- API rate limiting strategies: the core algorithms explained
- Decision rules: choosing an algorithm for your API
- Implementation patterns: Redis, Lua, and gateway-level limits
- Scaling and failure modes: shared state and local approximation
- Communicating limits to clients: headers and retry behaviour
- Monitoring and metrics to tune limits safely
- PODTECH operational notes for enterprise-grade APIs
- Author perspective: pragmatic trade-offs and three operational rules
- How PODTECH helps you build rate limiting that holds under load
- Sources
- FAQ
API rate limiting strategies: the core algorithms explained
Rate limiting is there to prevent a single client, a single bug, or a single attacker from knocking out infrastructure the rest of your users rely on. OWASP even lists unrestricted resource consumption as one of its core classes of API vulnerability, and each of the algorithms below is a different solution to the same problem: how do you, quickly and equitably, make a decision on whether this request should be allowed?
Token bucket provides each client with a bucket that refills at a constant rate and empties one token per request. Bursts up to the size of the bucket are allowed, after which the client is throttled to the refill rate. This makes it a natural fit for APIs where bursts (a batch job, a retry sweep) are legitimate traffic rather than abuse.
Leaky bucket enforces a constant output rate independent of request input rate. In policing mode, excess requests are dropped. In shaping mode, they wait in queue and empty smoothly. Use shaping when you're protecting a downstream system that really can't deal with variation. A good example would be a legacy database with a fixed connection limit.
Fixed window: counts requests in discrete blocks (e.g. per minute) and resets the counter at the boundary. It's simple and cheap, but a client can send a full quota at 0:59 and another full quota at 1:00, doubling the effective rate for two seconds. Acceptable for internal tooling; risky for public-facing limits.
Sliding window log keeps a timestamp for each request and counts the number of entries in a rolling window. It's precise, but memory consumption is linear with the request rate which can be costly at high rates.
A sliding window counter is a good approximation to the sliding log, combining two fixed-window counters and using a weighted average between them. It approaches log-level accuracy at a fraction of the memory cost, and is for that reason the default practical choice in most production systems.
| Algorithm | Memory cost | Accuracy | Burst tolerance | Best for |
|---|---|---|---|---|
| Token bucket | Low | Good | High (allows bursts up to a set capacity) | Public APIs where occasional bursts are legitimate |
| Leaky bucket | Low | Good | Low (produces smooth, steady output) | Protecting systems that cannot handle bursty traffic |
| Fixed window | Very low | Lower accuracy near window boundaries | May allow bursts at window edges | Internal or less critical limits |
| Sliding window log | High | Exact | Precise | Security-sensitive, low-volume endpoints |
| Sliding window counter | Low to moderate | Near exact | Controlled bursts with smoothing | General-purpose default for production |
Decision rules: choosing an algorithm for your API
The two questions that decide most of this: how much memory can you spend per client, and how much burst do you actually want to permit? Everything else is refinement.
- Choose the sliding window counter if you have a public API that requires fairness and predictable memory usage without the need to monitor specific request times.
- Change to token bucket when there's a valid reason for your clients to burst, like webhook consumers playing catch-up after a reconnect, or mobile apps that need to sync after being offline.
- Sliding window log: Try this when precision is worth the price, mostly on security-related endpoints where request volumes are modest: login, password reset, and similar flows.
- Use fixed window for internal services where simplicity and low overhead are more important than precision, and there is no danger of boundary abuse.
- Use leaky bucket in shaping mode to prevent traffic bursts to a downstream dependency that can not absorb variance at all, such as a legacy database or a third-party API with its own strict limits.
Three fast examples: a payments API validating a card authorisation will prefer sliding window counter with strict per-second and per-minute overlay, because combining short and long windows catches bursts a single window will miss. A public webhook receiver should use token bucket, as retry storms from partner systems are a regular pattern, not an attack. A telemetry ingestion endpoint ingesting data from thousands of sensors typically requires both: a token bucket at the edge to absorb bursts, and a sliding window counter centrally to enforce sustained quota.
Implementation patterns: Redis, Lua, and gateway-level limits
Redis is the default backing store for most rate limiters, as it's fast, provides the right data structures, and handles atomic operations cleanly. Redis's own tutorials map each algorithm to a specific structure: fixed window is well suited to a simple STRING counter with a TTL, sliding log fits a ZSET keyed by timestamp, sliding window counter works well as a pair of STRING keys, and token bucket maps naturally to a HASH storing token count and last refill time.
The fundamental problem is the atomicity problem. The basic "read count, check limit, increment count" sequence, if implemented naively as three separate Redis calls, has a race condition under concurrent requests. By executing Lua scripts inside Redis, they run atomically and close that gap entirely, avoiding the check-then-act race that makes multi-step limiter logic difficult.
Regarding the token bucket in particular: do not have a background timer that refills the tokens for each client. Lazy refill calculates elapsed time since last request and adds tokens on access, keeping the entire operation O(1) with no scheduler overhead. You only need to store two fields: tokens and last_refill.
Token buckets at the gateway and edge, including Cloudflare, API gateways, and service mesh sidecars, have the opposite benefit: they can enforce limits without a round trip to a central store, reducing latency. The downside is that each instance of the limit tracks the global limit independently, so the enforcement is only best-effort, not strict.
PRO TIP: If you are using Redis Cluster, use hash tags such as {client_id}.tokens to make sure a client's related keys end up on the same shard, and keep your Lua scripts small. Long scripts block the Redis event loop and silently pile up latency on every request queued behind it.
Scaling and failure modes: shared state and local approximation
Exact enforcement is provided by a single shared Redis instance, but that becomes a scalability bottleneck and a single point of failure at scale. Local, per-instance limiters remove that bottleneck, but reintroduce the N×L problem: if each of N instances locally enforces limit L, then the effective global limit is N×L, not L.
Sharding/consistent hashing shards client keys over multiple Redis nodes which minimizes the hot-key contention against any individual instance.
- Cell-based (or PoP-sharded) counters with local short-term mitigation can ease the load on the central store and help enforcement survive transient outages, at the expense of somewhat looser global accuracy.
- Fail-open allowing requests to pass when the limiter store is unavailable ensures availability but leaves you vulnerable to abuse during an outage.
- Fail-closed means better protection for the backend, but can escalate a Redis blip into an API outage for legitimate users.
Watch Redis latency and error rate, specifically, so you know when enforcement is degrading before it becomes an incident, not after.
The most common deployment of enterprise systems ends up somewhere in the middle. Short-term burst limits are applied locally or at the edge of the network, while longer-term quotas are reconciled against a shared store.
Communicating limits to clients: headers and retry behaviour
Each failed request should respond with 429 Too Many Requests with a Retry-After header as formalised in RFC 6585. In addition to the status code, clients require machine-readable state so that they can self-throttle intelligently, rather than resort to random guessing.
The minimum useful header set:
- Limit — the total quota for the current window.
- Remaining — how many requests are left before the client gets throttled.
- Reset — when the window refreshes.
- Retry-After — how long to wait before retrying, in seconds.
IETF's draft RateLimit and RateLimit-Policy headers standardise this into structured fields, separating current state from policy, which matters when a server enforces several limits at once.
The right behaviour on the client side to a 429 is exponential backoff with full jitter, a sensible fixed maximum retry count and, for retried writes, idempotency keys to prevent duplicate side effects. A 429 without a Retry-After header is a real danger: naive retrying clients without direction can synchronise into a retry storm, amplifying the exact load spike the limit was supposed to help guard against.
Monitoring and metrics to tune limits safely
Rate limits are never “set and forget.” Track these signals continuously:
- Breakdown of 429 rate as a percentage of total requests, by endpoint and client tier.
- Latency percentiles (p95/p99) on limiter checks themselves, since a slow limiter defeats its own purpose.
- Redis operation latency and error rate, an early warning sign of store degradation.
- Retry traffic volume, which spikes before a full outage if backoff isn’t working.
Launch any tightened limit first to a canary group. Monitor the dashboards for an increase in errors/retries and then stage the change to the broader client base instead of flipping it globally.
PODTECH operational notes for enterprise-grade APIs
On mission-critical telemetry, the limits can be stacked intentionally: a short term token bucket at the edge would absorb the bursts from sensors, while a sliding window counter at a central node enforces longer term quotas related to SLA guarantees. This stacking directly contributes to uptime monitoring in both datacentre telemetry ingestion and industrial IoT deployments.
A working checklist for teams standing this up:
- Identify what constitutes a client — API key, tenant, or IP — before choosing storage keys.
- Set per-endpoint profiles rather than one global limit.
- Instrument 429 rate and Redis latency from day one, not after the first incident.
- Run every limit change through a canary before wider rollout.
Pro Tip: If you’re shipping rate-limit changes in a new API version, make sure the timing aligns with your versioning strategy to prevent older clients from being throttled by rules they weren't built to follow.
Author perspective: pragmatic trade-offs and three operational rules
The majority of teams overthink the algorithm choice and underthink the rollout. The sliding window counter will serve you well in the vast majority of cases, but the algorithm matters less than three habits: keep defaults generous enough that developers don’t build workarounds, protect genuinely fragile downstreams with shaping rather than blanket limits, and never tighten a limit without measuring its effect on a canary group first. Machine-readable headers aren’t a nice-to-have. Without them, you’re forcing every client to guess, and guessing is what causes retry storms.
— Harry
How PODTECH helps you build rate limiting that holds under load
Getting the algorithm correct is one decision. Getting the architecture, monitoring and rollout process correct across dozens of endpoints and multiple client tiers is the harder, ongoing work, and it’s where most in-house teams run out of time. PODTECH builds this into enterprise systems as standard, not as an afterthought bolted on after the first outage.
Enterprisewide, PODTECH’s enterprise automation tools build layered limiting architecture, from edge token buckets to centrally enforced quotas, all rooted in the SLA commitments your business truly relies on. For tech teams serving ML inference pipelines behind your APIs, that discipline extends into machine learning development, where request throttling has to take into account model latency as well as traffic volume. If you’re looking at rate-limiting overhauls or building an API from the ground up, start with an architecture audit: map your endpoints, map your fragile downstreams, and lay out a rollout plan free from guesswork. Drop us a line about enterprise automation to scope the work.
Sources
- Redis tutorial: rate limiting howtos
- OWASP: Unrestricted resource consumption
- API rate limiting strategies: 2026 engineering reference
FAQ
What are some effective strategies for rate limiting?
The best practices include using an appropriate algorithm, sliding window counter for most APIs, token bucket where bursts are anticipated, layered short and long term quotas, and obvious client-facing headers.
How do you implement API rate limiting?
It is accomplished by selecting a storage backend, most commonly Redis, mapping your selected algorithm to the appropriate data structure, and using atomic Lua scripts to eliminate race conditions during concurrent limit evaluations.
How will you design rate limiting for an API?
First agree what is a client, choose an algorithm given your memory and burst-tolerance requirements, layer per-second and per-minute limits, and always respond with 429 and Retry-After/rate-limit headers.
What does API rate limiting mean?
API rate limiting is a means of restricting the number of requests a client can make to an API in a set time window. This can prevent overloading of the back-end systems. It also can guard against abuse or unintentional resource exhaustion. See OWASP's resource consumption cheat sheet for more on this.