Designing API rate limits that don't break integrations

Most rate limiters are built to solve one problem: stop a client from taking down the platform. A naive token bucket in front of your API gateway accomplishes that. What it won't do is tell you which partner just started silently failing at 2am because their retry logic doesn't understand your headers, or whether the account generating seven figures of annual revenue is dropping webhooks because their hourly batch job slams into your limit. We've inherited three of these situations at client platforms. In every case, the rate limiter was working exactly as designed. The design was wrong.

A 429 with no information is a support ticket waiting to happen

If your rate-limited response is a bare 429 Too Many Requests with no body and no headers, you've pushed the cost of understanding your system onto every integrator, forever. Every partner team reverse-engineers your limits by trial and error and hard-codes guesses into retry logic. Send X-RateLimit-Limit, X-RateLimit-Remaining, and X-RateLimit-Reset on every response, not just the rejected ones, so a client can throttle itself proactively instead of finding out reactively. Send Retry-After on the 429 itself, in seconds, and make it accurate. We've seen platforms send a Retry-After that doesn't match the actual reset, which teaches clients the header is unreliable and trains them to ignore it.

Decide what "the limit" actually protects, then price accordingly

Rate limits are usually set by picking a round number that feels safe, not by reasoning about what's actually being protected. Request volume protects your servers from being overwhelmed, but a request that triggers a full-text search or a report export costs two orders of magnitude more than one that reads a cached object by ID. A flat per-minute cap either starves your cheap endpoints or lets the expensive ones through at a volume that takes down a downstream database. Cost-weighted limiting, where each endpoint consumes a different number of tokens from the same bucket, is more work to build and explain, but it's the only version that doesn't force a choice between safe and usable for your heaviest integrations.

Tier the limit to the relationship, not just the plan

Self-serve consumers and negotiated enterprise integrations have fundamentally different risk profiles, and treating them identically is where most of the pain comes from. A self-serve developer hitting the default limit is annoying but low-stakes: they adjust their code, maybe complain in a support channel. An enterprise integrator hitting that same limit during their own customer's peak traffic event is a contract-level incident, the kind that surfaces in the next renewal conversation. Build at least three tiers: a conservative default for newly onboarded keys, a higher standard tier for verified accounts, and a negotiated tier for named integrators with limits set in the contract, not a config file nobody remembers to update. The negotiated tier needs a direct line to someone who can raise it same-day. "File a ticket and wait 48 hours" doesn't answer "our limit is failing customers right now."

Give clients room to recover, not just a wall to hit

A hard cutoff that goes from fully available to fully blocked the instant a counter crosses a threshold is the worst failure mode for anyone running a batch job, a backfill, or a burst-shaped workload like an end-of-month billing run. Two changes fix most of this. Use a sliding window or leaky-bucket algorithm instead of a fixed window, so a client isn't punished for two adjacent windows both landing near the edge of their allowance. And build in graceful degradation before hard rejection: at 80% of the limit, add latency or shed to a lower-priority queue, and only return 429 once the client is genuinely over budget.

Never ship a limit change without a grace window and a real notification

The single most common way rate limiting breaks an integration isn't the limit itself, it's changing it without warning. Tightening a limit that's been effectively unenforced for a year, because you finally turned on middleware that had been sitting in monitor-only mode, is a breaking change with the blast radius of an API version bump. It deserves the same treatment: an announced date, a migration path, and direct outreach to the accounts your usage data shows will be affected. Ship new or tightened limits in shadow mode first, logging what would have been rejected without rejecting it, for at least two weeks. That log tells you which accounts are going to break, before their support ticket does.

Rate limiting is infrastructure protection wearing a product-decision costume. Get the mechanics right and it's invisible to everyone who isn't abusing the platform. Get them wrong and your most valuable integrators, the ones with the traffic the system was built to withstand, are the first ones you break.