Rate limiting vs throttling

Your server is screaming, not because of a bug, but because of a success. A sudden viral tweet or a massive batch of automated requests has flooded your API. Without rate limiting or throttling in place, your database locks up, your CPU pegs at 100%, and the site goes down for everyone.

This is the nightmare scenario for any engineering team. To prevent it, you need to control the flow of traffic. However, engineers often use the terms “rate limiting” and “throttling” interchangeably. While they both deal with traffic management, they solve the problem using different philosophies.

Choosing the wrong one can lead to a degraded user experience or, worse, a complete system failure under pressure. This guide breaks down the technical nuances of both strategies and how to implement them in modern stacks like Laravel and Shopify.

The traffic surge crisis

The problem starts with resources. Every server has a finite amount of memory, compute, and bandwidth. When requests come in faster than your system can process them, a queue builds up. Once that queue exceeds the system’s capacity, the service fails.

This is not just about malicious actors or DDoS attacks. It is often about “noisy neighbors” in a multi-tenant environment or a poorly optimized loop in a client-side application. I have seen production environments crash simply because a mobile app’s “retry” logic was too aggressive during a minor network hiccup.

Rate limiting and throttling are your tools to enforce discipline. They ensure that no single user or service can monopolize your infrastructure. They are the gatekeepers of your cloud infrastructure.

Defining rate limiting: the hard stop

Rate limiting is a policy-based approach. It defines a strict contract: “You are allowed exactly X requests per Y amount of time.” Once a user reaches that limit, the gate shuts.

When a request exceeds the limit, the server immediately rejects it. This is typically done by returning an HTTP 429 “Too Many Requests” status code. The server does not spend any more resources on that request. It does not look up data in the database. It just says “no.”

Why use rate limiting?

Rate limiting is primarily about fairness and security. It protects your API gateway from abuse. If you are running a SaaS, you might have different tiers. A free user gets 60 requests per minute, while a premium user gets 1,000.

Rate limiting is easy to communicate to users. They can check their response headers (like X-RateLimit-Limit and X-RateLimit-Remaining) to see exactly where they stand. It is a binary state: you are either within your limit or you are blocked.

Defining throttling: the gentle brake

Throttling is a runtime behavior designed to “shape” traffic. Instead of a hard rejection, throttling slows down the processing of requests. Think of it like a funnel. You can pour a bucket of water into it all at once, but it only drips out at a controlled, steady speed.

In a throttled system, if you send too many requests, the server might add a delay to each response. It might queue the requests and process them as capacity becomes available. The goal is to smooth out spikes and avoid a “jagged” traffic pattern.

Why use throttling?

Throttling is excellent for protecting backend resources like databases or third-party APIs. If you know your database can only handle 500 writes per second, you might throttle incoming requests to stay just under that limit.

It provides a better user experience for occasional spikes. Instead of a 429 error, the user might just see a slightly longer loading time. However, if the traffic remains high for too long, the queue will eventually fill up, and the system will have to start rejecting requests anyway.

Key algorithms: token vs leaky bucket

To implement these strategies, engineers rely on specific mathematical models. Understanding these is crucial for fine-tuning your system performance.

1. Token bucket (common for rate limiting)

Imagine a bucket that holds “tokens.” Every time a request comes in, a token is removed. If the bucket is empty, the request is rejected. Tokens are added back to the bucket at a constant rate.

• Pros: It allows for “burstiness.” If the bucket is full, a user can send a quick burst of requests until the tokens run out.
• Cons: Hard to manage if the burst is too large for your downstream services.

2. Leaky bucket (common for throttling)

Imagine a bucket with a small hole at the bottom. Requests are poured into the bucket. They “leak” out of the hole at a constant rate to be processed. If the bucket overflows, new requests are dropped.

• Pros: It ensures a completely stable, predictable flow of traffic to your backend.
• Cons: It is very strict. It does not allow for bursts even if the system has idle capacity.

Practical implementation: Laravel and Shopify

How does this look in the real world? Let’s look at two ecosystems where these patterns are vital.

Rate limiting in Laravel

Laravel makes rate limiting simple through its RateLimiter facade and the throttle middleware. The built-in limiter uses a fixed-window counter backed by your cache store.

In Laravel 11 and 12, you define named limiters in the boot method of AppServiceProvider (the old RouteServiceProvider was dropped from the default skeleton):

use Illuminate\Cache\RateLimiting\Limit;
use Illuminate\Http\Request;
use Illuminate\Support\Facades\RateLimiter;

RateLimiter::for('api', function (Request $request) {
    return Limit::perMinute(60)->by($request->user()?->id ?: $request->ip());
});

This tells Laravel to allow 60 requests per minute per user ID or IP address. If the limit is hit, Laravel automatically throws a ThrottleRequestsException, which results in a 429 response. This is a classic “hard stop” rate limit.

Throttling in the Shopify API

Shopify uses a leaky bucket algorithm for its GraphQL Admin API on every plan, not just Plus. This is a clean example of throttling.

When you make a GraphQL call, the response carries cost details under extensions.cost, including the requestedQueryCost and a throttleStatus.

• GraphQL cost: Each query is assigned a calculated cost based on the fields and connection sizes you request.
• The bucket: Your app has a bucket of “points” — 100 per second restore on Standard, 200 on Advanced, 1,000 on Plus. Points are spent on each query and restore over time.
• The brake: If you spend points faster than they restore, Shopify returns a THROTTLED error.

For high-volume commerce, this means you have to build back-off logic into your application. A well-behaved client reads throttleStatus.currentlyAvailable and slows down before it hits the wall, rather than firing blindly and retrying on every THROTTLED error. When you do get throttled, wait long enough for restoreRate to refill the points your next query needs.

Architectural impact: security vs experience

When designing your system, you must decide where to place these controls.

At the edge: Implement rate limiting at the WAF or API gateway level. This stops malicious traffic before it even touches your application code. This saves money on compute costs.

Inside the application: Implement throttling within your service logic. Use it when calling external APIs or writing to a shared database. Use a queue system like Redis or Amazon SQS to hold requests that exceed your immediate capacity.

I have found that the most resilient systems use both. Rate limiting at the perimeter blocks the “bad actors,” while internal throttling ensures that your internal services don’t melt down during a legitimate traffic surge. This is especially important when building agentic commerce systems that may generate many automated API calls in a short period. And if those calls hit a paid LLM, the stakes shift from CPU to your bill — see rate limiting and the denial-of-wallet problem for the token-cost angle.

Takeaways

Managing traffic is about balance. You want to provide a fast experience for users while keeping your infrastructure healthy.

1. Rate limiting is for policy. Use it to enforce subscription tiers and block brute-force attacks.
2. Throttling is for stability. Use it to smooth out traffic spikes and protect your database or external API dependencies.
3. Return 429 codes. Always let the client know they are being limited. Include a Retry-After header so they know when they can try again.
4. Monitor your limits. Use dashboards to track how often users hit their limits. If 20% of your users are being limited, your limits might be too low — or your app might be inefficient.
5. Use Redis for state. Both rate limiting and throttling require a fast, central store to track request counts. Redis is the industry standard for this.

How do you decide between rejecting a request immediately or making the user wait 500ms longer to maintain system stability? If you’re hardening an API for production, here’s how I help teams ship it.