HTTP client rate limiting with Polly
One usually thinks of rate limiting in a server-side context, i.e., a service limits the rate of incoming requests. Well, we recently had to add client-side rate limiting to our service, i.e., limit the rate of outgoing requests to a specific upstream server. Fortunately, the Polly resilience library made it relatively simple to implement.
Rate limiting is one of the core strategies provided by Polly. To include it in the resilience pipeline, you only need to configure it to your needs:
pipeline = new ResiliencePipelineBuilder()
.AddRateLimiter(
new SlidingWindowRateLimiter(
new SlidingWindowRateLimiterOptions
{
PermitLimit = maxRequestsPerSecond,
SegmentsPerWindow = 10,
Window = TimeSpan.FromSeconds(1)
}
)
)
.Build();
The configuration above limits the number of requests per second. This pipeline can then be used to make the actual upstream calls:
return await pipeline.ExecuteAsync(async cancellationToken =>
await client.GetStringAsync(url, cancellationToken)
);
A call succeeds as long as the requests are below the configured rate limit. If a call exceeded the request rate limit, the pipeline would throw a RateLimiterRejectedException. After some time, the calls will succeed again. The following test snippet shows this behavior:
var tasks = Enumerable.Range(0, 5).Select(_ => client.GetAsync()).ToArray();
await Task.WhenAll(tasks).ShouldNotThrowAsync();
await client.GetAsync().ShouldThrowAsync<RateLimiterRejectedException>();
await Task.Delay(TimeSpan.FromSeconds(1));
await client.GetAsync().ShouldNotThrowAsync();
It's up to the surrounding code to decide how to handle this case. Our web service simply returns a 429 Too Many Requests response.
A sample project with full source code is available in my GitHub repository. A test is included to verify the desired behavior. Inside it, I use WireMock to mock the upstream server.
The Polly resilience library once again proved an invaluable tool for implementing non-trivial logic on top of outgoing HTTP requests. While you could always implement the whole thing yourself, relying on a well-established library instead can save you time. And the resulting code is less likely to introduce a difficult to find bug.
