Rate Limiting#
Melissa implements rate limiting (throttling) protections across our cloud services to maintain optimal performance and ensure equitable resource distribution among users. Rate Limiting helps prevent scenarios where excessive usage by one client could adversely impact others.
How Rate Limiting Affects You#
Rate limiting is a common concept for many web services and most users should already have a plan in place if they use these types of web services.
Under normal usage patterns, most clients will not experience rate limiting.
However, if your application sends a high volume of requests in a very short period, you may encounter rate limits.
When rate limited, the service responds with an HTTP status code 429 Too Many Requests
, indicating that you’ve surpassed the allowed request rate.
Example of a 429 Error Response:
HTTP/1.1 429 Too Many Requests
Content-Length: 0
Date: Mon, 22 Apr 2024 18:31:56 GMT
Server: Kestrel
In this scenario, the response body will be empty, and the 429
status code in the header signifies that your request rate has exceeded the permissible limit.
What are the actual limits?#
Our rate limiting system is designed to maintain server stability and may adjust based on various factors, including server capacity and user subscription levels. While specific rate limits can vary, the following general guidelines apply:
User Type |
Approximate Throughput Rate |
Annual Maximum Volume |
---|---|---|
Pay-as-you-go (Credit) User |
~5 records/second |
~158 million |
Self-Service Subscription |
~12 records/second |
~380 million |
Business Subscription |
~200 records/second |
~1.5 billion |
Enterprise Subscription |
~400 records/second |
~3 billion |
Note
These figures are subject to change. If you anticipate needing more than ~100 records per second or ~3 billion records per year, please contact your sales representative to discuss your requirements.
Recommended 429 Error Prevention#
There are a few simple strategies to mitigate the effects of rate limiting in your applications.
Single-threaded with batching (recommended):
Use a single worker thread to process requests.
Batch requests with 100 records per POST for optimal processing (where supported - please refer to documentation for the specific service).
You should generally not run into 429 errors, even if batching 100 records per request.
Our servers favor batching; your application may exceed single-record limits.
Multithreaded (generally for enterprise subscriptions):
If higher throughput is needed, consider upgrading to an enterprise plan to unlock higher limits to support multithreaded workloads.
Ensure the request rate across threads stays within the throttling limits. Requests are throttled across all requests using your subscription.
Manage throughput in queues.
If your request volume is difficult to predict due to traffic from multiple applications or servers, you may need to centralize traffic to a queue for optimal performance or assign a certain throughput cap to each server.
Conduct internal tests to evaluate your RPS performance:
Average RPS: Helps identify if you’re nearing the subscription limit.
Consider short-term aggregates in one-to-five second buckets. A sudden spike in requests could cause issues even if the rate over a longer period is low.
Max RPS: Indicates if you can scale up or down threads and batch sizes.
A low average RPS with a high maximum RPS may indicate that you can scale up but need to control short-term bursts or handle occasional 429 errors.
Recommended 429 Error Handling#
We recommend your application can handle the situation where an HTTP 429 error is returned:
If a
429
error is returned, please wait a second and re-send.If a
429
error is returned multiple times, scale down the speed you are sending records to our services.
Need Assistance?#
If you have questions or require support regarding rate limiting and its implications for your services, please visit our support page at https://www.melissa.com/company/product-support
By understanding and planning for rate limiting, you can ensure seamless and uninterrupted use of Melissa’s cloud services.