Skip to content
noughtdigital
Menu

Insights / Engineering

API rate limits are a product state

Plan bounded retries, work priorities and honest progress messages so an integration remains usable when a provider limits requests.

On this page

When an external API slows or limits your application, the effect reaches the user. A spinner that never finishes, a failed import and an empty dashboard are product behaviours, not merely infrastructure details.

Design the limited state before the integration becomes busy. The application should know what work can wait, what the user needs to see and how to recover without sending the same requests more aggressively.

Read the provider's actual contract

An HTTP 429 response indicates too many requests, and a response may include a Retry-After header describing when to try again. The header is not guaranteed on every response; see the MDN reference for 429.

Beyond that common mechanism, limits can vary by account, endpoint and type of operation. Confirm the provider's rules instead of inventing a universal requests-per-minute number in the application.

Keep rate limiting separate from other failures. An invalid credential, a malformed request and a temporary quota limit need different handling. Retrying them all with the same loop makes diagnosis harder and can prolong the problem.

Decide which work has priority

A background catalogue refresh and a user waiting for a current result should not necessarily compete equally. Identify the work the business needs immediately and the work that can be delayed or batched.

For a hypothetical inventory connector, a bulk refresh could pause while a smaller reconciliation job completes. That requires a shared view of capacity when several workers use the same account; independent per-process counters may not represent the provider's account-level limit.

Avoid making the visitor's browser responsible for coordinating repeated attempts. Server-side scheduling can preserve work across navigation and lets the application apply one consistent policy.

Bound the retries

Use the provider's timing guidance where available and a bounded backoff strategy where appropriate. Add a limit after which work becomes a visible failure or requires review. A retry policy should not be an infinite promise that something will eventually happen.

Before retrying a write, establish whether it is safe to repeat. A timeout can leave the application uncertain whether the provider completed the action. Use the provider's documented idempotency or reconciliation mechanism where one exists.

Keep enough state to explain progress: accepted, waiting, running, complete or failed. A user should be able to leave an import screen and return without accidentally starting the same job again.

Show an honest recovery path

If a report will take longer, say so. If a search cannot be completed, preserve the search inputs. If an import is paused, show the completed portion and what remains rather than discarding all progress behind a generic error.

In staging, test a 429 response with and without Retry-After, repeated limiting and a retry that succeeds. Check that concurrent workers do not produce a burst of requests as soon as the waiting period ends.

Record wait time and retry counts alongside ordinary failures. A connector that eventually succeeds but takes hours may still be failing the business expectation. Our API and integration engineering includes these delayed states so the connection remains useful when the provider is under pressure.

Keep reading

More from
the notebook.

All insights ↗

Put it into practice

Ready to build AI that actually works?

Let's discuss your AI engineering challenges and build something your users will love.