Rate limits

Every plan carries two rate limits: requests per minute and tokens per minute. They bound how fast requests go through the gateway, not what they cost; a line on spend is a cap. The figures below are the ones the platform serves for each plan, and on Free, Team, Growth and Scale they are the figures the gateway throttles at. Read them as indicative, never as a guarantee: each gateway node counts on its own.

The figures by plan

The gateway still throttles Enterprise traffic, at a default the platform does not publish, because no contract has agreed to that number. The platform serves no figure for Enterprise, only its note: Limits on this plan are set in your agreement. Your organization's plan decides the row, for every key the organization holds.

Where throttling sits

Before your provider is contacted, the gateway runs admission in order: it authenticates the key, refuses the request if the kill switch is on for the key or the organization, applies throttling, then checks every armed cap that covers the request (Cap). So:

• a request the kill switch refuses is never throttled;
• a throttled request reads no cap and carries no budget header;
• a request inside the rate limits can still be refused at a cap.

How a limit is counted

• Your organization, and each key. A request is counted twice: against the key it used and against your organization as a whole, and both are held to your plan's two figures. One key can use all of your organization's figure; your keys together share it.
• Up to two minutes' worth at once. Each limit holds up to twice its per-minute figure and refills at its per-minute figure, so after a quiet spell a burst of up to twice the figure is admitted.
• Tokens are charged at admission and corrected after the response. A request is charged tokens at admission from the body size it declares. After the response, the charge is corrected to the tokens your provider reported, where it reported any: an over-charge is credited back and an under-charge is charged forward. Requests per minute are never corrected; a request is one request.
• Each gateway node counts on its own. The counts live in the memory of the node that serves the request, and no node adds in another's, so the point at which your traffic is throttled can sit above or below a figure. That is why the figures are indicative.
• A fault in the limiter fails open. If the rate limiter itself faults, the request is passed through rather than refused (Fail-open and fail-closed).

What comes back

HTTP/1.1 429 Too Many Requests
retry-after: 12

{
  "error": {
    "message": "Rate limit reached for this organization.",
    "type": "rate_limit_error",
    "param": null,
    "code": "rate_limit_exceeded"
  }
}

• Retry-After is the whole number of seconds until the request would fit, from 1 to 60.
• Nothing is forwarded, queued or metered. There is no provider call, no cost and no receipt for a throttled request.
• Branch on error.code. An armed cap in pause mode also answers 429 with error.type rate_limit_error, but its code is budget_exceeded and the budget headers ride beside it (Refused in pause mode). rate_limit_exceeded is the rate limit and nothing else.

Where the platform serves them

GET /platform/v1/limits answers with your organization's row, and every role can read it:

The figures are configuration, served from one table: they say what your plan allows, never how much of it your traffic has used.