Rate limits
The ceilings a call is measured against, what a call that reaches one gets back, and how a higher ceiling is asked for.
What a new key carries
A key is issued with one ceiling of its own — how many streams it holds open at once. The two rate ceilings, requests and tokens, belong to the organization and are shared by every key it issues. Nothing is switched on to get them, nothing in a request asks for them, and a key that has just been minted is already under them:
| Requests | Tokens | Concurrent streams | |
|---|---|---|---|
| Organization | 500 per second | 5,000,000 per minute | none of its own |
| Key | none of its own | none of its own | 16 |
A key carries no rate ceiling of its own by default, so issuing several is free — one per project, environment or agent: a new key takes nothing away and meets nothing new. Where one key needs a narrower ceiling — a test bench that must not eat the shared budget, say — it is set on that key alone; it applies in addition to the organization's and is never wider than it.
The organization's budget is shared, not a sum of what the keys are allowed: a call is refused for either of the organization's ceilings on the traffic of all its keys together — a quiet key can be turned away for the organization's request ceiling as readily as for its token one.
The ceilings are counted separately and on different clocks — one over a second, the other over a minute — so a workload can sit comfortably inside the request ceiling and still be over the token one. A handful of very large calls every second is exactly the shape that does it, and it is the shape that surprises people, because the request count looks fine.
How many streams are open at once
The rate ceilings count how often calls arrive. The stream ceiling counts how many are open right now: a stream takes a slot from the moment it is admitted until it ends, is abandoned by the client, or expires. A key's seventeenth simultaneous stream is refused with a 429, and the sixteen already running carry on.
Only streams take a slot. An ordinary non-streaming call takes none and frees none, however long it runs, and counting tokens takes nothing at all.
Sixteen is chosen for a client that keeps several calls in flight by itself: an agent running subagents or parallel tool calls reaches about ten requests on one key, and the ceiling sits above that rather than against it. It sits below the organization's request ceiling so that the whole of it can be taken at once: sixteen streams are opened by sixteen requests, and a client with no other traffic fills them inside a single second. Nothing in the arithmetic forbids more — concurrency accumulates across seconds — but a stream ceiling above the request ceiling would be filled in instalments rather than in a burst, and the rate ceiling would refuse on the way.
The stream ceiling a key is issued with belongs to the key and not to the organization: two keys of one organization each hold sixteen streams and do not take them from each other. The organization carries no stream default, unlike both rate ceilings, which are its alone.
A configured organization-wide stream ceiling does exist as a possibility: if one was set for you it is shared by every key and is checked before the key's own. It never appears by itself and nothing above assumes one — but where there is one, a second key does not get around it, and it is raised separately.
Which is also why one key shared between several agents is not the same as a key per agent: the sixteen slots are divided among everything using that key. A client that runs out of them gets a second key more cheaply than a raise — unless its organization carries a stream ceiling of its own, which the keys share.
When a ceiling is reached
A call that would cross any of the ceilings is refused rather than queued: the status is 429, and the envelope's type is rate_limit_error. The refusal is decided before the call reaches a provider, so nothing upstream sees it. It names the ceiling the call crossed, that ceiling's value, the room left under it, and the instant before which retrying is pointless.
The request ceiling can be counted as calls arrive. The token ceiling cannot, because what a call will cost is not known until it has been answered.
So the platform estimates the call's tokens before it goes upstream and counts the estimate against the minute, then reconciles that estimate against the usage the answer actually reports.
When that reconciliation lands, the minute carries the real figure rather than the guess, and a run of calls that came in far under their estimates leaves room behind it.
Two cases end differently, and both are deliberate:
- The minute has already rolled over — The correction is aimed at the exact minute the call was charged to, so once a later call has carried a counter into a new minute there is nothing left to correct. The counter is no longer on that minute. Time passing does not do this; a further call does. And the two counters move on independently, so a correction can still land on one of them and find nothing on the other.
- Images — The estimate really does stand: an operation that reports no tokens at all keeps what it was charged. Image generation is priced in images, not tokens, so correcting it to zero would refund the whole estimate and let that surface cost nothing against the ceiling.
Streaming does not change the arithmetic: a stream that has begun is a call that has been counted. Counting tokens does not enter it at all — that operation calls no provider and consumes no quota, which is what makes it safe to run in front of a large request.
Back off before retrying a 429, lengthen the wait with every attempt, and cap the number of attempts.
Sizing a call in advance
The token ceiling runs into the fact that what a call costs is not known until it has been answered. One half of that can be learned before you send: counting tokens on the Messages dialect takes a subset of an ordinary call's members — model, messages, system, tools, tool_choice, thinking, metadata and context_management — and answers with a deterministic local estimate of the input tokens. The members that describe the generation rather than the input are refused on this operation: max_tokens, stream, output_config, stop_sequences, temperature and top_p.
It calls no provider, creates no request and no reservation, touches no balance and spends no rate quota. So you can put it in front of a large request without paying for it in money or in room under the ceiling.
It does not count the output half and cannot: that half is bounded by you, through the output ceiling the request names, or the surface's published default when it names none.
Asking for more
A raise is a conversation, not a setting. None of the ceilings is editable in the console, and no member of a request asks for a higher one. Write to support with the traffic you need admitted — the shape of it, not only the peak — and your organization's ceilings are changed for you. The same goes for a narrower ceiling on one key.