Limits at a glance
Tokens per minute counts each running request at about 2.0 to 2.3 million tokens, whatever its size, so the default allows two requests at a time. See How tokens per minute works.
The values for Requests per minute, Tokens per minute, and Concurrent requests are defaults. If DeepMask raised them for your organization, the API limits card shows your values. For limits on keys, logs, and exports, see Account limits.
Token throughput
Tokens per minute limits your organization’s token throughput. All your keys share it, and the default is 5,000,000. DeepMask applies it in two ways:- Running requests: the requests from all your keys that are running right now can’t count for more than your Tokens per minute value. A new request that would go over it returns
429withAPI_TPM_LIMIT. - Actual tokens: your organization’s actual token use per minute is also limited to the Tokens per minute value. A request refused for that reason gets
429withAPI_RATE_LIMIT.
How tokens per minute works
While a request is running, DeepMask counts it toward Tokens per minute at a fixed size for its model, not at its actual size. That size is about 2.0 to 2.3 million tokens, depending on the model’s maximum output. A request stops counting as soon as it finishes. A short prompt therefore counts the same as a long one, and a lowermax_tokens value doesn’t lower the count.
In practice, this makes Tokens per minute act as a second limit on parallel requests. With the default limit of 5,000,000, two requests can run at the same time, whichever models they use. A third request doesn’t fit until one of them finishes. That matches the default Concurrent requests limit, so you usually reach API_CONCURRENCY_LIMIT first. If your organization has a higher concurrency limit, API_TPM_LIMIT can be the limit you reach instead.
To get the most out of your throughput:
- Run at most as many requests in parallel as your Concurrent requests value allows. See Limit parallel requests in your app.
- Ask for shorter responses when you can. A request that finishes sooner frees its share of Tokens per minute sooner.
- Don’t start requests you might abandon. Once DeepMask has sent a request to the model, disconnecting doesn’t stop it. DeepMask completes it, and it keeps counting toward Tokens per minute until it finishes.
Rate limits
DeepMask limits how many requests your organization, each key, and each model can handle. When a request goes over one of these limits, it’s refused right away with a429 error. It isn’t queued. See 429: rate and capacity limits for each code.
Requests per minute
Requests per minute counts the requests from all your keys in the last 60 seconds. The default is 60.- Requests that DeepMask refuses before processing them don’t count, for example for invalid credentials, an invalid request, a reached limit, or a spending cap.
- Requests refused later, for example by the model provider, still count.
- Requests blocked by a spending cap still appear in the request log, even though they don’t count here.
429 with API_RPM_LIMIT. Some requests over your organization’s request volume get API_RATE_LIMIT instead. Handle both the same way: slow down and retry with backoff.
Concurrent requests
Concurrent requests counts the requests from all your keys that are running right now. The default is 2.- A streamed request counts until its stream ends.
- A request keeps counting after your app disconnects or times out, because DeepMask still completes it. A retry sent right after a timeout can therefore return
429withAPI_CONCURRENCY_LIMIT. - A new request over the limit returns
429withAPI_CONCURRENCY_LIMIT. - Many requests that arrive at the same moment can also return
429withAPI_CONCURRENCY_LIMIT, or409withAPI_ACCOUNT_BUSY, even below your limits. Spread them out over a short time.
Limit parallel requests in your app
To avoid429 errors, keep the number of requests your app runs in parallel at or below your Concurrent requests value. If several apps use your organization’s keys, they share this limit, so split it between them.
The examples below send three prompts but run at most two requests at a time. The Node.js example uses top-level await, so run it as an ES module, for example a .mjs file.
Per-key flood guard
Each key also has a flood guard of 120 requests per minute. This is twice the default Requests per minute limit, so with the default limits you normally reach your organization’s limit first. With higher limits, a single key can reach its flood guard first. See Requesting higher limits. The flood guard counts every request the key sends, including refused requests and calls toGET /v1/models. A client that retries too fast can therefore reach it even while your organization’s limits are refusing its requests. A request over the flood guard returns 429 with API_RATE_LIMIT.
A 429 from the flood guard includes a Retry-After header. Other 429 responses don’t. Responses also include RateLimit and RateLimit-Policy headers, for example RateLimit-Policy: 120;w=60. They describe only this per-key flood guard, not your organization’s Requests per minute, Tokens per minute, or Concurrent requests limits.
Per-key requests in flight
Each key can have up to 8 requests in flight at the same time. The request bodies that one key has in flight can total up to 64 MiB.- The count of requests in flight includes calls to
GET /v1/models. - A body sent without a
Content-Lengthheader counts as 32 MiB. - A request keeps counting until DeepMask finishes it, even if your client disconnects or times out.
- A request over either limit returns
429withAPI_CONCURRENCY_LIMIT. These refusals don’t appear in the request log.
429 until one of them finishes.
Model capacity
Each model has a capacity limit, shared by all organizations that use it through the API. DeepMask sets this limit, and the app doesn’t show it. When a model is at its capacity limit, requests for that model return429 with API_MODEL_CAPACITY. This can happen even when your organization is below its own limits. Your other models keep working. Retry with backoff, or switch to another model your key can use.
The model provider can also limit requests. You then get 429 with API_RATE_LIMIT, and you aren’t billed. Retry with backoff.
Token limits
Token limits apply to each request on its own. For each model’s values, see Available models.Context window
Your prompt and the model’s output share the model’s context window. A long prompt therefore leaves less room for output. DeepMask doesn’t check the length of your prompt itself. The model provider refuses a prompt that doesn’t fit with a4xx status and API_UPSTREAM_REJECTED, and you aren’t billed for it. See Available models for each model’s context window.
Maximum output tokens
max_tokens sets the maximum number of output tokens for a response. max_completion_tokens works the same way.
- Range: from 1 up to the model’s Max output tokens value, listed under Available models. A larger value returns
400withAPI_REQUEST_UNBOUNDED. - Default: for most models, DeepMask adds no limit of its own if you omit
max_tokens. The model’s default output length applies, and it can be lower than the model’s maximum.glm-4.7andglm-4.7-flashstop at 65,536 output tokens unless you setmax_tokens. - Reasoning tokens: hidden reasoning tokens count toward
max_tokensand are billed as output. See Reasoning and thinking tokens.
Request size and duration
These limits also apply to each request on its own.Request body and payload size
- Request body: a request body can be up to 32 MiB. A larger body returns
413withAPI_REQUEST_INVALID. - Uncompressed bodies only: a body with a
Content-Encodingheader such asgzipreturns400withAPI_REQUEST_INVALID. The OpenAI SDKs send uncompressed bodies. - Payload: messages, tool definitions,
tool_choice, andstoptogether can be up to 32 MiB as UTF-8 JSON. Above that, the request returns400withAPI_REQUEST_UNBOUNDED. The base64 text of your images counts toward both limits.
stop sequences, see Request body.
Images
On models with image input, each request can include up to 20 images and 20 MB of decoded image data. The count and the size include the images in every message, including earlier turns of the conversation. Each side of an image can be up to 8,000 pixels. The size limit for each image depends on the model. A request over an image limit returns400 with API_IMAGE_INVALID. See Image limits for the limits of each model.
Request duration
A request can run for up to 15 minutes. What happens at the limit depends on whether the response has started:- Before the response starts: the request returns
504withAPI_INFERENCE_UNAVAILABLE. - During a stream: DeepMask closes the stream without an error event. Treat a stream without a chunk that has a
finish_reasonas incomplete. See Errors in streamed responses.
timeout=960 in Python (seconds) or timeout: 960_000 in Node.js (milliseconds). See Retries in the OpenAI SDKs.
For tasks that need longer, ask for a shorter response or split the task into smaller requests.
Budget limits
Two spending caps limit what your keys can spend on model costs each month:- The Gateway spending cap covers all keys in your organization combined. The default is €100, and you can raise it up to the maximum cap (€500 by default).
- An optional key spending cap covers one key, up to your Gateway spending cap.
400 or 429 with API_SPENDING_CAP or API_KEY_SPENDING_CAP. Don’t retry it, because it keeps failing until the cap changes or resets. See Spending caps for how to set both caps, and Spending caps: 400 or 429 for the status codes.
Account limits
Above the limit for changes, the API page shows “Too many API management changes. Retry shortly.”
Fair use
All organizations share the capacity behind the DeepMask API. DeepMask keeps that capacity fair through the limits on this page, and it enforces them automatically:- Limits are per organization. Requests per minute, Tokens per minute, Concurrent requests, and the Gateway spending cap apply to all keys in your organization combined. Creating more keys doesn’t add to your organization’s limits.
- Each key has its own limits too. The flood guard and the limits on requests in flight are the same for every key. They stop a single key from sending more than the API can handle.
- Model capacity is shared. All organizations share each model’s capacity. A busy model can refuse your requests even when you’re below your own limits.
- Unclear outcomes have a 24-hour allowance. Up to 20 requests in any 24 hours that end without a clear result are released at no charge. On their own, they don’t pause your requests. Past that, each further one pauses your requests until DeepMask checks it. See Unclear outcomes.
- Retries need backoff. Retry refused requests with backoff instead of sending them again right away. See Retry with backoff.
- Chat and API limits are separate. API requests don’t count toward your DeepMask Chat message limits, and Chat messages don’t count toward your API limits.
Unclear outcomes
A request has an unclear outcome when it ends without a clear result. Examples are a server error or a timeout at the model provider, a stream that breaks off midway, and a request that reaches the 15-minute limit.- No charge: DeepMask releases a request with an unclear outcome at no charge. On its own, an unclear outcome doesn’t pause your organization’s requests.
- Late usage: if usage for that request still arrives later, DeepMask bills it like any other usage. This can happen when the provider finished the response after the error.
- Usage that needs checking: if DeepMask can’t verify that usage, for example because it’s recorded as a failed attempt, DeepMask checks it first. Your requests pause while it does. See When requests are paused.
- 24-hour allowance: up to 20 unclear outcomes for your organization in any 24 hours are released this way. All your keys count together. Unclear outcomes older than 24 hours stop counting. The app doesn’t show this count.
- Over the allowance: from the 21st unclear outcome within 24 hours, DeepMask checks each further one before it decides what to charge. Your organization’s API requests pause until DeepMask resolves them.
- After a pause: when DeepMask resolves a pause, the count doesn’t reset. Each further unclear outcome pauses your requests again until fewer than 20 remain within the last 24 hours.
503, and the API page shows a notice. Don’t retry in a loop. Contact support so DeepMask can resolve the requests. See When requests are paused for the codes your apps get and other reasons for a pause.
A disconnect or timeout in your own app isn’t an unclear outcome. If DeepMask already sent the request to the model, it still completes and bills it. If your app disconnects before that, the request isn’t sent or billed.
Retry with backoff
A request over a limit is refused right away, so an immediate retry usually fails again. It also counts toward your key’s flood guard. Retry with exponential backoff and jitter, and wait at least as long as anyRetry-After header says. Don’t retry spending cap codes, because they keep failing until the cap changes or resets. If all your keys return 503 with API_ADMISSION_UNAVAILABLE, check the API page for a notice before you retry.
See Retrying failed requests for which errors to retry and an example of your own retry logic.
Requesting higher limits
To ask for higher limits for your organization, contact support. You can ask for higher Requests per minute, Tokens per minute, and Concurrent requests values, and for a higher maximum Gateway spending cap.- Say which limits you need raised and describe your expected traffic, for example how many requests you send per minute and how many run at the same time.
- A higher Concurrent requests value helps only if Tokens per minute also allows that many requests to run at once. Each running request counts for about 2.0 to 2.3 million tokens, so mention both limits when you ask. See How tokens per minute works.
- Raising your organization’s limits doesn’t change the per-key limits. One key can still send up to 120 requests per minute and have 8 requests in flight. To use higher limits, spread your traffic over several keys.
- A higher limit doesn’t raise a model’s shared capacity, which DeepMask sets for each model. If you need many requests on one model at the same time, say so when you ask.
Related pages
Errors and retries
Look up error codes and learn which errors to retry.
Spending caps
Limit monthly spend for your organization and for each key.
Usage and logs
See your API limits, review requests, and export a CSV.
Models
Compare context windows, output limits, and image input by model.