Skip to main content
This page collects the main limits that apply to the DeepMask API. It covers how much traffic your organization can send, how large each request can be, and how much your keys can spend. A request that DeepMask refuses because of a limit isn’t billed. For how DeepMask keeps shared capacity fair for everyone, see Fair use. Most limits are shared by all keys in your organization. After you enable API billing, organization admins see the current traffic limits in the API limits card on the Usage tab. See API limits. For what each error code means and how to handle it, see Errors and retries.

Limits at a glance

Tokens per minute counts each running request at about 2.0 to 2.3 million tokens, whatever its size, so the default allows two requests at a time. See How tokens per minute works. The values for Requests per minute, Tokens per minute, and Concurrent requests are defaults. If DeepMask raised them for your organization, the API limits card shows your values. For limits on keys, logs, and exports, see Account limits.

Token throughput

Tokens per minute limits your organization’s token throughput. All your keys share it, and the default is 5,000,000. DeepMask applies it in two ways:
  • Running requests: the requests from all your keys that are running right now can’t count for more than your Tokens per minute value. A new request that would go over it returns 429 with API_TPM_LIMIT.
  • Actual tokens: your organization’s actual token use per minute is also limited to the Tokens per minute value. A request refused for that reason gets 429 with API_RATE_LIMIT.

How tokens per minute works

While a request is running, DeepMask counts it toward Tokens per minute at a fixed size for its model, not at its actual size. That size is about 2.0 to 2.3 million tokens, depending on the model’s maximum output. A request stops counting as soon as it finishes. A short prompt therefore counts the same as a long one, and a lower max_tokens value doesn’t lower the count. In practice, this makes Tokens per minute act as a second limit on parallel requests. With the default limit of 5,000,000, two requests can run at the same time, whichever models they use. A third request doesn’t fit until one of them finishes. That matches the default Concurrent requests limit, so you usually reach API_CONCURRENCY_LIMIT first. If your organization has a higher concurrency limit, API_TPM_LIMIT can be the limit you reach instead. To get the most out of your throughput:
  • Run at most as many requests in parallel as your Concurrent requests value allows. See Limit parallel requests in your app.
  • Ask for shorter responses when you can. A request that finishes sooner frees its share of Tokens per minute sooner.
  • Don’t start requests you might abandon. Once DeepMask has sent a request to the model, disconnecting doesn’t stop it. DeepMask completes it, and it keeps counting toward Tokens per minute until it finishes.

Rate limits

DeepMask limits how many requests your organization, each key, and each model can handle. When a request goes over one of these limits, it’s refused right away with a 429 error. It isn’t queued. See 429: rate and capacity limits for each code.

Requests per minute

Requests per minute counts the requests from all your keys in the last 60 seconds. The default is 60.
  • Requests that DeepMask refuses before processing them don’t count, for example for invalid credentials, an invalid request, a reached limit, or a spending cap.
  • Requests refused later, for example by the model provider, still count.
  • Requests blocked by a spending cap still appear in the request log, even though they don’t count here.
A request over the limit returns 429 with API_RPM_LIMIT. Some requests over your organization’s request volume get API_RATE_LIMIT instead. Handle both the same way: slow down and retry with backoff.

Concurrent requests

Concurrent requests counts the requests from all your keys that are running right now. The default is 2.
  • A streamed request counts until its stream ends.
  • A request keeps counting after your app disconnects or times out, because DeepMask still completes it. A retry sent right after a timeout can therefore return 429 with API_CONCURRENCY_LIMIT.
  • A new request over the limit returns 429 with API_CONCURRENCY_LIMIT.
  • Many requests that arrive at the same moment can also return 429 with API_CONCURRENCY_LIMIT, or 409 with API_ACCOUNT_BUSY, even below your limits. Spread them out over a short time.

Limit parallel requests in your app

To avoid 429 errors, keep the number of requests your app runs in parallel at or below your Concurrent requests value. If several apps use your organization’s keys, they share this limit, so split it between them. The examples below send three prompts but run at most two requests at a time. The Node.js example uses top-level await, so run it as an ES module, for example a .mjs file.

Per-key flood guard

Each key also has a flood guard of 120 requests per minute. This is twice the default Requests per minute limit, so with the default limits you normally reach your organization’s limit first. With higher limits, a single key can reach its flood guard first. See Requesting higher limits. The flood guard counts every request the key sends, including refused requests and calls to GET /v1/models. A client that retries too fast can therefore reach it even while your organization’s limits are refusing its requests. A request over the flood guard returns 429 with API_RATE_LIMIT. A 429 from the flood guard includes a Retry-After header. Other 429 responses don’t. Responses also include RateLimit and RateLimit-Policy headers, for example RateLimit-Policy: 120;w=60. They describe only this per-key flood guard, not your organization’s Requests per minute, Tokens per minute, or Concurrent requests limits.

Per-key requests in flight

Each key can have up to 8 requests in flight at the same time. The request bodies that one key has in flight can total up to 64 MiB.
  • The count of requests in flight includes calls to GET /v1/models.
  • A body sent without a Content-Length header counts as 32 MiB.
  • A request keeps counting until DeepMask finishes it, even if your client disconnects or times out.
  • A request over either limit returns 429 with API_CONCURRENCY_LIMIT. These refusals don’t appear in the request log.
The limit of 8 requests is four times the default Concurrent requests limit, so with the default limits you normally reach your organization’s limit first. With large requests, such as requests with images, the 64 MiB limit can come first. For example, two requests with 20 MB of images each count about 54 MB together. A third such request returns 429 until one of them finishes.

Model capacity

Each model has a capacity limit, shared by all organizations that use it through the API. DeepMask sets this limit, and the app doesn’t show it. When a model is at its capacity limit, requests for that model return 429 with API_MODEL_CAPACITY. This can happen even when your organization is below its own limits. Your other models keep working. Retry with backoff, or switch to another model your key can use. The model provider can also limit requests. You then get 429 with API_RATE_LIMIT, and you aren’t billed. Retry with backoff.

Token limits

Token limits apply to each request on its own. For each model’s values, see Available models.

Context window

Your prompt and the model’s output share the model’s context window. A long prompt therefore leaves less room for output. DeepMask doesn’t check the length of your prompt itself. The model provider refuses a prompt that doesn’t fit with a 4xx status and API_UPSTREAM_REJECTED, and you aren’t billed for it. See Available models for each model’s context window.

Maximum output tokens

max_tokens sets the maximum number of output tokens for a response. max_completion_tokens works the same way.
  • Range: from 1 up to the model’s Max output tokens value, listed under Available models. A larger value returns 400 with API_REQUEST_UNBOUNDED.
  • Default: for most models, DeepMask adds no limit of its own if you omit max_tokens. The model’s default output length applies, and it can be lower than the model’s maximum. glm-4.7 and glm-4.7-flash stop at 65,536 output tokens unless you set max_tokens.
  • Reasoning tokens: hidden reasoning tokens count toward max_tokens and are billed as output. See Reasoning and thinking tokens.
See Output length for more detail.

Request size and duration

These limits also apply to each request on its own.

Request body and payload size

  • Request body: a request body can be up to 32 MiB. A larger body returns 413 with API_REQUEST_INVALID.
  • Uncompressed bodies only: a body with a Content-Encoding header such as gzip returns 400 with API_REQUEST_INVALID. The OpenAI SDKs send uncompressed bodies.
  • Payload: messages, tool definitions, tool_choice, and stop together can be up to 32 MiB as UTF-8 JSON. Above that, the request returns 400 with API_REQUEST_UNBOUNDED. The base64 text of your images counts toward both limits.
See Request size for how much room images leave for the rest of a request. For limits on individual fields, such as up to 128 tools or 4 stop sequences, see Request body.

Images

On models with image input, each request can include up to 20 images and 20 MB of decoded image data. The count and the size include the images in every message, including earlier turns of the conversation. Each side of an image can be up to 8,000 pixels. The size limit for each image depends on the model. A request over an image limit returns 400 with API_IMAGE_INVALID. See Image limits for the limits of each model.

Request duration

A request can run for up to 15 minutes. What happens at the limit depends on whether the response has started:
  • Before the response starts: the request returns 504 with API_INFERENCE_UNAVAILABLE.
  • During a stream: DeepMask closes the stream without an error event. Treat a stream without a chunk that has a finish_reason as incomplete. See Errors in streamed responses.
A request that reaches the limit has an unclear outcome. See Unclear outcomes for how DeepMask handles it. The OpenAI SDKs time out after 10 minutes by default and then retry the request. DeepMask bills both requests. Set the client timeout a little above 15 minutes: timeout=960 in Python (seconds) or timeout: 960_000 in Node.js (milliseconds). See Retries in the OpenAI SDKs. For tasks that need longer, ask for a shorter response or split the task into smaller requests.

Budget limits

Two spending caps limit what your keys can spend on model costs each month:
  • The Gateway spending cap covers all keys in your organization combined. The default is €100, and you can raise it up to the maximum cap (€500 by default).
  • An optional key spending cap covers one key, up to your Gateway spending cap.
Both caps count model costs before the 3% fee and tax. Both reset at 00:00 UTC on the 1st of each month. A request over a cap is refused before any model runs, so it isn’t billed. Requests that are already running can still take spend slightly over a cap. A cap refusal returns 400 or 429 with API_SPENDING_CAP or API_KEY_SPENDING_CAP. Don’t retry it, because it keeps failing until the cap changes or resets. See Spending caps for how to set both caps, and Spending caps: 400 or 429 for the status codes.

Account limits

Above the limit for changes, the API page shows “Too many API management changes. Retry shortly.”

Fair use

All organizations share the capacity behind the DeepMask API. DeepMask keeps that capacity fair through the limits on this page, and it enforces them automatically:
  • Limits are per organization. Requests per minute, Tokens per minute, Concurrent requests, and the Gateway spending cap apply to all keys in your organization combined. Creating more keys doesn’t add to your organization’s limits.
  • Each key has its own limits too. The flood guard and the limits on requests in flight are the same for every key. They stop a single key from sending more than the API can handle.
  • Model capacity is shared. All organizations share each model’s capacity. A busy model can refuse your requests even when you’re below your own limits.
  • Unclear outcomes have a 24-hour allowance. Up to 20 requests in any 24 hours that end without a clear result are released at no charge. On their own, they don’t pause your requests. Past that, each further one pauses your requests until DeepMask checks it. See Unclear outcomes.
  • Retries need backoff. Retry refused requests with backoff instead of sending them again right away. See Retry with backoff.
  • Chat and API limits are separate. API requests don’t count toward your DeepMask Chat message limits, and Chat messages don’t count toward your API limits.
If your workload needs more than your limits allow, ask for higher limits. See Requesting higher limits.

Unclear outcomes

A request has an unclear outcome when it ends without a clear result. Examples are a server error or a timeout at the model provider, a stream that breaks off midway, and a request that reaches the 15-minute limit.
  • No charge: DeepMask releases a request with an unclear outcome at no charge. On its own, an unclear outcome doesn’t pause your organization’s requests.
  • Late usage: if usage for that request still arrives later, DeepMask bills it like any other usage. This can happen when the provider finished the response after the error.
  • Usage that needs checking: if DeepMask can’t verify that usage, for example because it’s recorded as a failed attempt, DeepMask checks it first. Your requests pause while it does. See When requests are paused.
  • 24-hour allowance: up to 20 unclear outcomes for your organization in any 24 hours are released this way. All your keys count together. Unclear outcomes older than 24 hours stop counting. The app doesn’t show this count.
  • Over the allowance: from the 21st unclear outcome within 24 hours, DeepMask checks each further one before it decides what to charge. Your organization’s API requests pause until DeepMask resolves them.
  • After a pause: when DeepMask resolves a pause, the count doesn’t reset. Each further unclear outcome pauses your requests again until fewer than 20 remain within the last 24 hours.
During a pause, requests from all your keys return 503, and the API page shows a notice. Don’t retry in a loop. Contact support so DeepMask can resolve the requests. See When requests are paused for the codes your apps get and other reasons for a pause. A disconnect or timeout in your own app isn’t an unclear outcome. If DeepMask already sent the request to the model, it still completes and bills it. If your app disconnects before that, the request isn’t sent or billed.
To stay well within the allowance, don’t resend a request in a loop when it keeps failing with 502 or 504, or when its stream keeps breaking off. Wait and retry with backoff, switch to another model, or ask for a shorter response. The OpenAI SDKs retry 502 and 504 automatically, twice by default. Each retry that fails the same way counts as another unclear outcome, so one failing call can count up to three times. For long or repeatedly failing requests, set a lower max_retries (Python) or maxRetries (Node.js).

Retry with backoff

A request over a limit is refused right away, so an immediate retry usually fails again. It also counts toward your key’s flood guard. Retry with exponential backoff and jitter, and wait at least as long as any Retry-After header says. Don’t retry spending cap codes, because they keep failing until the cap changes or resets. If all your keys return 503 with API_ADMISSION_UNAVAILABLE, check the API page for a notice before you retry. See Retrying failed requests for which errors to retry and an example of your own retry logic.

Requesting higher limits

To ask for higher limits for your organization, contact support. You can ask for higher Requests per minute, Tokens per minute, and Concurrent requests values, and for a higher maximum Gateway spending cap.
  • Say which limits you need raised and describe your expected traffic, for example how many requests you send per minute and how many run at the same time.
  • A higher Concurrent requests value helps only if Tokens per minute also allows that many requests to run at once. Each running request counts for about 2.0 to 2.3 million tokens, so mention both limits when you ask. See How tokens per minute works.
  • Raising your organization’s limits doesn’t change the per-key limits. One key can still send up to 120 requests per minute and have 8 requests in flight. To use higher limits, spread your traffic over several keys.
  • A higher limit doesn’t raise a model’s shared capacity, which DeepMask sets for each model. If you need many requests on one model at the same time, say so when you ask.
Once DeepMask changes your limits, the API limits card on the Usage tab shows the new values. Click Refresh to see them.

Errors and retries

Look up error codes and learn which errors to retry.

Spending caps

Limit monthly spend for your organization and for each key.

Usage and logs

See your API limits, review requests, and export a CSV.

Models

Compare context windows, output limits, and image input by model.