Term
Rate Limit (AI)
Provider-enforced cap on requests or tokens per time window — it protects infrastructure and ensures fair usage across customers.
Rate Limit (AI) — in more detail
Rate limits are upper bounds that AI API providers set per account, model and time window. Two axes are usually enforced at once: requests per minute (RPM) and tokens per minute (TPM); some providers add a tokens-per-day cap (TPD) for daily quotas. Exceeding a limit returns HTTP 429 with a Retry-After header.
Example / practical context
Anyone processing high volumes against the OpenAI, Anthropic or Google APIs hits rate limits regularly. The standard mitigation is exponential backoff with jitter — wait after a 429 and retry with progressively longer pauses. For predictable high loads, providers offer tier systems (limits rise with usage history and prepayment) and alternative modes such as the batch API, which uses a separate, larger quota. On the application side, it helps to batch requests, pick smaller models for simple tasks and enforce a per-job token budget.
Delineation from similar terms
A rate limit is not the same as the context window: the context window caps tokens inside a single request, while the rate limit caps the sum across many requests. It is also distinct from a quota — typically a hard monthly ceiling, whereas rate limits operate over much shorter windows (seconds, minutes).
Entdecke mehr
Headless without an API bill — how do you reach the best AI models for automation in 2026?
Provider comparison mid-2026: who has a headless mode, whose subscription still covers it — and why BYOK is the most stable foundation.
Glossar$/MTok (Cost per Million Tokens)
Standard pricing unit for AI APIs — cost in US dollars per one million processed tokens. Listed separately for input, output and sometimes cache.
LexikonFunction Calling / Tool Use
How an LLM uses tools: define a tool as a schema, the model picks the function and arguments, the result returns to the chat — the basis of every agent.