Term
Thinking Budget
The thinking budget (token budget) is the maximum number of internal reasoning tokens a reasoning model may spend on "thinking" per request — controlling the trade-off between answer depth and cost or latency.
Thinking Budget — explained in more detail
Before producing the actual answer, reasoning models generate an internal chain of intermediate reasoning steps (reasoning tokens — see Chain-of-Thought). These tokens are usually hidden from the user, yet they are produced, consume compute time, and — crucially — they are typically billed like ordinary output tokens. The thinking budget is the cap on this: it sets how many tokens the model may invest at most in that internal thinking before it writes the final answer.
The lever behind it is a direct trade-off. A larger budget gives the model more room to work through multi-step problems cleanly and raises accuracy on hard tasks (mathematics, code analysis, longer proof chains). At the same time it increases latency and cost. A tight budget yields faster, cheaper answers but can produce premature or incomplete results on demanding tasks. In practice the budget is not a “more is always better” dial: beyond a task-dependent point, extra thinking adds little value and just burns tokens.
Example / Practical context
Providers implement the concept differently. Google Gemini expects a concrete token count for thinking_budget as a soft upper limit. Anthropic long used the same hard-token-cap logic with budget_tokens (extended thinking); in current models (as of June 2026, from Opus 4.6 onward) that fixed token value is replaced by “adaptive thinking” plus an effort parameter (low/medium/high/max) — budget_tokens is rejected there. OpenAI drops the token count entirely and controls reasoning depth solely via reasoning_effort (low/medium/high), thereby setting the internal budget indirectly.
Cost relevance: because reasoning tokens are billed like output tokens, a visibly short 500-token answer can total 2,000 to 3,000 charged tokens once the thinking is counted. In workloads such as code review, practitioners report a 4-to-5x token overhead — which is why the budget is increasingly planned as a real line item.
Distinction from related concepts
The thinking budget caps the internal reasoning tokens before the answer; max_tokens (the output-token limit) instead caps the visible answer length — they are different limits and can clip one another. Reasoning Effort is the abstract, tier-based variant of the same control (low/medium/high instead of an exact token count). Chain-of-Thought describes the underlying step-by-step reasoning method that the thinking budget caps in quantity.
Sources
- Anthropic, Extended Thinking & adaptive thinking / effort (as of June 2026)
- Google Cloud, Vertex AI — Thinking (
thinking_budget) - OpenAI,
reasoning_effort(low/medium/high)
Entdecke mehr
AI Workflows by Keyword: How We Make Recurring Routines Enforceable
A typed keyword triggers a fixed AI routine — and every single step must be committed before the next one appears. Why that's the actual trick.
GlossarReasoning Effort
Reasoning Effort is a control parameter on modern reasoning models that sets how much internal step-by-step thinking (thinking tokens) a model spends before answering — higher levels raise quality but also latency and cost.
LexikonLLM Hallucinations — Causes and Remedies
Why LLMs confidently invent falsehoods, what types of hallucinations exist, and which remedies actually help — RAG, source enforcement, verification.