Concept dictionary
Inference-Time Scaling
Inference-time scaling is deliberately giving a model or AI system more computation after a request arrives, such as through longer reasoning, multiple candidates, search, verification, or revision, to improve the result.
- Review state
- Reviewed in POC
- Freshness
- Current
- Version
- 1
Also called
- Inference-Time Scaling
- Test-Time Scaling
- Test-Time Compute Scaling
- Inference-Time Compute Scaling
Remember it this waySpend more compute on this answer, not on retraining the model.
In plain language
Instead of changing the model through more training, the system spends more effort on this particular request. It might explore several solution paths, use a verifier to compare them, revise an answer, or continue reasoning before it responds.
Why it matters
Inference-time scaling creates a new tradeoff among quality, latency, and cost. It can help on difficult reasoning tasks and let systems allocate extra effort selectively, but the method used to search or choose among answers matters as much as the raw amount of computation.
Example
For a difficult math problem, a system generates several candidate solutions, checks intermediate steps with a verifier, and spends its extra compute on the most promising path before returning one answer.
Common misconception
More inference compute does not guarantee a better answer. Longer output can contain more error, repeated samples can share the same mistake, and a weak selector can choose the wrong candidate. Research shows that verification and compute allocation are central to whether scaling helps.
Related terms
Sources
- Snell et al., “Scaling LLM Test-Time Compute Optimally Can Be More Effective than Scaling Parameters for Reasoning”studies search and adaptive response methods for allocating compute during inference.
- Setlur et al., “Scaling Test-Time Compute Without Verification or RL Is Suboptimal”examines why added test-time computation depends on effective verification and learning methods.
Learn more
- Contrast the one-request computation in [[inference-time-scaling|Inference-Time Scaling]] with broader [[scaling-laws|Scaling Laws]].
- Look for a dependable correctness signal through [[verification|Verification]] or a [[verifiable-domain|Verifiable Domain]].