Inference-Time Compute Allocation

This field studies how to distribute limited prediction-time resources—such as reasoning tokens, candidate samples, model calls, search steps, or GPU time—across questions, reasoning paths, and agent actions. Common approaches use difficulty or uncertainty estimates to route tasks, adapt sampling, guide verifier-based search, stop early, or escalate to stronger models; systems work also schedules requests against latency and budget constraints. Evaluations span mathematical reasoning and coding benchmarks as well as retrieval, embodied planning, and multi-tenant inference. A central challenge is that allocation gains depend on whether available signals predict the value of extra computation, and apparent improvements can vanish when costs, selection rules, runtime conditions, and held-out evaluation are controlled.