Inference-Time Compute Allocation

Follow this topic Ask about this topic

Momentum

10 papers in the last four weeks, against 2 the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 53

All topics

DriftSched: Adaptive QoS-Aware Scheduling under Runtime Token Drift for Multi-Tenant GPU Inference
Jun 2, 2026Kathiravan PalaniappanLLM Inference LLM Inference Serving
Independent Researcher (University of Colorado Colorado Springs Alumni)
ATLAS: Agentic Test-time Learning-to-Allocate Scaling
Jun 1, 2026Peijia Qin, Qi Cao, Pengtao XieTest-Time Scaling Inference-Time Compute Allocation
University of California, San Diego
CRB-Guided Framework Design and Resource Allocation for Indoor mmWave ISCC Systems
May 28, 2026Zhonghao Liu, Yahao Ding, Yinchao Yang +1Integrated Sensing and Communication Wireless Resource Allocation
∗King’s College London
Uncertainty-Aware Budget Allocation for Adaptive Test-Time Reasoning
May 26, 2026Manh Nguyen, Sunil Gupta, Hung LeEfficient Language Model Reasoning Adaptive Sampling
Applied Artificial Intelligence Initiative Deakin University, Australia
Anticipate and Learn: Unleashing Idle-Time Compute in Proactive Agents
May 25, 2026Haoyi Hu, Qirong Lyu, Xianghan Kong +7AI Agent Evaluation AI Agent Benchmarks
1Shanghai Jiao Tong University · 2Tencent
Computational Challenges in Token Economics: Bridging Economic Theory and AI System Design
May 17, 2026Ou Wu, Yingjun DengLLM Inference Cost-Aware Inference
Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences, China · Hefei Institutes of Physical Science, Chinese Academy of Sciences, China
Reasoning Can Be Restored by Correcting a Few Decision Tokens
May 16, 2026Changshuo Shen, Leheng Sheng, Yuxin Chen +2Efficient Language Model Reasoning Inference-Time Compute Allocation
University of Science and Technology of China, Hefei, China · National University of Singapore, Singapore
Nice Fold or Hero Call: Learning Budget-Efficient Thinking for Adaptive Reasoning
May 12, 2026Zhaomeng Zhou, Lan Zhang, Junyang Wang +2RL for Language Model Reasoning Efficient Language Model Reasoning
University of Science and Technology of China · The Chinese University of Hong Kong
Compute Where it Counts: Self Optimizing Language Models
May 11, 2026Yash Akhauri, Mohamed S. AbdelfattahLLM Inference Efficiency RL for Language Models
Cornell University
Statistical Scouting Finds Debate-Safe but Not Debate-Useful Cases: A Matched-Ceiling Study of Open-Weight LLM Reasoning Protocols
May 10, 2026Julia Hu, Alfred Shen, Kumar LakshmipathiLLM Evaluation Multi-Agent Debate
Amazon Web Services
When Independent Sampling Outperforms Agentic Reasoning
May 8, 2026Yihe Dong, Boris ShigidaCost-Aware Inference Competitive Programming
Princeton University
CA-SQL: Complexity-Aware Inference Time Reasoning for Text-to-SQL via Exploration and Compute Budget Allocation
May 8, 2026James Petullo, Nianwen XueInference-Time Search Inference-Time Scaling
Brandeis University / Waltham, MA
Same Signal, Opposite Meaning: Direction-Informed Adaptive Learning for LLM Agents
May 7, 2026Ziming Li, Jiatan Huang, Xiaoguang Guo +2LLM Agent Training Inference-Time Compute Allocation
University of Connecticut · New Jersey Institute of Technology
On Time, Within Budget: Constraint-Driven Online Resource Allocation for Agentic Workflows
May 7, 2026Xinglin Wang, Zishen Liu, Shaoxiong Feng +9Stochastic Optimization Cost-Aware Inference
School of Computer Science, Beijing Institute of Technology · Xiaohongshu Inc
BitCal-TTS: Bit-Calibrated Test-Time Scaling for Quantized Reasoning Models
May 7, 2026Sai Babu Patarlapalli, Surya Teja AvvaruLLM Inference 4-Bit Quantization
Adaptive Test-Time Compute Allocation with Evolving In-Context Demonstrations
Apr 22, 2026Bowen Zuo, Dongruo Zhou, Yinglun ZhuEfficient Inference Test-Time Optimization
University of California, Riverside · 2Indiana University Bloomington
Weak-Link Optimization for Multi-Agent Reasoning and Collaboration
Apr 17, 2026Haoyu Bian, Chaoning Zhang, Jiaquan Zhang +4Multi-Agent Collaboration Inference-Time Compute Allocation
University of Electronic Science and Technology of China, Chengdu 611731, China · Beihang University, Beijing 100191, China · Xi’an University of Architecture and Technology, Xi’an 710064, China
Adaptive Test-Time Compute Allocation for Reasoning LLMs via Constrained Policy Optimization
Apr 16, 2026Zhiyuan Zhai, Bingcong Li, Bingnan Xiao +2Test-Time Scaling Constrained Optimization
Fudan University · ETH Zurich · Guangming Lab
What If We Allocate Test-Time Compute Adaptively?
Feb 1, 2026Ahsan Bilal, Ahmed Mohsin, Muhammad Umer +4Inference-Time Search Test-Time Scaling
University of Oklahoma, OK, USA · Stanford University, CA, USA · Universitat Pompeu Fabra, Barcelona, Spain +2
More Bang for the Buck: Improving the Inference of Large Language Models at a Fixed Budget using Reset and Discard (ReD)
Jan 29, 2026Sagi Meir, Tommer D. Keidar, Noam Levi +2LLM Inference Efficiency LLM Evaluation
School of Chemistry, Tel Aviv University, Tel Aviv 6997801, Israel · The Center for Physics and Chemistry of Living Systems, Tel Aviv University, Tel Aviv 6997801, Israel · School of Physics and Astronomy, Tel Aviv University, Tel Aviv 6997801, Israel +1
Optimal Self-Consistency for Efficient Reasoning with Large Language Models
Nov 15, 2025Austin Feng, Marius Alonso, Ambroise Odonnat +2Self-Consistency Decoding Inference-Time Scaling
1Yale University, New Haven, CT, USA · 2Noah’s Ark Lab, Paris, France · 3Inria, Rennes, France +1
Think Right: Learning to Mitigate Under-Over Thinking via Adaptive, Attentive Compression
Oct 2, 2025Joykirat Singh, Justin Chih-Yao Chen, Archiki Prasad +3RL for Language Model Reasoning LLM Inference Acceleration
UNC Chapel Hill · The University of Texas at Austin · Microsoft Research
TIP-Search: Time-Predictable Inference Scheduling for Market Prediction under Uncertain Load
May 30, 2025Xibai WangOnline Convex Optimization Real-Time Scheduling
NeuroQuant Labs Limited Hong Kong SAR, China

Jun 2, 2026, cs.PFJ/K move · Enter open · S save

DriftSched: Adaptive QoS-Aware Scheduling under Runtime Token Drift for Multi-Tenant GPU Inference

Kathiravan Palaniappan

Independent Researcher (University of Colorado Colorado Springs Alumni)

LLM Inference LLM Inference Serving Inference-Time Compute Allocation

Open paper AskPDF arXiv

The rapid growth of large language model (LLM) inference services has increased the demand for efficient multi-tenant GPU scheduling. While modern inference runtimes such as vLLM improve throughput through continuous batching and optimized memory management, accurately estimating the runtime cost of heterogeneous inference requests remains challenging. In practice, admission-time workload estimates may deviate from observed execution behavior, leading to workload misclassification, queue imbalance, increased tail latency, and degraded Quality-of-Service (QoS). This paper presents DriftSched, a QoS-aware scheduling framework for multi-tenant LLM inference serving on NVIDIA L4 GPUs. DriftSched combines workload classification, token-budget estimation, tenant-aware queue management, and an online feedback mechanism to refine workload estimates using runtime observations. The framework evaluates FIFO, Priority, Weighted, Shortest-Job-First (SJF), and Aging Priority scheduling policies under heterogeneous multi-tenant workloads. Experimental results show that adaptive calibration reduces workload estimation error by an average of 38.8% (MAE) and 40.5% (RMSE), improving workload classification stability. Among all evaluated schedulers, SJF achieves the best overall performance, reducing median end-to-end latency by approximately 42% and P99 latency by approximately 16% relative to FIFO under sustained GPU contention. The results further indicate that scheduler selection has a greater impact on latency behavior than runtime calibration alone, while accurate workload characterization largely eliminates systematic estimation drift. This work contributes a reproducible framework for studying workload-estimation fidelity and QoS-aware scheduling in multi-tenant GPU inference systems.

Momentum

Related topics

Latest papers 53