Cost-Aware Inference

Follow this topic Ask about this topic

Momentum

24 papers in the last four weeks, up 50% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 183

All topics

One Policy, Any Budget: Internalizing Budget-Aware Search via Reinforcement Learning
Sep 1, 2026Xiaowei Sun, Jin Li, Yili Hong +2Cost-Aware Inference Constrained RL
College of Computer Science and Artiﬁcial Intelligence, Fudan University · College of Software Engineering, Southeast University · School of Artiﬁcial Intelligence, Shanghai Jiao Tong University
Drift-Aware LLM Routing with Sparse Contexts and Shared Budgets
Sep 1, 2026Cheung Hao Lee, Patrick WongCost-Aware Inference Non-Stationary Bandits
The Differential Reasoning Router: Operationalizing Cost-Aware LLM Annotation in E-commerce
Aug 31, 2026Cheng Lyu, Jingyue Zhang, Vinny DeGenova +2LLM-Assisted Annotation Human-in-the-Loop Annotation
Wayfair
The Shadow Price of Intelligence: Quality Degradation in LLM Inference as a Supply Chain Problem
Aug 25, 2026Elioth SanabriaLLM Serving LLM Inference Scheduling
Department of Decision and Technology Analytics Lehigh University - College of Business
SPADE: Speculative Decoding for Precise and Low Cost Distributed Edge Cloud Inference
Aug 13, 2026Divya Jyoti Bajpai, Kishan Kumar Upadhyay, Manjesh Kumar HanawalEdge Computing Cost-Aware Inference
MLiONS, Department of IEOR, IIT Bombay Mumbai, Maharashtra-400076, India
Error-Aware Reverse Auction Mechanism for Large Language Model Routing
Aug 13, 2026Haolong Chen, Zhengyuan Xin, Liang Zhang +2Cost-Aware Inference LLM Routing
Shenzhen International Center for Industrial and Applied Mathematics · Shenzhen Research Institute of Big Data · The Chinese University of Hong Kong, Shenzhen +2
How to Spend Your Oracle Budget: Practical Guidance for Protein Structure Prediction Models
Aug 12, 2026Aleksandra Kalisz, Jack Simons, Krisztina Sinkovics +4Protein Structure Prediction Inference-Time Optimization
InstaDeep Ltd · University of Oxford · Lancaster University
SCOPE-Router: Cost-Aware Open-Set VLM Routing for Execution-Oriented Tasks
Aug 12, 2026Tao Yu, Yifei Qu, Zhiqing Cui +14Adaptive Model Routing Cost-Aware Inference
1CASIA · 3NUS · 2UCAS +1
Total Recall at What Cost? Benchmarking the Serving Cost of Agentic Memory Systems
Aug 12, 2026Natchanon Pollertlam, Witchayut KornsuwannawitLLM Serving LLM Agent Memory
Bricks Technology · Thailand
Lifecycle-Optimal Tokenization: Vocabulary Size as a Deployment-Regime-Dependent Infrastructure Parameter
Aug 11, 2026Rima Mittal, Ankit Gubrani, Satyanarayana KakolluLLM Inference Efficiency LLM Serving
CARB: A Characterization-Guided Framework for CNN Inference Cost Prediction and Deployment Screening
Aug 11, 2026Linh Nguyen, Zhixin PanCost-Aware Inference Convolutional Neural Networks
Florida State University
RAISE: Diagnosing Acquisition Collapse in Costly LLM Signals
Aug 11, 2026Ying Yuan, Yu Wang, Yize Cheng +1Cost-Aware Inference LLM Routing
University of California, San Diego · Independent Researcher · University of Maryland, College Park
Beyond Detection Accuracy: Measuring Explanation Cost, Stability, and Utility for Resource-Aware IoT Intrusion Detection
Aug 11, 2026Abdurrahman TolaySHAP Feature Attribution IoT Intrusion Detection
Independent Researcher, Istanbul, Türkiye
BDH-CQ: In-Context Learning with Recurrent Latent Reasoning
Aug 10, 2026Björn Engdahl, Adrian Kosowski, Jan Chorowski +6Cost-Aware Inference Efficient Language Model Reasoning
New York University
Can We Optimize the Performance-Carbon Emission Break-Even Point?: The Quest for Greener LLMs
Aug 9, 2026Sourav Das, Tanmay Joshi, Kripabandhu GhoshFine-Tuning LLM Fine-Tuning
IIIT Kalyani · BITS Pilani Goa · IISER Kolkata
Task-to-Model Optimization for Enterprise LLM Coding Assistants: A Data-Driven Framework for Cost-Optimal Routing
Aug 9, 2026Srinivasan Manoharan, Junhua Zhao, Fangbo Tu +6Cost-Aware Inference LLM Routing
PayPal
CoBa: Cost-Effective Test-Time Scaling via Compute-Balanced Routing
Aug 7, 2026Yan Zhou, Yue Ouyang, Kaiyang Zheng +1Cost-Aware Inference Test-Time Scaling
School of Mathematics and Statistics, Changsha University of Science and Technology, Changsha, China · School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, China.
Pre-Inference Routing for Cost-Efficient Document Field Extraction
Aug 6, 2026Sreerekha RajendranAdaptive Model Routing Document Information Extraction
Independent Researcher Salt Lake City, Utah, United States
Activity Frames: Deterministic Screen-Activity Compilation for Agent Memory and Replay
Aug 6, 2026Nossa IyamuComputer-Use Agents LLM Agent Memory
Independent Researcher
Capability-Gated Planning: Cost-to-Goal Discovery and the Limits of Myopic Experiment Selection
Aug 5, 2026Ahmed Hassoon, Mark DredzeBelief-Space Planning Cost-Aware Inference
Johns Hopkins University
AFD-Ledger: Deployment Provisioning for Attention--FFN Disaggregation
Aug 5, 2026Chengyu Qiu, Xiao Fu, Fengcun Li +6Mixture-of-Experts Language Models Disaggregated LLM Serving
Tsinghua University · Meituan · Beijing, China
BAP-SQL: Budget-Aware Observation Planning for Agentic Text-to-SQL
Aug 3, 2026Chong Peng, Pin Qian, Su Wang +2Cost-Aware Inference Text-to-SQL
Microsoft · Carnegie Mellon University · Georgia Institute of Technology
Uncertainty Is Not Enough: Value-of-Information Routing for Mixtures of LoRA Experts
Aug 3, 2026Tom Saliencro, Rohan Desai, Priya Nair +2Adaptive Model Routing Selective Prediction
University of California, Irvine · University of Washington
BRiG-AFA: Bellman Risk-to-Go Learning for Non-Myopic Active Feature Acquisition
Aug 3, 2026Jiaorong Feng, Qian Li, Ying LiCost-Aware Inference Active Learning
Curtin Business School, Curtin University, Perth, Western Australia, Australia · †Present affiliation: Independent Researcher. · School of Electrical Engineering, Computing and Mathematical Sciences, Curtin University, Perth, Western Australia, Australia
CARE: A Cascaded Framework for Efficient and Reliable Time Series Anomaly Detection
Aug 3, 2026Zemin Chao, Qianhui Xu, Jianhe Cen +3Efficient Neural Network Inference Cost-Aware Inference
Massive Data Computing Lab, Harbin Institute of Technology
Computing with Agentic Oracles
Aug 2, 2026Jie WangCost-Aware Inference
Richard A. Miner School of Computing and Information Sciences, University of Massachusetts, Lowell, MA 01854, USA.
When Does LLM Orchestration Pay Off? A Controlled Evaluation of Accuracy, Cost, and Task Difficulty
Aug 1, 2026Nicolas Leins, Nico Pelleriti, Jana Gonnermann-Müller +1LLM Evaluation LLM Self-Refinement
Zuse Institute Berlin & TU Berlin, Berlin, Germany · Zuse Institute Berlin & Weizenbaum Institute Berlin, Berlin, Germany
HetRoute Heterogeneous and Cost-aware Collaborative Routing Framework for Distributed Edge MoE Inference
Aug 1, 2026Xin Yuan, Ning Li, Wenchao Xu +2Edge Inference Cost-Aware Inference
School of Ocean Engineering, Harbin Institute of Technology, Heilongjiang, China · School of Artificial Intelligence, University of Science and Technology Beijing, China · Division of Integrative Systems and Design, Hong Kong University of Science and Technology, Hong Kong +1
ARES: Adaptive Reasoning-Effort Steering for PPA- and Cost-Aware RTL Optimization with LLM Agents
Jul 30, 2026Stef Cuyckens, Mihaela Jivanescu, Jun Yin +2Inference-Time Optimization Large Language Model-Guided Optimization
KU Leuven Leuven, Belgium · Nokia Bell Labs Antwerp, Belgium
Budget-Aware LLM Discovery via Cost-Calibrated Frontier Utility
Jul 29, 2026Yansen Zhang, Yilu Liu, Tianyu Liu +6Inference-Time Search Automated Algorithm Discovery
International Digital Economy Academy (IDEA) · City University of Hong Kong · Mohamed bin Zayed University of Artificial Intelligence (MBZUAI)

Sep 1, 2026, cs.AIJ/K move · Enter open · S save

One Policy, Any Budget: Internalizing Budget-Aware Search via Reinforcement Learning

Xiaowei Sun, Jin Li, Yili Hong, Yikun Fu+1

College of Computer Science and Artiﬁcial Intelligence, Fudan University · College of Software Engineering, Southeast University · School of Artiﬁcial Intelligence, Shanghai Jiao Tong University

Cost-Aware Inference Constrained RL RL for LLM Agents RL for Tool Use

Open paper AskPDF arXiv

While reinforcement learning has enabled LLM-based search agents to invoke external tools, existing methods train under fixed budgets and cannot adapt when constraints vary at deployment. We propose AnySearch, a framework that enables a single policy to perform budget-aware search under any budget constraint through a training scaffold and curriculum reinforcement learning. In the first phase, we train the agent with explicit budget state injection and structured reasoning prompts that guide efficient allocation under linearly decaying budgets. In the second phase, the scaffold is removed and the agent learns to operate autonomously under adaptively sampled budget constraints, matching inference conditions. Both phases are optimized with a composite reward that couples answer accuracy with budget efficiency through absolute and relative signals, where an adaptive weight amplifies the efficiency signal for high-accuracy queries and attenuates it for low-accuracy ones. Extensive experiments on seven general and multi-hop QA benchmarks show that our method outperforms baselines across all budget scales, generalizes to unseen constraints beyond the training range, and achieves superior tool productivity without excessive token overhead. Our code is available at https://github.com/xwsun01/AnySearch.

Momentum

Related topics

Latest papers 183