Inference Cost

Follow this topic Ask about this topic

Recent momentum

-57%

9 papers in the last 28 days · 0.1% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this topic, kept on the site without email delivery.

Period ending 2026-09-21

2 new papers

A weekly snapshot of new work published in Inference Cost.

Period ending 2026-09-14

3 new papers

A weekly snapshot of new work published in Inference Cost.

Period ending 2026-09-07

4 new papers

A weekly snapshot of new work published in Inference Cost.

114 papers

Latest in Inference Cost

Browse all topics →

Visual-OPSD: Cross-Modal On-Policy Self-Distillation for Efficient Unified Multimodal Reasoning
Jun 17, 2026Pengyu Li, Zhitao Gao, Lingling Zhang +5Multimodal Reasoning Visual Reasoning
Towards Scalable Customization and Deployment of Multi-Agent Systems for Enterprise Applications
Jun 16, 2026Paresh Dashore, Shreyas Kulkarni, Uttam Gurram +5Multi-Agent Systems Enterprise Systems
Stepwise Token Selection for Efficient Multimodal Large Language Models
Jun 14, 2026Landi He, Shawn Young, Lijian XuVisual Token Pruning Unified Multimodal Models
Are Online Skill and Memory Modules Always Worth Their Tokens? A Budget-Constrained Study of Web Agents
Jun 12, 2026Sina Hajimiri, Masih Aminbeidokhti, Jose Dolz +4Token Budgets Inference Cost
AliyunConsoleAgent: Training Web Agents in Real-World Cloud Environments via Distillation and Reinforcement Learning
Jun 8, 2026Bojie Rong, Zheyu Shen, Qiaoping Wang +7Cp-Agent Cloud
Fast and Robust On-Device Speaker Diarization: Relative Minimum Cluster Size for Stride-Accelerated Pipelines
Jun 7, 2026Fumiaki YamaguchiSpeaker Diarization Inference Cost
What Your Posts Reveal: A Benchmark and Agentic Framework for User-Level Privacy Leakage on Social Media
Jun 5, 2026Zifan Peng, Yini Huang, Aiwen Lu +7Efficient Privacy Loss AI Privacy Risks and Protection
CRAFT: Cost-aware Refinement And Front-aware Tuning of Prompts
Jun 3, 2026Shanu Kumar, Shubhanshu Khandelwal, Akhila Yesantarao Venkata +3Cost-Aware Inference Cost
DOT-MoE: Differentiable Optimal Transport for MoEfication
Jun 1, 2026Udbhav Bamba, Arnav Chavan, Aryamaan Thakur +2Mixture-Of-Expert Inference Inference Cost
Learning to Construct Practical Agentic Systems
May 29, 2026Aditya Kumar, Zhihan Lei, Jerry Yan +6Agentic Systems Inference Cost
UI-KOBE: Knowledge-Oriented Behavior Exploration for Lightweight Graph-Guided GUI Agents
May 28, 2026Yuxiang Chai, Han Xiao, Xinyu Fu +3Graphical User Interface Agents Graphical User Interface
CosmicFish-HRM: Adaptive Reasoning via Hierarchical Recurrent Mechanisms in Compact Language Models
May 27, 2026Venkat Akhil LakkapragadaLarge Models Inference Cost
Pair-In, Pair-Out: Latent Multi-Token Prediction for Efficient LLMs
May 26, 2026Wenhui Tan, Minghao Li, Xiaoqian Ma +5Next-Token Prediction LLM Inference Efficiency
Uncertainty-Aware Budget Allocation for Adaptive Test-Time Reasoning
May 26, 2026Manh Nguyen, Sunil Gupta, Hung LeLarge Language Model Reasoning Token Budgets
Argo: Efficient Importance Labeling for Enterprise Email Systems
May 20, 2026Siddhant Ray, Ganesh Ananthanarayanan, Kevin Chian +5Multi-Label Classification Phishing
The Economics of AI Inference: Inflation Dynamics, Welfare Costs, and Optimal Monetary Policy under the Inference-Cost Phillips Curve
May 19, 2026Gustav Olaf Yunus Laitinen-Fredriksson Lundström-ImanovOnline Inference Economies
Prompt Compression in Diffusion Large Language Models: Evaluating LLMLingua-2 on LLaDA
May 18, 2026Sterling Huang, Abigayle Brown, Jiyoo Noh +4Large Language Model Compression Diffusion Language Models
PAIR: Prefix-Aware Internal Reward Model for Multi-Turn Agent Optimization
May 18, 2026Wonjoong Kim, Yeonjun In, Sangwu Park +2Group Relative Policy Optimization External Reward Models
TabPFN-MT: A Natively Multitask In-Context Learner for Tabular Data
May 16, 2026Cormac Cureton, Narges ArmanfardTabular Learning Tabpfn
BoLT: A Benchmark to Democratize Black-box Optimization Research for Expensive LLM Tasks
May 16, 2026Ruth Wan Theng Chew, Zhiliang Chen, Apivich Hemachandra +1Benchmarking Large Language Models Bayesian Optimization
Effort as Ceiling, Not Dial: Reasoning Budget Does Not Modulate Cognitive Cost Alignment Between Humans and Large Reasoning Models
May 16, 2026Yueqing Hu, Tianhong WangLarge Reasoning Models Inference Cost
Context, Reasoning, and Hierarchy: A Cost-Performance Study of Compound LLM Agent Design in an Adversarial POMDP
May 15, 2026Igor Bogdanov, Chung-Horng Lung, Thomas Kunz +3Large Language Model Agents Partially Observable Markov Decision Process
MO-CAPO: Multi-Objective Cost-Aware Prompt Optimization
May 15, 2026Jan Büssing, Moritz Schlager, Timo Heiß +2Inference Cost Prompt Engineering
Think When Needed: Adaptive Reasoning-Driven Multimodal Embeddings with a Dual-LoRA Architecture
May 14, 2026Longxiang Zhang, Weilong Dai, Guanghao Zhang +2Multimodal Embeddings Multimodal Learning
G $^2$ TR: Generation-Guided Visual Token Reduction for Separate-Encoder Unified Multimodal Models
May 12, 2026Junxian Li, Kai Liu, Zizhong Ding +4Visual Tokens Unified Multimodal Models
Rethinking Supervision Granularity: Segment-Level Learning for LLM-Based Theorem Proving
May 12, 2026Shuo Xu, Jiakun Zhang, Junyu Lai +2Theorem Step-Level Supervision
OTT-Vid: Optimal Transport Temporal Token Compression for Video Large Language Models
May 12, 2026Minseok Kang, Minhyeok Lee, Jungho Lee +6Token Compression Neural Video Codecs
UCCI: Calibrated Uncertainty for Cost-Optimal LLM Cascade Routing
May 11, 2026Varun KotteLarge Language Model Routing Inference Cost
Uncertainty-Aware Token Importance Estimation in Spiking Transformers
May 10, 2026Wenxuan Liu, Zecheng Hao, Tong Bu +2Transformer Architectures Importance
AESOP: Adversarial Execution-path Selection to Overload Deep Learning Pipelines
May 9, 2026Tingxi Li, Mingfang Ji, Ravishka Shemal Rathnasuriya +3Online Inference Multiple Attack Surfaces
Arrow: A Foundation Model for Causal Discovery
May 8, 2026Ryan Thompson, He Zhao, Daniel M. Steinberg +1Causal Foundation Models Causal Attention
Switchcraft: AI Model Router for Agentic Tool Calling
May 8, 2026Sharad Agarwal, Pooria Namyar, Alec Wolman +3Large Language Model Routing Agentic Tool Use
Policy-Guided Stepwise Model Routing for Cost-Effective Reasoning
May 7, 2026Wenwen Si, Insup Lee, Osbert BastaniInference Cost Chain-of-Thought Reasoning
Towards Generation-Efficient Uncertainty Estimation in Large Language Models
May 7, 2026Mingcheng Zhu, Yu Liu, Tingting ZhuLarge Language Model Uncertainty Inference Cost
Budgeted Attention Allocation: Cost-Conditioned Compute Control for Efficient Transformers
May 7, 2026Amrit NidhiTransformer Attention Transformer Architectures
A Few Good Clauses: Comparing LLMs vs Domain-Trained Small Language Models on Structured Contract Extraction
May 7, 2026Nicole Lincoln, Nick Whitehouse, Jaron Mar +1Small Language Models Legal Domain
Zero-Shot Confidence Estimation for Small LLMs: When Supervised Baselines Aren't Worth Training
May 4, 2026Luong N. NguyenLarge Language Model Uncertainty Zero-Shot Benchmark
Quantization Undoes Alignment: Bias Emergence in Compressed LLMs Across Models and Precision Levels
May 2, 2026Plawan Kumar Rath, Rahul MaliakkalLarge Language Model Quantization Quantization
When Less is Enough: Efficient Inference via Collaborative Reasoning
May 1, 2026Yilei Chen, Sharut Gupta, Yannis Paschalidis +2Efficient Inference Inference Cost
Affordance Agent Harness: Verification-Gated Skill Orchestration
May 1, 2026Haojian Huang, Jiahao Shi, Yinchuan Li +1Affordances Inference Cost
Length Value Model: Scalable Value Pretraining for Token-Level Length Modeling
Apr 29, 2026Zhen Zhang, Changyi Yang, Zijie Xia +11Token-Level Uncertainty Inference Cost
Pruning via Causal Attribution Preserves Reasoning Performance in Large Language Models
Apr 27, 2026Amogh Sheth, Biruk Assefa, Yi Wen Huang +2Large Language Model Reasoning Pruning
RouteNLP: Closed-Loop LLM Routing with Conformal Cascading and Distillation Co-Optimization
Apr 26, 2026Dongxin Guo, Jikun Wu, Siu Ming YiuLarge Language Model Routing Inference Cost
MTRouter: Cost-Aware Multi-Turn LLM Routing with History-Model Joint Embeddings
Apr 26, 2026Yiqun Zhang, Hao Li, Zihan Wang +6Large Language Model Routing Inference Cost
Mixture of Heterogeneous Grouped Experts for Language Modeling
Apr 25, 2026Zhicheng Ma, Xiang Liu, Zhaoxiang Liu +5Mixture-of-Experts Models Inference Cost
Large Language Models Decide Early and Explain Later
Apr 24, 2026Ayan Datta, Zhixue Zhao, Bhuvanesh Verma +3Large Language Model Reasoning Chain-of-Thought Reasoning
On the Quantization Robustness of Diffusion Language Models in Coding Benchmarks
Apr 22, 2026Aarav Gupta, Gururaj Deshpande, Chandreyi ChakrabortyLarge Language Model Quantization Quantization
Expert Upcycling: Shifting the Compute-Efficient Frontier of Mixture-of-Experts
Apr 21, 2026Chaitanya Dwivedi, Binxuan Huang, Himanshu Gupta +3Mixture-of-Experts Models Experts
Apriel-Reasoner: RL Post-Training for General-Purpose and Efficient Reasoning
Apr 2, 2026Rafael Pardinas, Ehsan Kamalloo, David Vazquez +1Large Reasoning Models Inference Cost
Towards Reliable Local Security Agents: Verifiable Post-Training for Linux Privilege Escalation
Mar 18, 2026Philipp Normann, Andreas Happe, Jürgen Cito +1Security Small Language Models
CRISP: Compressed Reasoning via Iterative Self-Policy Distillation
Mar 5, 2026Hejian Sang, Yuanda Xu, Zhengze Zhou +3Large Reasoning Models Unsupervised On-Policy Self-Distillation
Evaluating AGENTS.md: Are Repository-Level Context Files Helpful for Coding Agents?
Feb 12, 2026Thibaud Gloaguen, Niels Mündler, Mark Müller +2Coding Agents Repository
Controllably Efficient Language Models
Nov 7, 2025Jatin Prakash, Aahlad Puli, Rajesh RanganathLLM Inference Efficiency Inference Cost
Distributionally Robust Optimization via Iterative Algorithms in Continuous Probability Spaces
Dec 29, 2024Linglingzhi Zhu, Yunqin Zhu, Yao XieRobust Optimization Distributional Learning

Jun 17, 2026 · cs.CVJ/K move · Enter open · S save

Visual-OPSD: Cross-Modal On-Policy Self-Distillation for Efficient Unified Multimodal Reasoning

Pengyu Li, Zhitao Gao, Lingling Zhang, Muye Huang+4

1Xi’an Jiaotong University · 3Shaanxi Province Key Laboratory of Big Data Knowledge Engineering · 2MOE KLINNS Lab, Xi’an Jiaotong University · 4Sun Yat-sen University.

Unified multimodal models (UMMs) interleave generated ''visual thoughts'' (VTs) with text reasoning to improve spatial tasks. This incurs roughly an order-of-magnitude inference cost from multi-step diffusion. We find this cost yields limited direct benefit. On ThinkMorph, removing or noising VTs barely changes accuracy across nine benchmarks. Once rendered, attention concentrates on the VT regardless of content. Yet a KL diagnostic shows that conditioning on a privileged VT trace shifts the model's completion distribution. This suggests the generation pathway encodes useful reasoning beyond the rendered pixels. Motivated by this gap, we propose Visual On-Policy Self-Distillation(Visual-OPSD). Teacher and student share identical weights but differ in context: the teacher sees privileged VTs while the student sees only the question. Token-level JSD distillation on on-policy student trajectories transfers the teacher's reasoning to a text-only student. Across nine benchmarks, Visual-OPSD improves over its generative teacher by

+3.40

pp with

14.3\times

speedup (10.0s vs. 142.8s per sample) and outperforms same-scale VLMs by

+63.83

pp on VSP. A Gaussian-noise control (

+0.40

pp vs.

+10.28

pp for real VTs) and

58.4\%

closure of the KL gap confirm that gains come from the semantic content of the generation pathway.

Multimodal Reasoning Visual Reasoning On-Policy Distillation Inference Cost Image Editing

Open paper Ask about it PDF