Efficient Inference

Follow this topic Ask about this topic

Recent momentum

+75%

7 papers in the last 28 days · 0.1% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this topic, kept on the site without email delivery.

Period ending 2026-09-21

4 new papers

A weekly snapshot of new work published in Efficient Inference.

Period ending 2026-09-14

2 new papers

A weekly snapshot of new work published in Efficient Inference.

54 papers

Latest in Efficient Inference

Browse all topics →

When2Think: Learning Difficulty-Aware Length Control for Efficient Hybrid Reasoning Models
Sep 17, 2026Jaejun Shim, HyunJin Kim, Young Jin Kim +1Large Reasoning Models Efficient Inference
The Free Inference Dimension: Complexity Measure for Zero-Collision Navigation under Hypothesis Mixtures
Sep 15, 2026Luiz Carlos Castro Guedes, Edward Hermann HaeuslerCollision-Free Trajectories Model-Free Reinforcement Learning
LCAP: Population-Informed Latent Chip Adaptation from Few Output Probes for Photonic Neural Networks
Sep 15, 2026Tianyu Gao, Guantian ZhengPhotonic Devices Parameter-Efficient Adaptation
Attention Quantization for Tabular Foundation Models
Sep 14, 2026Jonas M. Kübler, Benjamin Jäger, Klemens Flöge +2Tabular Foundation Models Key-Value Cache Quantization
Self-Orchestrating Language Models: Leveraging Semantic Dependence for Efficient Inference
Sep 13, 2026Tian JinLLM Inference Efficiency Autoregressive Decoding
Lightning Weave: Improving the Accuracy-Efficiency Frontier of Reasoning Models through Capability Composition
Sep 13, 2026Yecheng Wu, Song Han, Han CaiLarge Reasoning Models Efficient Inference
Why Is Video Still So Expensive? A Survey of Inference-Efficiency Mechanisms in Video and Audiovisual LLMs
Sep 9, 2026Killian Steunou, Yannis Tevissen, Mounîm A. El YacoubiAudio-Visual Consistency Vision Encoders
Depth-adaptive Inference of Looped Language Models via Continuous Depth Batching
Aug 10, 2026Kristian Schwethelm, Daniel Rueckert, Georgios KaissisLLM Inference Efficiency Well-Tuned Static Batching Policy
BnBERT-iPET: Sparse Few-Shot Language Modeling for Bengali via Lottery Ticket Pruning
Aug 5, 2026Sajib Hossain, Md Kamrus Samad, Anan Ghosh +2Bert-Based Models Bangla
Adaptive Two-Stage Visual Token Pruning for Efficient Inference in Video-Language Models
Aug 4, 2026Paribesh Regmi, Qingshuang Chen, Chi Zhang +3Visual Token Pruning Video-Language Models
From Data to Device: ELMOD An Efficient German-First 2.7B Language Model for Mobile Inference
Jul 27, 2026Darina Gold, Alexander Schwirjow, Viktor Haag +4Efficient Inference Resource-Constrained Edge Devices
Are All Tokens Necessary for Visual Place Recognition? An Empirical Study of Token Reduction for Efficient Inference
Jul 17, 2026Tong Jin, Yunpeng Liu, Shuyu Hu +4Visual Place Recognition Visual Tokens
Constrained Decoding for Diffusion Language Models via Efficient Inference over Finite Automata
Jul 8, 2026Meihua Dang, Stefano ErmonDiffusion Language Models Efficient Inference
Quantize the Target, Quantize the Drafter: Efficient Inference with Qwen3.5-4B
Jul 5, 2026Jaeyeon Kim, Jewon Lee, Bo-Kyeong KimLLM Inference Efficiency Qwen3
ROSA: A Robotics Foundation Model Serving System for Robot Factories
Jul 1, 2026Wenqi Jiang, Jason Clemons, Rowland O'Flaherty +5Robotics Robot Systems
Coverage-Driven KV Cache Eviction for Efficient and Improved Inference of LLM
Jun 28, 2026Shuvendu Roy, Mengyao Zhai, Hossein Hajimirsadeghi +1Key-Value Cache Eviction LLM Inference Efficiency
KernelSight-LM: A Kernel-Level LLM Inference Simulator
Jun 26, 2026Xiteng Yao, Taeho Kim, Hengzhi Pei +7LLM Inference Efficiency Efficient Inference
Brevity is the Soul of Inference Efficiency: Inducing Concision in VLMs via Data Curation
Jun 24, 2026DatologyAI, :, Matthew L. Leavitt +8Efficient Inference Data-Curation
MemoryWAM: Efficient World Action Modeling with Persistent Memory
Jun 18, 2026Sizhe Yang, Juncheng Mu, Tianming Wei +8Efficient World-Action Model World Models
ADaPT: Token-Level Decoupling for Efficient Large Reasoning Models
Jun 18, 2026Tingyun Li, Zishang Jiang, Jinyi Han +8Large Reasoning Models Efficient Inference
Multi-task LLMs for Bug Classification: Efficient Inference with Auxiliary Decoding Heads
Jun 8, 2026Nikolai RozanovBug Code Generation
OctoT2I: A Self-Evolving Agentic Text-to-Image Router
Jun 1, 2026Xu Jiang, Bin Chen, Gehui Li +3Text-To-Image Real-Time Inference
EvoCut: Multi-Layer Evolution-Aware Visual Token Compression for Efficient Large Vision-Language Models
Jun 1, 2026Hongyu Lu, Feng Zhang, Wenwei Jin +5Token Compression Visual Tokens
Efficient Test-time Inference for Generative Planning Models with OCL Search
May 30, 2026Robert Gieselmann, Mihai Samson, Federico Pecora +1Classical Planning Efficient Inference
Bridging Maximum Likelihood and Optimal Transport for Efficient Inference and Model Selection in Stochastic Block Models
May 27, 2026Simon Queric, Cédric Vincent-Cuaz, Charles Bouveyron +1Maximum Likelihood Model Selection
Semiparametrically Efficient Inference for Kernel Measures of Noise Heterogeneity
May 26, 2026Jakub Wornbard, Zikai Shen, Dimitri Meunier +1Heteroskedasticity Semiparametric Framework
Boosting Inference with Guided Reasoning: Stochastic Exploration for Recursive Models
May 24, 2026Andrew Corbett, Archit Sood, Anna Tzatzopoulou +2Reasoning Trajectory Recursion
Your Embedding Model is SMARTer Than You Think
May 24, 2026Jianrui Zhang, Hyun Jung Lee, Sukanta Ganguly +3Multimodal Retrieval Contrastive Learning
JT-SAFE-V2: Safety-by-Design Foundation Model with World-Context Data
May 23, 2026Junlan Feng, Fanyu Meng, Chong Long +12Large Language Model Safety Wireless Foundation Models
Can AI Guess What You Know? Performance Comparison of Large Language Models for Human Domain Knowledge Estimation From Communication Logs
May 21, 2026Ko Watanabe, Shoya IshimaruKnowledge-Intensive Domains Efficient Inference
PSD: Pushing the Pareto Frontier of Diffusion LLMs via Parallel Speculative Decoding
May 15, 2026Shengyin Sun, Yiming Li, Renxi Liu +7Diffusion Language Models Speculative Decoding
PreFT: Prefill-only finetuning for efficient inference
May 14, 2026Andrew Lanpouthakoun, Aryaman Arora, Zhengxuan Wu +4Parameter-Efficient Fine-Tuning Prefill
LEAD: Length-Efficient Adaptive and Dynamic Reasoning for Large Language Models
May 10, 2026Songtao Wei, Yi Li, Zhikai Li +7Large Reasoning Models Efficient Inference
Efficient LLM Reasoning via Variational Posterior Guidance with Efficiency Awareness
May 10, 2026Zizhao Chen, Yuying Li, Siting Lin +1Efficient Inference Large Language Model Reasoning
DARE: Difficulty-Adaptive Reinforcement Learning with Co-Evolved Difficulty Estimation
May 9, 2026Yang Zhou, Can Jin, Zihan Dong +7Efficient Inference
Budgeted LoRA: Distillation as Structured Compute Allocation for Efficient Inference
May 5, 2026Mohammed Sabry, Anya BelzEfficient Inference Low-Rank Adaptation
Exact and Approximate Algorithms for Polytree Learning
May 5, 2026Juha Harviainen, Frank Sommer, Manuel SorgeBayesian Networks Approximation Algorithms
Rethinking Model Selection in VLM Through the Lens of Gromov-Wasserstein Distance
May 2, 2026Muyang Li, Yucheng Liu, Jianbo Ma +3Vision Encoders Vision-Language Alignment
When Less is Enough: Efficient Inference via Collaborative Reasoning
May 1, 2026Yilei Chen, Sharut Gupta, Yannis Paschalidis +2Efficient Inference Inference Cost
DARE: Diffusion Language Model Activation Reuse for Efficient Inference
May 1, 2026Natalia Frumkin, Bokun Wang, Hung-Yueh Chiang +3Diffusion Language Models LLM Inference Efficiency
Density Field State Space Models: 1-Bit Distillation, Efficient Inference, and Knowledge Organization in Mamba-2
Apr 28, 2026Chirag ShindeBidirectional Mamba Efficient Inference
Fix Initial Programs and Iteratively Refine Repair Instructions Toward Non-Elimination Multi-Turn Program Correction
Apr 27, 2026Yuto Tanaka, Issei SatoIterative Refinement Efficient Inference
Reasoning Primitives in Hybrid and Non-Hybrid LLMs: Do Architectural Differences Yield Advantages in State-Tracking and Recall?
Apr 23, 2026Shivam Rawat, Lucie Flek, Florian Mai +1Large Language Model Reasoning Search-Augmented Reasoning
Efficient Test-Time Inference via Deterministic Exploration of Truncated Decoding Trees
Apr 22, 2026Xueyan Li, Johannes Zenn, Ekaterina Fadeeva +3Efficient Inference Reasoning Traces
DeInfer: Efficient Parallel Inferencing for Decomposed Large Language Models
Apr 20, 2026You-Liang Huang, Xinhao Huang, Chengxi Liao +1LLM Inference Efficiency Efficient Inference
SecureRouter: Encrypted Routing for Efficient Secure Inference
Apr 16, 2026Yukuan Zhang, Mengxin Zheng, Qian LouGraph Neural Network Inference Homomorphic Encryption
DEEP-GAP: Deep-learning Evaluation of Execution Parallelism in GPU Architectural Performance
Apr 16, 2026Kathiravan PalaniappanMulti-Gpu Systems Nvidia Nemotron
Channel-wise Retrieval for Multivariate Time Series Forecasting
Apr 7, 2026Junhyeok Kang, Jun Seo, Soyeon Park +4Multivariate Time Series Forecasting Efficient Inference
A Replicate-and-Quantize Strategy for Plug-and-Play Load Balancing of Sparse Mixture-of-Experts LLMs
Feb 23, 2026Zijie Liu, Jie Peng, Jinhao Duan +7Load Balancing Mixture-Of-Expert Inference
Beyond Tokens: Semantic-Aware Speculative Decoding for Efficient Inference by Probing Internal States
Feb 3, 2026Ximing Dong, Shaowei Wang, Dayi Lin +2Speculative Decoding LLM Inference Efficiency
What If We Allocate Test-Time Compute Adaptively?
Feb 1, 2026Ahsan Bilal, Ahmed Mohsin, Muhammad Umer +4Test-Time Scaling Efficient Inference
DAGGER: Distractor-Aware Graph Generation for Executable Reasoning in Math Problems
Jan 11, 2026Zabir Al Nazi, Shubhashis Roy Dipta, Sudipta KarMathematical Reasoning Math Problems
Entropy-Aware Token Rejection for Improving Speculative Decoding
Dec 29, 2025Tiancheng Su, Meicong Zhang, Guoxiu HeSpeculative Decoding Efficient Inference
TERC: A Transfer Entropy Redundancy Criterion for State Variable Selection in Reinforcement Learning
Jan 21, 2024Charles Westphal, Stephen Hailes, Mirco MusolesiEntropy Regularized Reinforcement Learning Proximal Policy Optimization

Sep 17, 2026 · cs.AIJ/K move · Enter open · S save

When2Think: Learning Difficulty-Aware Length Control for Efficient Hybrid Reasoning Models

Jaejun Shim, HyunJin Kim, Young Jin Kim, JinYeong Bak

Sungkyunkwan University · Microsoft

Large Reasoning Models (LRMs) achieve strong performance on complex tasks but exhibit systematic inefficiency: they often overthink easy problems and underthink hard ones. Existing approaches based on uniform length penalties or rigid routing incur an efficiency tax, trading reduced computation on easy instances for accuracy loss on hard instances. We formulate efficient reasoning as an instance-adaptive computation allocation problem and propose When2Think, a post-training framework for hybrid reasoning that dynamically allocates computation based on problem difficulty. Our method introduces Instance-level Difficulty-Aware Control (IDAC), a reward-shaping mechanism that leverages pre-computed reference statistics (accuracy and token usage) to regulate reasoning depth. Combined with verifier-based rewards and batch-wise standardized advantages, IDAC enables stable critic-free optimization without learned reward models or online reference-model queries. When2Think encourages direct answering on easy instances while preserving extended reasoning on hard instances, thereby learning when to use System 1 (NoThink) versus System 2 (Think). Experiments on mathematical benchmarks demonstrate improved accuracy-efficiency trade-offs: on AIME24, Pass@3 increases by 10.0% while token usage is reduced by 27.9% relative to the base model, and on AIME25, When2Think achieves 40.0% Pass@3, outperforming compression and routing-only baselines.

Large Reasoning Models Efficient Inference Verifiable Rewards

Open paper Ask about it PDF