Efficient Language Model Inference

Follow this topic Ask about this topic

Momentum

37 papers in the last four weeks, up 185% on the four weeks before. 0.4% of all new papers.

Jul 13Week of Sep 28

Latest papers 206

All topics

Less Languages, Less Tokens: An Efficient Unified Logic Cross-lingual Chain-of-Thought Reasoning Framework
Apr 22, 2026Chenyuan Zhang, Qiguang Chen, Xie Chen +6Cross-Lingual Reasoning in Language Models CoT Reasoning
Harbin Institute of Technology, Shenzhen · Shanghai Innovation Institute · Central South University +3
Neural Garbage Collection: Learning to Forget while Learning to Reason
Apr 20, 2026Michael Y. Li, Jubayer Ibn Hamid, Emily B. Fox +1LLM Compression Memory-Efficient Optimization
Stanford University
WISV: Wireless-Informed Semantic Verification for Distributed Speculative Decoding in Device-Edge LLM Inference
Apr 20, 2026Zixuan Liu, Zhiyong Chen, Nan Xue +4Speculative Decoding Efficient Language Model Inference
Cooperative Medianet Innovation Center and the Department of Electronic Engineering, Shanghai Jiao Tong University, Shanghai 200240, China
Efficient Test-Time Scaling via Temporal Reasoning Aggregation
Apr 19, 2026Jiakun Li, Xingwei He, Kefan Li +3Test-Time Scaling Efficient Language Model Inference
School of Computer Science and Engineering, Beihang University · 3Hangzhou Innovation Institute, Beihang University · 2Qingdao Research Institute, Beihang University
Layer-wise MoE Routing Locality under Shared-Prefix Code Generation: Token-Identity Decomposition and Compile-Equivalent Fork Redundancy
Apr 19, 2026Shun-ichiro Hayashi, Daichi Mukunoki, Tetsuya Hoshino +1Expert Routing Efficient Language Model Inference
Graduate School of Informatics, Nagoya University, Japan · Information Technology Center, Nagoya University, Japan
RankGuide: Tensor-Rank-Guided Routing and Steering for Efficient Reasoning
Apr 17, 2026Jiayi Tian, Yupeng Su, Ryan Solgi +2LLM Routing Efficient Language Model Inference
University of California, Santa Barbara · Intel Labs
SCATR: Simple Calibrated Test-Time Ranking
Apr 16, 2026Divya Shyamal, Marta Knežević, Lan Tran +3Test-Time Scaling Best-of-N Sampling
EPFL · MIT
Towards Faster Language Model Inference Using Mixture-of-Experts Flow Matching
Apr 16, 2026Aihua LiNon-Autoregressive Text Generation Flow Matching
Duke University
TrigReason: Trigger-Based Collaboration between Small and Large Reasoning Models
Apr 16, 2026Yi Zhao, Yajuan Peng, Cam-Tu Nguyen +4Efficient Language Model Reasoning LLM Inference Acceleration
AGI Institute, School of Computer Science, Shanghai Jiao Tong University, Shanghai, China · Shanghai Key Laboratory for Intelligent Information Processing, Fudan University · State Key Laboratory for Novel Software Technology, Nanjing University +2
EnComp: Lightweight Encoder-Only Context Compression for Retrieval-Augmented Question Answering
Mar 10, 2026Thao Do, Dinh Phu Tran, An Vo +2Efficient Language Model Inference Knowledge-Intensive QA
Korea Advanced Institute of Science and Technology (KAIST), Daejeon 34141, South Korea
How Small Can 6G Reason? Scaling Tiny-to-Small Language Models for AI-Native Networks
Mar 2, 2026Mohamed Amine Ferrag, Abderrahmane Lakas, Merouane Debbah6G Networks Small Language Models
Department of Computer and Network Engineering, United Arab Emirates University, UAE · Research Institute for Digital Future, Khalifa University, UAE
Dynamic Model Routing and Cascading for Efficient LLM Inference: A Survey
Feb 23, 2026Yasmin Moslem, John D. KelleherLLM Routing Efficient Language Model Inference
ADAPT Centre School of Computer Science & Statistics Trinity College Dublin Dublin, Ireland
Framework of Thoughts: A Foundation Framework for Dynamic and Optimized Reasoning based on Chains, Trees, and Graphs
Feb 18, 2026Felix Fricke, Simon Malberg, Georg GrohInference-Time Optimization Tree of Thoughts
School of Computation, Information and Technology Technical University of Munich
Arapai: An Offline-First LLM Architecture for Adaptive Learning in Low-Connectivity Environments
Feb 14, 2026Joseph Walusimbi, Ann Move Oguti, Joshua Benjamin Ssentongo +1On-Device Language Model Inference Intelligent Tutoring Systems
Department of Electronics and Computer Engineering, Soroti University, Soroti, Uganda
Latent Chain-of-Thought as Planning: Decoupling Reasoning from Verbalization
Jan 29, 2026Jiecong Wang, Hao Peng, Zhanyi Wang +2LLM Planning Efficient Language Model Reasoning
Beihang University · Didi Chuxing
Step-Tagging: Toward controlling the generation of Language Reasoning Models through step monitoring
Dec 16, 2025Yannis Belkhiter, Seshu Tirupathi, Giulio Zizzo +1LLM Inference Efficiency Efficient Language Model Reasoning
1IBM Research Europe, Dublin, Ireland · 2Trinity College Dublin, Ireland · 3ADAPT Research Centre, Dublin, Ireland
Optimal Self-Consistency for Efficient Reasoning with Large Language Models
Nov 15, 2025Austin Feng, Marius Alonso, Ambroise Odonnat +2Self-Consistency Decoding Inference-Time Scaling
1Yale University, New Haven, CT, USA · 2Noah’s Ark Lab, Paris, France · 3Inria, Rennes, France +1
Thinking While Speaking: Inference-Time Knowledge Transfer for Responsive and Intelligent Conversational Voice Agents
Nov 10, 2025Vidya Srinivas, Zachary Englhardt, Vikram Iyer +1Conversational Agents Efficient Language Model Inference
Paul G. Allen School of Computer Science & Engineering
Correctness Forensics for Batch Speculative Decoding: Diagnosing the Ragged Tensor Problem
Oct 26, 2025Ranran Haoran Zhang, Soumik Dey, Ashirbad Mishra +3Speculative Decoding Efficient Language Model Inference
The Pennsylvania State University, University Park, PA, USA · eBay Inc, San Jose, CA, USA.
A Vision Toward Energy-Efficient Domain-Specific Artificial Intelligence Models and Agents
Oct 24, 2025Abhijit Chatterjee, Niraj K. Jha, Jonathan D. Cohen +6Energy-Efficient ML Lifelong Learning Agents
School of Electrical and Computer Engineering, Georgia Institute of Technology · Department of Electrical and Computer Engineering, Princeton University · Department of Psychology & Neuroscience Institute, Princeton University +4
Stop Before You Fail: Operational Capability Boundaries for Mitigating Unproductive Reasoning in Large Reasoning Models
Sep 29, 2025Qingjie Zhang, Yujia Fu, Yang Wang +5Efficient Language Model Reasoning Efficient Language Model Inference
Tsinghua University · 2Ant Group
Toward Efficient Uncertainty in LLMs through Evidential Knowledge Distillation
Jul 24, 2025Lakshmana Sri Harsha Nemani, P. K. Srijith, Tomasz KuśmierczykLLM Uncertainty Estimation Uncertainty-Aware Knowledge Distillation
Department of Computer Science and Engineering, Indian Institute of Technology Hyderabad, India · Jagiellonian University, Faculty of Mathematics and Computer Science, Kraków, Poland
Adaptive Activation Steering for Efficient LLM Reasoning via Closed-Loop PID Control
Jun 23, 2025Aryasomayajula Ram BharadwajLanguage Model Steering Efficient Language Model Inference
Independent Researcher
Adaptive GoGI-Skip: Coupling Goal-Gradient Importance with Dynamic Uncertainty for Efficient Reasoning
May 13, 2025Ren ZhuangGradient-Based Attribution LLM Pruning
School of Information Science and Technology, Hangzhou Normal University
A Survey of Transformer-based Language Models with Focus on Efficiency
May 15, 2024Wazib Ansar, Saptarsi Goswami, Amlan ChakrabartiLLM Compression Energy-Efficient ML
A. K. Choudhury School of IT University of Calcutta, Kolkata, India · Department of Computer Science Bangabasi Morning College Kolkata, India
Almost Free State Prediction Separation
Date pendingJohn Langford, Nathan Godey, Giovanni Monea +5Language Model Pretraining Efficient Language Model Training
Microsoft · Cornell University

Apr 22, 2026, cs.CLJ/K move · Enter open · S save

Less Languages, Less Tokens: An Efficient Unified Logic Cross-lingual Chain-of-Thought Reasoning Framework

Chenyuan Zhang, Qiguang Chen, Xie Chen, Zhuotao Tian+5

Harbin Institute of Technology, Shenzhen · Shanghai Innovation Institute · Central South University+3

Cross-Lingual Reasoning in Language Models CoT Reasoning Efficient Language Model Reasoning Efficient Language Model Inference Cross-Lingual Transfer

Open paper AskPDF arXiv

Cross-lingual chain-of-thought (XCoT) with self-consistency markedly enhances multilingual reasoning, yet existing methods remain costly due to extensive sampling of full trajectories across languages. Moreover, multilingual LLM representations vary strongly by language, hindering direct feature comparisons and effective pruning. Motivated by this, we introduce UL-XCoT, the first efficient unified logic cross-lingual reasoning framework that minimizes redundancy in token usage and latency, yielding the greatest efficiency under limited sampling budgets during inference. Specifically, UL-XCoT (1) achieves less languages by selecting, per query, a small candidate language set in a language-invariant unified logic space, (2) enables less tokens by monitoring logic-space trajectory dynamics during decoding to prune low-quality reasoning paths, and (3) aggregates the remaining high-quality trajectories via voting. Experiments on PolyMath across 18 languages and MMLU-ProX-Lite across 29 languages with DeepSeek-R1-DistillQwen-7B demonstrate that UL-XCoT achieves competitive accuracy while sharply cutting over 50% decoding token cost versus prior sampling baselines. UL-XCoT also delivers more stable gains on low-resource languages, underscoring consistently superior robustness where standard XCoT self-consistency method fails.

Momentum

Related topics

Latest papers 206