Efficient Language Model Inference

Follow this topic Ask about this topic

Momentum

37 papers in the last four weeks, up 185% on the four weeks before. 0.4% of all new papers.

Jul 13Week of Sep 28

Latest papers 206

All topics

Geometric Latent Reasoning Induces Shorter Generations in LLMs
Jun 1, 2026Shashi Kumar, Yacouba Kaloga, Petr Motlicek +2Efficient Language Model Reasoning Continuous Latent Reasoning
Idiap Research Institute, Switzerland · EPFL, Switzerland · BUT, Czech Republic
PortBERT: Navigating the Depths of Portuguese Language Models
Jun 1, 2026Raphael Scheible-Schmitt, Henry He, Armando B. MendesLanguage Model Pretraining Efficient Language Model Training
School of Computation, Information and Technology, Technical University of Munich, Munich, Germany · IS2E - Intelligent Systems, Science and Engineering, LIACC polo on Azores University, Ponta Delgada, Portugal
Unveiling the Entropy Dynamics of Chain-of-Thought Reasoning
Jun 1, 2026Ting Xu, Xu He, Yupu Lu +4CoT Reasoning Test-Time Scaling
The Chinese University of Hong Kong · Huawei Technologies Ltd · School of Computing and Data Science, The University of Hong Kong
ProbeScale: Probing Analysis to Optimize Neural Scaling Laws for Efficient Small Language Model Inference
Jun 1, 2026Sourav DasSmall Language Models Efficient Language Model Inference
Department of Computer Science and Engineering Indian Institution of Information Technology Kalyani Kalyani, West Bengal, India
Semantic Triplet Restoration: A Novel Protocol for Hierarchical Table Understanding in Large Language Models
May 29, 2026Yibin Zhao, Fangxin Shang, Dingrui Yang +1Table QA Table Structure Recognition
Taiyuan University of Technology · AI Lab, Qifu Technology, Beijing, China · AI Lab, Greensea Technology, Shenzhen, China
Trading Complexity for Expressivity Through Structured Generalized Linear Token Mixing
May 29, 2026Erwan Fagnou, Paul Caillon, Blaise Delattre +1Transformer Expressivity Autoregressive Language Modeling
Miles Team, LAMSADE, Université Paris Dauphine-PSL, Paris, France · Department of Computer Science, School of Computing, Institute of Science Tokyo, Tokyo, Japan · ESPCI PSL, Paris, France
SLAT: Segment-Level Adaptive Trimming for Efficient CoT Reasoning
May 29, 2026Jian Yao, Xiongcai Luo, Ran Cheng +1RL for Language Model Reasoning Efficient Language Model Inference
Department of Data Science and Artificial Intelligence, The Hong Kong Polytechnic University · Bytedance · The Hong Kong Polytechnic University-Daya Bay Technology and Innovation Research Institute, Huizhou,Guangdong Province, China.
Teaching Language Models to Check Grounded Claim Factuality with Human Test-Taking Strategies
May 28, 2026Yuxuan Ye, Raul Santos-Rodriguez, Edwin SimpsonLLM Grounding Claim Verification
Intelligent Systems Laboratory University of Bristol United Kingdom
DynaGraph: Lightweight Multi-Model Interaction Framework via Dynamic Topological Reconfiguration
May 28, 2026Yanxing Guo, Zihao Zheng, Fangzhou Wu +4Multi-Agent LLM Systems LLM Routing
Peking University · Nanjing University · Beijing Advanced Innovation Center for Integrated Circuits +2
Rubric-Guided Process Reward for Stepwise Model Routing
May 28, 2026Shenghao Ye, Yu Guo, Zhengheng Li +2Process Reward Models RL for Language Model Reasoning
University of Science and Technology of China · Southeast University
Robust and Efficient Guardrails with Latent Reasoning
May 27, 2026Siddharth Sai, Xiaofei Wen, Muhao ChenLLM Guardrails Content Moderation
University of California, Davis
HRBench: Benchmarking and Understanding Thinking-Mode Switch Strategies in Hybrid-Reasoning LLMs
May 27, 2026Yansong Ning, Mianpeng Liu, Jingwen Ye +2LLM Evaluation Adaptive Inference
AI Thrust, The Hong Kong University of Science and Technology (Guangzhou) · AIPD, Tencent
CIRF: Tokenizing Chain-of-Thoughts into Reusable Functional Units for Efficient Latent Reasoning in Large Language Models
May 27, 2026Yukyung Lee, Yumeng Shen, Jinhyeong Park +2CoT Reasoning Efficient Language Model Reasoning
Boston University · Hankuk University of Foreign Studies
PrunePath: Towards Highly Structured Sparse Language Models
May 27, 2026Zhexuan Gu, Zixun Fu, Yancheng YuanMixture-of-Experts Pruning LLM Pruning
Department of Applied Mathematics, The Hong Kong Polytechnic University
Self-Consistency via Marginal Sharpening
May 27, 2026Aleksei Arzhantsev, Otmane Sakhi, Nicolas ChopinEfficient Language Model Reasoning Reasoning Consistency in Language Models
Criteo AI Lab, CREST IP Paris · Criteo AI Lab · ENSAE, Institut Polytechnique de Paris
The Shape of Overthinking: Backtracking Bursts in Long Reasoning Traces
May 27, 2026Navid Rezazadeh, Arash Gholami DavoodiReasoning Evaluation Efficient Language Model Inference
University of California, Irvine · Carnegie Mellon University
MobileMoE: Scaling On-Device Mixture of Experts
May 26, 2026Yanbei Chen, Hanxian Huang, Ernie Chang +5Mixture-of-Experts Language Models On-Device Language Model Inference
Meta AI
Decision-Making with Lightweight Confidence-Aware Language Model for Autonomous Driving
May 25, 2026Ruoyu Yao, Ruiguo Zhong, Pei Liu +3Confidence Estimation in Language Models Language Model Distillation
The Hong Kong University of Science and Technology (Guangzhou), Guangzhou 511453, China
RAGe: A Retrieval-Augmented Generation Evaluation Framework
May 23, 2026Larissa Guder, João Pedro de Moura, Arthur Accorsi +5Retrieval-Augmented Generation RAG Evaluation
School of Technology, Pontifical Catholic University of Rio Grande do Sul, Av. Bento Gonçalves, Porto Alegre, Rio Grande do Sul, Brazil.
OnePred: Next-Query Prediction via Recursive Intent Memory in Multi-Turn Conversations
May 22, 2026Jiangwang Chen, Bowen Zhang, Zixin Song +4Long-Term Conversational Memory Efficient Language Model Inference
1Tsinghua University · 2Qwen Applications Business Group of Alibaba
HARNESS-LM: A Three-Phase Training Recipe for Harnessing SLMs in Sponsored Search Retrieval
May 22, 2026Vipul Gupta, Shikhar Mohan, Lakshya Kumar +4Online Advertising Small Language Models
Microsoft AI India
Limited Marginal Benefit of Reasoning-Heavy LLM Deployment in ESG Narrative Scoring: A 4-Model Consensus Study on Japanese Listed Firms
May 22, 2026Hiroyuki KokubuLLM Evaluation Language Model Ensembles
Kansai University
PathCal: State-Aware Reflection-Marker Calibration for Efficient Reasoning
May 21, 2026Lingyu Jiang, Zirui Li, Shuo Xing +6Efficient Language Model Reasoning Efficient Language Model Inference
1Tohoku University · 2Texas A&M University · 3Worcester Polytechnic Institute
Single-Pass, Depth-Selective Reading for Multi-Aspect Sentiment Analysis
May 20, 2026Yan Xia, Zhuangzhuang Pan, Amirrudin Kamsin +1Aspect-Based Sentiment Analysis Efficient Language Model Inference
1Universiti Malaya, Malaysia · 2Suzhou University of Technology, China · 3VinUniversity, Vietnam
Distributional Alignment as a Criterion for Designing Task Vectors in In-Context Learning
May 20, 2026Jihoon Kwon, Jiwon Choi, Jy-yong SohnTask Vectors Distribution Matching
Seoul National University · Yonsei University
When Do LLMs Reason? A Dynamical Systems View via Entropy Phase Transitions
May 20, 2026Wei Xia, Haoqing Wang, Zhi-Hong Deng +1Language Model Decoding CoT Reasoning
Samsung Research, Beijing, China · State Key Laboratory of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University
CopT: Contrastive On-Policy Thinking with Continuous Spaces for General and Agentic Reasoning
May 19, 2026Dachuan Shi, Hanlin Zhu, Xiangchi Yuan +4Agentic Reasoning Efficient Language Model Inference
Georgia Tech · UC Berkeley · Stanford University +1
Flash PD-SSM: Memory-Optimized Structured Sparse State-Space Models
May 18, 2026Aleksandar Terzić, Francesco Carzaniga, Nicolas Menet +4Efficient Language Model Inference Sequence Modeling
1IBM Research – Zurich · Department of Computer Science, ETH Zürich · *Research conducted at IBM Research – Zurich.
SNLP: Layer-Parallel Inference via Structured Newton Corrections
May 18, 2026Ligong Han, Kai Xu, Hao Wang +1Transformer Inference Parallel Decoding
1Red Hat AI Innovation · 2MIT-IBM Watson AI Lab · 3Core AI, IBM
Stop When Reasoning Converges: Semantic-Preserving Early Exit for Reasoning Models
May 17, 2026Dehai Min, Giovanni Vaccarino, Huiyi Chen +3Early Stopping CoT Reasoning
University of Illinois Chicago · Politecnico di Milano · University of Illinois Urbana-Champaign +1

Jun 1, 2026, cs.CLJ/K move · Enter open · S save

Geometric Latent Reasoning Induces Shorter Generations in LLMs

Shashi Kumar, Yacouba Kaloga, Petr Motlicek, Ina Kodrasi+1

Idiap Research Institute, Switzerland · EPFL, Switzerland · BUT, Czech Republic

Efficient Language Model Reasoning Continuous Latent Reasoning Efficient Language Model Inference Representation Geometry in Language Models LLM Reasoning Latent Reasoning

Open paper AskPDF arXiv

Large language models solve complex problems by generating lengthy chains of explicit reasoning tokens. While effective, this makes reasoning expensive, length-sensitive, and constrained to (discrete) natural language. While latent reasoning offers a continuous alternative, determining useful structures for intermediate latent states is an open challenge. In this paper, we formulate latent reasoning as a geometric path-approximation problem within the model's pretrained token-embedding space. We introduce Geometric Latent Reasoning (GLR), which uses a lightweight transition head to predict iterative direction updates in embedding space. Using textual chain-of-thought traces as anchors, GLR learns to approximate discrete reasoning trajectories while permitting continuous deviations from exact token embeddings. Evaluations on mathematical reasoning benchmarks using Qwen3 models reveal an emergent phenomenon: geometric latent reasoning induces substantially shorter generations without an explicit length objective. By replacing early explicit reasoning with continuous latent steps, models often reach correct answers using substantially fewer total generation steps. These findings suggest that continuous trajectories act as compact intermediate reasoning states, exposing a new tradeoff between latent computation budget, output length, and accuracy.

Momentum

Related topics

Latest papers 206