Latent CoT Reasoning

CoT: Chain-of-Thought

Latent chain-of-thought reasoning asks whether language models can perform multi-step computation in hidden states or compact learned tokens, potentially reducing the cost of generating long verbal explanations. Approaches include recurrent hidden-state reasoning, discrete or continuous latent tokens, and methods that compress or selectively retain explicit steps; training uses distillation, process supervision, reinforcement learning, and alignment with written reasoning traces. Work spans math and symbolic benchmarks as well as multimodal, structured-data, and tool-using tasks, with methods increasingly combining latent exploration with explicit verification. A central open question is whether latent states genuinely guide answers and remain monitorable, rather than merely correlating with success, while preserving the accuracy and flexibility of explicit chain-of-thought.

Latest papers

30 papers1 paper in the last four weeks, down 75% on the four weeks before. 0.0% of all new papers.

Does Latent Chain-of-Thought Actually Reason? A Single, Unifying Taxonomy for Latent Reasoning
Oct 15, 2026Jingjie Qiu, Sheldon Yu, Xunyi Jiang +6Latent CoT Reasoning CoT Evaluation
University of California, San Diego · New York University · University of Virginia +1
Next Thoughts Are Distributions: Generative Autoregressive Reasoning in the Latent Space
Sep 27, 2026Yang Li, Yi Wang, Shiyuan Huang +3Numerical Reasoning in Language Models Continuous Latent Reasoning
Rutgers University · Amazon · University of Illinois at Urbana-Champaign
Structural Process Supervision for Latent Chain-of-Thought Reasoning
Sep 9, 2026Yiqi Li, Xu Chen, Chen Ju +5Representation Learning Process Supervision
Shanghai Jiao Tong University · Taobao & Tmall Group
Think Wider: Mitigating Latent Rank Collapse in Implicit Chain-of-Thought Reasoning
Sep 7, 2026Yuwen Hao, Menglin YangSpectral Regularization Continuous Latent Reasoning
The Hong Kong University of Science and Technology (Guangzhou)
Latent Recurrent Thoughts: Recurrent Refinement of Proposed Latents for Reasoning with Frozen LLMs
Sep 1, 2026Zhaoliang Chen, Jie FuEfficient Language Model Reasoning Continuous Latent Reasoning
Emory University · IQuest Research
Does Out-of-Sight Equal Out-of-Mind in CoT Monitorability?
Aug 5, 2026Pedro Ferreira, Wilker Aziz, Ivan TitovCoT Monitoring Latent CoT Reasoning
University of Amsterdam · University of Edinburgh
Final Checkpoints Are Not Enough: Analyzing Latent Reasoning Faithfulness Along Training Trajectories
Jul 7, 2026Hengyu Jin, Shu Yang, Di WangCoT Faithfulness Continuous Latent Reasoning
Tongji University · Provable Responsible AI and Data Analytics (PRADA) Lab · King Abdullah University of Science and Technology
CoLT: Teaching Multi-Modal Models to Think with Chain of Latent Thoughts
Jun 30, 2026Lianyu Hu, Shengqian Qin, Zeqin Liao +4Latent CoT Reasoning Efficient Language Model Reasoning
Nanyang Technological University, Singapore · Shanghai Jiao Tong University, China · NKIARI, Shenzhen Futian, China +2
Bridging the Gap Between Latent and Explicit Reasoning with Looped Transformers
Jun 30, 2026Ying Fan, Anej Svete, Kangwook LeeCoT Reasoning Recurrent Transformers
Microsoft Research · 2ETH Zürich · 3KRAFTON +1
Interpreting Latent CoT Reasoning as Dynamical Systems
Jun 20, 2026Sabari Iyyappan Duraipandian, Shreya Sanjay Boyane, Manju Nagesh +3Latent Dynamics Modeling LLM Interpretability
1San Jose State University. · 2Worcester Polytechnic Institute. · 3George Mason University. +1
What Makes Effective Supervision in Latent Chain-of-Thought: An Information-Theoretic Analysis
Jun 18, 2026Xinghao Chen, Chak Tou Leong, Wenjin Guo +3Continuous Latent Reasoning Process Supervision
Ningbo Institute of Digital Twin, Eastern Institute of Technology, Ningbo, China · Department of Computing, The Hong Kong Polytechnic University, Hong Kong, China.
Demystifying Hidden-State Recurrence: Switchable Latent Reasoning with On-Policy Reinforcement Learning
Jun 11, 2026Jiayu Yang, Chao Chen, Shengen Wu +6RL for Language Model Reasoning Mechanistic Interpretability
HKUST(GZ) · University of Cambridge · NTU +2
Observable Patterns Are Not Explanations: A Causal-Geometric Analysis of Latent Reasoning Models
Jun 10, 2026Darpan Aswal, Thomas Palmeira Ferraz, Yongxin Zhou +1Continuous Latent Reasoning Causal Interpretability
Université Grenoble Alpes, CNRS, Grenoble INP, LIG · Université Paris-Saclay · NAVER LABS Europe
LoRi: Low-Rank Distillation for Implicit Reasoning
Jun 3, 2026Ryan Solgi, Jiayi Tian, Zheng ZhangCoT Distillation Mathematical Reasoning
University of California-Santa Barbara, USA
Adaptive Latent Agentic Reasoning
Jun 1, 2026Dongwon Jung, Peng Shi, Yi Zhang +2Agentic Reasoning LLM Agents
University of California, Davis · University of Waterloo · Greenshoe, Inc.
The Chain Holds, the Answer Folds: Trace-Answer Dissociation in Reasoning Models Under Adversarial Pressure
May 27, 2026Yubo Li, Ramayya Krishnan, Rema PadmanCoT Faithfulness Reasoning Evaluation
Carnegie Mellon University
Transformers Provably Learn to Internalize Chain-of-Thought
May 27, 2026Yixiao Huang, Hanlin Zhu, Zixuan Wang +4Transformer CoT Reasoning
UC Berkeley · Princeton University
CIRF: Tokenizing Chain-of-Thoughts into Reusable Functional Units for Efficient Latent Reasoning in Large Language Models
May 27, 2026Yukyung Lee, Yumeng Shen, Jinhyeong Park +2CoT Reasoning Efficient Language Model Reasoning
Boston University · Hankuk University of Foreign Studies
Unified Synthesis of Compositional Speech and Sound from Free-Form Text Prompts
May 27, 2026Yuyue Wang, Xihua Wang, Xin Cheng +2Text-to-Audio Generation Audio Generation
Renmin University of China
Selective Latent Thinking: Adaptive Compression of LLM Reasoning Chains
May 25, 2026Hui Xie, Jie Liu, Ziyue Qiao +1Efficient Language Model Reasoning LLM Inference Acceleration
Eindhoven University of Technology, Netherlands · School of Computing and Information Technology, Great Bay University, China
Latent Chain-of-Thought Improves Structured-Data Transformers
May 11, 2026Carson Dudley, Samet OymakCoT Reasoning Time Series Forecasting
RuPLaR : Efficient Latent Compression of LLM Reasoning Chains with Rule-Based Priors From Multi-Step to One-Step
May 10, 2026Xiaocheng Luo, Kang Wang, Zaifu Zhan +2LLM Compression CoT Compression
School of Computer Science and Technology, Soochow University, Suzhou, 215006, Jiangsu, PR China · Department of Electrical and Computer Engineering, University of Minnesota, Minneapolis, 55455, MN, USA
LaTER: Efficient Test-Time Reasoning via Latent Exploration and Explicit Verification
May 8, 2026Xuan Li, Yining Wang, Yuchen Liu +7CoT Reasoning Efficient Language Model Reasoning
University of Science and Technology of China · Unisound AI Technology Co., Ltd · MAIS, Institute of Automation, Chinese Academy of Sciences
Thinking Without Words: Efficient Latent Reasoning with Abstract Chain-of-Thought
Apr 24, 2026Keshav Ramji, Tahira Naseem, Ramón Fernandez AstudilloEfficient Language Model Reasoning Implicit Reasoning in Language Models
IBM Research AI
Think in Latent Thoughts: A New Paradigm for Gloss-Free Sign Language Translation
Apr 16, 2026Yiyang Jiang, Li Zhang, Xiao-Yong Wei +1Sign Language Translation Video-Language Models
The Hong Kong Polytechnic University · Sichuan University
CrystaL: Spontaneous Emergence of Visual Latents in MLLMs
Feb 24, 2026Yang Zhang, Danyang Li, Yuxuan Li +4Vision-Language Models Large Vision-Language Models
VCIP, School of Computer Science, Nankai University · Xiamen University · Nankai International Advanced Research Institute (Shenzhen Futian)
Beyond Dense States: Sparse Transcoders as Causally Testable Operators for LLM Latent Reasoning
Feb 2, 2026Yadong Wang, Haodong Chen, Yu Tian +3Sparse Autoencoders Continuous Latent Reasoning
Nanjing University of Aeronautics and Astronautics · State Key Laboratory of Ocean Sensing, ZJU-Hangzhou Global Scientific and Technological Innovation Center, Zhejiang University, Hangzhou, 311215, China · Security Capability Center
Latent Chain-of-Thought as Planning: Decoupling Reasoning from Verbalization
Jan 29, 2026Jiecong Wang, Hao Peng, Zhanyi Wang +2LLM Planning Efficient Language Model Reasoning
Beihang University · Didi Chuxing
Triggering Chain-of-Thought via Latent Feature Interventions in Large Language Models
Jan 12, 2026Zhenghao He, Guangzhi Xiong, Bohan Liu +2LLM Interpretability CoT Reasoning
University of Virginia, USA
Reasoning Beyond Language: A Comprehensive Survey on Latent Chain-of-Thought Reasoning
May 22, 2025Xinghao Chen, Anhao Zhao, Heming Xia +7CoT Reasoning LLM Reasoning
Department of Computing, The Hong Kong Polytechnic University · Ningbo Digital Twin Institute, Eastern Institute of Technology, Ningbo, China

Oct 15, 2026J/K move · Enter open · S save

Does Latent Chain-of-Thought Actually Reason? A Single, Unifying Taxonomy for Latent Reasoning

Jingjie Qiu, Sheldon Yu, Xunyi Jiang, Jiajin Liu+5

University of California, San Diego · New York University · University of Virginia+1

Latent CoT Reasoning CoT Evaluation LLM Interpretability

Open paper AskPDF arXiv

Latent chain-of-thought replaces written reasoning steps with continuous hidden states, giving models additional steps to compute before answering. It remains unclear what computation these steps actually perform. High task accuracy and decodable intermediate answers do not establish that a model uses its latent chain to reach an answer. We study this question across twenty-one model-task configurations covering three training recipes, three reasoning tasks, and two backbones. We combine chain deletion and content replacement with probes and trajectory analysis to test whether the chain affects the answer, what it encodes, and how its states change. On logical tasks, COCONUT often carries an answer that is already readable before the chain begins. Its states settle early, and deleting the chain leaves predictions almost unchanged. Per-step supervision produces an ordered layout of reasoning steps and can make the model depend on the chain. Yet even an ordered chain can act as a placeholder, with deletion hurting performance while content replacement leaves answers unchanged. On arithmetic tasks, all tested recipes show dependence on chain content. These results show why readable latent states alone are insufficient evidence of reasoning. We introduce a taxonomy that separates what a chain represents from how the model uses it. Seen this way, earlier reports no longer compete: placeholder tokens, collapsed superposition and step-dependent causal roles are each consistent with a different kind of chain, each obtained on a different task, training recipe or measurement. Once a study says which property it tested, whether the model uses the chain, what the chain holds, or how its state moves, claims about latent reasoning can be compared.