EndoCoT: Scaling Endogenous Chain-of-Thought Reasoning in Diffusion Models
Authors: Xuanlang Dai, Yujie Zhou, Long Xing, Jiazi Bu, Xilin Wei, Yuhong Liu, Beichen Zhang, Kai Chen, Yuhang Zang
Organizations: Shanghai AI Laboratory · Xi’an Jiaotong University · Shanghai Jiaotong University · University of Science and Technology of China · Fudan University · The Chinese University of Hong Kong
Recently, Multimodal Large Language Models (MLLMs) have been widely integrated into diffusion frameworks primarily as text encoders to tackle complex tasks such as spatial reasoning. However, this paradigm suffers from two critical limitations: (i) MLLMs text encoder exhibits insufficient reasoning depth. Single-step encoding fails to activate the Chain-of-Thought process, which is essential for MLLMs to provide accurate guidance for complex tasks. (ii) The guidance remains invariant during the decoding process. Invariant guidance during decoding prevents DiT from progressively decomposing complex instructions into actionable denoising steps, even with correct MLLM encodings. To this end, we propose Endogenous Chain-of-Thought (EndoCoT), a novel framework that first activates MLLMs' reasoning potential by iteratively refining latent thought states through an iterative thought guidance module, and then bridges these states to the DiT's denoising process. Second, a terminal thought grounding module is applied to ensure the reasoning trajectory remains grounded in textual supervision by aligning the final state with ground-truth answers. With these two components, the MLLM text encoder delivers meticulously reasoned guidance, enabling the DiT to execute it progressively and ultimately solve complex tasks in a step-by-step manner. Extensive evaluations across diverse benchmarks (e.g., Maze, TSP, VSP, and Sudoku) achieve an average accuracy of 92.1%, outperforming the strongest baseline by 8.3 percentage points. The code and dataset are publicly available at https://internlm.github.io/EndoCoT/.
Figures & tables
Figure 1: EndoCoT enables endogenous chain-of-thought reasoning. (a) Radar plot showing EndoCoT outperforms baselines across all benchmarks. (b) On visual reasoning tasks requiring generalization (maze size, Sudoku font), previous work [ 11 ] fails on novel domains while EndoCoT consistently generalizes correctly. (c) Vanilla denoising (left) commits to solutions early without reasoning, while our approach (right) enables interpretable, step-by-step reasoning chains.
Figure 2: (a) Layer-wise sensitivity across Vision Encoder, LLM, and DiT components (red: high sensitivity, white: low sensitivity). (b) Limited single-step reasoning : DiT performs spatial grounding but trajectory violates constraints. (c) Static-guidance failure : Dense topologies cause attention entropy to become diffuse.
Figure 3: Overview of EndoCoT . (a) Training: We propose a progressive two-stage training strategy: the first stage trains the model to fit both intermediate and final states at each reasoning step, capturing the full multi-step trajectory; the second stage freezes gradients on intermediate states and optimizes only the terminal state, refining generation quality while preserving learned reasoning dynamics (b) Inference: the model iteratively updates latent representations.
Figure 4: Overview of notations and iterative thought guidance module. EndoCoT iteratively refines latent states hτ through the MLLM fϕ , then conditions the DiT fψ at each reasoning step τ to generate intermediate visual outputs Iτ .
Figure 5: Step-by-step reasoning process across four distinct tasks. Our model incrementally resolves complex visual reasoning tasks through intermediate reasoning steps. For each task (Maze, TSP, Sudoku, VSP), we show the initial input (leftmost), intermediate refinement steps, and the final optimal solution (rightmost).
Models
Maze
TSP
Sudoku
VSP
VSP-Super
Avg
8
16
32
12
15
18
45
40
35
30
3
4
5
6
7
8
16
32
▼ Zero-Shot
ThinkGen [ 15 ]
0
0
0
0
0
0
0
0
0
0
44
4
11
13
9
10
0
0
5.1
ChronoEdit [ 33 ]
1
1
26
0
0
0
0
0
0
0
60
20
12
7
16
13
1
1
8.8
Qwen3-VL-8B [ 35 ]
1
0
0
0
0
0
0
0
0
0
64
46
33
21
12
21
1
0
11.1
Qwen-Image-Edit-2511 [ 32 ]
0
0
0
0
0
0
0
0
0
0
50
55
44
16
23
23
0
0
11.7
Table 1: Evaluation results across visual reasoning tasks . Our model establishes a new state-of-the-art across both task-specific and unified training settings, demonstrating exceptional robustness to increasing task complexity.
Table 7
Models
Accuracy
Path Repetition (%)
Time
8
16
32
8
16
32
DiffThinker [ 11 ]
100
97
56
100.00
100.00
73.74
15.72
Ours ( τ=2 )
100
72
11
100.00
89.15
45.26
16.02
Ours ( τ=5 )
100
94
27
100.00
98.49
63.90
16.54
Ours ( τ=10 )
100
99
49
100.00
99.93
82.33
17.52
Ours ( τ=20 )
100
100
74
100.00
100.00
96.47
19.27
Table 4: Inference-Time CoT Scaling on Maze Benchmarks. Comparison of Accuracy and Path Repetition Rate across different token budgets ( τ ). Performance consistently improves, particularly on challenging Maze-32.
Figure 9
Figure 8: Qualitative comparison of path reasoning. While MLLM-Only enables the model to understand the basic topological structure of the maze compared to the untuned baseline, it still lacks sufficient visual reasoning representations. Consequently, it exhibits erratic and wandering paths.
Figure 9: Progressive image editing results. Left: step-by-step object addition, where a stone lantern and a deer are sequentially introduced into the scene. Right: object transformation, where the deer is progressively modified into a sheep.
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Task
Maze
Sudoku
TSP
VSP
Total
Size
75K
40K
30K
37.4K
182.4K
Appendix
Table 5: Dataset Composition. Overview of the sample size for each task in our constructed dataset, totaling 182.4K instances.
Models
Accuracy (ACC)
Path Repetition (%)
8
16
32
8
16
32
Single-Stage
39
42
14
83.49
91.34
90.34
Two-Stage
100
100
90
100.00
100.00
98.13
Appendix
Table 6: Single-stage training and multi-stage training on the Maze task.
Figure 10: Terminal Consolidation’s Excessive Training Results.
Figure 11: OOD Test Result. EndoCoT demonstrates stronger generalization than baseline methods when solving Sudoku puzzles with varying resolutions and font styles.
General Settings
LoRA Target Modules
Hyperparameter
Value
Component
Modules
Base Model
Qwen Image Edit 2511
DiT Attn.
to_q/k/v , add_proj , to_out
Learning Rate
1×10−4
DiT FFN
img/txt_mlp.net.2
Training Epochs
5
DiT Mod.
img/txt_mod.1
LoRA Rank
32
Text Enc. Attn.
q/k/v/o_proj
LoRA Target
DiT + Text Enc.
Text Enc. FFN
gate/up/down_proj
Appendix
Table 7: Training hyperparameters and LoRA configuration.
Figure 12: Gradient difference between reasoning and non-reasoning datasets.
Figure 13: Overview of the multi-step image editing dataset generation pipeline.
Figure 14: Pseudocode for Dataset Generation
Figure 15: Qualitative comparison of EndoCoT and other methods on TSP .
Figure 16: Qualitative comparison of EndoCoT and other methods on VSP .
Figure 17: Qualitative comparison of EndoCoT and other methods on Sudoku .
Figure 18: Qualitative comparison of EndoCoT and other methods on Maze .
Multimodal chain-of-thought reasoning studies how AI models combine visual, audio, and textual evidence through intermediate reasoning to solve complex tasks.
Chain-of-thought (CoT) reasoning has enabled multi-modal large language models (MLLMs) to tackle complex visual reasoning tasks by generating explicit intermediate reasoning steps in natural language. However, this text-based reasoning paradigm is inherently slow at inference time with even thousands of tokens and fundamentally constrained by the expressiveness of natural language. In this paper, we propose CoLT, (Chain of Latent Thoughts), a novel framework that teaches multi-modal models to reason through a chain of latent thought representations instead of verbose text tokens, which can perform thinking with as few as 3 steps. Naively forcing the model to think with latent states easily produces meaningless semantics and makes training unstable. To effectively regulate the latent reasoning process, we introduce a lightweight external decoder that provides step-level supervision for each latent reasoning step in two complementary directions: a forward mode that decodes latent thoughts into the textual reasoning of the next step, and a backward mode that aligns decoder hidden states with the model's latent thoughts given preceding textual context. We further incorporate internal supervision that encourages coherent step-by-step latent transitions. The decoder and internal supervision are removed during inference to maintain high efficiency of latent reasoning. Extensive experiments on eight benchmarks demonstrate that CoLT not only outperforms existing latent reasoning methods such as CODI and SIM-CoT, but also surpasses latent visual reasoning approaches that rely on auxiliary images with costly annotation requirements. Compared to text CoT methods, CoLT can notably reduce the inference time by 10.1× and text decoding time by 22.6×. Code is released at https://github.com/hulianyuyy/CoLT.
Lianyu Hu, Shengqian Qin, Zeqin Liao +4
Nanyang Technological University, Singapore · Shanghai Jiao Tong University, China · NKIARI, Shenzhen Futian, China +2
Diffusion large language models (dLLMs) are emerging as promising alternatives to autoregressive (AR) LLMs. Recently, this paradigm has been extended to multimodal tasks, leading to the development of diffusion multimodal large language models (dMLLMs). These models are expected to retain the reasoning capabilities of LLMs while enabling faster inference through parallel generation. However, when combined with Chain-of-Thought (CoT) reasoning, dMLLMs exhibit two critical issues. First, we observe that dMLLMs often generate the final answer token at a very early timestep. This trend indicates that the model determines the answer before sufficient reasoning, leading to degraded reasoning performance. Second, during the initial timesteps, dMLLMs show minimal dependency on visual prompts, exhibiting a fundamentally different pattern of visual information utilization compared to AR vision-language models. In summary, these findings indicate that dMLLMs tend to generate premature final answers without sufficiently grounding on visual inputs. To address these limitations, we propose Position and Step Penalty (PSP) and Visual Reasoning Guidance (VRG). PSP penalizes tokens in later positions during early timesteps, delaying premature answer generation and encouraging progressive reasoning across timesteps. VRG, inspired by classifier-free guidance, amplifies visual grounding signals to enhance the model's alignment with visual evidence. Extensive experiments across various dMLLMs demonstrate that our method achieves up to 7.5% higher accuracy while delivering more than 3x speedup compared to reasoning with four times more diffusion steps.
Keuntae Kim, Mingyu Kang, Yong Suk Choi
Department of Computer Science, Hanyang University · Department of Artificial Intelligence, Hanyang University
The effectiveness of Chain-of-Thought (CoT) prompting in Multimodal Large Language Models (MLLMs) remains uncertain: across several visual reasoning benchmarks, CoT prompting often degrades performance compared to direct prompting. In this paper, we provide a systematic analysis of CoT behavior in three modern MLLM families across model scales on datasets requiring step-wise visual evidence. Our analysis identifies two recurring failure modes: premature answer commitment and limited direct visual-token access during rationale generation. We further find that standard CoT-style Supervised Fine-Tuning (CoT-SFT) can mitigate these issues only partially, while often increasing reliance on textual priors and reducing counterfactual visual dependence. Motivated by these findings, we propose Attentive-CoT (Att-CoT), an attention-guided fine-tuning objective that encourages CoT trajectories to delay answer commitment while maintaining sustained visual-token access. Att-CoT can be plugged into any CoT-SFT training run without architectural changes. Experiments on three visual reasoning benchmarks across six MLLMs show that Att-CoT enhances CoT performance over standard fine-tuning.