Multimodal large language models increasingly use visual chain-of-thought (Visual CoT) to reason about spatial, temporal, and embodied environments. By generating intermediate reasoning images, Visual CoT provides an intuitive mechanism for visual foresight but introduces substantial inference overhead, which is particularly problematic for proactive video reasoning. We ask whether models can learn to think visually during training while reasoning directly at inference. We introduce Internalized Visual Thinking (IVT), a post-training framework that jointly optimizes textual prediction and next-embedding prediction over unlabeled videos. Given a partially observed video, IVT predicts latent representations of future frames together with the target textual answer, encouraging the model to capture motion, object transitions, interactions, and latent intent. At inference, IVT generates the answer directly without synthesizing or re-encoding future frames. We conduct controlled studies across target representations, decoder designs, prediction horizons, data mixtures, training curricula, and predictive objectives. IVT improves over direct-answer fine-tuning on all six evaluation settings while retaining the same inference pathway. Compared with explicit Visual CoT, IVT achieves comparable or better performance and reduces average end-to-end latency by more than 5x. Together, our findings suggest that explicit pixel-space generation at inference time, as used in visual chain-of-thought, may not be necessary for effective proactive video reasoning. Predictive world modeling can be internalized during training to produce multimodal reasoners that are both more accurate and substantially more efficient.
Figures & tables
Figure 1: Three post-training paradigms for proactive video reasoning. Answer-Only SFT maps the observed input directly to the predicted textual state. Visual CoT first generates an explicit future visual state in pixel space and then conditions on it to produce the answer, so it pays for the image generation cost at inference. In contrast, our Internalized Visual Thinking (IVT) predicts the future visual state as a latent embedding target only during training ; at inference it generates the answer directly, without synthesizing an intermediate image, and therefore executes the same computation graph as Answer-Only SFT.
Latency ( ↓ )
Answer Quality ( ↑ )
Task
Method
Mean
P95
ROUGE-L
BERTScore
METEOR
CIDEr
Early Event Pred.
Answer-Only SFT
1.07
1.79
34.4
41.3
24.3
1.43
Visual CoT (generated frame)
6.56 (6.2 × )
8.72 (4.9 × )
36.4 ( + 5.9%)
43.2 ( + 4.5%)
29.8 ( + 22.4%)
1.69 ( + 17.9%)
Visual CoT (oracle frame)
—
—
41.9 ( + 21.6%)
48.3 ( + 17.0%)
34.2 ( + 40.6%)
2.05 ( + 43.4%)
Next Event Pred.
Answer-Only SFT
1.32
2.05
47.2
50.2
35.3
2.14
Visual CoT (generated frame)
6.55 (5.0 × )
7.12 (3.5 × )
46.8 ( − 0.8%)
49.9 ( − 0.7%)
36.9 ( + 4.3%)
2.13 ( − 0.6%)
Table 1: Accuracy–efficiency trade-off of Visual CoT. For each task, we report performance averaged across the validation subsets of Ego-Exo4D, Ego4D, and EPIC-KITCHENS-100. Conditioning on a generated future frame improves early-event prediction but provides limited benefit for next-event prediction, while substantially increasing mean inference latency. Replacing the generated frame with the ground-truth future frame improves both tasks, revealing a considerably higher upper bound when accurate future visual context is available. This gap suggests that the effectiveness of Visual CoT depends strongly on the fidelity of the generated future state, as intermediate images that fail to preserve plausible scene dynamics or task-relevant visual information may provide weak or misleading conditioning signals.
Figure 2: Comparison of different target representations. We report ROUGE-L, BERTScore, METEOR, and CIDEr, averaged across the three benchmarks for early-event and next-event prediction. The dashed line denotes the Answer-Only SFT baseline. Predicting a Flux-VAE latent consistently achieves the strongest performance across all metrics and both tasks, whereas semantic targets provide smaller and less consistent gains.
Figure 3: Effect of the understanding-to-prediction data-mixture ratio on Ego-Exo4D early-event prediction. We report performance throughout training for three sampling ratios. Understanding-heavy mixtures ( 3:1 and 5:1 ) improve more rapidly at early checkpoints but subsequently plateau or decline. In contrast, the balanced 1:1 mixture improves steadily and achieves the strongest result on all four metrics at 20k steps.
Figure 4: Comparison of predictive objectives on Ego-Exo4D early-event prediction. We compare rectified-flow matching and direct feature regression for VAE and DINOv2 future-state targets. Direct regression substantially outperforms flow matching for DINOv2 and is also competitive with flow matching for VAE across most checkpoints and metrics.
Method
ROUGE-L
BERTScore
METEOR
CIDEr
Answer-Only SFT
30.6
32.5
22.6
1.60
Two-Stage Training
28.6 ( − 6.5%)
30.7 ( − 5.6%)
21.3 ( − 5.5%)
1.54 ( − 3.5%)
Joint Training
34.5 ( + 12.6%)
36.5 ( + 12.4%)
27.4 ( + 21.4%)
2.02 ( + 26.3%)
Table 2: Joint optimization is more effective than sequential training. We report the results on Ego-Exo4D early-event prediction. Relative changes are computed against the Answer-Only SFT baseline. Two-stage training underperforms Answer-Only SFT on every metric, whereas joint training achieves the strongest result across all metrics. Best value in each column is shown in bold .
Figure 5: Interaction between decoder architecture and prediction horizon on Ego-Exo4D early-event prediction. We report the experimental results for each prediction horizon H , defined as the number of predicted future frames. The dashed line denotes the Answer-Only SFT baseline. Both architectures outperform it across all horizons. The shared Dense decoder performs best at short horizons, whereas the MoE architecture is comparatively less sensitive to horizon and reaches its strongest results around H=3 .
Early Event Pred. (top-1)
Next Event Pred. (top-5)
Method
ROUGE-L
BERTScore
METEOR
CIDEr
ROUGE-L
BERTScore
METEOR
CIDEr
External models fine-tuned on the same data
LLaVA-NeXT-Video-7B
24.3
5.5
17.4
1.11
44.2
42.3
33.7
2.40
VideoLLaMA3-7B
14.6
17.4
7.7
0.31
30.2
26.3
19.6
1.30
Qwen2.5-VL-7B
34.9
36.0
26.6
1.75
48.0
45.9
37.9
2.77
Qwen3-VL-8B
37.0
38.1
28.5
1.90
52.1
49.5
42.0
3.09
Table 3: Results on the full Ego-Exo4D validation set. We evaluate early-event prediction with top-1 decoding and next-event prediction with top-5 decoding using ROUGE-L, BERTScore, METEOR, and CIDEr. Answer-Only SFT uses the same matched backbone and training setup as IVT but removes future-state prediction, retaining only the textual prediction objective. External models are included as reference points, while bold and underlined values denote the best and second-best results within the controlled BAGEL-7B-based block.
Early Event Pred. (top-1)
Next Event Pred. (top-5)
Method
ROUGE-L
BERTScore
METEOR
CIDEr
ROUGE-L
BERTScore
METEOR
CIDEr
External models fine-tuned on the same data
LLaVA-NeXT-Video-7B
36.8
30.3
27.2
1.49
46.3
46.3
33.1
1.98
VideoLLaMA3-7B
15.6
19.9
9.8
0.44
24.3
22.2
13.3
0.63
Qwen2.5-VL-7B
35.8
38.6
26.0
1.39
49.2
47.9
35.6
2.12
Qwen3-VL-8B
39.5
41.9
29.1
1.59
52.9
51.8
39.7
2.37
Table 5: Results on the full EPIC-KITCHENS-100 validation set. We evaluate early-event prediction with top-1 decoding and next-event prediction with top-5 decoding using ROUGE-L, BERTScore, METEOR, and CIDEr. Answer-Only SFT uses the same matched backbone and training setup as IVT but removes future-state prediction, retaining only the textual prediction objective. External models are included as reference points, while bold and underlined values denote the best and second-best results within the controlled BAGEL-7B-based block.
Table 6: Training hyperparameters. Default configuration used for the target-representation ablations and main IVT experiments. Architecture and curriculum variants modify only the corresponding design choices described in the main paper.
Table 11
Answer-Only SFT
Visual CoT
IVT (Ours)
Task
Benchmark
Mean
P95
Mean
P95
Mean
P95
Early Event Prediction
Ego-Exo4D
1.42
2.45
7.08
10.54
1.52
2.34
Ego4D
0.83
1.27
6.38
7.28
0.81
1.12
EPIC-KITCHENS-100
0.95
1.64
6.24
8.34
0.87
1.30
Next Event Prediction
Ego-Exo4D
1.37
2.40
7.70
8.62
1.25
1.91
Ego4D
0.92
1.49
3.72
4.01
0.82
0.98
Appendix
Table 9: End-to-end inference latency. Per-sample latency is measured on a single B200 GPU with batch size 1 and bf16 precision over 200 validation examples per benchmark. Mean and P95 latency are reported in seconds. IVT retains an inference profile comparable to Answer-Only SFT, whereas Visual CoT incurs substantially higher latency due to intermediate future-frame generation and visual re-encoding.