cs.CVNov 7, 2026

Mental Imagery That Matters: Curing Latent Collapse in Multimodal Reasoning

Authors: Mi Luo, Rohan Dugad, Zihui Xue, Alex Dimakis, Kristen Grauman

Organizations: The University of Texas at Austin · UC Berkeley · Bespoke Labs

Abstract

Mental simulation, the ability to reason by constructing and updating internal visual states, is central to flexible human cognition, but remains elusive in current multimodal large language models. Latent multimodal reasoning offers a promising route toward this capability by interleaving text with continuous visual latent blocks, allowing models to maintain internal visual workspaces without decoding them into pixels. In this paper, we ask whether these latent visual states actually matter. Through a direct perturbation analysis across representative latent-reasoning MLLMs, we find that they often do not: dropping, replacing, or shuffling generated visual latents has only minor effects on the model's own final-answer probability, while equivalent perturbations to the linguistic reasoning trace are substantially more damaging. We call this failure mode Mental Simulation Collapse (MSC): the model appears to perform interleaved visual reasoning, but routes most answer-relevant computation through text. We further diagnose that MSC follows naturally from existing training objectives: visual alignment can make generated latents resemble helper-image embeddings, but it does not require the decoder to rely on them; subsequent text-only supervision further leaves the semantic role of the latent pathway unidentifiable. To address this, we introduce AnchorSim (Anchored Mental Simulation), a contrastive distillation framework that trains models to make their generated visual latents consequential. Across multiple visual reasoning benchmarks, AnchorSim improves performance, while diagnostic re-evaluation shows that visual perturbations now meaningfully affect predictions. Our results expose a fundamental failure mode of latent multimodal reasoning and demonstrate that mental simulation must be anchored to model behavior in order to matter.

Figures & tables

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Visual Latents Know More Than They Say: Unsilencing Latent Reasoning in MLLMs

    May 4, 2026Xin Zhang, Qiqi Tao, Jiawei Du +2Inference-Time OptimizationTest-Time Optimization

  2. Fill the GAP: A Granular Alignment Paradigm for Visual Reasoning in Multimodal Large Language Models

    May 12, 2026Yanting Miao, Yutao Sun, Dexin Wang +8Multimodal Large Language ModelsLatent Visual Reasoning

  3. LanteRn: Latent Visual Structured Reasoning

    Mar 26, 2026André G. Viveiros, Nuno Gonçalves, Matthias Lindemann +1Latent Visual ReasoningVLM Reasoning