Flow-of-Thought: A Framework for Visual Reasoning

Mariia Baidachna; Nicolas Pugeault

cs.CVOct 7, 2026

Flow-of-Thought: A Framework for Visual Reasoning

Authors: Mariia Baidachna, Nicolas Pugeault

Organizations: University of Glasgow

Abstract

Mental imagery, ``seeing with the mind's eye'' is an essential aspect of human cognition. Despite rapid progress Large Language Models (LLMs) and Vision Transformers (ViTs) still underperform on tasks requiring spatial understanding. To address this, we introduce Flow-of-Thought (FoT), a framework that integrates the generation of visual sketches as intermediate reasoning steps, mimicking mental imagery in humans. We train coordinate-aware trajectory flow fields on $SO(2)$ group orbits and cumulative shortest paths, then freeze the learned dynamics; same vs. different decisions compare competing generative hypotheses using foreground-weighted reconstruction energy. On locked tests FoT reaches 100.0% accuracy on Tetris and 99.0% on colored shapes. Under frozen transfer, the orbit-trained 2D flow improves over its endpoint-only control on BLINK Multi-view (72.2% vs. 63.9% on 133 public validation pairs), supporting continuous visual traces as an effective and interpretable representation for spatial reasoning in some out-of-distribution settings.

Figures & tables

Figure 1 : Training and inference in FoT. Top: the rotation example follows the rendered path

x_{t}=\gamma_{a}(t)

with target transport

u^{*}(p;\omega)=(-\omega p_{y},\omega p_{x})

. The field predicts

\hat{u}_{t}=v_{\theta}(x_{t},t,c,a)

and minimizes

\ell(\hat{u}_{t},u_{t}^{*})

; thus only

\theta

is learned. For mazes, the same slots contain the trace

\tau(t)

, map condition

m

, and additive field

v_{\theta}(\tau(t),t,m)

. Bottom:

A_{\mathcal{D}}

maps an arbitrary compatible item to

(x_{0},c,\mathcal{H},\mathcal{A})

. Each hypothesis/action is integrated under the same frozen field, producing the trace

\{x(t)\}

and endpoint

T_{\hat{\theta}}(x,a)=x(1)

. Rotation pairs use the paper’s exact SAME/DIFFERENT energies

E_{\rm same}

and

E_{\rm diff}

; dense tasks return

x(1)

. Only

A_{\mathcal{D}}

and the readout are dataset-specific.

Figure 2 : Representative source–target pairs for Tetris, colored shapes, and 3D Blocks ( Ganis and Kievit, 2015 ) ; each labeled group shows source on the left and target on the right. Maze inputs are shown with their generated traces in Figure 5 .

Task	Metric	ViT	DINOv3	Qwen3.5	GPT-4o	GPT-5.6	Claude 4.6	GPT-4V †	P 2 +Qwen3VL †	FoT
Tetris	Acc.	61.0	60.0	52.0	52.4 ‡	72.0	46.2 ‡	–	–	100.0
Colored	Acc.	62.0	49.0	51.0	50.0 ‡	98.0	51.6 ‡	–	–	99.0
3D Blocks	Acc.	–	50.4	51.3	50.0	57.3 ⋆	53.8	–	–	61.5
BLINK	Acc.	–	–	27.8	–	–	–	58.7	94.0	72.2
Maze	Trace acc.	–	–	59.0	50.6 ‡	75.0 ⋆	50.0 ‡	–	–	–
	End/prefix IoU	–	–	–	–	–	–	–	–	97.5/84.2

Table 1 : Results (%). † Published; ‡ historical items; ⋆ incomplete and excluded.

Figure 3 : Audited accuracy with 95% confidence intervals. Open markers are published point estimates without compatible reported uncertainty; intervals are not pairwise significance tests. Maze is omitted because its dense trajectory metrics are not accuracy-equivalent.

Figure 4 : Frozen-flow audits: (a) Tetris SAME, (b) colored-shape DIFFERENT, and (c) 3D Blocks transfer. $H_{\mathrm{same}}$ and $H_{\mathrm{diff}}$ are the best endpoint reconstructions for the competing branches; the remaining columns show the chosen branch at arbitrary times.

Figure 5 : Maze trajectory audit on a held-out maze. Alternating ground-truth and predicted states show accurate cumulative path construction together with faint premature activation.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Task	Supervision	Accuracy (%)	Endpoint IoU
Tetris	Rotation orbit	100.0 [96.3, 100.0]	0.876
Tetris	Endpoint only	94.0 [87.5, 97.2]	0.757
Tetris	Linear pixel path	39.0 [30.0, 48.8]	0.532
Colored	Rotation orbit	99.0 [94.6, 99.8]	0.832
Colored	Endpoint only	85.0 [76.7, 90.7]	0.633
Colored	Linear pixel path	51.0 [41.3, 60.6]	0.300

Appendix

Table 2: Matched path-supervision ablation. Accuracy is the seed-ensemble result with a Wilson 95% interval; endpoint IoU is measured at held-out non-grid angles.

Figure 6 : Matched path ablations and no-intermediate OOD controls. Boxes span three training seeds and retain all raw points; black circles and bars denote seed means and two-sided Student-

t

95% intervals. Open diamonds on accuracy panels denote seed-ensemble decisions with Wilson intervals over items.

Task	Supervision	Identity MSE	Inverse MSE	Composition MSE
Tetris	Rotation orbit	0.000987	0.002526	0.001138
Tetris	Endpoint only	0.001061	0.017123	0.012587
Tetris	Linear path	0.011518	0.031338	0.044970
Colored	Rotation orbit	0.001899	0.009159	0.004895
Colored	Endpoint only	0.001855	0.030288	0.014789
Colored	Linear path	0.009657	0.053068	0.026190

Appendix

Table 3: Frozen approximate

SO(2)

consistency audit. Lower defects are better. The analytic renderer supplies endpoint ground truth only and is not part of the decision.

Figure 7 : Seed distributions for identity, inverse/cycle, composition, and non-grid endpoint defects. The model is approximately group-consistent; equivariance is not built into the architecture.

Explore similar work

MentalThink: Shaping Thoughts in Mental SVG World
Jul 3, 2026Kangheng Lin, Jisheng Yin, Dingming Li +11Think Imagination
State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications · University of Chinese Academy of Sciences · StepFun
Imaginative Perception Tokens Enhance Spatial Reasoning in Multimodal Language Models
Jun 2, 2026Mahtab Bigverdi, Linjie Li, Weikai Huang +8Spatial Reasoning Perceptually
University of Washington · 2Allen Institute for AI · 3Microsoft +1
SketchVLM: Vision language models can annotate images to explain thoughts and guide users
Apr 23, 2026Brandon Collins, Logan Bolton, Hung Huy Nguyen +3Visual Reasoning Sketches
Auburn University · Adobe Research

Jul 3, 2026, cs.AIJ/K move · Enter open · S save

MentalThink: Shaping Thoughts in Mental SVG World

Kangheng Lin, Jisheng Yin, Dingming Li, En Yu+10

State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications · University of Chinese Academy of Sciences · StepFun

Think Imagination Thoughts Scalable Vector Graphic Sketches Multi-Turn Reinforcement Learning+1

Open paper AskPDF arXiv

We introduce MentalThink, a visual-symbolic reasoning paradigm that equips Multimodal LLMs (MLLMs) with an executable mechanism for "mental" visualization. The core of MentalThink is a think-with-SVG pipeline, where the model learns to generate, render, and interpret scalable vector graphics (SVG) code as an intermediate visual representation for multi-turn reasoning. By creating structured vector sketches, the model can externalize spatial hypotheses, inspect them through deterministic rendering, and reason within a constrained geometric space, effectively mimicking the human process of mental imagery. We instantiate this paradigm through a two-stage training framework, combining Supervised Fine-Tuning (SFT) for SVG syntactic alignment with multi-turn Reinforcement Learning (RL) to encourage iterative inspection, revision, and refinement of intermediate visual hypotheses. Extensive evaluations demonstrate that MentalThink achieves superior performance on spatial understanding and reasoning benchmarks (e.g., 55.1% on VSIBench, 76.0% on MindCube), showing that executable vector graphics provide a verifiable visual workspace for dynamic perspective taking, visual reflection, and compositional scene construction.

Abstract

Explore similar work

MentalThink: Shaping Thoughts in Mental SVG World

Imaginative Perception Tokens Enhance Spatial Reasoning in Multimodal Language Models

SketchVLM: Vision language models can annotate images to explain thoughts and guide users