Organizations: Shanghai Qi Zhi Institute · Institute for Interdisciplinary Information Sciences, Tsinghua University · Georgia Institute of Technology · Xiongan AI Institute
Recent work has explored visual reasoning through image generation, where models solve visual-reasoning problems by directly generating solution images. Intuitively, iterative samplers such as flow matching are well suited to this task, since each sampling step may resemble one reasoning step. However, in this work, from a circuit complexity perspective, we show that this intuition is misleading. We prove that as long as the model outputs satisfy certain mild regularity conditions, the trajectory of the vanilla flow-matching sampler can be simulated by a constant number of sequential rounds of parallel model evaluations. Consequently, additional sampling steps cannot increase the expressive power of a constant-depth backbone model under the standard complexity-theoretic model of transformers. We then reveal that self-conditioning, which conditions the model on the previous step's estimate of the final image, plays a previously unrecognized role in removing this bottleneck. Controlled experiments on diverse visual-reasoning tasks show that flow matching with self-conditioning achieves higher accuracy and greater reasoning depth than vanilla flow matching, and its performance improves with more sampling steps. Theoretically, we prove that with self-conditioning, a constant-depth backbone can achieve expressive power equivalent to that of space-bounded Turing machines. Our study suggests a design principle for future visual reasoning research: carry more information between sampling steps, so that additional steps translate into deeper reasoning.
Figures & tables
Figure 1: Controlled experiments on three synthetic tasks: line tracing, long multiplication, and long text rendering (§ 4.2 ). The top row shows an example input and target output for each task, and the bottom row plots performance against training FLOPs. Across all three tasks, self-conditioning improves the performance of flow matching by large margins.
Figure 2
Figure 2: Overview of self-conditioning. Each prediction zi+1 is obtained by extrapolating the state xi along its velocity vi to t=1 .
Figure 3: SelfCondFM ’s performance scales strongly with NFE, while that of VanillaFM saturates early.
Method
VSP-Super
Maze
TSP
Sudoku
Jigsaw
Avg.
VanillaFM
89.0
64.3
51.0
60.5
64.7
65.9
SelfCondFM
92.0
(+3.0)
70.7
(+6.4)
66.7
(+15.7)
80.0
(+19.5)
74.3
(+9.6)
76.7
(+10.8)
Table 1: Controlled experiments on the DiffThinker tasks (accuracy, %). Under matched training FLOPs and NFE, SelfCondFM outperforms VanillaFM on every task.
Method
VSP-Super
Maze
TSP
Sudoku
Avg.
16
32
8
16
32
12
15
18
45
40
35
30
Task-specific training
Qwen3-VL-8B (SFT)
61
8
53
37
0
59
60
43
30
17
2
0
30.8
Qwen3-VL-8B (GRPO)
0
0
0
0
0
0
0
0
0
0
0
0
0.0
DiffThinker
99
80
100
100
65
76
72
59
97
94
55
13
75.8
EndoCoT
99
85
100
100
90
77
77
73
100
100
95
64
88.3
Table 2: Large-scale comparison on the DiffThinker tasks (accuracy, %). Each column header is a difficulty level, following the evaluation of Dai et al. (2026) , from which the baseline numbers are taken. The best results are in bold and the second best are underlined. For unified training, each method uses a single adapter trained jointly on data from all four tasks.
Appendix figures & tables17 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Character error rate (CER) of parsed text from the long text rendering evaluation, against cumulative training FLOPs. SelfCondFM keeps improving throughout training, whereas VanillaFM plateaus nearly an order of magnitude above the final SelfCondFM value.
Figure 6: Training curves of line tracing with 3 different seeds, plotting mean LCP on the held-out set of 20 instances at 100 NFE against training FLOPs. For each method, the curve shows the mean over the 3 runs, and the shaded band shows the mean ± one standard deviation. SelfCondFM saturates at 50.0±0.0 , while VanillaFM reaches 5.6±2.7 at the matched budget.
Figure 7: Training curves of long multiplication with 3 different seeds, plotting the mean accuracy on the held-out set under 50 NFE against training FLOPs. As in Figure 6 , each curve and its shaded band show the mean and the mean ± one standard deviation over the 3 runs. VanillaFM reaches (35.0±8.9)% while SelfCondFM is higher at (90.0±5.3)% under the fixed budget.
Figure 8: Examples on line tracing. For visualization, each trajectory is truncated shortly after the first error of VanillaFM ; the full trajectory contains 50 stages. VanillaFM first fails at stages 8 and 14 in (a) and (b), respectively, by incorrectly switching colors at line crossings. Black boxes mark the errors. In contrast, SelfCondFM preserves each line’s color throughout all 50 stages in both examples (LCP =50 ).
Figure 9: Examples on long multiplication. VanillaFM makes accumulation errors in both cases, resulting in incorrect final answers, with the erroneous digits highlighted in red boxes. In contrast, SelfCondFM produces the correct product for both examples.
Figure 10: Examples on long text rendering. VanillaFM produces text errors as well as line-break and whitespace errors, yielding CERs of 0.0106 and 0.0410 in (a) and (b), respectively. Red boxes highlight text errors, and blue dashed boxes indicate line-break and whitespace errors. In contrast, SelfCondFM renders both passages correctly with zero CER.
Figure 11: Mean LCP in the NFE scaling comparison of Figure 3 , plotted against cumulative inference FLOPs. Both methods have zero mean LCP at NFE=1 . SelfCondFM reaches 49.84 at 8.29×1011 FLOPs. VanillaFM reaches 7.66 at 2.74×1011 FLOPs, ending below 8.0 up to 2.28×1012 FLOPs (which corresponds to NFE=50 ).
Method
2
3
4
Avg.
Gemini-3-Flash
71
16
0
29.0
GPT-5
30
0
0
10.0
Qwen3-VL-8B
7
0
0
2.3
Qwen3-VL-8B (SFT)
95
56
9
53.3
Qwen3-VL-8B (GRPO)
6
0
0
2.0
DiffThinker
99
98
80
92.3
Appendix
Table 3: Jigsaw accuracy (%) at difficulty levels 2 to 4. “Avg.” is the unweighted mean over the three levels. The best results are in bold and the second best are underlined.
Figure 12: VSP-Super examples on 16×16 and 32×32 grids. The VanillaFM routes contain disconnected segments. In contrast, the SelfCondFM routes correctly connect the endpoints.
Figure 13: Maze examples on 8×8 and 16×16 grids. In (a), VanillaFM leaves separate path fragments; in (b), its route stops before reaching the blue endpoint. In contrast, SelfCondFM completes both routes.
Figure 14: TSP examples with 12 and 15 cities. VanillaFM fails to find the optimal tour in both cases, yielding excess tour lengths of +0.45 and +0.67, respectively. In contrast, SelfCondFM produces the optimal solution for both instances.
Figure 15: Sudoku examples with 35 and 45 given digits. VanillaFM produces incorrect solutions with explicit Sudoku constraint violations: row 3 in (a) contains two 2s, while row 5 in (b) contains two 1s. The red boxes highlight these errors. In contrast, SelfCondFM correctly solves both puzzles.
Figure 16: Jigsaw examples on 2×2 and 3×3 tile grids. VanillaFM fails in both examples: it swaps the two bottom tiles of (a), giving the order 3, 2 instead of 2, 3, and swaps the first two tiles in the top row of (b), giving 1, 7 instead of 7, 1. SelfCondFM matches the reference permutation in both examples. Tile identifiers are part of the original images.
Figure 17: Maximum joint ℓ∞ empirical Lipschitz estimate among 30 held-out trajectories at each problem length.
Figure 18: Normalized attention mass of the first layer to each input source. Endpoint prediction maintains a nontrivial share throughout the trajectory, declining nearly monotonically from 31.3% to 20.0% , thus peaking at early steps, where noise dominates.
Figure 19: Parallel local computation. Accuracy of partial products on long multiplication at every NFE from 1 to 50 : both methods keep nearly all partial products correct within a single generated worksheet.
Figure 20: In-place error correction on long multiplication. The condition changes from w to w′ after step k , and the curves show the accuracy of matching w′ as k varies: the output of SelfCondFM is redirected in place to the new problem when the swap falls in the first half of the trajectory, whereas VanillaFM follows the new problem only partially.
Flow-matching models generate samples through iterative velocity predictions, but typically do not revisit these predictions using the intermediate representations that produced them. We introduce CorrFlow, a self-correcting flow-matching framework that reuses this information through a lightweight refiner. Under rectified-flow interpolation, the error of the one-step clean prediction equals the realized velocity residual scaled by the noise level. Motivated by this identity, the refiner learns to predict residual corrections from the base model's clean prediction, conditioning, noise level and hidden states from the same forward pass. The correction is added to the base velocity before each sampling update, so subsequent predictions are made from the corrected state. This requires no additional base-model evaluations and only one lightweight refiner evaluation per step. Our theoretical analyses establish conditions under which feature reuse outperforms raw-input affine predictors and refiner training achieves greater residual-risk reduction than continued base-model training under a matched training compute budget. Experiments on class-conditional image generation, text-to-video generation and image-to-video generation show consistent improvements over the corresponding base models, including gains in visual quality and physical consistency. Project Page: https://fogradio.github.io/corrflow-project
Hangwei Zhang, Nathan Rouillé, Chuanxia Zheng
PVG, Nanyang Technological University · Télécom Paris, Institut Polytechnique de Paris
Flow Matching enables high-quality visual generation via continuous-time dynamics, but inference remains costly due to multiple sequential function evaluations. Existing acceleration methods reduce the number of function evaluations but often introduce additional training overhead, degrade quality, or fail to account for input-dependent variability. We propose COFLOW, an inference-time method that adaptively selects the step counts each generation based on the prompt features. Our context-aware COFLOW is trained online with an unsupervised reward that balances inference efficiency and generation fidelity. Our method is plug-and-play, requiring no retraining of the underlying generative model. It generalizes to image and video generation, achieving over 2.5x speedup while preserving perceptual and semantic quality. We further provide a theoretical analysis establishing an O(1/K) forward-Euler discretization error bound under standard regularity conditions.
Flow matching enables language generation in few steps, but whether additional integration steps improve reasoning remains unclear. We prove that a flow parameterized by a two-layer Transformer can solve graph reachability, with the required number of integration steps increasing with the target's distance from the root. Yet, standard flow language models can fail to benefit from additional steps on reasoning tasks. We attribute this limitation to objectives that supervise each time point independently, without explicitly training successive steps to build on one another. To address this, we instead train through the model's own latent rollout over a randomly sampled subinterval of [0, 1], decoding only at the endpoint. On ProsQA, this raises accuracy to 97% and enables performance to improve with additional integration steps. For the longer rollouts required by reasoning tasks such as Sudoku and Maze, retracting the latent state onto a sphere stabilizes the dynamics and yields substantial gains over baselines with more than three times as many parameters. Sampling multiple rollouts further improves performance when paired with a parameter-free selection score, although reliable selection remains challenging for longer answers. Together, these results establish a theoretical basis for reasoning with flows and show how rollout training, stable latent dynamics, and rollout selection help realize this capacity in practice.
Faissal Izermine, Hanru Bai, Oscar Davis +1
Max Planck Institute for Intelligent Systems · ELLIS Institute Tübingen · ETH Zurich +3