Chain-of-thought (CoT) reasoning has proven effective in improving the reasoning abilities of large language models by breaking complex tasks into step-by-step solutions. In recent years, researchers have attempted to extend CoT reasoning to visual generative models to improve their ability to handle complex tasks. However, existing CoT methods for text-to-image generation still rely mainly on textual reasoning and therefore cannot fully use and preserve the correct semantic information already present in an image, as even the most detailed language cannot capture all visual information. With advances in image generation, models such as FLUX and BAGEL can generate images conditioned on visual inputs. Recent work has attempted to carry images forward as context in the chain of thought, but this introduces another limitation: unsuitable earlier images can strongly interfere with the model, while repeated whole-image regeneration introduces new hallucinations at each round and causes them to accumulate. To address these issues, we propose MLIE-CoT, a visual chain-of-thought framework based on multi-level image editing. MLIE-CoT learns to select the repair granularity from three levels, preserving correct semantics whenever possible during reasoning while reducing the effect of incorrect content. To improve the ability of multimodal models to route errors of different severity to appropriate repair levels, we construct an image-repair dataset covering different error cases and use it to train a repair-level router. Extensive experiments demonstrate that MLIE-CoT significantly outperforms both its base models and methods using fixed repair strategies, producing clear improvements across multiple benchmarks.
Figures & tables
Figure 1: Failure cases caused by negative visual anchors and collateral changes during image-conditioned repair.
Figure 2: Quality degradation caused by repeated whole-image regeneration. Across both sequences, successive corrections introduce artifacts and overwrite content that was initially correct.
Figure 3: Representative results for the three routed repair levels.
Figure 4: Overview of the MLIE-CoT framework. The analyzer identifies mismatches between the prompt and current image. A frozen multimodal reasoning backbone feeds its representation to a trainable three-way prediction head, which selects L1 local editing with cropped regions, L2 whole-image editing conditioned on the current image, or L3 regeneration by the prompt refiner and generator. The critic examines the new image: a pass returns the final image and terminates, whereas a failure sends the result back for another diagnosis and repair round.
Level
Action
Typical scenario
L0
Accept current image
The current image satisfies the constraints in the target prompt.
L1
Local repair
Errors in color, attributes, counts, or a few objects that affect only a small number of regions.
L2
Whole-image editing
Spatial relations, background, or layout require adjustment, but the current image remains worth preserving.
L3
Text-conditioned regeneration
The main concept or overall composition is incorrect, or multiple connected errors are difficult to edit one by one.
Table 4: Fixed-level and router ablations on the same set of initial images.
Figure 5: Effect of the number of MLIE-CoT steps on generation quality and cost for BAGEL and FLUX.2 on GenEval and WISE.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Comparison
Base [95% CI]
MLIE-CoT [95% CI]
Δ [95% CI]
Bagel on GenEval
0.8082 [0.7791, 0.8359]
0.8566 [0.8279, 0.8840]
+0.0484 [+0.0430, +0.0555]
Bagel on WISE
0.7040 [0.6780, 0.7300]
0.7600 [0.7330, 0.7860]
+0.0560 [+0.0400, +0.0720]
Flux.2 on GenEval
.848 [.819, .876]
.884 [.857, .909]
+.036 [+.018, +.054]
Flux.2 on WISE
.734 [.708, .760]
.778 [.752, .803]
+.044 [+.028, +.060]
Appendix
Table S1: Paired uncertainty and significance results for the two specified same-prompt comparisons. Raw/Holm-adjusted randomization p -values are .1100/.1100 for WISE and .0003/.0006 for GenEval. The WISE interval for the improvement includes zero and is not significant after correction; the GenEval improvement remains significant.
Text-to-image generation creates images from natural-language prompts using generative models, with research focused on fidelity, control, efficiency, and safety.
Multimodal large language models (MLLMs) excel at visual reasoning but rely on text-based chain-of-thought (CoT), lacking interpretable visual intermediates. Existing methods use opaque tokens or external tools, missing key properties. We propose Gen-VCoT, a framework using expert vision models to generate RGB images as reasoning intermediates. It has three stages: visual grounding (SAM segmentation), geometric reasoning (Marigold depth maps), and semantic reasoning (Qwen2-VL integration). An adaptive router selects reasoning depth. Evaluations show Gen-VCoT improves spatial (25% better) and depth (50% better) questions, but may hurt simple factual queries. Text CoT outperforms visual intermediates on CLEVR (91.2% vs 62.5%), showing task-dependent optimal representations. Gen-VCoT establishes a new paradigm for interpretable multimodal reasoning.
Zhiqiang Zhou, Junliang Dai, Xu ling
Hunan Chemical Industry Vocational and Technical College, Zhuzhou, Hunan 412000, China
Unified multi-modal large language models (MLLMs) have achieved strong text-to-image generation quality, but still struggle with structure-aware prompt following, where object counts, spatial relations, attribute bindings, and coarse layouts must be preserved. We attribute this limitation in part to the entanglement of structural planning and appearance rendering within a single conditioning stream. To address this issue, we propose Implicit Visual Chain-of-Thought (IV-CoT), a latent visual reasoning framework for query-conditioned image generation. IV-CoT decomposes the visual conditioning queries into a structural-to-semantic cascade, where structural queries first form a latent visual plan and semantic queries then render appearance conditioned on this plan. To guide the structural queries, we introduce training-only sketch supervision, which encourages them to capture structure from sketches without requiring sketch extraction or intermediate decoding at inference time. IV-CoT performs implicit CoT reasoning in a single forward pass and achieves superior results on GenEval and T2I-CompBench. Visualizations and analyses demonstrate that the learned structural and semantic queries play complementary roles in structure-aware generation.
Zixuan Li, Haokun Lin, Yicheng Xiao +10
NLPR, Institute of Automation, Chinese Academy of Sciences · The University of Hong Kong · Ant Group
Chain-of-thought (CoT) reasoning has proven effective for enhancing problem-solving in large language models. However, when applied to multimodal LLMs (MLLMs), existing CoT approaches suffer from a fundamental limitation: they perform reasoning entirely in text without accessing visual features during the reasoning process. After initial visual encoding, image information becomes inaccessible, forcing models to reason based solely on whatever was captured in the initial description, which forms a `vision-blind reasoning' paradigm that limits fine-grained visual extraction, error verification, and adaptive attention. We propose Text-Visual Interleaved Chain-of-Thought (TVI-CoT), a framework that enables explicit interleaving of textual reasoning and visual feature access through learnable control tokens <THINK>, <LOOK> and <ANSWER>. These tokens allow dynamic switching between reasoning and visual grounding, attending to relevant image regions conditioned on the evolving reasoning state. Experiments on eight benchmarks demonstrate state-of-the-art results among MLLM-based CoT methods and notable performance boost compared to the baseline: +6.1% on MMMU, +3.8% on MathVerse, +3.4% on MathVista, and +3.4% on ScienceQA. Code is available at https://github.com/hulianyuyy/TVI-CoT.
Lianyu Hu, Xiaoyu Ma, Zeqin Liao +1
College of Computing and Data Science of Nanyang Technological University, Singapore.