Latent Reasoning in Continuous Space for Unified Multimodal Models

Byungwoo Jeon; Yoonwoo Jeong; Hyunseok Lee; Minsu Cho; Jinwoo Shin

cs.CVOct 30, 2026

Latent Reasoning in Continuous Space for Unified Multimodal Models

Authors: Byungwoo Jeon, Yoonwoo Jeong, Hyunseok Lee, Minsu Cho, Jinwoo Shin

Organizations: KAIST · POSTECH · RLWRLD

Abstract

Unified Multimodal Models (UMMs) achieve remarkable success on diverse multimodal understanding and generation tasks, yet they still struggle to effectively leverage language-based chain-of-thought reasoning for image generation. As a result, existing reasoning-augmented UMMs often fail to inherit the test-time scaling and fine-grained controllability observed in large language models. We attribute this failure to a fundamental modality mismatch: reasoning is performed in a discrete token space, whereas image generation operates in a continuous space. To bridge this gap, we introduce a Latent Reasoning framework in Continuous space (LARC), which alleviates the token-level bottleneck by reasoning over continuous hidden states. Specifically, the training framework consists of two stages: (i) curriculum supervised fine-tuning (SFT), and (ii) information-gain reinforcement learning (RL). First, the curriculum SFT stage gradually converts token-level reasoning into latent reasoning. Subsequently, the information-gain RL stage self-evolves latent reasoning without ground truth supervision. Experiments across diverse text-to-image generation benchmarks show that LARC consistently improves generation quality, prompt following, and compositional fidelity. Notably, LARC achieves state-of-the-art performance on GenEval with a 7%p gain over BAGEL and demonstrates clear test-time scaling behavior, outperforming token-based generation under both parallel and sequential test-time scaling strategies.

Figures & tables

Figure 1: Test-time scaling. We investigate the effect of the number of latent during test-time. Under controlled experiments, LARC performs remarkable test-time scaling behavior , outperforming the best-of-N sampling on image generation [ 1 ] , image editing [ 27 ] , and visual reasoning [ 75 ] tasks.

Figure 2: Overview of LARC. Our framework, LARC, consists of two stages. (1) Stage 1 adapts supervised fine-tuning (SFT) while gradually introducing latent thought on continuous space where the last hidden state of UMM becomes the next input embedding of the model. (2) Stage 2 adapts self-evolving reinforcement learning (RL) with information-gain reward derived from the internal knowledge. $\pi_{\text{ref}}$ denotes SFT model trained in Stage 1. Dotted line is task-specific option.

Type	Method	Single Obj. $\uparrow$	Two Obj. $\uparrow$	Counting $\uparrow$	Colors $\uparrow$	Position $\uparrow$	Attr. Binding $\uparrow$	Overall $\uparrow$
Reference	GPT-4o [ 23 ]	0.99	0.92	0.85	0.92	0.75	0.61	0.84
	SD3-medium [ 8 ]	0.99	0.94	0.72	0.89	0.33	0.46	0.74
	FLUX.1-dev [ 30 ]	0.99	0.81	0.79	0.74	0.20	0.47	0.67
UMM	Discrete UMM
	Janus-Pro [ 5 ]	0.99	0.89	0.59	0.90	0.79	0.66	0.80
	Emu3 [ 64 ]	0.98	0.71	0.34	0.81	0.17	0.21	0.54

Table 1: LARC achieves strong text-to-image generation performance on GenEval. We report accuracy (%) on GenEval subcategories and the overall score. Bold and underline indicate the best and second-best results within each group, respectively.

Type	Method	Global $\uparrow$	Entity $\uparrow$	Attribute $\uparrow$	Relation $\uparrow$	Other $\uparrow$	Overall $\uparrow$
Reference	SD3-medium [ 8 ]	87.90	91.01	88.83	80.70	88.68	84.08
Reference	FLUX.1-dev [ 30 ]	82.10	89.50	88.70	91.10	89.40	84.00
UMM	Discrete UMM
	Janus-Pro [ 5 ]	86.90	88.90	89.40	89.32	89.48	84.19
	Emu3 [ 64 ]	85.21	86.68	86.84	90.22	83.15	80.60
	Continuous UMM

Table 2: LARC improves prompt-following fidelity on DPG-Bench. We report scores across global, entity-level, attribute, relation, and other categories. Higher is better.

Method	NP $\uparrow$	T&P $\uparrow$	Short $\uparrow$	Medium $\uparrow$	Long $\uparrow$	Overall $\uparrow$
	Discrete UMM
Janus-Pro [ 5 ]	0.557	0.533	0.609	0.548	0.515	0.552
BLIP-o3 [ 4 ]	0.719	0.671	0.754	0.712	0.674	0.706
	Continuous UMM
Show-o2 [ 68 ]	0.825	0.778	0.825	0.819	0.807	0.811
BAGEL [ 6 ]	0.776	0.734	0.782	0.769	0.759	0.764

Table 3: LARC substantially improves instruction-guided image generation on OneIG-Bench. We report performance across prompt types and prompt lengths. NP refers natural language prompt. T&P denotes tag-based and phrase-based prompt. Higher is better.

Method	LC-T $\uparrow$	LC-I $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
Janus-Pro [ 5 ]	19.769	0.801	21.192	0.856	0.097
Emu3 [ 64 ]	19.536	0.803	22.036	0.868	0.092
Show-o2 [ 68 ]	20.350	0.831	24.661	0.919	0.065
BAGEL [ 6 ]	20.434	0.842	24.370	0.917	0.069
Uni-CoT [ 43 ]	20.215	0.837	23.925	0.904	0.088
LARC (ours)	22.509	0.880	25.710	0.953	0.049

Table 4: LARC improves image editing on CompBench. We report performance across score of CLIP [ 44 ] for text (LC-T) and image (LC-I), PSNR, SSIM and LPIPS. LC-T refers local CLIP score between predicted foreground and descriptions while LC-I refers CLIP similarity between predicted foreground and ground-truth image. Higher is better except for LPIPS.

Method	MME [ 14 ] $\uparrow$	MMB [ 39 ] $\uparrow$	MMMU [ 72 ] $\uparrow$	MIRA [ 75 ] $\uparrow$
Janus-Pro [ 5 ]	1567.1	79.2	41.0	4.9
Emu3 [ 64 ]	1487.3	67.4	31.6	4.1
Show-o2 [ 68 ]	1620.5	79.3	48.9	6.9
BAGEL [ 6 ]	1687.0	85.0	55.3	7.5
Uni-CoT [ 43 ]	1691.7	85.4	55.5	10.6
LARC (ours)	1725.3	88.1	56.2	11.7

Table 5: LARC consistently enhances multimodal understanding of UMM. We report matching score for MME, MMB, MMMU, and MIRA.

Method	Reasoning space	T2I gen. $\uparrow$	Comp. gen. $\uparrow$	Image editing $\uparrow$	Visual reasoning $\uparrow$
Baseline [ 6 ]	Discrete	0.81	0.76	20.43	7.5
Token-level Reasoning	Discrete	0.82	0.79	20.81	7.1
Latent Reasoning	Continuous	0.83	0.81	21.46	9.7

Table 6: Effect of latent reasoning compared to token-level reasoning. We report scores on GenEval for T2I generation, OneIG-Bench for compositional generation, CompBench for image editing, and MIRA for visual reasoning. Token-level Reasoning denotes standard Chain-of-Thought on discrete reasoning space. Latent Reasoning refers LARC in Stage 1 trained by SFT.

Method	T2I gen. $\uparrow$	Comp. gen. $\uparrow$	Image editing $\uparrow$	Visual reasoning $\uparrow$
Baseline [ 6 ]	0.81	0.76	20.43	7.5
Naive	0.83	0.79	21.02	9.9
Interleaved (ours)	0.83	0.81	21.46	9.7

Table 7: Ablation on latent reasoning strategy. We report scores on GenEval for T2I generation, OneIG-Bench for compositional generation, CompBench for image editing, and MIRA for visual reasoning. Naive refers latent reasoning without text segments proposed by the prior work [ 20 ] .

Method	T2I gen. $\uparrow$	Comp. gen. $\uparrow$	Image editing $\uparrow$	Visual reasoning $\uparrow$
Baseline [ 6 ]	0.81	0.76	20.43	7.5
Stage 1	0.83	0.81	21.46	9.7
Stage 1 & 2	0.88	0.87	22.51	11.7

Table 8: Ablation on training stages. We report scores on GenEval for T2I generation, OneIG-Bench for compositional generation, CompBench for image editing, and MIRA for visual reasoning.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Hyperparameter	Stage 1 (SFT)	Stage 2 (RL)
Optimizer	AdamW	AdamW
Algorithm	-	GRPO
Learning rate	1e-5	1e-6
Global batch size	-	64
Max number of tokens	$6\times 10^{5}$	-
Weight decay	$10^{-2}$	0.0

Appendix

Table 9: Hyperparameters for Stage 1 and Stage 2.

Source dataset	Task	# samples
FLUX-Reason [ 12 ]	Text-to-Image	500K
WEAVE [ 10 ]	Text-to-Image	100K
ScaleEdit [ 3 ]	Image Editing	400K
Total	–	1M

Appendix

Table 10: Composition of the 1M reasoning dataset.

Method	$N{=}1$	$N{=}2$	$N{=}4$	$N{=}6$	$N{=}8$	$N{=}10$
BAGEL [ 6 ] + Best-of- $N$	46.5	92.8	187.1	275.5	367.7	469.0
	$L{=}1$	$L{=}2$	$L{=}4$	$L{=}6$	$L{=}8$	$L{=}10$
LARC (ours)	46.8	46.9	47.0	47.3	47.6	48.0

Appendix

Table 11: Clock-time analysis under test-time scaling.

$\lambda_{\text{div}}$	Cos. Sim.	GenEval $\uparrow$	OneIG-Bench $\uparrow$	MIRA $\uparrow$
$0.0$	0.97	0.83	0.81	9.8
$0.05$	0.87	0.85	0.84	10.9
$0.1$ (ours)	0.79	0.88	0.87	11.7
$0.2$	0.71	0.85	0.85	11.4
$0.5$	0.65	0.82	0.82	10.6

Appendix

Table 12: Effect of the latent diversity reward. We report the average pairwise cosine similarity (Cos. Sim.) among latent steps within a trajectory, and accuracy on GenEval, OneIG-Bench, and MIRA.

Figure 3: Qualitative comparison during test-time. We compare BAGEL [ 6 ] with best-of- $N$ sampling to LARC on test-time scaling with $L$ latent steps on text-to-image generation task. First row presents $N=L=1$ , second row presents $N=L=4$ , and third row presents $N=L=10$ .

Figure 4: Qualitative comparison during test-time. We compare BAGEL [ 6 ] with best-of- $N$ sampling to LARC on test-time scaling with $L$ latent steps on image editing task. First row presents $N=L=1$ , second row presents $N=L=4$ , and third row presents $N=L=10$ .

Continuous Latent Reasoning

Continuous latent reasoning studies how models can compute through continuous hidden states instead of spelling out every reasoning step.

See all 82 papers

Latent Action Control for Reasoning-Guided Unified Image Generation
May 16, 2026Fuxiang Zhai, Sixiang Chen, Yingjin Li +4Unified Multimodal Models Compositional T2I Generation
The Hong Kong University of Science and Technology (Guangzhou)
UReason: Benchmarking Reasoning-to-Generation Alignment in Unified Multimodal Models
Feb 9, 2026Cheng Yang, Chufan Shi, Bo Shui +7Unified Multimodal Models Image Generation
University of California San Diego · University of Southern California · University of Illinois Urbana-Champaign +1
Ask, Solve, Generate: Self-Evolving Unified Multimodal Understanding and Generation via Self-Consistency Rewards
Jun 25, 2026Ritesh Thawkar, Shravan Venkatraman, Omkar Thawakar +5Unified Multimodal Models Multimodal Generation
1Mohamed bin Zayed University of Artificial Intelligence · 4Linköping University · 3Australian National University +1

May 16, 2026J/K move · Enter open · S save

Latent Action Control for Reasoning-Guided Unified Image Generation

Fuxiang Zhai, Sixiang Chen, Yingjin Li, Shuaibo Li+3

The Hong Kong University of Science and Technology (Guangzhou)

Unified Multimodal Models Compositional T2I Generation RL for Generative Models Controllable Image Generation

Open paper AskPDF arXiv

Unified multimodal models can encode visual understanding and image generation within a shared backbone, yet understanding does not automatically translate into control: models may infer objects, relations, or knowledge cues but fail to instantiate them in the generated image. We propose Latent Action Control (LAC), which makes reasoning actionable by representing it as hidden continuous actions inside a unified generator. Given a prompt, LAC rolls out a role-structured latent trajectory for planning, internal visual drafting, diagnosis, and refinement, and injects these actions into the hidden stream that conditions flow-based generation, without producing reasoning tokens or intermediate images. Since such action trajectories are unobserved, LAC learns them through prior-guided variational latent action alignment from training-only rendered semantic priors, draft image features, and supervised halting signals, followed by Latent-Flow GRPO to align the latent-to-image rollout with terminal visual feedback. This provides a control path from inferred relations, bindings, and knowledge cues to the generation process. Instantiated on BAGEL-7B-MoT, LAC consistently improves compositional and knowledge-grounded generation across GenEval, WISE, and T2I-CompBench, with the largest gains on spatial relations, attribute binding, and world-knowledge-sensitive prompts. Ablations and latent interventions show that the learned action trajectory is consumed by the generator, suggesting that unified generation benefits when understanding is not only encoded, but made actionable during generation.

Abstract

Continuous Latent Reasoning

Explore similar work

Latent Action Control for Reasoning-Guided Unified Image Generation

UReason: Benchmarking Reasoning-to-Generation Alignment in Unified Multimodal Models

Ask, Solve, Generate: Self-Evolving Unified Multimodal Understanding and Generation via Self-Consistency Rewards