Mutually Adversarial Self-Training with Evolving Data for Unified Multimodal Models

Wentao Zhou; Weijie Gan; Jiayun Wang

cs.CVSep 28, 2026

Mutually Adversarial Self-Training with Evolving Data for Unified Multimodal Models

Authors: Wentao Zhou, Weijie Gan, Jiayun Wang

Organizations: Georgia Institute of Technology · Washington University in St. Louis

Abstract

Unified multimodal models (UMMs) combine image generation and visual understanding in a shared backbone. Since generation and understanding are inverse tasks, recent studies self-train UMMs by letting the two branches cooperatively supervise each other. We introduce MATE (Mutually Adversarial self-Training with Evolving data), a reinforcement-learning-based post-training framework in which the two branches instead challenge each other, and the challenges evolve as the model trains. MATE lets generation and understanding take turns to be challenger and solver. Given an image, the understanding branch proposes several candidate descriptions that the generation branch must turn back into similar images, and vice versa. The candidates are screened for consistency with the image or prompt they were proposed from, and the solver is trained on the candidate it handles worst. The adversary thus comes from the model's own outputs, and no separate adversary is trained. Moreover, the candidates that defeat one branch become the sources of the next challenges to the other in the next epoch, which keeps the challenges evolving with the model and turns the training into self-play in data space. On Janus-Pro-1B, MATE improves GenEval by 2.4 points, DPG-Bench by 1.7 points, and the average over nine understanding benchmarks by 0.7 points, while strengthening consistency across repeated image-text cycles.

Figures & tables

Figure 1: Self-improvement through mutual challenges. (a) In other self-improving methods such as SUDER ( Hong et al., 2025 ) , one branch scores an output to reward its producer. (b) Our MATE selects valid outputs that challenge the opposite branch, trains that solver, and uses the selected outputs to construct new challenges in the next epoch. In both panels, $\mathsf{G}_{{\bm{\theta}}}$ and $\mathsf{U}_{{\bm{\theta}}}$ share one backbone and the examples are real training records. (c) MATE and Janus-Pro-1B on the 11 benchmarks of the main tables. Each axis uses its own truncated linear range, shared by both methods, so polygon areas do not measure overall performance. MME-C is in cognition points, and other scores are percentages.

Figure 2: MATE: mutual challenge and data evolution. (a) Mutual challenge. The lower path exchanges the challenger and solver roles of the upper path. (1) The challenger proposes $N$ candidates. (2) A fixed scoring function admits candidates above $\tau$ , or the highest-scoring candidate if none passes. (3) The solver makes $M$ attempts per admitted candidate, scored by its scoring function $\mathsf{R}$ and input-consistency term $\lambda\mathsf{C}$ . (4) The lowest mean score $\bar{r}$ identifies the selected challenge. (5) GRPO reuses all $M$ existing attempts for that challenge. Inputs and selected challenges are records from (b), and other candidates and attempts are shown symbolically. (b) Data evolution. Consecutive image and text records from one training trajectory illustrate ${\bm{I}}^{(0)}\!\to\!{\bm{p}}^{(1)}\!\to\!{\bm{I}}^{(2)}\!\to\!{\bm{p}}^{(3)}$ . Each arrow applies the procedure in (a) and advances one epoch, so a selected output becomes the input of the opposite branch. Quoted text is taken from actual model outputs and shortened for display.

Method	# Params	GenEval $\uparrow$ (per category)						GenEval $\uparrow$	DPG-Bench $\uparrow$
Method	# Params	Single Obj.	Two Obj.	Counting	Colors	Position	Color Attr.	Overall	Overall
Generation only
SD3-Medium ( Esser et al., 2024 )	2B	99.0	94.0	72.0	89.0	33.0	60.0	74.0	84.1
LlamaGen ( Sun et al., 2024 )	0.8B	71.0	34.0	21.0	58.0	7.0	4.0	32.0	–
Sana-0.6B ( Xie et al., 2025a )	0.6B	99.0	76.0	64.0	88.0	18.0	39.0	64.0	83.6
Sana-1.6B ( Xie et al., 2025a )	1.6B	99.0	77.0	62.0	88.0	21.0	47.0	66.0	84.8

Table 1: Main results on image generation.

Figure 3: Generation samples. Images for the same prompt from Janus-Pro-1B and the three self-training methods applied to it; three prompts are from GenEval and one from DPG-Bench. One selected sample per model and prompt is shown from four generated samples, using the same sampling settings. The DPG-Bench prompt asks for a giraffe and a zebra standing side by side in a field of wildflowers; its displayed text is shortened.

Method	# Params	POPE	MME-C	MMB	SEED	MMStar	V ∗ Bench	CV-2D	CV-3D	SEED2+	Avg.
Understanding only
LLaVA-v1.5 ( Liu et al., 2024 )	7B	86.1	302.1	72.9	65.8	33.1	45.0	62.0	64.3	41.3	56.5
DeepSeek-VL-1.3B ( Lu et al., 2024 )	1.3B	85.9	225.0	74.5	66.0	39.9	42.9	66.2	62.7	43.7	56.7
LLaVA-OV-0.5B ( Li et al., 2025 )	0.5B	87.8	237.5	67.1	66.6	37.7	39.3	38.6	50.6	52.6	52.2
UMMs
Janus ( Wu et al., 2025a )	1.3B	84.1	238.6	70.0	62.9	37.6	41.4	49.1	58.7	38.5	52.5

Table 2: Main results on visual understanding. Higher is better on every benchmark.

Figure 4: Understanding samples. Six questions from POPE, SEED-Bench, V ∗ Bench, and CV-Bench. Multiple-choice answers show the selected option’s text; three questions are shortened and images cropped for display. A check mark denotes agreement with the reference answer.

Figure 5: An evolving training input under MATE. An original dataset pair (left) is followed through five epochs. Selected images and descriptions become inputs for the next epoch. The image and text in each column come from separate training directions and need not form a matched pair. Descriptions are shortened for display. Figure 7 in Appendix C shows a second example.

Figure 6: Round-trip consistency and ablation. (a) Left: SigLIP cosine between the original image

{\bm{I}}_{0}

and the image

{\bm{I}}_{k}

after

k

rounds. Right: SigLIP text cosine between the first description

{\bm{p}}_{1}

and the

k

-th description

{\bm{p}}_{k}

. Points are means over 32 held-out samples and whiskers are standard errors. (b) Understanding and generation averages of Janus-Pro-1B, MATE, and three variants of MATE.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Other branch supplies	Tasks optimized	Training inputs	Hard cases selected for	Added module
SUDER ( Hong et al., 2025 )	likelihood	$\mathsf{G}$ , $\mathsf{U}$	corpus	–	–
GvU ( Pan et al., 2026 )	likelihood	$\mathsf{G}$	corpus	–	–
UniRL ( Mao et al., 2025 )	answers	$\mathsf{G}$ , $\mathsf{U}$	constructed prompts	–	–
SILMM ( Qu et al., 2025 )	answers	$\mathsf{G}$	model-written prompts	–	–
HermesFlow ( Yang et al., 2025 )	answers	$\mathsf{G}$ , $\mathsf{U}$	corpus	–	–
SRUM ( Jin et al., 2026 )	ratings	$\mathsf{G}$	corpus	–	–

Appendix

Table 3: Comparison of MATE with the closest post-training methods for UMMs. Hard-case selection refers to choosing training inputs by the difficulty they pose to the task being optimized.

Training
Candidates per input $N$	4	Learning rate	$10^{-6}$
Rollouts per candidate $M$	8	GRPO clipping range	0.2
Admission threshold $\tau$	0.5	Reference penalty	0.01 (low-variance KL)
Standard-deviation floor $\epsilon$	$10^{-6}$	Trainable parameters	all (2.1B)
Epochs	5	Hardware	6 $\times$ NVIDIA B200
Effective batch size	48 (24 inputs per step)	Consistency weight $\lambda$	0.1

Appendix

Table 4: Training hyperparameters and evaluation settings.

	Und. Avg. $\uparrow$	Gen. Avg. $\uparrow$
Janus-Pro-1B	56.9	78.3
MATE	57.6	80.3
w/o consistency term	57.5	80.5
All admitted candidates	57.5	80.6
Two separate networks	66.3	64.1
$\Delta$ vs. own base models	$-$ 0.9	$+$ 2.1

Appendix

Table 5: Additional variants of MATE, with Janus-Pro-1B and MATE for reference.

Figure 7: A second evolving training input under MATE, displayed as in Figure 5 . Descriptions are shortened for display.

Figure 8: One complete round of MATE, in the format of Figure 2 (a). All candidates, attempts, and scores of one training pair in one epoch (pair 4, epoch 2 of the main run). Upper row: from the source description ${\bm{p}}$ , the generation branch proposes $N=4$ images ${\bm{I}}_{1},\dots,{\bm{I}}_{4}$ ; each is checked against ${\bm{p}}$ ( $\mathsf{R}_{\mathsf{G}}$ , admission threshold $\tau=0.5$ ); the understanding branch writes $M=8$ descriptions of each admitted image, scored by $\mathsf{R}_{\mathsf{U}}$ (the eight scores and one example are shown); the image with the lowest mean score $\bar{r}$ is selected and its eight attempts train $\mathsf{U}_{{\bm{\theta}}}$ with GRPO. Lower row: from the source image ${\bm{I}}$ , the understanding branch proposes $N=4$ descriptions ${\bm{p}}_{1},\dots,{\bm{p}}_{4}$ , checked against ${\bm{I}}$ ( $\mathsf{R}_{\mathsf{U}}$ ); the generation branch produces $M=8$ images from each admitted description, scored by $\mathsf{R}_{\mathsf{G}}$ ; the description with the lowest $\bar{r}$ is selected and its attempts train $\mathsf{G}_{{\bm{\theta}}}$ . The two selected outputs become this pair’s inputs in the next epoch. Descriptions are shortened for display.

Explore similar work

Jun 25, 2026cs.CV

Ask, Solve, Generate: Self-Evolving Unified Multimodal Understanding and Generation via Self-Consistency Rewards

Most unified large multimodal models (LMMs) that support both visual understanding and image generation still rely on curated post-training supervision, such as human annotations, preference labels, or external reward models. We ask whether a unified LMM can improve both abilities autonomously using only unlabeled images. We propose a self-evolving training framework with three internal roles: a Proposer that generates visual questions, a Solver that answers and evaluates them, and a Generator that synthesizes images. Training uses only self-derived consistency signals, without human annotations, preference labels, or task-trained external reward/judge models. To stabilize learning, we introduce Solver Token Entropy (STE), a continuous difficulty signal based on token-level prediction uncertainty that remains useful even when sample-level consistency becomes unreliable. For image generation, we design a multi-scale internal evaluation scheme that combines question-answer fidelity scoring with cycle-consistent captioning. This creates a solver-mediated coupling, where better visual understanding enables more reliable generation assessment and stronger internal training signals. The framework preserves the same role decomposition, reward logic, and training schedule across diffusion-based BLIP3o, rectified-flow BAGEL, and autoregressive VARGPT-v1.1 architectures, requiring only each backbone's native prompting and generation interface. Across eight understanding metrics, our method consistently improves over the corresponding base models. On BAGEL, it achieves a

+3.5\%

absolute gain on MMMU and improves GenEval image generation performance from

82\%

85\%

. Code and models are publicly released.

Ritesh Thawkar, Shravan Venkatraman, Omkar Thawakar +5

1Mohamed bin Zayed University of Artificial Intelligence · 4Linköping University · 3Australian National University +1

Multimodal Generation Large Multimodal Models Multimodal Understanding Visual Representations Self-Consistency

Ask PDF

May 25, 2026cs.CV

DIVA: Harnessing the Representation Divergence in Unified Multimodal Models for Mutual Reinforcement

Unified Multimodal models (UMMs) built on a single architecture have shown impressive performance in both understanding and generation. We identify a fundamental challenge that lies in inductive biases induced by distinct supervision signals: generation branch prefers high-fidelity, fine-grained representations capable of reconstruction, while the understanding favours semantically discriminative embeddings that remain invariant to task-irrelevant factors. Consequently, optimizing these complementary but non-equivalent objectives within a monolithic backbone leads to mutual impairment instead of enhancement. In this paper, we first analyze the root cause of this interference in unified backbones and reveal a complementary structure in their internal representations. Motivated by the observation, we propose DIVA, a self-improved post-training framework that transforms the representation divergence into interior synergy. By explicitly factorizing the visual representation into shared and unique components based on two complementary information flow, DIVA enables both the understanding and generation branches to achieve beneficial transferring while preserving the integrity of unique information from cross-flow interference via mutual information estimation. Despite its generality, our method consistently achieves improvements across visual understanding (+7.82%) and generation (+8.46%). The official code is available at: https://github.com/Jayyy-H/DIVA.

Renjie Lu, Xulong Zhang, Xiaoyang Qu +2

Ping An Technology (Shenzhen) Co., Ltd., Shenzhen, China · University of Science and Technology of China, Hefei, China.

Multimodal Model Inductive Bias

Ask PDF

Sep 30, 2026cs.AI

UniEvo-VL: An On-policy Self-Distillation Training Recipe for Multimodal Model Self-improvement

Modern multimodal models bring generation and understanding into a single unified system, which enables them to provide and learn from their own feedback. Motivated by this unified capacity, we introduce UniEvo-VL, a self-evolving framework for multimodal models to learn from this constructive self-correction feedback during test-time compute. Instead of relying on a separate, often larger, teacher, we leverage their self-critiques as privileged information and ask a single multimodal model to act as both teacher and student with different contexts. The student only sees the vanilla question, while the teacher conditions on the privileged critique. Then training minimizes the per-state divergence between their denoising diffusion distributions over the student's own sampling trajectories. Experiments demonstrate that UniEvo-VL improves the image generation capabilities of multimodal models, while maintaining their sensitivity to additional reflection information. Specifically, we build on top of the open-source Qwen-image-2512 and observe a significant performance gain from 0.747 to 0.808 on GenEval and from 32.97 to 35.53 on GenEval2 Soft-TIFA. Moreover, attempts with more powerful external critics (e.g., GPT5.6-Luna) show that multimodal models with strong judge capabilities can anticipate a higher self-evolving ceiling. Last but not least, mixed text-rendering outcomes show that our self-improvements may not be uniform across different tasks. Our study aims to shed light on the current hot recursive self-improvement research line to enhance the user experience when using multimodal models without external supervision or guidance.

Fang Wu, Da Xing, Yanjie Huang +16

Stanford University · Johns Hopkins University · Independent Researcher +7

Multimodal Model Self-Evolution Feedback Divergence

Ask PDF