Mutually Adversarial Self-Training with Evolving Data for Unified Multimodal Models

Wentao Zhou; Weijie Gan; Jiayun Wang

cs.CVSep 28, 2026

Mutually Adversarial Self-Training with Evolving Data for Unified Multimodal Models

Authors: Wentao Zhou, Weijie Gan, Jiayun Wang

Organizations: Georgia Institute of Technology · Washington University in St. Louis

Abstract

Unified multimodal models (UMMs) combine image generation and visual understanding in a shared backbone. Since generation and understanding are inverse tasks, recent studies self-train UMMs by letting the two branches cooperatively supervise each other. We introduce MATE (Mutually Adversarial self-Training with Evolving data), a reinforcement-learning-based post-training framework in which the two branches instead challenge each other, and the challenges evolve as the model trains. MATE lets generation and understanding take turns to be challenger and solver. Given an image, the understanding branch proposes several candidate descriptions that the generation branch must turn back into similar images, and vice versa. The candidates are screened for consistency with the image or prompt they were proposed from, and the solver is trained on the candidate it handles worst. The adversary thus comes from the model's own outputs, and no separate adversary is trained. Moreover, the candidates that defeat one branch become the sources of the next challenges to the other in the next epoch, which keeps the challenges evolving with the model and turns the training into self-play in data space. On Janus-Pro-1B, MATE improves GenEval by 2.4 points, DPG-Bench by 1.7 points, and the average over nine understanding benchmarks by 0.7 points, while strengthening consistency across repeated image-text cycles.

Figures & tables

Figure 1: Self-improvement through mutual challenges. (a) In other self-improving methods such as SUDER ( Hong et al., 2025 ) , one branch scores an output to reward its producer. (b) Our MATE selects valid outputs that challenge the opposite branch, trains that solver, and uses the selected outputs to construct new challenges in the next epoch. In both panels, $\mathsf{G}_{{\bm{\theta}}}$ and $\mathsf{U}_{{\bm{\theta}}}$ share one backbone and the examples are real training records. (c) MATE and Janus-Pro-1B on the 11 benchmarks of the main tables. Each axis uses its own truncated linear range, shared by both methods, so polygon areas do not measure overall performance. MME-C is in cognition points, and other scores are percentages.

Figure 2: MATE: mutual challenge and data evolution. (a) Mutual challenge. The lower path exchanges the challenger and solver roles of the upper path. (1) The challenger proposes $N$ candidates. (2) A fixed scoring function admits candidates above $\tau$ , or the highest-scoring candidate if none passes. (3) The solver makes $M$ attempts per admitted candidate, scored by its scoring function $\mathsf{R}$ and input-consistency term $\lambda\mathsf{C}$ . (4) The lowest mean score $\bar{r}$ identifies the selected challenge. (5) GRPO reuses all $M$ existing attempts for that challenge. Inputs and selected challenges are records from (b), and other candidates and attempts are shown symbolically. (b) Data evolution. Consecutive image and text records from one training trajectory illustrate ${\bm{I}}^{(0)}\!\to\!{\bm{p}}^{(1)}\!\to\!{\bm{I}}^{(2)}\!\to\!{\bm{p}}^{(3)}$ . Each arrow applies the procedure in (a) and advances one epoch, so a selected output becomes the input of the opposite branch. Quoted text is taken from actual model outputs and shortened for display.

Method	# Params	GenEval $\uparrow$ (per category)						GenEval $\uparrow$	DPG-Bench $\uparrow$
Method	# Params	Single Obj.	Two Obj.	Counting	Colors	Position	Color Attr.	Overall	Overall
Generation only
SD3-Medium ( Esser et al., 2024 )	2B	99.0	94.0	72.0	89.0	33.0	60.0	74.0	84.1
LlamaGen ( Sun et al., 2024 )	0.8B	71.0	34.0	21.0	58.0	7.0	4.0	32.0	–
Sana-0.6B ( Xie et al., 2025a )	0.6B	99.0	76.0	64.0	88.0	18.0	39.0	64.0	83.6
Sana-1.6B ( Xie et al., 2025a )	1.6B	99.0	77.0	62.0	88.0	21.0	47.0	66.0	84.8

Table 1: Main results on image generation.

Figure 3: Generation samples. Images for the same prompt from Janus-Pro-1B and the three self-training methods applied to it; three prompts are from GenEval and one from DPG-Bench. One selected sample per model and prompt is shown from four generated samples, using the same sampling settings. The DPG-Bench prompt asks for a giraffe and a zebra standing side by side in a field of wildflowers; its displayed text is shortened.

Method	# Params	POPE	MME-C	MMB	SEED	MMStar	V ∗ Bench	CV-2D	CV-3D	SEED2+	Avg.
Understanding only
LLaVA-v1.5 ( Liu et al., 2024 )	7B	86.1	302.1	72.9	65.8	33.1	45.0	62.0	64.3	41.3	56.5
DeepSeek-VL-1.3B ( Lu et al., 2024 )	1.3B	85.9	225.0	74.5	66.0	39.9	42.9	66.2	62.7	43.7	56.7
LLaVA-OV-0.5B ( Li et al., 2025 )	0.5B	87.8	237.5	67.1	66.6	37.7	39.3	38.6	50.6	52.6	52.2
UMMs
Janus ( Wu et al., 2025a )	1.3B	84.1	238.6	70.0	62.9	37.6	41.4	49.1	58.7	38.5	52.5

Table 2: Main results on visual understanding. Higher is better on every benchmark.

Figure 4: Understanding samples. Six questions from POPE, SEED-Bench, V ∗ Bench, and CV-Bench. Multiple-choice answers show the selected option’s text; three questions are shortened and images cropped for display. A check mark denotes agreement with the reference answer.

Figure 5: An evolving training input under MATE. An original dataset pair (left) is followed through five epochs. Selected images and descriptions become inputs for the next epoch. The image and text in each column come from separate training directions and need not form a matched pair. Descriptions are shortened for display. Figure 7 in Appendix C shows a second example.

Figure 6: Round-trip consistency and ablation. (a) Left: SigLIP cosine between the original image

{\bm{I}}_{0}

and the image

{\bm{I}}_{k}

after

k

rounds. Right: SigLIP text cosine between the first description

{\bm{p}}_{1}

and the

k

-th description

{\bm{p}}_{k}

. Points are means over 32 held-out samples and whiskers are standard errors. (b) Understanding and generation averages of Janus-Pro-1B, MATE, and three variants of MATE.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Other branch supplies	Tasks optimized	Training inputs	Hard cases selected for	Added module
SUDER ( Hong et al., 2025 )	likelihood	$\mathsf{G}$ , $\mathsf{U}$	corpus	–	–
GvU ( Pan et al., 2026 )	likelihood	$\mathsf{G}$	corpus	–	–
UniRL ( Mao et al., 2025 )	answers	$\mathsf{G}$ , $\mathsf{U}$	constructed prompts	–	–
SILMM ( Qu et al., 2025 )	answers	$\mathsf{G}$	model-written prompts	–	–
HermesFlow ( Yang et al., 2025 )	answers	$\mathsf{G}$ , $\mathsf{U}$	corpus	–	–
SRUM ( Jin et al., 2026 )	ratings	$\mathsf{G}$	corpus	–	–

Appendix

Table 3: Comparison of MATE with the closest post-training methods for UMMs. Hard-case selection refers to choosing training inputs by the difficulty they pose to the task being optimized.

Training
Candidates per input $N$	4	Learning rate	$10^{-6}$
Rollouts per candidate $M$	8	GRPO clipping range	0.2
Admission threshold $\tau$	0.5	Reference penalty	0.01 (low-variance KL)
Standard-deviation floor $\epsilon$	$10^{-6}$	Trainable parameters	all (2.1B)
Epochs	5	Hardware	6 $\times$ NVIDIA B200
Effective batch size	48 (24 inputs per step)	Consistency weight $\lambda$	0.1

Appendix

Table 4: Training hyperparameters and evaluation settings.

	Und. Avg. $\uparrow$	Gen. Avg. $\uparrow$
Janus-Pro-1B	56.9	78.3
MATE	57.6	80.3
w/o consistency term	57.5	80.5
All admitted candidates	57.5	80.6
Two separate networks	66.3	64.1
$\Delta$ vs. own base models	$-$ 0.9	$+$ 2.1

Appendix

Table 5: Additional variants of MATE, with Janus-Pro-1B and MATE for reference.

Figure 7: A second evolving training input under MATE, displayed as in Figure 5 . Descriptions are shortened for display.

Figure 8: One complete round of MATE, in the format of Figure 2 (a). All candidates, attempts, and scores of one training pair in one epoch (pair 4, epoch 2 of the main run). Upper row: from the source description ${\bm{p}}$ , the generation branch proposes $N=4$ images ${\bm{I}}_{1},\dots,{\bm{I}}_{4}$ ; each is checked against ${\bm{p}}$ ( $\mathsf{R}_{\mathsf{G}}$ , admission threshold $\tau=0.5$ ); the understanding branch writes $M=8$ descriptions of each admitted image, scored by $\mathsf{R}_{\mathsf{U}}$ (the eight scores and one example are shown); the image with the lowest mean score $\bar{r}$ is selected and its eight attempts train $\mathsf{U}_{{\bm{\theta}}}$ with GRPO. Lower row: from the source image ${\bm{I}}$ , the understanding branch proposes $N=4$ descriptions ${\bm{p}}_{1},\dots,{\bm{p}}_{4}$ , checked against ${\bm{I}}$ ( $\mathsf{R}_{\mathsf{U}}$ ); the generation branch produces $M=8$ images from each admitted description, scored by $\mathsf{R}_{\mathsf{G}}$ ; the description with the lowest $\bar{r}$ is selected and its attempts train $\mathsf{G}_{{\bm{\theta}}}$ . The two selected outputs become this pair’s inputs in the next epoch. Descriptions are shortened for display.

Explore similar work

Ask, Solve, Generate: Self-Evolving Unified Multimodal Understanding and Generation via Self-Consistency Rewards
Jun 25, 2026Ritesh Thawkar, Shravan Venkatraman, Omkar Thawakar +5Multimodal Generation Large Multimodal Models
1Mohamed bin Zayed University of Artificial Intelligence · 4Linköping University · 3Australian National University +1
DIVA: Harnessing the Representation Divergence in Unified Multimodal Models for Mutual Reinforcement
May 25, 2026Renjie Lu, Xulong Zhang, Xiaoyang Qu +2Multimodal Model Inductive Bias
Ping An Technology (Shenzhen) Co., Ltd., Shenzhen, China · University of Science and Technology of China, Hefei, China.
UniEvo-VL: An On-policy Self-Distillation Training Recipe for Multimodal Model Self-improvement
Sep 30, 2026Fang Wu, Da Xing, Yanjie Huang +16Multimodal Model Self-Evolution
Stanford University · Johns Hopkins University · Independent Researcher +7

Jun 25, 2026, cs.CVJ/K move · Enter open · S save

Ask, Solve, Generate: Self-Evolving Unified Multimodal Understanding and Generation via Self-Consistency Rewards

Ritesh Thawkar, Shravan Venkatraman, Omkar Thawakar, Abdelrahman Shaker+4

1Mohamed bin Zayed University of Artificial Intelligence · 4Linköping University · 3Australian National University+1

Multimodal Generation Large Multimodal Models Multimodal Understanding Visual Representations Self-Consistency

Open paper AskPDF arXiv

Most unified large multimodal models (LMMs) that support both visual understanding and image generation still rely on curated post-training supervision, such as human annotations, preference labels, or external reward models. We ask whether a unified LMM can improve both abilities autonomously using only unlabeled images. We propose a self-evolving training framework with three internal roles: a Proposer that generates visual questions, a Solver that answers and evaluates them, and a Generator that synthesizes images. Training uses only self-derived consistency signals, without human annotations, preference labels, or task-trained external reward/judge models. To stabilize learning, we introduce Solver Token Entropy (STE), a continuous difficulty signal based on token-level prediction uncertainty that remains useful even when sample-level consistency becomes unreliable. For image generation, we design a multi-scale internal evaluation scheme that combines question-answer fidelity scoring with cycle-consistent captioning. This creates a solver-mediated coupling, where better visual understanding enables more reliable generation assessment and stronger internal training signals. The framework preserves the same role decomposition, reward logic, and training schedule across diffusion-based BLIP3o, rectified-flow BAGEL, and autoregressive VARGPT-v1.1 architectures, requiring only each backbone's native prompting and generation interface. Across eight understanding metrics, our method consistently improves over the corresponding base models. On BAGEL, it achieves a

+3.5\%

absolute gain on MMMU and improves GenEval image generation performance from

82\%

85\%

. Code and models are publicly released.

Abstract

Explore similar work

Ask, Solve, Generate: Self-Evolving Unified Multimodal Understanding and Generation via Self-Consistency Rewards

DIVA: Harnessing the Representation Divergence in Unified Multimodal Models for Mutual Reinforcement

UniEvo-VL: An On-policy Self-Distillation Training Recipe for Multimodal Model Self-improvement