Multimodal instruction following (MMIF) is crucial for building generalist agents. However, current training paradigms rely heavily on Supervised Fine-Tuning (SFT), which often leads to surface-level pattern matching and degrades general capabilities. While Reinforcement Learning with Verifiable Rewards (RLVR) offers a promising alternative, its scalability in MMIF is severely bottlenecked by the scarcity of high-quality, RL-ready multimodal data. To bridge this gap, we present MIFS (Multimodal Instruction Following Synthesis), a systematic pipeline designed to generate RL-ready multimodal data. Specifically, MIFS introduces a generative constraint protocol to synthesize diverse raw samples, followed by a learnability-aware distillation mechanism that filters data based on RL training dynamics to ensure stable policy optimization. Furthermore, a code-based verifier provides high-precision reward signals for policy learning. The resulting dataset comprises 90k samples across 8 constraint categories and 14 task domains. Empirical evaluations demonstrate that MIFS-trained MLLMs achieve an average improvement of 8.13% on four MMIF benchmarks and a 3× faster training convergence compared to using raw data. Crucially, our approach mitigates the generalization trade-offs typical of SFT, preserving core visual capabilities while significantly boosting instruction-following precision.
Figures & tables
Dataset
#Sample
#Constraint
#Domain
Eval?
Evaluator
SFT?
RLVR?
MIA-BENCH ( Qian et al. )
400
8
15
✓
LLM
✗
✗
CrafText ( Volovikova et al. 2025 )
3,924
-
4
✓
Rule
✗
✗
MMMT-IF ( Sun et al. 2025 )
990
4
13
✓
Rule
✗
✗
MM-IFInstruct ( Ding et al. 2025 )
23k
32
13
✓
LLM&Rule
✓
✗
MIFS (Ours)
90,838
8
14
✓
Rule
✓
✓
Table 1: Comparison of Multimodal Instruction Following (MM-IF) datasets. MIFS distinguishes itself by providing the largest scale of programmatically verifiable samples, specifically engineered for RLVR beyond simple SFT.
Figure 1: The overall framework of the Multimodal Instruction Synthesis suitable for both SFT and RL training. It comprises: (1) Instruction Composition, where the raw multimodal IF and their verifier are synthesized via LLMs; (2) Multi-Step Quality Distillation, featuring a three-tier filtering mechanism that prunes samples based on constraint density, model-perceived difficulty, and RL training dynamics; and (3) Sample Curation, illustrating data characteristics and partitioning strategy.
Figure 2: RL training dynamic filtering. Retaining aligned samples (i.e., those with low variance (a) ) and removing outlier samples (i.e., those with high variance (b)).
Figure 3: Empirical analysis of multi-step quality distillation. It showcases: (a) Stepwise contraction of data scale across steps; (b) Monotonic alignment gains in RL training;
Figure 4: The distribution of constraint types. The dataset comprises 8 verifiable constraint categories with an average density of 7.1 constraints per sample, ensuring high-constraint complexity.
Split
# Samples
Jaccard ≥0.8
Cosine ≥0.8
MIFS-SFT
83,280
0.00%
2.61%
MIFS-RL
60,048
0.00%
4.73%
MIFS-Eval
1,200
0.00%
1.31%
Table 2: Semantic overlap and near-duplicate rates across dataset splits. Minimizing instance redundancy prevents shortcut, safeguarding policies during RLVR training.
Figure 5: The hierarchical distribution of task diversity. The chart features a nested structure: the inner ring classifies tasks into four high-level cognitive domains, segment size reflects task frequency.
Method
Benchmarks
Average
MIABench
MM-IFEval
MIFS-Eval
IFEval
Qwen3-VL-4B
Base
92.2±0.2
58.5±0.4
72.8±0.4
79.1±0.2
75.6
MIFS-RL
95.5±0.1
66.8±0.5
85.5±0.4
87.4±0.3
83.8 ↑8.2
Qwen3-VL-8B
Base
91.2±0.3
62.7±0.5
74.5±0.3
81.2±0.2
77.4
Table 3: Performance comparison on Instruction Following benchmarks. We report scores for MIABench, MM-IFEval, MIFS-Eval, and IFEval, along with their group-wise average. All metrics are reported as the mean±std over 3 independent runs.
Method
OCR
MMVet
MMBen
Avg.
Qwen3-VL-4B
Base Model
86.50
65.60
84.41
78.84
MIFS-RL
87.52
67.32
83.24
79.36 ↑0.5
Qwen3-VL-8B
Base Model
88.00
69.72
85.30
81.01
MIFS-SFT
78.62
57.32
70.30
68.75 ↓12.3
Table 4: Performance on General Visual Capability benchmarks (i.e., OCRBench, MMVet, and MMBench).
Figure 6: Training process visualization: (a) Raw data and (b) final MIFS-RL data. The MIFS-RL curriculum demonstrates significantly faster convergence ( ∼3× ) and higher test performance.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Configuration
Value
Optimizer
AdamW ( β1=0.9,β2=0.95 )
Learning Rate
2×10−5
Learning Rate Scheduler
Cosine decay
Weight Decay
0.1
Global Batch Size
128
Precision
BFloat16
Appendix
Table 5: Hyper-parameters for SFT training.
Hyperparameter
Value
Data & Rollout
Global Batch Size
128
Max Prompt/Resp.
2k / 4k
Sampling Temp. τ
1.0
Top- p
0.6
Rollout Group Size G
8
Appendix
Table 6: Hyperparameters for GRPO training.
Code: Pass
Code: Fail
Human: Pass
35 (TP)
3 (FN)
Human: Fail
1 (FP)
11 (TN)
Metrics
Agreement: 92.0% | Precision: 97.22%
Appendix
Table 7: Confusion Matrix: Automated Verifier vs. Human Judgment (N=50).
Constraint Type
Definition
Example
Keyword Inclusion
Mandatory integration of specific lexical items.
“Must include the words ’sustainability’ and ’innovation’.”
Negative Constraint
Prohibiting the use of specific words, phrases, or topics.
“Do not use the word ’very’ or mention price details.”
Length Constraint
Quantifiable limits on the number of characters, words, or sentences.
“The response must be between 50 and 80 words long.”
Structural Constraint
Requirements regarding the organizational layout or components.
“Format the response into three distinct bullet points.”
Punctuation Constraint
Rules governing the usage or exclusion of specific punctuation marks.
“Each sentence must end with a semicolon instead of a period.”
Positioning Constraint
Placing specific content at predefined locations within the text.
“The output must conclude with the phrase ’End of Report’.”
Appendix
Table 8: Definitions and Examples of Constraint Types in our Dataset.
Dataset / Method
MIABench
MM-IFEval
MIFS-Eval
IFEval
Average
Qwen3-8B (Text-Only)
Base
-
-
-
85.77
-
RECAST-SFT
-
-
-
87.14
-
RECAST-RL
-
-
-
88.65
-
VERIF-RL
-
-
-
86.51
-
Qwen3-VL-8B (Multimodal)
Appendix
Table 9: Comparison with additional baselines. MIFS demonstrates superior performance in the multimodal setting compared to both standard SFT and text-only RL approaches.
Multimodal instruction-following models require training data that is accurate, diverse, verifiable, and challenging. Existing synthesis pipelines typically follow a one-pass generate-and-filter paradigm, discarding feedback from failed samples, verifier outcomes, and target-model errors. We present VISA (Visual Instruction Synthesis Agent), an agentic framework that reformulates multimodal instruction synthesis as a self-evolving loop. At each round, VISA analyzes an image to filter incompatible constraints and discover new verifiable ones, samples diversity- and difficulty-aware constraint sets from persistent memory, generates candidate instructions, and verifies the resulting samples with executable tools and structured large language model judges. Failed samples trigger diagnostic-guided recovery, while accepted samples are probed against the target model to estimate difficulty. The resulting verifier signals and target-model failure profiles are written back to memory, allowing subsequent rounds to adaptively expand the constraint space, reduce template repetition, and focus on unresolved model weaknesses. The same verifier contracts further provide reward signals for reinforcement learning without a separately trained reward model. Experiments on MM-IFEval show that VISA consistently improves multimodal instruction following over strong baselines, while preserving general multimodal capability across seven public benchmarks.
Multimodal Large Language Models (MLLMs) are increasingly being applied in a wider range of real-world scenarios. However, due to the substantial labor cost, creating high-quality multimodal instruction datasets for MLLMs remains a significant challenge. Although some methods propose to generate instruction data, they often face limitations in modality support and struggle with generating multi-round instructions. To address these problems, we introduce UniData, a universal instruction generation pipeline, to transform simple user requirements into multi-round, multimodal instructions. Specifically, UniData first expands user requirements into multiple diverse events. Using these events, UniData then integrates an any-to-any large model for multimodal instruction generation. Finally, UniData enhances data quality by correcting irrelevant and redundant inference flow, leveraging correlations between instruction rounds. To train this pipeline, we also build UniDataset, a dataset comprising 20,000 entries across nine modalities for improved multimodal generation. Our experiments demonstrate that UniData achieves SOTA performance in data quality and can also enhance the understanding and generation capabilities of other multimodal models.
Jiaqi Tang, Yi-Feng Wu, Yuting Zhang +12
The Hong Kong University of Science and Technology · ATH, Alibaba Group · The Hong Kong University of Science and Technology (Guangzhou) +2
RL has emerged as a powerful paradigm for enhancing the instruction following capabilities of LLMs. While existing training recipes achieve substantial gains, we find that they suffer from exploration bias towards easy instructions when the training data has multiple instructions in a prompt. This bias is caused by two main reasons: 1) the policy model's initial ability to satisfy hard instructions is too low to trigger successful exploration during RL training, so the optimization is biased towards easy instructions; and 2) canonical RL training recipes typically employ a cumulative reward (the number of instructions fulfilled), treating all instructions equally, which biases the policy model towards fulfilling easy instructions to obtain the same amount of reward. To address these issues, we first propose two metrics to measure the exploration bias in instruction following and then introduce a two-stage framework to alleviate it: 1) Behavioral Bootstrapping, a lightweight rejection sampling fine-tuning stage before RL to activate hard instructions; and 2) Scarcity-Aware Rewards, a new RL reward function that assigns rewards to instructions based on their empirical scarcity. Experiments show that the proposed metrics are highly correlated with model performance, and our methods unleash the potential of RL training: our best models outperform the baselines by a significant margin across three verifiable instruction following benchmarks. We release codes at https://github.com/mianzhang/MulIF.