Multimodal instruction-following models require training data that is accurate, diverse, verifiable, and challenging. Existing synthesis pipelines typically follow a one-pass generate-and-filter paradigm, discarding feedback from failed samples, verifier outcomes, and target-model errors. We present VISA (Visual Instruction Synthesis Agent), an agentic framework that reformulates multimodal instruction synthesis as a self-evolving loop. At each round, VISA analyzes an image to filter incompatible constraints and discover new verifiable ones, samples diversity- and difficulty-aware constraint sets from persistent memory, generates candidate instructions, and verifies the resulting samples with executable tools and structured large language model judges. Failed samples trigger diagnostic-guided recovery, while accepted samples are probed against the target model to estimate difficulty. The resulting verifier signals and target-model failure profiles are written back to memory, allowing subsequent rounds to adaptively expand the constraint space, reduce template repetition, and focus on unresolved model weaknesses. The same verifier contracts further provide reward signals for reinforcement learning without a separately trained reward model. Experiments on MM-IFEval show that VISA consistently improves multimodal instruction following over strong baselines, while preserving general multimodal capability across seven public benchmarks.
Figures & tables
Figure 1: Overview of VISA. The framework iteratively performs perception, planning and execution, reflection, and memory update, allowing verifier feedback and target-model difficulty signals to guide subsequent synthesis rounds.
Model / Training
Params
MM-IFEval
C-Level
P-Level
Avg.
Qwen3.5-9B
9B
63.9
60.0
63.0
MiniCPM-V-4.5
8B
68.8
44.0
62.6
InternVL3.5-8B
8B
61.8
39.0
56.1
InternVL3.5-4B
4B
57.0
33.0
51.0
Qwen3.5-4B
4B
62.7
55.0
60.8
Table 1: Main results on MM-IFEval. We report C-Level accuracy, P-Level accuracy, and their average score; higher values indicate better instruction-following performance.
Method (Qwen3.5-4B)
C-Level
P-Level
Avg.
Base
62.7
55.0
60.8
Static Pipeline
64.9
46.0
60.2
+ Reflection
65.6
48.0
61.2
+ Reflection + Memory
67.1
51.0
63.1
Full VISA
67.2
54.0
63.9
Table 2: Ablation results on MM-IFEval with Qwen3.5-4B under SFT, showing the incremental contribution of reflection, memory, and the full VISA pipeline.
Model / Training
MMBench
MMStar
MM-Vet
HallusionBench
MathVista
OCRBench
AI2D
Avg.
Qwen3.5-4B
78.4
61.7
47.6
56.0
78.8
84.9
86.2
70.5
+ VISA-SFT-15k
81.2
63.2
49.1
54.1
77.8
85.2
86.4
71.0
+ VISA-RL-15k
78.8
62.5
50.5
57.5
87.9
85.6
87.6
72.9
Table 3: General multimodal capability results on Qwen3.5-4B. VISA preserves or improves broad multimodal performance across seven public benchmarks, with all evaluations conducted in non-thinking mode.
Figure 2: Evolution of the constraint-verifier registry in VISA. Left: cumulative growth of constraint types across self-evolution rounds, showing that most discoveries occur early and gradually saturate. Right: category distribution of the final 310 constraint types.
Figure 3: Diversity analysis of VISA-synthesized instructions. Left: conditional co-occurrence among the ten most frequent constraint categories, where each cell reports P(j∣i) as a percentage. Right: t-distributed stochastic neighbor embedding (t-SNE) projection of high-dimensional instruction embeddings, colored by dominant constraint category.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 4: Distribution of the number of constraints per synthesized sample, showing that most accepted samples combine multiple jointly satisfiable requirements rather than a single constraint.
Table 4: Constraint categories in the final VISA registry. Representative types are illustrative examples; “Seed” and “+Disc.” denote initially defined and dynamically discovered constraint types, respectively.