Visual agents that interleave reasoning with tool invocation have posed a promising direction for multimodal tasks. However, learning effective tool invocation usually relies on curated question-answer pairs or expert demonstrations at scale, which are expensive to acquire. In this study, we propose Marvo, a simple-yet-effective self-evolution approach that enables learning tool invocation from unlabeled images. Marvo consists of two agents: (1) a challenger that generates question-answer pairs from unlabeled images, and (2) a solver that learns tool invocation by solving questions through reinforcement learning. To encourage effective tool invocation, a good training question should be difficult enough to warrant tool use, yet remain within the solver's capabilities. To this end, we introduce a margin reward to the challenger, defined as solver's performance difference with and without access to tools. With the margin reward, we create an adaptive curriculum that encourages the solver to learn effective tool invocation. Across benchmarks covering agentic reasoning, visual search, mathematical, and general real-world reasoning, Marvo improves average accuracy by up to 7.1% --- demonstrating margin reward as a preferred way to scale question complexity without human supervision.
Figures & tables
Figure 1: Flowchart of Marvo. (a) Marvo employs an evolution loop between a challenger and a solver, driven by the Margin Reward. (b) The Margin Reward quantifies the solver’s performance gap with and without tools, encouraging the challenger to propose questions that require tool use.
Figure 2: Question Distribution.
Figure 3: Equilibrium States.
Agentic
Visual Search
Mathematical Reasoning
General
TIR-Bench
V*
HR4K
MathVerse
MathVision
WeMath
RWQA
Overall
Gemini-2.5-Pro
28.9
83.8
87.3
82.9
63.8
78.0
78.0
71.6
o3
26.9
95.7
-
-
73.3
80.6
-
-
Qwen2.5-7B
16.0
78.5
71.6
33.8
25.6
34.6
57.5
34.8
Qwen3-4B
-
74.9
73.5
42.3
44.6
-
70.9
-
DeepEyesV2
-
81.8
77.9
52.7
28.9
38.1
-
-
Table 1: Main results. For each self-evolution process (one model family with one method), the best value in each column is in bold. Parenthesized values are the change versus the previous round. Overall is the mean over the seven benchmarks.
Figure 4: Training Dynamics. (a-b) Reward progress in challenger and solver training phase. (c) Per-round difficulty of challenger-generated datasets, as measured by model at each round.
Figure 5: Marvo across post-training stages. The gray curve is the reported PyVision-RL training progress. Colored curves are Marvo experiments launched from different checkpoints.
Figure 6: Mean number of tool turns during evaluation. Initializing from Instruct model, Marvo incentivizes tool invocation without human-curated data supervision.
Table 8
Agentic
Visual Search
Mathematical Reasoning
TIR-Bench
V*
MathVision
WeMath
Qwen3-4B-SFT
22.3
78.0
38.8
57.4
Round 1
22.6 (+0.3)
78.5 (+0.5)
35.1 (–3.7)
57.7 (+0.3)
Round 2
22.4 (–0.2)
79.1 (+0.5)
35.8 (+0.6)
59.0 (+1.3)
Round 3
23.2 (+0.7)
77.5 (–1.6)
38.0 (+2.1)
62.5 (+3.5)
Table 4: Per-round solver performance under Eq. ( 6 ) setup, where H2 is a truncated variant of H1 . Parenthesized values are the change versus the previous round.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Visual Agent Flowcharts
Figure 8: Left: SFT hyperparameters. Right: SFT training loss.
Value
Run / data
Rounds R
3
Images per round B
1,280 (cyclic disjoint slices; pool 44,643, seed 42)
Image long-edge cap
1,536 pixels
Challenger (GRPO)
Steps per round m
10
Appendix
Table 5: Training Hyperparameters.
Figure 9: Resolution Distribution of Training Image Pool.
Figure 10: Image-embedding space of generated questions vs. evaluation benchmarks across training rounds. Each panel shows a shared 2D projection (PCA + t-SNE over Qwen3-VL-4B vision-tower embeddings) of 100 images sampled from each of seven evaluation benchmarks (colored points) and 100 images underlying the questions generated in the indicated training round (red rings).
Round 5
Round 6
Round 7
Round 8
WeMath
63.2
62.1
64.2
64.3
MathVision-Mini
27.3
24.0
23.4
23.7
V*
77.5
78.5
72.3
74.4
TIR-Bench
21.7
20.7
21.1
21.0
Appendix
Table 6: Performance of Marvo across evolution rounds 5–8 on four benchmarks. Scores are reported in percentages; the best result for each benchmark is achieved no later than round 6, after which performance degrades.
Round
1
2
3
4
5
6
7
8
n
1201
1280
1281
1280
1281
1280
1280
1280
Judge Acc. (%)
68.5
62.2
66.1
67.1
58.7
46.1
46.2
43.6
Appendix
Table 7: Accuracy of the proposed solver training questions {(Ii,xi,yi)}i=1N at each evolution round, probed by Qwen3.8-27B. Despite consensus-based filtering, question quality degrades from 68.5% to 43.6%, mirroring the performance degradation in Table 6 .
Figure 11: Example Trajectory of Challenger Agent.