Organizations: School of Computer Science and Engineering, Southeast University, Nanjing 211189, China · Key Laboratory of Computer Network and Information Integration (Ministry of Education), Southeast University, Nanjing 211189, China · University of Science and Technology of China, Hefei, China · University of Bath, Bath, United Kingdom
Long-horizon visual agents must do more than inspect images: they must turn observations into evidence that remains usable as a task unfolds. Existing active-perception methods teach agents where and how to look, while a final-answer reward gives little structure for deciding which observations satisfy dependent subgoals or should be retained for later use. We present ViStep-RL (Visual Stepwise Reinforcement Learning), which models execution as a dependency-linked graph of evidence-bearing subgoals. A verifier tracks which subgoals are supported by valid evidence, and changes in graph potential provide a net-progress learning signal; an active memory protocol preserves evidence across turns and links retrieval to downstream use. On ViInteract, ViStep-RL reaches 60.0% strict completion with Qwen2.5-VL-7B, compared with 10.0% for the base model and 46.5% after supervised fine-tuning. Controlled studies show that incremental task-state rewards outperform cumulative rewards by 3.2 points, while active evidence writing improves strict completion by 7.75 points over automatic writing. Under a common interaction framework, the task-trained ViStep-RL policy achieves 59.5% answer accuracy, 24.0 points above the strongest of six general-purpose reference models. Held-out episodes confirm the gain at both Qwen scales, and gains extend to three external visual reasoning benchmarks.
Figures & tables
Figure 1: ViStep-RL: from visual observations to verified task progress. (a) Source-grounded examples illustrate object counting, pose measurement, and chart reading. (b) Tool observations are stored with provenance and retrieved after visual context changes. (c) Schematic changes in strict verification: new evidence establishes progress, unchanged verification earns no additional graph credit, and invalidation retracts dependent memory verification. Auxiliary retrieval coverage is defined separately in the training potential.
Dataset / suite
Tool interface
Scale and coverage
Evaluation capabilities
Benchmark instances
Suite task families
Visual input
Agent interaction
Progress scoring
Evidence-path verification
Mantis-Eval [ 14 ]
None
217
4
✓
–
–
–
MuirBench [ 35 ]
None
2,600
12
✓
–
–
–
VisualWebArena [ 15 ]
Browser tools
910
3
✓
✓
–
–
OSWorld [ 41 ]
GUI tools
369
5
✓
✓
–
–
AgentBoard [ 21 ]
Environment tools
1,013
9
–
✓
✓
–
Table 1: Scale, task coverage, and evaluation capabilities. Instances count benchmark candidates. Task families count top-level categories across the full suite, including all ten ViInteract execution domains. ViInteract combines visual interaction, intermediate progress scoring, and provenance- and prerequisite-checked evidence paths to evaluate complete task execution across multiple steps.
Model / Backbone
Evaluation metrics ↑
Answer
Strict
Graph
Tool
Protocol
State
(a) Direct API evaluation
GPT-4o
12.0
0.5
25.9
49.2
67.4
67.4
Claude Opus 4.6
18.0
11.5
37.6
54.6
95.9
95.6
Qwen3-VL-Plus
3.0
2.5
16.0
19.3
49.3
49.3
Qwen3.8-Flash
9.5
8.5
33.2
41.6
54.1
54.0
Table 2: Main results on ViInteract. API models are evaluated directly in (a) and with the shared Harness in (b). Qwen training stages use native agent execution in (c), where Answer and Strict use Core200 and the remaining four metrics use Process500. Scores are percentages ( ↑ ). Bold marks the best value within each API group and each Qwen backbone.
Training variant
Design components
Process500
Holdout200
Progress
Other nodes
Answer node
Strict ↑
Graph ↑
Strict ↑
Graph ↑
Cumulative reward
Cumulative
✓
✓
22.6
49.4
12.0
33.6
No progress reward
None
✓
✓
23.8
54.1
12.5
33.6
No prerequisite gating
Incremental
×
×
25.2
51.7
14.5
35.0
No answer prerequisite gate
Incremental
✓
×
23.4
51.4
14.5
35.3
ViStep-RL
Incremental
✓
✓
25.8
51.8
15.5
36.3
Table 3: Ablation of progress rewards and verification constraints. Progress specifies the training reward. The two gate columns indicate where prerequisites apply during training. All variants share matched training budgets and are scored with the same full verifier on identical episodes within each panel. Strict and Graph are percentages; higher is better.
Figure 2: Action budget and visual horizon. (a, b) Qwen2.5-VL-3B SFT and ViStep-RL on 200 episodes per budget. (c) Qwen2.5-VL-7B Strict completion on Core200, grouped by visual turns; episode counts are shown below group labels, and numbers above the SFT and ViStep-RL bars give completed episodes. T+A uses terminal reward with action updates; G+T uses graph reward with trajectory updates.
Figure 3: Category-wise performance on external benchmarks. Accuracy changes for Qwen2.5-VL-3B ViStep-RL relative to matched references on (a) MMLongBench-Doc, (b) MuirBench and (c) MMSI-Bench. Teal bars indicate improvements, while coral bars indicate declines. Cam., Obj., and Reg. denote camera, object, and region.
(a) Writing × retrieval, N=200 /cell
Write / retrieval
Answer
Strict
Graph
Tool
Active / Active
24.5
17.0
49.0
69.6
Active / BM25
22.0
14.5
48.5
70.8
Automatic / Active
16.5
8.5
43.2
60.4
Automatic / BM25
15.0
7.5
43.6
62.0
(b) Evidence interventions, Holdout200
Table 4: Effects of memory and evidence interventions. (a) Fixed 3B policy and memory budget on Core200. (b) Holdout200; Answer scores final responses independently of evidence checks.
Base
ViStep-RL
Direct
Harness
Direct
Harness
Answer (%)
16.5
18.0
18.0
29.0
Parse fail (%)
4.5
3.0
5.0
0.5
Input tokens
2,798.17
22,197.81
2,796.97
24,249.02
Output tokens
324.58
703.60
455.61
903.83
Calls
4.815
15.850
4.815
15.235
Table 5: Answer quality and cost on Core200. Both 3B policies receive the same public inputs. Direct supplies observation notes; Harness also provides a tool controller. Native execution is evaluated in Table 2 (c). Costs are per episode.
Figure 4: Human evaluation of stored evidence on 200 paired Core200 episodes (7B, three annotators, majority vote). Error bars are 95% bootstrap intervals; the right column is the gain over SFT.
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Core200
Holdout200
Source component
1
2
3
4–5
6–8
9–12
13+
1
2
3
4–5
6–8
9–12
13+
COCO Detection
5
15
15
15
–
–
–
5
14
14
–
–
–
–
TextOCR
2
8
12
6
7
–
–
2
8
12
6
3
–
–
ChartQA
2
10
8
10
–
–
–
2
10
8
3
–
–
–
COCO Pose
1
3
3
4
9
–
–
1
3
3
4
–
–
–
LVIS Segmentation
–
3
2
2
8
–
–
–
3
2
2
8
–
–
Appendix
Table S1 : Exact panel quotas used by the deterministic builder. A dash denotes an empty component/length cell. The component totals reproduce the Core200 and Holdout200 panels; the resulting manifests and source-pool hash identify the released instances.
Component / domain
N
Primary execution requirement
COCO Detection / visual-tool
50
locate and verify objects
TextOCR / OCR-memory
35
extract, write, retrieve text
ChartQA / multi-chart
30
compare chart evidence
COCO Pose / pose-memory
20
measure pose relations
Memory Control / temporal memory
16
preserve state across turns
Memory Control / selective identity
2
retain identity evidence
Appendix
Table S2: ViInteract benchmark card. Counts refer to the frozen Core200 composition.
Setting
Multi-turn visual input
Active visual operation
Typed tools
Persistent evidence
Dependency-gated success
Primary learning target
ViGoRL [ 29 ]
✓
Grounding / zoom
–
–
–
Spatially grounded reasoning
MGPO [ 12 ]
✓
Multi-turn crop
–
–
–
High-resolution visual reasoning
DeepEyes [ 51 ]
✓
Image operations
–
–
–
Thinking with images
ACTIVE-o3 [ 54 ]
✓
Active perception
Modular sensing
–
–
Information acquisition
MuirBench [ 35 ]
Multi-image
–
–
–
–
Multi-image understanding
MMLongBench-Doc [ 22 ]
Long visual document
–
–
–
–
Long-context document understanding
Appendix
Table S3 : Positioning against active visual reasoning settings. The table compares the execution contract and learning target. A checkmark denotes an explicit part of the reported setting; a dash denotes that the capability is outside the benchmark or method target.
Source component
Core200
Holdout200
AITW temporal
17
54
ChartQA
30
23
COCO detection
50
33
COCO pose
20
11
LVIS segmentation
15
15
Memory control
18
18
Appendix
Table S4: Panel composition by source component and episode length. Counts are frozen before the corresponding evaluation matrix.
Release
Images/ep.
Turns/ep.
Nodes/ep.
Unique images
ID
Core200
1–16
1–16
3–33
781
4b71
Holdout200
1–16
1–16
3–33
787
8034
Mean (Core)
4.03
4.82
10.80
–
–
Mean (Holdout)
4.05
4.66
10.12
–
–
Appendix
Table S5 : Frozen release statistics. Core200 and Holdout200 are disjoint evaluation releases; prefixes identify the payload manifests recorded in the release ledger.
Field
Stored content
Visibility
id , episode_id
Stable episode and source identifiers
Public
dataset_version , partition , split
Release and partition metadata
Public
domain , release_component
Task family and source component
Public
source_sequence_id , images
Source sequence and ordered image records
Public
student_visible
Instruction, current image, legal actions, observations
Public
tool_state
Typed tool outputs and action provenance
Public
Appendix
Table S6 : Top-level ViInteract release schema. Public fields are visible to the policy; hidden fields are consumed only by the verifier and scorer.
Field
Stored content
Runtime use
node_id
Stable node identifier
Dependency references
type
Evidence, memory, retrieval, computation, or answer
Selects verifier
depends_on
Parent node identifiers
Fixed-point ordering
expected
Source value, annotation, or deterministic predicate
Evidence match
evidence_refs
Producing tool-call identifiers
Provenance check
weight
Optional positive node weight
Potential normalization
Appendix
Table S7 : Task-program fields and verification rules. Hidden fields are consumed by the runtime and scorer; they are excluded from the policy observation.
Step
Operation and emitted state
1
Sample a rollout group under the current policy.
2
Execute each action and record the public observation, typed tool result, memory provenance, and verifier state.
3
Recompute the dependency fixed point and evaluate δt and the execution cost.
4
Check variation in rollout returns, then compare transition rewards within phase, turn, and progress bins.
5
Select actions by absolute weight and apply the clipped token loss with the frozen SFT reference.
Appendix
Table S8: Qwen policy update. The verifier evaluates every environment transition; each selected action receives its own signed weight.
Stage
Actions
Action tokens
Time (s)
Base
19.54
1,386.8
44.87
SFT
15.76
1,921.0
72.46
ViStep-RL
15.00
1,732.2
67.43
Appendix
Table S9: Interaction efficiency on Core200. Qwen2.5-VL-7B; values are per episode.
Figure S1 : Same evidence, different task completion. Matched Qwen2.5-VL-7B trajectories on one TextOCR episode. All stages read and store the same evidence; after image eviction, only ViStep-RL retrieves all required records and answers correctly. Counts cover the full recorded trajectories.
Criterion
Fleiss’ κ
Unanimous (%)
Evidence validity
0.744
88.2
Provenance validity
0.716
79.7
Downstream support
0.674
75.7
Appendix
Table S10: Inter-annotator agreement over all 600 traces.
API model
Difference
95% CI
GPT-4o
+24.5
[17.5, 31.5]
Claude Opus 4.6
+29.0
[21.0, 37.0]
Qwen3-VL-Plus
+28.0
[20.0, 36.0]
Qwen3.8-Flash
+24.0
[16.5, 31.5]
Gemini 3.8 Flash
+25.0
[17.0, 32.5]
Kimi K2.5
+37.5
[29.5, 45.5]
Appendix
Table S11: Paired advantage of the trained Qwen2.5-VL-7B with Harness over each API model with Harness. Differences are percentage points on the same 200 items.