cs.CVMar 16, 2026
SaveGym-V: A Unified Vision Environment System for Agentic Vision Research
Organizations: National University of Singapore. · University of Washington. · The Hong Kong Polytechnic University. · The University of Hong Kong.
Abstract
As agentic systems increasingly rely on reinforcement learning from verifiable rewards, standardized ``gym'' infrastructure has become essential for rapid iteration, reproducibility, and fair comparison. Vision agents lack such infrastructure, limiting systematic study of what drives their learning and where current models fall short. We introduce Gym-V, a unified platform of 179 procedurally generated visual environments across 10 domains with controllable difficulty, enabling controlled experiments that were previously infeasible across fragmented toolkits. Using it, we find that observation scaffolding is more decisive for training success than the choice of RL algorithm, with captions and game rules determining whether learning succeeds at all. Cross-domain transfer experiments further show that training on diverse task categories generalizes broadly while narrow training can cause negative transfer, with multi-turn interaction amplifying all of these effects. Gym-V is released as a convenient foundation for training environments and evaluation toolkits, aiming to accelerate future research on agentic VLMs.
Figures & tables
| Interaction Workflows | Infra. | |||||||||
| Platform | #Env | ST | MT | MA | Eval | Off. | Tool | Diff. | MM | Eaas |
| Text-centric | ||||||||||
| TextArena ( Guertler et al., 2025 ) | 74 | |||||||||
| Reasoning Gym ( Stojanovski et al., 2025 ) | 100+ | |||||||||
| RLVE ( Zeng et al., 2025 ) | 400 | |||||||||
| GEM ( Liu et al., 2025 ) | 24 | |||||||||
Table 1: Comparison of gym-style environment platforms. = supported, = not supported. ST: single-turn; MT: multi-turn; MA: multi-agent; Eval: evaluation wrapper; Off.: offline data; Tool: tool-augmented; Diff.: difficulty levels; MM: multimodal; Eaas: evaluation-as-a-service.
Figure 1: Overview of Gym-V. Top: 105 single-turn and 74 multi-turn environments across 10 categories. Bottom: a unified reset / step interface shared by interactive environments, offline datasets, and evaluation benchmarks.
Figure 2: Evaluation fidelity of Gym-V against official pipelines. (a) VLM evaluation on Qwen2.5-VL-7B-Instruct reproduces official VLMEvalKit scores. (b) Text-to-image and image-editing evaluation closely matches official GenExam, RISE, and GenEval results.
| Category | #Envs | Description |
| Single-Turn (105) | ||
| Algorithmic | 21 | Grid-based tasks that require algorithmic reasoning—BFS traversal, cellular automata simulation, and game-theoretic computation—over visually rendered grids (e.g. GridBFS ). |
| ARC | 3 | Abstract pattern induction tasks that challenge models to infer transformation rules from few visual input–output grid pairs and generalize to unseen instances (e.g. ArcAgi ). |
| Cognition | 10 | Perceptual inference tasks involving 3D mental rotation, visual pattern completion, and spatial reasoning from rendered scenes (e.g. RubiksCube-QA ). |
| Geometry | 8 | Geometric reasoning tasks requiring computation of convex hulls, enclosing circles, and areas over visually rendered point sets and shapes (e.g. ConvexHull ). |
| Graphs | 23 | Graph-theoretic reasoning tasks including traversal, shortest-path computation, spanning trees, coloring, and matching over visually rendered graphs (e.g. ShortestPath ). |
Table 2: Overview of the Gym-V environment suite (179 environments across 10 categories). All environments share the reset / step interface and support deterministic seeding.
Table 3: Left: Single-turn evaluation ( ) across 7 domain categories . Right: Multi-turn evaluation ( ) across Games (12 envs), Spatial/2D (6 Minigrid envs), and Spatial/3D (3 MiniWorld envs). Avg: mean over all 10 columns reported in this table. Best result per column in bold . Note that Minigrid (Sp./2D) uses shaped episodic returns that include negative penalties (e.g., stepping into hazards), so mean@3 returns can be below zero.
Figure 4: Training reward curves for GRPO, GSPO, and SAPO across 12 single-turn (rows 1–3) and 4 multi-turn (row 4) environments. Smoothed curves (bold) are overlaid on raw trajectories (translucent). All environments exhibit learnable reward signals; no single algorithm dominates uniformly. Multi-turn games show slower convergence and lower absolute returns, reflecting the compounding difficulty of sequential decision-making.
Figure 5: Training reward curves for context modeling (top row) and rules injection (bottom row) ablations on four multi-turn games. Top : with context (3-turn, red; 5-turn, blue) vs. without context (green). Bottom : with rules (green) vs. without rules (red). Smoothed curves (bold) are overlaid on raw trajectories (translucent).
Figure 6: Training reward curves comparing image only (green) vs. image + caption (red) on two single-turn and two multi-turn environments. Adding textual captions yields a substantial and consistent improvement across all four environments. The benefit is most pronounced on perception-heavy tasks such as longest_path , where caption narrows the visual grounding gap early in training, and on multi-turn games ( minesweeper , frozenlake ), where natural-language state descriptions help the agent maintain situational awareness across interaction steps.
| Train Eval | Algo. | Cogn. | Geom. | Graph | Logic | Puzzle |
|---|---|---|---|---|---|---|
| Baseline | 3.6 | 31.6 | 5.3 | 1.2 | 36.7 | 7.5 |
| Algorithmic | 4.5 +0.9 | 47.1 +15.5 | 7.4 +2.1 | 1.6 +0.5 | 45.8 +9.2 | 8.1 +0.5 |
| Cognition | 5.7 +2.2 | 48.3 +16.7 | 8.5 +3.2 | 1.7 +0.5 | 42.5 +5.8 | 12.2 +4.7 |
| Geometry | 2.5 -1.1 | 36.2 +4.6 | 11.9 +6.6 | 0.8 -0.3 | 42.6 +5.9 | 4.1 -3.4 |
| Graphs | 3.7 +0.1 | 40.3 +8.7 | 8.1 +2.8 | 1.3 +0.1 | 45.1 +8.4 | 11.0 +3.5 |
| Logic | 5.8 +2.3 | 42.6 +11.0 | 7.2 +1.9 | 1.1 -0.1 | 45.6 +9.0 | 10.5 +3.0 |
Table 4: Cross-domain transfer matrices ( ) for (a) single-turn categories and (b) multi-turn games. Each row is a training source; each column an evaluation target. Subscripts show absolute change from zero-shot baseline. Diagonal entries (bold) denote in-domain performance.
Appendix figures & tables14 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Training reward curves comparing three prompt conditions across four multi-turn games. Top row : 3-turn context window; Bottom row : 5-turn context window. Green: no context, no rules; Red: context only, no rules; Blue: context with rules. Providing interaction context without rules substantially outperforms the no-context baseline and, in several environments ( Game2048 , FrozenLake , Minesweeper ), approaches or matches the with-rules setting, demonstrating that context enables implicit rule discovery through interaction experience.
Figure 8: Sampled observations from 8 retro arcade environments over 200 VLM agent steps. Despite exhausting the full step budget, the game state barely progresses in any environment.
Figure 9: Algorithmic. Left: BinaryMatrix — the agent performs an algorithmic operation (e.g., counting connected components) on a rendered binary grid; output is a single integer or transformed grid. Right: RottenOranges — the agent determines the minimum time for all oranges to rot via BFS propagation on a grid; output is a single integer.
Figure 10: Cognition. Left: RubiksCube-QA — the agent observes a rendered 3D Rubik’s Cube and answers questions about face colors after rotations. Right: OddOneOutPoly — the agent identifies which polygon differs from the rest based on a visual property (e.g., symmetry, number of sides).
Figure 11: Geometry. Left: LargestIsland — the agent observes a binary grid (blue = water, green = land) and must find the maximum area of a 4-directionally connected island; output is a single integer. Right: VisibleLine — given lines plotted on a 2D plane, the agent identifies which lines are visible from (i.e., lie on the upper envelope); output is the space-separated indices of visible lines.
Figure 12: Graphs. Left: ShortestPath — the agent computes the shortest path between two nodes in a weighted graph; output is the node sequence. Right: LongestPath — the agent finds the longest simple path in a graph; output is the path length.
Figure 13: Logic. Left: Thermometers — the agent fills thermometer-shaped regions in a grid to satisfy row/column sum constraints. Right: CircuitLogic — the agent traces a Boolean logic circuit and determines the output given input values.
Figure 14: Puzzles. Left: KnightSwap — the agent swaps the positions of two sets of knights on a small chessboard using legal knight moves; output is the move sequence. Right: TowerOfHanoi — the agent produces the move sequence to transfer all discs to the target peg; output is a sequence of (source, target) moves.
Figure 15: Games. Left: Sokoban — the agent pushes boxes onto goal positions by issuing directional commands ( up/down/left/right ) in a multi-turn grid puzzle. Right: Chess — the agent plays against a built-in opponent, outputting moves in UCI notation (e.g., e2e4 ) each turn.
Figure 16: Spatial. Left: DoorKey (MiniGrid, 2D) — the agent navigates a top-down grid-world to find a key, unlock a door, and reach the goal. Right: CollectHealth (MiniWorld, 3D) — the agent navigates a first-person 3D room to collect health items; actions include move_forward and turn .
Figure 17: Temporal. Left: StreetsOfRage2 — a side-scrolling beat-em-up requiring continuous combat inputs. Right: GoldenAxe — a hack-and-slash arcade game.
Figure 18: Per-environment zero-shot evaluation heatmap across all evaluated environments and 9 models. Rows are grouped by category; columns correspond to models ordered by overall performance. Scores are percentages ( ); darker shades indicate higher performance.
| Environment | Description |
|---|---|
| Algorithmic (21 environments) | |
| AdditionTable | Find the correct base and digit mappings for a system where distinct letters represent digits, given addition equations. |
| BinaryMatrix | Compute the taxicab (Manhattan) distance to the nearest 0 for each cell in a binary matrix. |
| CirculatingGrid | Modify the minimum number of directional arrows in a toroidal grid so that every cell eventually loops back to itself. |
| CoinSquareGame | Determine the optimal strategy in a two-player coin-removal game where players take leftmost coins and sum their values. |
| FaceRightWay | Make all arrows in a binary array face right using the minimum number of fixed-window flip operations. |
Table 5: Complete catalog of single-turn environments in Gym-V, grouped by category.
| Environment | Description |
|---|---|
| Games (31 environments) | |
| Alquerque | Capture all opponent pieces or block their moves on a 5 5 connected board. |
| Breakthrough | Advance pieces to the opponent’s home row; move forward or capture diagonally. |
| Chess | Play standard chess against an opponent, outputting moves in UCI notation. |
| ConnectFour | Drop pieces into a 7-column, 6-row grid; connect four in a line to win. |
| ConnectFourMultiAgent | Two-agent Connect Four where both sides are controlled by separate models. |
Table 6: Complete catalog of multi-turn environments in Gym-V, grouped by category.
More in this topic
AI Agent Benchmarks
AI agent benchmarks measure how reliably autonomous systems plan, use tools, interact, and complete tasks across controlled and real-world settings.
See all 1,494 papers