Multimodal agentic coding requires robust visual reasoning to analyze observations and evaluate generated artifacts. Current frameworks largely rely on a Thinking-with-Images paradigm, where agents depend on their intrinsic vision to execute visually grounded tasks. However, this architectural coupling introduces a fundamental bottleneck. Agent performance remains strictly bounded by internal perceptual limitations. Furthermore, text-only agents lack native perception. Without specialized tools, they cannot perform visual reasoning. To overcome these constraints, we introduce the Vision-as-a-Tool (VaaT) paradigm. Prior visual tool-use merely manipulates images for an agent's intrinsic vision to read. In contrast, VaaT reframes general-purpose multimodal large language models (MLLMs) as callable external utilities. As a result, text-only agents can completely offload visual reasoning during agentic execution. Meanwhile, vision-capable agents can dynamically augment their native perception on demand. We instantiate this paradigm in VHarness, a unified framework driven by five atomic actions. We comprehensively evaluate VHarness across Image-to-HTML, Image-to-SVG, and Image-to-3D generation tasks. Experimental results demonstrate that VHarness consistently improves visual fidelity and geometric accuracy across both text-only and multimodal coding agents. These findings highlight that modularizing visual reasoning into callable utilities provides a flexible and scalable foundation for future agentic coding systems.
Figures & tables
Figure 2: Qualitative comparisons across Image-to-HTML, Image-to-SVG, and Image-to-3D tasks. From left to right, we show the full reference input, the enlarged ground truth, and the corresponding rendered code artifacts from VHarness (Ours) and the Baseline . Red dashed lines highlight key layout boundaries and geometric contours. Compared with the baseline, VHarness faithfully preserves curved text wrapping in HTML, isometric contours and fine details in SVG, and organic branching geometry in 3D.
(a) Image-to-HTML
Model
VHarness
2D Perceptual
2D Structural
SigLIP2 ↑
DINOv2 ↑
LPIPS ↓
SSIM l. ↑
SSIM g. ↑
Px MSE ↓
Edge P ↑
Edge R ↑
Edge F1 ↑
GPT-5.5
w/o
0.5773
0.8881
0.2277
0.6789
0.6950
0.0500
0.5464
0.5335
0.5389
w/
0.6319
0.9046
0.1820
0.6930
0.7201
0.0461
0.5867
0.5792
0.5820
GPT-5.6-sol
w/o
0.6527
0.9055
0.1635
0.7197
0.7496
0.0414
0.6625
0.6433
0.6517
w/
0.6859
0.9131
0.1414
0.7385
0.7663
0.0385
0.7009
0.6883
0.6936
Table 1: Quantitative comparison of final rendered artifacts with ( w/ ) and without ( w/o ) VHarness across Image-to-HTML, Image-to-SVG, and Image-to-3D (100 cases per task). ∗ Due to inference budget constraints, the text-only coding agent GLM-5.2 is evaluated on a random subset of 50 cases per task. Arrows indicate the preferred direction for each metric, and bold marks the best result in each column.
Figure 3: Illustrative examples of the five atomic actions in VHarness across Image-to-HTML, Image-to-SVG, and Image-to-3D tasks. Using representative visual inputs and highlighted regions on the left, the right panel illustrates the input queries and outputs of each callable action.
Figure 4: Refinement trajectories of GPT-5.6-sol with DINOv2 on Image-to-3D. Dashed segments extend the baseline’s final scores beyond termination.
Metric
w/o VHarness
w/ VHarness
SigLIP2 ↑
0.7368
0.7529
DINOv2 ↑
0.9428
0.9486
LPIPS ↓
0.0572
0.0473
SSIM l. ↑
0.8903
0.8987
SSIM g. ↑
0.9229
0.9356
Px MSE ↓
0.0061
0.0051
Table 2: Ablation study with GPT-6-astra on the Image-to-SVG benchmark. Bold marks the better result in each row.
Appendix figures & tables17 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Examples of workspace inputs provided to the coding agent in Image-to-HTML. For each test case, the workspace supplies the target webpage screenshot ( Visual Input , target.png , left), the pre-extracted graphic files stored under ./assets/ ( Visual Assets , middle), and the JSON manifest ( provided_assets.json , right) specifying each asset’s filename, relative path, and native pixel dimensions without spatial coordinates or layout metadata.
Figure 6: Representative benchmark test cases for Image-to-HTML (top) and Image-to-SVG (bottom).
Figure 7: Representative benchmark test cases in 3DCodeBench.
Figure 9: Exact backend prompts for the single-image perception actions ( grep , and inspect ).
Figure 10: Exact backend prompts for the artifact verification actions ( align , transcribe and diff ).
Figure 11: Exact skill instructions for the w/o VHarness baseline on Image-to-HTML and Image-to-SVG.
Figure 12: Exact skill instructions for the w/o VHarness baseline on Image-to-3D.
Figure 13: Complete skill instructions for the Freestyle ablation on Image-to-HTML.
Figure 14: Complete skill instructions for the Freestyle ablation on Image-to-3D.
Model
Context Window
Reasoning Effort
GPT-5.5 [ 15 ]
200K
high
GPT-5.6-sol [ 14 ]
200K
high
Claude Opus 4.8 [ 1 ]
200K
high
GLM-5.2 [ 27 ]
200K
high
GLM-5.3-Flash [ 28 ]
200K
high
GPT-6-astra (Ablation)
200K
low
Appendix
Table 4: Summary of model configurations and inference hyperparameters across the main experiments and ablation study. All coding agents operate under a fixed 200K context window.
Model
VHarness
HTML
SVG
3D
GPT-5.5
w/o
52,792
30,983
37,886
w/
173,374
195,148
237,750
GPT-5.6-sol
w/o
23,825
16,301
17,918
w/
62,005
64,841
104,543
Claude Opus 4.8
w/o
14,132
14,153
14,522
w/
62,859
82,688
92,321
Appendix
Table 5: Average recorded output tokens per case by the coding agent Pθ across Image-to-HTML, Image-to-SVG, and Image-to-3D.
Figure 15: Atomic action invocation distributions across tasks. Proportions are computed by normalizing the average per-case call counts across actions.
Task
transcribe
grep
inspect
align
diff
Total
Image-to-HTML
7.8
199.5
264.8
14.3
246.5
733.0
Image-to-SVG
11.2
29.4
155.9
5.4
164.5
366.4
Image-to-3D
3.8
34.3
53.5
0.3
351.8
443.8
Appendix
Table 6: Average number of atomic visual action calls per case invoked by the text-only coding agent GLM-5.2 under VHarness across Image-to-HTML, Image-to-SVG, and Image-to-3D.
(a) Image-to-HTML
Model
Read
Bash
Write
Edit
Other
VHarness
GPT-5.5
72.5
58.3
3.3
74.7
13.6
187.4
GPT-5.6-sol
39.1
46.4
2.4
64.8
10.1
103.0
Claude Opus 4.8
48.7
37.6
3.2
43.3
0.0
87.4
GLM-5.2
0.0
1091.7
46.5
361.4
26.6
733.0
GLM-5.3-Flash
23.7
179.7
27.8
32.7
0.2
33.1
Appendix
Table 7: Average per-case invocations of native harness tools ( Read , Bash , Write , Edit , and Other ) and total VHarness atomic visual actions across Image-to-HTML, Image-to-SVG, and Image-to-3D. Other includes task-management and MCP resource tools.
(a) Image-to-HTML
Model
VHarness
2D Perceptual
2D Structural
SigLIP2 ↑
DINOv2 ↑
LPIPS ↓
SSIM l. ↑
SSIM g. ↑
Px MSE ↓
Edge P ↑
Edge R ↑
Edge F1 ↑
GPT-5.5
w/o
0.5973
0.8939
0.2129
0.6757
0.7122
0.0500
0.5817
0.5637
0.5717
w/
0.6400
0.9074
0.1807
0.6911
0.7284
0.0476
0.6169
0.5996
0.6073
GPT-5.6-sol
w/o
0.6459
0.9074
0.1694
0.6971
0.7453
0.0448
0.6526
0.6317
0.6410
w/
0.6871
0.9128
0.1452
0.7265
0.7657
0.0412
0.7062
0.6876
0.6960
Appendix
Table 8: Quantitative comparison on the same 50 cases evaluation testset. GPT-5.5 and GPT-5.6-sol are evaluated with ( w/ ) and without ( w/o ) VHarness on all 50 cases per task. Arrows indicate the preferred direction for each metric, and bold marks the best displayed result in each column.
Figure 16: Cross-model and cross-metric refinement trajectories on Image-to-3D. We plot DINOv2 (left) and global SSIM (right) across render iterations for GPT-5.5 (top) and GPT-5.6-sol (bottom). Because the w/o VHarness baseline terminates early, the dashed line extends its final score for visual comparison.
Figure 17: Multi-step refinement trajectories of the text-only coding agent GLM-5.2 with VHarness on Image-to-HTML (top) and Image-to-SVG (bottom), showing selected intermediate renders from the initial draft to the final output alongside zoomed-in regional details.
The fundamental goal of agentic visual reasoning is to improve the success rate of multimodal large language models (MLLMs) on complex tasks. We rethink agentic visual reasoning through two key dimensions of tool use: Mode Adaptiveness and Tool Effect. Mode Adaptiveness characterizes whether an MLLM recognizes when tools are necessary and invokes them accordingly, avoiding unnecessary computational overhead while improving performance on problems requiring tool assistance. Tool Effect characterizes whether tools extend the model's capabilities on problems unsolvable through tool-free reasoning without introducing errors on problems it can already solve. Our analysis quantifies these properties and reveals that existing models exhibit limited Mode Adaptiveness, while tool-use gains on hard examples are largely offset by harm on easy ones. Motivated by these observations, we propose Beacon, a novel agentic visual reasoning model trained with supervised fine-tuning (SFT) and reinforcement learning (RL). Its RL stage combines Necessity-Aware Adaptive Reward and Hint-Guided Capability Expansion. Necessity-Aware Adaptive Reward encourages tool-free solutions when they succeed while preserving full reward for successful tool use when tool-free rollouts fail. Hint-Guided Capability Expansion uses verified, answer-free expert hints to recover learning signals from all-wrong rollout groups, aiming to extend tool-use capability on the hardest problems. Across 13 benchmarks, Beacon achieves the highest average score among the evaluated open-source models and ranks first on 11 benchmarks. On five diagnostic benchmarks, it improves the average tool-available accuracy over its tool-free accuracy by 1.96 points and achieves the largest tool-gain minus tool-harm score (+3.14 points). These results show Beacon's advanced performance, Mode Adaptiveness, and the net benefit of tool use.
We present GLM-5V-Turbo, a step toward native foundation models for multimodal agents. As foundation models are increasingly deployed in real environments, agentic capability depends not only on language reasoning, but also on the ability to perceive, interpret, and act over heterogeneous contexts such as images, videos, webpages, documents, GUIs. GLM-5V-Turbo is built around this objective: multimodal perception is integrated as a core component of reasoning, planning, tool use, and execution, rather than as an auxiliary interface to a language model. This report summarizes the main improvements behind GLM-5V-Turbo across model design, multimodal training, reinforcement learning, toolchain expansion, and integration with agent frameworks. These developments lead to strong performance in multimodal coding, visual tool use, and framework-based agentic tasks, while preserving competitive text-only coding capability. More importantly, our development process offers practical insights for building multimodal agents, highlighting the central role of multimodal perception, hierarchical optimization, and reliable end-to-end verification.
Recent advancements in Multimodal Large Language Models (MLLMs) have incentivized models to ``think with images'' by actively invoking visual tools during multi-turn reasoning. The common Reinforcement Learning (RL) practice of relying on outcome-based rewards ignores the fact that textual plausibility often masks executive failure, meaning that models may exhibit intuitive textual reasoning while executing imprecise or irrelevant visual actions within their agentic reasoning trajectories. This reasoning-action discrepancy introduces noise that accumulates throughout the multi-turn reasoning process, severely degrading the model's multimodal reasoning capabilities and potentially leading to training collapse. In this paper, we introduce Multimodal Agentic Policy Optimization (MAPO), bridging the gap between textual reasoning and visual actions generated by models within their Multimodal Chain-of-Thought (MCoT). Specifically, MAPO mandates the model to generate explicit textual descriptions for the visual content obtained via tool usage. We then employ a novel advantage estimation that couples the semantic alignment between these descriptions and the actual observations with the task reward. Theoretical findings are provided to justify the rationale behind MAPO, which inherently reduces the variance of gradients, and extensive experiments demonstrate that our method achieves superior performance across multiple visual reasoning benchmarks.
Wenhao Yang, Yu Xia, Jinlong Huang +10
National Key Laboratory for Novel Software Technology, Nanjing University, China · School of Artificial Intelligence, Nanjing University, China · AI Business, Alibaba Group +5