ViSkill: Reinforcing VLM Agents with Evolving Visual-Native Skills

Hongxing Li; Dingming Li; Yixin Li; Yong Du; Wenqi Zhang; Weiming Lu; Jun Xiao; Yueting Zhuang

cs.CVOct 8, 2026

ViSkill: Reinforcing VLM Agents with Evolving Visual-Native Skills

Authors: Hongxing Li, Dingming Li, Yixin Li, Yong Du, Wenqi Zhang, Weiming Lu, Jun Xiao, Yueting Zhuang, +1 more

Organizations: Zhejiang University

Abstract

Skill-augmented agents improve sample efficiency by distilling successful trajectories into reusable strategies. Yet most existing approaches remain text-centric, linearizing spatial layouts and action-state correspondences into language that loses critical geometric structure. Recent efforts have begun incorporating visual evidence, but construct and update skills separately from policy optimization, leaving their mutual improvement underexplored. We propose ViSkill, a visual-native skill learning framework that encodes successful interactions as composite visual skill cards directly accessible to VLM agents. Retrieved skills guide both inference and reward shaping, while successful trajectories are distilled back into the library, forming a closed feedback loop in which skill accumulation and policy improvement reinforce each other. An optional cold-start mechanism further accelerates early-stage learning. Evaluated on Sokoban, FrozenLake, and PrimitiveSkill, ViSkill achieves an overall success rate of 0.89, rising to 0.91 with cold-start initialization, outperforming all evaluated proprietary and open-source baselines while converging faster than standard PPO. Our code is available at https://github.com/ZJU-REAL/ViSkill.

Figures & tables

Figure 1: Text-centric skills linearize spatial experience into language, causing lossy abstraction. ViSkill instead represents reusable experience as visual skill cards that preserve spatial structure natively.

Figure 2: Overview of ViSkill. The agent retrieves a visual skill $z_{k^{\star}}$ from the skill library to guide interaction, collects trajectory $\tau$ , and distills eligible trajectories into new skills. The skill-guided reward $R(\tau)$ and PPO jointly optimize the policy.

Proprietary Models
Method	Sokoban	FrozenLake	PrimitiveSkill						Overall
Method	Sokoban	FrozenLake	Place	Stack	Drawer	Align	Swap	Average	Overall
GPT-5	0.70	0.77	1.00	0.63	0.00	1.00	0.55	0.64	0.70
o3	0.60	0.78	1.00	0.63	0.00	1.00	0.82	0.69	0.69
o4-mini	0.44	0.82	1.00	0.50	0.00	0.75	0.33	0.52	0.59
GPT-4o	0.43	0.54	0.50	0.63	0.00	0.88	0.94	0.59	0.52

Table 1: Success rates on Sokoban, FrozenLake, and PrimitiveSkill. Bold and underlined entries indicate the best and second-best results, respectively.

Figure 3: Training dynamics of PPO, ViSkill, and ViSkill + Cold-Start across Sokoban, FrozenLake, and PrimitiveSkill. ViSkill converges faster and achieves higher success rates than PPO across all environments, with cold-start initialization providing additional gains in early-stage learning.

Table 5

$\bm{\lambda}$	$\mathbf{0.0}$	$\mathbf{0.6}$	$\mathbf{0.8}$	$\mathbf{1.0}$
Sokoban	$0.77$	$0.71$	0.82	$0.77$
FrozenLake	$0.80$	$0.79$	0.84	$0.81$

Table 4: Impact of

\lambda

on success rate.

Figure 8: Transfer performance of evolved skill libraries on the unoptimized base model, with and without joint policy optimization.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 9: Geometry-based skill retrieval on three held-out Sokoban configurations. Each row shows a query state alongside its top-five retrieved skills, ordered by composite score, with the selected skill highlighted in green.

Figure 10: Representative visual skill cards from Sokoban, FrozenLake, and PrimitiveSkill. Each card combines a rendered strategy description with annotated trajectory frames and action labels into a single composite image.

Optimization
Hyperparameter	Value
Policy backbone	Qwen2.5-VL-3B-Instruct
Algorithm	PPO
Advantage estimator	GAE
PPO mini-batch size	$32$
Actor learning rate	$1\times 10^{-6}$

Appendix

Table 5: Shared training hyperparameters across all environments.

Hyperparameter	Sokoban	FrozenLake	PrimitiveSkill
Total training steps	$300$	$300$	$60$
Training GPUs	$4$	$2$	$2$
Environment GPUs	–	–	$1$
Maximum prompt length	$5{,}000$	$5{,}000$	$3{,}000$
Maximum response length	$4{,}000$	$4{,}000$	$10{,}000$

Appendix

Table 6: Environment-specific training hyperparameters.

Figure 11: Task-wise success-rate dynamics across the five PrimitiveSkill manipulation tasks. Both ViSkill variants reach full success faster than PPO, with cold-start initialization further accelerating early-stage learning.

Figure 12: Skill retrieval and distillation dynamics on Sokoban and FrozenLake. Retrieval fraction (left) and newly admitted skills per step (right). Retrieval becomes nearly ubiquitous as the library grows, while admissions diminish once common configurations are covered.

Figure 13: Qualitative comparison with PPO on Sokoban and FrozenLake. For each configuration, PPO reaches a failure state while ViSkill retrieves a relevant visual skill card and completes the task.

Threshold	Sokoban	FrozenLake
Solver-Based	0.82	0.84
Static-10	0.84	0.78

Appendix

Table 7: Success rates under different quality thresholds.

Extractor	Sokoban	FrozenLake
Rule-Based	0.82	0.84
DINOv2	0.80	0.83

Appendix

Table 8: Success rates with different feature extractors.

Explore similar work

Jul 17, 2026cs.RO

SkillNav: Score-Level Skill Intervention for Zero-Shot Object Goal Navigation

Vision-Language Model (VLM) agents have advanced zero-shot object-goal navigation, yet single-frame reasoning leaves them without the cross-step behavioral awareness an embodied navigator requires, producing recurring failures such as dead-end stalls, in-room loops, and circuitous approaches to detected targets. Prompt-based remedies inflate token budgets across multi-submodule episodes and still struggle to encode inherently spatial signals such as angles, map cells, and viewpoint coordinates. In this paper, we propose SkillNav, an extensible behavioral skill framework for VLM-based navigation that treats the curiosity value map already maintained by modern VLM navigators as a writable substrate on which composable skills inscribe behavioral memory at zero token cost. Skills are stratified into three tiers by their level of behavioral authority, namely soft scaling for proportional reweighting, lower-bound boost for region-level guarantees, and hard override for threshold-triggered forced actions, and cooperate across tiers under a fixed composition order that establishes a predictable, declared priority among skills. This design turns capability improvement into skill registration: new behaviors plug in without retraining the VLM or disturbing existing skills, opening a path for continual refinement. A minimal prompt channel complements the score-level skills with category-level semantic hints, yielding a dual-representation design in which spatial memory lives on the map and semantic memory in short prompts. Training-free, SkillNav establishes new state-of-the-art SPL across MP3D (25.5), HM3D v0.1 (39.3), and HM3D v0.2 (43.2), improving SPL by up to 6.0 absolute over the strongest prior method, and achieves the highest Success Rate on HM3D v0.1 (69.7) and v0.2 (75.9).

Ruijie Sang, Yiqun Duan, Pinhan Fu +3

D-Robotics · HAI Center, University of Technology Sydney · School of Computer Science, Wuhan University

Vision-Language Navigation Object-Goal Navigation

Ask PDF

Sep 1, 2026cs.RO

EmbodiedSkills: A Unified Framework for Orchestrating, Training, and Deploying VLA Agents

Vision-language-action (VLA) models map visual observations and language instructions directly to robot actions, but long-horizon tasks require more than action prediction. An agent must coordinate perception, planning, execution, progress verification, and recovery as the physical state evolves. An action prediction or a model-generated skill decision does not, by itself, guarantee that the proposed operation is valid in the current state or that its outcome will be verified. We propose EmbodiedSkills, a unified framework that treats each skill decision as an execution proposal: the runtime checks its prerequisites before execution and verifies the outcome afterward. A shared executable-skill interface connects high-level skill selection, bounded low-level VLA execution, and post-action verification within a single agent loop. Because this interface remains fixed, low-level VLA policies can be replaced or adapted without changing the agent loop. The interface also records planning, execution, verification, and recovery events as structured trajectories, which provide supervision for individual components and can support optional online adaptation when interactive feedback is available. We instantiate EmbodiedSkills with Qwen3-VL and OpenPI/pi0.5 on RoboTwin 2.0 and LIBERO. Task-adapted low-level VLA policies achieve an average success rate of 86.20% across 50 RoboTwin 2.0 tasks and 97.40% across the four LIBERO suites. These results establish the execution performance of the task-adapted low-level VLA policies used in EmbodiedSkills. On four memory-dependent RMBench tasks, the same task-adapted execution approach achieves 12.5% average success. The framework provides a trainable and inspectable agent layer for turning these policies into closed-loop embodied systems.

Wei Wang, Wenqiao Zhang, Yutong Lin +14

College of Computer Science and Technology, Zhejiang University · Nanjing University of Aeronautics and Astronautics · Cornell University +3

Robotic Control Large Language Model-Based Robot Planning Robotics Vision-Language-Action Models

Ask PDF

May 18, 2026cs.CV

AtlasVA: Self-Evolving Visual Skill Memory for Teacher-Free VLM Agents

Vision-language model (VLM) agents increasingly rely on memory-augmented reinforcement learning to reuse experience across long-horizon tasks, yet most existing frameworks store memory as text and depend on proprietary teacher models to summarize or refine it. This design is poorly matched to spatial decision making: geometric priors are compressed into lossy language, and sparse interaction is often supervised through delayed textual feedback rather than dense visually grounded signals. We argue that reusable experience for VLM agents should remain visually grounded. Based on this insight, we propose \textbf{AtlasVA}, a teacher-free visual skill memory framework that organizes memory into three complementary layers: spatial heatmaps, visual exemplars, and symbolic text skills. AtlasVA further evolves danger and affinity atlases directly from trajectory statistics and lightweight grid heuristics, and reuses these self-evolving atlases as potential-based shaping rewards for reinforcement learning. This unifies perception, memory, and optimization without external LLM supervision. Experiments on \textsc{Sokoban}, \textsc{FrozenLake}, 3D embodied navigation, and 3D robotic manipulation benchmarks show that AtlasVA consistently outperforms text-centric memory baselines and competitive VLM agents, with especially strong gains on spatially intensive tasks. Homepage: https://wangpan-ustc.github.io/AtlasvaWeb

Pan Wang, Yihao Hu, Xiujin Liu +3

Ant Group · University of Science and Technology of China · Westlake University +2

Memory-Augmented VLMs Reward Shaping Visually Grounded Reasoning VLMs for Robotics Memory-Augmented Agents RL for VLMs

Ask PDF