cs.CVOct 8, 2026

ViSkill: Reinforcing VLM Agents with Evolving Visual-Native Skills

Authors: Hongxing Li, Dingming Li, Yixin Li, Yong Du, Wenqi Zhang, Weiming Lu, Jun Xiao, Yueting Zhuang, +1 more

Organizations: Zhejiang University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Skill-augmented agents improve sample efficiency by distilling successful trajectories into reusable strategies. Yet most existing approaches remain text-centric, linearizing spatial layouts and action-state correspondences into language that loses critical geometric structure. Recent efforts have begun incorporating visual evidence, but construct and update skills separately from policy optimization, leaving their mutual improvement underexplored. We propose ViSkill, a visual-native skill learning framework that encodes successful interactions as composite visual skill cards directly accessible to VLM agents. Retrieved skills guide both inference and reward shaping, while successful trajectories are distilled back into the library, forming a closed feedback loop in which skill accumulation and policy improvement reinforce each other. An optional cold-start mechanism further accelerates early-stage learning. Evaluated on Sokoban, FrozenLake, and PrimitiveSkill, ViSkill achieves an overall success rate of 0.89, rising to 0.91 with cold-start initialization, outperforming all evaluated proprietary and open-source baselines while converging faster than standard PPO. Our code is available at https://github.com/ZJU-REAL/ViSkill.

Figures & tables

Figure 1: Text-centric skills linearize spatial experience into language, causing lossy abstraction. ViSkill instead represents reusable experience as visual skill cards that preserve spatial structure natively.
Figure 2: Overview of ViSkill. The agent retrieves a visual skill zk⋆z_{k^{\star}} from the skill library to guide interaction, collects trajectory τ\tau , and distills eligible trajectories into new skills. The skill-guided reward R(τ)R(\tau) and PPO jointly optimize the policy.
MethodSokobanFrozenLakePrimitiveSkillOverall
PlaceStackDrawerAlignSwapAverage
Proprietary Models
GPT-50.700.771.000.630.001.000.550.640.70
o30.600.781.000.630.001.000.820.690.69
o4-mini0.440.821.000.500.000.750.330.520.59
GPT-4o0.430.540.500.630.000.880.940.590.52
Table 1: Success rates on Sokoban, FrozenLake, and PrimitiveSkill. Bold and underlined entries indicate the best and second-best results, respectively.
Figure 3: Training dynamics of PPO, ViSkill, and ViSkill + Cold-Start across Sokoban, FrozenLake, and PrimitiveSkill. ViSkill converges faster and achieves higher success rates than PPO across all environments, with cold-start initialization providing additional gains in early-stage learning.
Table 5
Figure 6
λ\bm{\lambda}0.0\mathbf{0.0}0.6\mathbf{0.6}0.8\mathbf{0.8}1.0\mathbf{1.0}
Sokoban0.770.770.710.710.820.770.77
FrozenLake0.800.800.790.790.840.810.81
Table 4: Impact of λ\lambda on success rate.
Figure 8
Figure 8: Transfer performance of evolved skill libraries on the unoptimized base model, with and without joint policy optimization.
Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 9: Geometry-based skill retrieval on three held-out Sokoban configurations. Each row shows a query state alongside its top-five retrieved skills, ordered by composite score, with the selected skill highlighted in green.
Figure 10: Representative visual skill cards from Sokoban, FrozenLake, and PrimitiveSkill. Each card combines a rendered strategy description with annotated trajectory frames and action labels into a single composite image.
HyperparameterValue
Optimization
Policy backboneQwen2.5-VL-3B-Instruct
AlgorithmPPO
Advantage estimatorGAE
PPO mini-batch size3232
Actor learning rate1×10−61\times 10^{-6}
Appendix
Table 5: Shared training hyperparameters across all environments.
HyperparameterSokobanFrozenLakePrimitiveSkill
Total training steps3003003003006060
Training GPUs442222
Environment GPUs––11
Maximum prompt length5,0005{,}0005,0005{,}0003,0003{,}000
Maximum response length4,0004{,}0004,0004{,}00010,00010{,}000
Appendix
Table 6: Environment-specific training hyperparameters.
Figure 11: Task-wise success-rate dynamics across the five PrimitiveSkill manipulation tasks. Both ViSkill variants reach full success faster than PPO, with cold-start initialization further accelerating early-stage learning.
Figure 12: Skill retrieval and distillation dynamics on Sokoban and FrozenLake. Retrieval fraction (left) and newly admitted skills per step (right). Retrieval becomes nearly ubiquitous as the library grows, while admissions diminish once common configurations are covered.
Figure 13: Qualitative comparison with PPO on Sokoban and FrozenLake. For each configuration, PPO reaches a failure state while ViSkill retrieves a relevant visual skill card and completes the task.
ThresholdSokobanFrozenLake
Solver-Based0.820.84
Static-100.840.78
Appendix
Table 7: Success rates under different quality thresholds.
ExtractorSokobanFrozenLake
Rule-Based0.820.84
DINOv20.800.83
Appendix
Table 8: Success rates with different feature extractors.

Explore similar work

CardsList

  1. SkillNav: Score-Level Skill Intervention for Zero-Shot Object Goal Navigation

    Jul 17, 2026Ruijie Sang, Yiqun Duan, Pinhan Fu +3Vision-Language NavigationObject-Goal Navigation

    D-Robotics · HAI Center, University of Technology Sydney · School of Computer Science, Wuhan University

  2. EmbodiedSkills: A Unified Framework for Orchestrating, Training, and Deploying VLA Agents

    Sep 1, 2026Wei Wang, Wenqiao Zhang, Yutong Lin +14Robotic ControlLarge Language Model-Based Robot Planning

    College of Computer Science and Technology, Zhejiang University · Nanjing University of Aeronautics and Astronautics · Cornell University +3

  3. AtlasVA: Self-Evolving Visual Skill Memory for Teacher-Free VLM Agents

    May 18, 2026Pan Wang, Yihao Hu, Xiujin Liu +3Memory-Augmented VLMsReward Shaping

    Ant Group · University of Science and Technology of China · Westlake University +2