cs.CVNov 16, 2026

What Makes a Good Move? Learning to Act from Experience in Visual Worlds

Authors: Cehao Yang, Ziyi He, Jing Yi, Jian Guo, Hui Xiong

Organizations: HKUST(GZ) · HKU · LIGHTSPEED · IDEA Research

Abstract

Learning visual policies requires turning interaction experience into guidance for individual decisions. Trajectory outcomes obscure action quality, while unguided exploration can revisit familiar situations. We present LightAct, an experience-learning framework that connects skill-guided exploration with policy internalization. Interaction segments are summarized into experience entries and consolidated into a game-specific skill that guides root collection toward broader state coverage. At each root, candidate actions are compared through short rollouts under a shared frozen policy. Environment-reported score gains are averaged across continuations and normalized into relative action rewards, producing reusable supervision for reinforcement learning. Policy training uses a compact dataset of just 1,024 decision roots collected in a single source game and paired with their action rewards. Across 170 GameWorld tasks spanning 34 games, LightAct improves overall progress by 23.6, 29.2, and 13.0 percentage points at the 2B, 4B, and 9B scales, respectively, with our strongest model competitive with reported proprietary-model results. Ablations show broader root-state coverage with skill guidance and consistent gains across three policy optimizers. Case studies show how rollouts distinguish actions with identical immediate gains.

Figures & tables

Explore similar work

CardsList
  1. WorldGuide: Learning Success-Failure Boundaries in Latent World Models for Vision-Language-Action Policies

    Sep 28, 2026Lin Liu, Lu Zhang, Ziying Song +6Robotic ManipulationVision-Language-Action Models

  2. Back on Track: Deployment-time Policy Improvement via World Model Planning

    Oct 14, 2026Colton Crosby, Hao Wang, Mac Schwager +1Robot Policy AdaptationWorld Model-Based Planning