Self-Correcting VLA: Online Action Refinement via Sparse World Imagination

Chenyv Liu; Wentao Tan; Lei Zhu; Fengling Li; Jingjing Li; Guoli Yang; Heng Tao Shen

cs.ROFeb 25, 2026

Self-Correcting VLA: Online Action Refinement via Sparse World Imagination

Authors: Chenyv Liu, Wentao Tan, Lei Zhu, Fengling Li, Jingjing Li, Guoli Yang, Heng Tao Shen

Organizations: Tongji University · University of Technology Sydney · University of Electronic Science and Technology of China · Advanced Institute of Big Data

Abstract

Standard vision-language-action (VLA) models rely on fitting statistical data priors, limiting their robust understanding of underlying physical dynamics. Reinforcement learning enhances physical grounding through exploration yet typically relies on external reward signals that remain isolated from the agent's internal states. World action models have emerged as a promising paradigm that integrates imagination and control to enable predictive planning. However, they rely on implicit context modeling, lacking explicit mechanisms for self-improvement. To solve these problems, we propose Self-Correcting VLA (SC-VLA), which achieve self-improvement by intrinsically guiding action refinement through sparse imagination. We first design sparse world imagination by integrating auxiliary predictive heads to forecast current task progress and future trajectory trends, thereby constraining the policy to encode short-term physical evolution. Then we introduce the online action refinement module to reshape progress-dependent dense rewards, adjusting trajectory orientation based on the predicted sparse future states. Evaluations on challenging robot manipulation tasks from simulation benchmarks and real-world settings demonstrate that SC-VLA achieve state-of-the-art performance, yielding the highest task throughput with 16% fewer steps and a 9% higher success rate than the best-performing baselines, alongside a 14% gain in real-world experiments. Code is available at https://github.com/Kisaragi0/SC-VLA.

Figures & tables

Figure 1: We present Self-Correcting VLA (SC-VLA), a novel framework designed to enhance physical grounding through intrinsic self-improvement. The model is equipped with Sparse World Imagination (SPI) to forecast task progress and future trajectory trends, and Online Action Refinement (OAR) to dynamically optimize policies via residual adjustments and reshaped rewards. SC-VLA achieves superior performance on ManiSkill and real-world ARX5 benchmarks, surpassing baselines in both success rate and execution throughput.

Figure 2: The architecture of Self-Correcting VLA. The framework consists of two stages: Stage I (Top) utilizes a VLM and DiT-based backbone to generate base actions and sparse world imagination, decoded from the final output (Layer $N$ ) and intermediate features (Layer $M$ ), respectively. Stage II (Bottom) implements Online Action Refinement, where a Residual RL Module optimizes the final action by learning a residual term. This process is guided by endogenous dense rewards derived from the dynamic weighting of imagination consistency (Progress and $\Delta$ State ) without external supervision.

Model	ManiSkill Tasks				Avg
Model	Stack Cube	Place Sphere	LiftPeg Upright	Peg Insertion	Avg
DP ‡	0.46	0.90	0.10	0.00	0.36
DP †	0.88	1.00	0.80	0.40	0.77
ACT ‡	0.50	0.88	0.60	0.12	0.52
ACT †	0.64	0.90	0.46	0.04	0.51
$\pi_{0}$	0.66	0.86	0.48	0.22	0.55

Table 1: Success rates of all methods in ManiSkill. Due to the absence of language guidance in DP, ACT, we evaluate them under two settings: ‡ denotes a single multi-task policy trained on all tasks simultaneously. † denotes training independent specialist models for each of the four tasks separately.

Model	ManiSkill Tasks				Avg
Model	Stack Cube	Place Sphere	LiftPeg Upright	Peg Insertion	Avg
DP ‡	162	146	202	800	327
DP †	132	125	197	233	172
ACT ‡	148	131	207	250	184
ACT †	198	126	203	230	189
$\pi_{0}$	265	179	331	331	276

Table 2: Average completion length over successful episodes.

Model	ManiSkill Tasks				Avg
Model	Stack Cube	Place Sphere	LiftPeg Upright	Peg Insertion	Avg
SC-VLA w/o state	0.88	1.00	0.84	0.42	0.78
SC-VLA w/o prog	0.92	1.00	0.80	0.50	0.80
SC-VLA w/o state_prog	0.78	1.00	0.72	0.40	0.72
SC-VLA(SPI)	0.96	1.00	0.82	0.50	0.82

Table 3: Ablation study on different imagination components.

Figure 3: Hardware platforms and visualizations of sampled tasks. The setup is equipped with a wristed camera and a third-person camera.

Figure 4: Ablation study on the effectiveness of sparse world imagination rewards and dynamic weight scheduling. We visualize the performance curves starting from the main training phase, excluding the data collection and residual warm-up periods. See Appendix C for further details.

Model	Real World Tasks				Avg
Model	Stack Cube	Place Sphere	Push Cube	Peg Insertion	Avg
DP ‡	0.30	0.40	0.45	0.00	0.28
GR00T N1.5	0.75	0.45	0.80	0.30	0.57
SC-VLA(SPI)	0.85	0.60	1.00	0.40	0.71

Table 4: Success rates on real-world tasks using the ARX5 arm.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Tasks	Success Criteria	Max Steps
StackCube	The task is considered successful if the horizontal offset of the red cube relative to the green cube does not exceed the half-diagonal of the cube plus 5 mm, the vertical distance between the centers of the two cubes equals one cube edge length with a tolerance of 5 mm, and the red cube remains stationary and is not grasped by the robot end-effector.	800
PlaceSphere	The task is considered successful if the horizontal offset of the sphere relative to the center of the container does not exceed 5 mm, the vertical position satisfies that the contact point between the bottom of the sphere and the container floor has an error within 5 mm, and the sphere is not grasped by the robot end-effector.	500
LiftPegUpright	The task is considered successful if the peg’s orientation is close to the upright configuration, with a deviation around the vertical axis not exceeding 0.08 rad, and the vertical position of the peg’s center of mass deviates from its half-length by no more than 5 mm.	800
PegInsertionside	The task is considered successful if the tail end of the peg lies within the coordinate frame of the box hole, with an offset along the insertion direction no greater than 2 cm, and offsets along the other two orthogonal directions both smaller than 8 mm.	800

Appendix

Table 5: Detailed definitions and success criteria for the four manipulation tasks used in our evaluation.

Figure 5: Schematic of the multi-stage training protocol and residual weight schedule. The training process is divided into three distinct phases to ensure stability: (1) Buffer Warm-up for gathering demonstrations, (2) Residual Injection for gradually introducing the RL policy, and (3) the Main Training Phase . The curve illustrates the evolution of the residual weight

\lambda

, transitioning from pure imitation to full residual learning.

Hyperparameter	Value
Batch size	32
Training steps	50,000
Learning rate	$1\times 10^{-4}$
Model init seed	42
Optimizer	AdamW

Appendix

Table 6: SC-VLA base model hyperparameter settings.

	Shared Parameters		Task-Specific Parameters
Tasks	Hyperparameter	Value	Hyperparameter	Value
StackCube	Random seed 0 Replay buffer size Total steps Discount factor $\gamma$ 0.97 Target smoothing $\tau$ 0.01 Batch size $B$ 1024 Policy learning rate $1\times 10^{-4}$ Critic learning rate $1\times 10^{-4}$ Entropy coeff. $\alpha$ 0.2 Gradient clipping 50 Update-to-Data (UTD) 0.5 Training frequency 64 Guide weight $w_{\text{guide}}$ 0.6		Total timesteps	500,000
			Learning start steps	30,000
			Exploration steps	100,000
			Residual scale (train)	0.01
			Residual scale (eval)	0.005

Appendix

Table 7: SC-VLA residual model hyperparameter settings.

Figure 6: Visualization results of Maniskill simulated tasks.

Figure 7: Visualization results of real-world tasks.

Explore similar work

Find Something You Can't Do: Agentic Real-World Reinforcement Learning for Self-Improving VLA Models
Sep 25, 2026Yuan Fang, Zechu Li, Haolei Tong +2Robotic RL Agentic RL
TU Darmstadt · University of Augsburg · Tongji University +1
Imagine-RL: Residual-Confidence-Guided Cross-Attention for World-Model-Augmented VLA Reinforcement Learning
Sep 21, 2026Kejia Hu, Wentong Zhai, Bo Zhao +1Robotic Manipulation Vision-Language-Action Models
Department of Electric Engineering, Korea Advanced Institute of Science and Technology · University Of Science and Technology Beijing · Shanghai Jiao Tong University +1
Imagining Recovery: Inference-Time Counterfactual Realignment for Vision-Language-Action Models
Aug 14, 2026Yanyan Zhang, Disheng Liu, Kai Ye +6Vision-Language-Action Models Robustness of VLA Models
Case Western Reserve University Cleveland, OH, USA

Sep 25, 2026J/K move · Enter open · S save

Find Something You Can't Do: Agentic Real-World Reinforcement Learning for Self-Improving VLA Models

Yuan Fang, Zechu Li, Haolei Tong, Puze Liu+1

TU Darmstadt · University of Augsburg · Tongji University+1

Robotic RL Agentic RL RL for VLMs Robotic Manipulation

Open paper AskPDF arXiv

Vision--language--action (VLA) models provide strong priors for robotic manipulation but are typically deployed as frozen policies, unable to improve from their own failures. Real-world reinforcement learning (RL) offers a path to continued improvement, yet manual environment resets and task-success supervision hinder autonomous learning. We introduce FIND, an agentic real-world RL framework that closes the loop between scene understanding, weakness-aware practice, self-evaluation, and policy improvement in a persistent workspace. FIND reframes autonomous practice as a scene-conditioned, performance-aware task-selection problem: instead of restoring a predefined scene after each rollout, it uses the resulting scene to determine what to practice next. A vision--language agent identifies feasible tasks from a predefined library, prioritizes those with lower recent success rates, and evaluates outcomes using paired pre- and post-execution observations. We instantiate FIND with a frozen

π_{0.5}

VLA and residual off-policy RL. Across eight real-world manipulation tasks, the independent human-assessed success rate improves from

55\%

71.9\%

. A representative run completes 456 autonomous episodes within 6 hours of interaction, requiring 30 scene-recovery interventions and no human-provided reward labels during online learning. Ablations and systematic evaluations further examine key design choices, agent evaluation accuracy, and human intervention requirements. Our website is made publicly available at: FIND.github.io.

Abstract

Explore similar work

Find Something You Can't Do: Agentic Real-World Reinforcement Learning for Self-Improving VLA Models

Imagine-RL: Residual-Confidence-Guided Cross-Attention for World-Model-Augmented VLA Reinforcement Learning

Imagining Recovery: Inference-Time Counterfactual Realignment for Vision-Language-Action Models