Boosting Vision-Language-Action Finetuning with Feasible Action Neighborhood Prior

Haochen Niu; Kanyu Zhang; Shuyu Yin; Qinghai Guo; Peilin Liu; Fei Wen

Figure 1: Geometric structure of policy distribution on a ManiSkill task. (a) After the SFT warm-up stage, the policy learned a narrow, peaked distribution with a minimal FAN, resulting in poor generalization. (b) Subsequent RFT with PPO broadens the distribution, leading to improved task success; (c) Our FAN-PPO method explicitly guides the policy towards a robust Gaussian shape, achieving the highest success rate and demonstrating superior generalization.

Method	In-Distribution	Out-of-Distribution
Method	In-Distribution	Vision	Semantic	Execution	Avg.
RL4VLA [ 31 ]	88.5	74.0	61.8	46.2	60.7
OpenVLA + SFT [ 23 ]	78.1 $\pm$ 3.1	76.6 $\pm$ 1.9	57.4 $\pm$ 0.9	40.4 $\pm$ 0.8	58.1
OpenVLA + FAN-SFT (Ours)	89.8 $\pm$ 0.8	81.7 $\pm$ 1.1	63.5 $\pm$ 1.5	44.8 $\pm$ 0.5	63.3
$\triangle$ Improvement	+11.7	+5.1	+6.1	+4.4	+5.2

Table 1: Comparison of SFT results on the ManiSkill benchmark. Values denote success rates (%).

Figure 2: SFT performance on OpenVLA with and without FAN-guided regularization across different OOD tasks on ManiSkill.

Figure 3: SFT performance on OpenVLA with and without FAN-guided regularization across different data sizes on in-distribution and three OOD tasks (Vision, Semantic, Execution).

Figure 4: Spatial robustness on LIBERO-Spatial , comparing OpenVLA finetuned with SFT (left) versus our FAN-SFT (right). Color indicates success rate; the black and red dashed lines are the equal-success-rate contours for each method, respectively.

Figure 5: Qualitative comparison under spatial perturbation. Vanilla SFT remains biased toward the seen position, whereas FAN-SFT correctly adapts to the perturbed target location.

Method	In-Distribution	Out-of-Distribution
Method	In-Distribution	Vision	Semantic	Execution	Avg.
RL4VLA [ 31 ]	93.8	76.6	72.8	78.3	75.9
OpenVLA + PPO	95.9 $\pm$ 3.2	80.1 $\pm$ 0.1	79.7 $\pm$ 2.0	85.8 $\pm$ 1.8	81.9
OpenVLA + FAN-PPO (Ours)	97.4 $\pm$ 0.7	85.0 $\pm$ 4.0	86.7 $\pm$ 1.3	92.6 $\pm$ 1.5	88.1
$\triangle$ Improvement	+1.5	+4.9	+7.0	+6.9	+6.2
OpenVLA-OFT + PPO	92.3 $\pm$ 2.5	84.9 $\pm$ 1.1	49.0 $\pm$ 0.6	55.9 $\pm$ 1.2	63.3

Table 2: Comparison of RFT results on the ManiSkill benchmark. Values denote success rates (%).

Figure 6: Performance comparison of OpenVLA and OpenVLA-OFT trained with PPO and FAN-PPO across various OOD tasks.

Figure 7: Training curves of rollout and evaluation success rates during RFT of OpenVLA. Our FAN-guided regularization significantly improved the sample efficiency of OpenVLA.

Figure 8: Training curves of rollout and evaluation success rates during RFT of OpenVLA-OFT. Our FAN-guided regularization significantly improved the sample efficiency in evaluation and obtained better performance in training process of OpenVLA-OFT.

Figure 9: Real-world physical platform. (a) JAKA robot platform; (b) the task of placing the object into a box from unseen spatial positions, to evaluate the model’s spatial robustness.

Method	Task-1	Task-2	Task-3	Task-4
OpenVLA [ 23 ]	19/30	7/30	7/30	1/30
OpenVLA + FAN-SFT	22/30	12/30	17/30	7/30

Table 3: Real-world evaluation results (Successes / Total Trials).

Base Configuration (adopted from RL4VLA)
# GPUs	4 $\times$ NVIDIA A100 (80GB VRAM)
learning rate (LR)	5e-4
total batch size	40 (10 per GPU)
grad accumulation steps	1
input images	1 third-person camera
input image size	224 $\times$ 224 px

Table 4: Hyperparameters for SFT OpenVLA on the ManiSkill benchmark. We adopt the configuration from RL4VLA for our baseline and extend it with a regularization coefficient

\alpha

for our method (FAN-SFT).

Base Configuration (adopted from OpenVLA)
# GPUs	2 $\times$ NVIDIA A100 (80GB VRAM)
learning rate (LR)	5e-4
total batch size	48 (24 per GPU)
grad accumulation steps	1
input images	1 third-person camera
input image size	224 $\times$ 224 px

Table 5: Hyperparameters for SFT OpenVLA on the LIBERO benchmark. We adopt the configuration from OpenVLA for our baseline and extend it with a regularization coefficient

\alpha

for our method (FAN-SFT).

Base Configuration (adopted from OpenVLA-OFT)
# GPUs	4 $\times$ NVIDIA A100 (80GB VRAM)
learning rate (LR)	5e-4
total batch size	32 (8 per GPU)
grad accumulation steps	2
input images	1 third-person camera, 1 wrist-mounted camera
input image size	224 $\times$ 224 px

Table 6: Hyperparameters for SFT OpenVLA-OFT on the LIBERO benchmark. We adopt the configuration from OpenVLA-OFT for our baseline and extend it with a regularization coefficient

\alpha

for our method (FAN-SFT).

Variant	Definition
Vision
Table	unseen table
Texture-w	dynamic texture (weak)
Texture-s	dynamic texture (strong)
Noise-w	dynamic noise (weak)
Noise-s	dynamic noise (strong)

Table 7: Definitions of the OOD task variants. The first five variants correspond to vision OOD, the middle eight to semantic OOD, and the last three to execution OOD.

Method	In-Distribution	Out-of-Distribution
Method	In-Distribution	Vision	Semantic	Execution	Avg.
RL4VLA † [ 31 ]	88.5	74.0	61.8	46.2	60.7
OpenVLA + SFT [ 23 ]	78.1 $\pm$ 3.1	76.6 $\pm$ 1.9	57.4 $\pm$ 0.9	40.4 $\pm$ 0.8	58.1
OpenVLA + FAN-SFT (Ours)	89.8 $\pm$ 0.8	81.7 $\pm$ 1.1	63.5 $\pm$ 1.5	44.8 $\pm$ 0.5	63.3
$\triangle$ Improvement	+11.7	+5.1	+6.1	+4.4	+5.2

Table 8: Comparison of supervised fine-tuning (SFT) results on the ManiSkill benchmark. Values denote success rates (%). Results marked with † are taken directly from the original RL4VLA paper. Since the SFT dataset used in RL4VLA is not publicly available, we reproduce its setting by collecting an SFT dataset of the same size in the same environments using a motion planner. Based on this reproduced setup, we obtain the OpenVLA + SFT baseline as our reproduction of RL4VLA, and further implement our proposed OpenVLA + FAN-SFT method.

	Task	RL4VLA(SFT) †	OpenVLA+SFT	OpenVLA+FAN-SFT	$\Delta$
Vision	Table	86.5	80.6 $\pm$ 5.5	84.4 $\pm$ 2.9	+3.8
	Texture-w	79.7	80.0 $\pm$ 3.4	85.9 $\pm$ 3.1	+5.9
	Texture-s	63.0	72.5 $\pm$ 5.8	73.4 $\pm$ 4.5	+0.9
	Noise-w	77.1	81.3 $\pm$ 4.0	87.2 $\pm$ 1.8	+5.9
	Noise-s	63.5	66.9 $\pm$ 2.3	74.1 $\pm$ 3.2	+7.2
Semantic	Obj.	53.6	42.5 $\pm$ 4.1	49.4 $\pm$ 4.8	+6.9

Table 9: Detailed SFT results on the ManiSkill benchmark (OOD tasks).

Method	Spatial	Object	Goal	Long	Avg.
Octo [ 15 ]	78.9	85.7	84.6	51.1	75.1
OpenVLA [ 23 ]	84.7	88.4	79.2	53.7	76.5
WorldVLA [ 8 ]	87.6	96.2	83.4	60.0	81.8
TriVLA [ 32 ]	91.2	93.8	89.8	73.2	87.0
UniVLA [ 7 ]	96.5	96.8	95.6	92.0	95.2
OpenVLA-OFT [ 22 ]	95.2	94.2	95.2	93.2	94.5

Table 10: Detailed SFT results on the LIBERO Benchmark.

Figure 10: Simulation rollouts on four manipulation tasks from the LIBERO-Spatial benchmark under disturbances applied to the plate (i.e., the target placement location). For Tasks 1 and 2, the plate is perturbed along the

x

-axis; for Tasks 3 and 4, the plate is perturbed along the

y

-axis. In each subfigure, the top row shows trajectories generated by the baseline OpenVLA + SFT policy, and the bottom row shows trajectories generated by our proposed OpenVLA + FAN-SFT policy, which exhibits stronger robustness to target-location disturbances.

Regularization coeff. $\alpha$	In-Distribution	Out-of-Distribution
Regularization coeff. $\alpha$	In-Distribution	Vision	Semantic	Execution	Avg.
0.0	78.1	76.3	57.6	41.0	58.3
1e-4	81.8	76.2	58.0	44.6	59.6
1e-3	87.0	75.6	60.9	44.2	60.2
1e-2	89.6	77.5	62.9	46.5	62.3
2e-2	88.4	77.5	62.9	49.1	63.2

Table 11: Sensitivity of OpenVLA + FAN-SFT to the regularization coefficient

\alpha

on ManiSkill.

Method	In-Distribution	Out-of-Distribution
Method	In-Distribution	Vision	Semantic	Execution	Avg.
Ori	78.1	76.6	57.4	40.4	58.1
+ $\epsilon=0.05$	82.8	75.9	62.3	42.2	60.1
+ $\epsilon=0.1$	81.3	69.1	60.3	39.4	56.3
+ $\epsilon=0.2$	79.7	68.1	51.8	42.2	54.0
Ours	89.8	81.7	63.5	44.8	63.3

Table 12: Comparison to label smoothing on ManiSkill.

Base Configuration
# GPUs	1 $\times$ NVIDIA A100 (80GB VRAM)
learning rate (LR)	1e-4 for policy; 3e-3 for value head
mini-batch size	8 for OpenVLA; 12 for OpenVLA-OFT
grad accumulation steps	20
LoRA rank	32
input images	1 third-person camera

Table 13: Hyperparameter setup for PPO in our experiments. We follow the configuration from the original implementation, including common parameters and those specific to our FAN-PPO.

	Task	RL4VLA (RFT) †	OpenVLA +PPO	OpenVLA +FAN-PPO	$\Delta$	OpenVLA-OFT +PPO	OpenVLA-OFT +FAN-PPO	$\Delta$
Vision	Table	84.4	91.3 $\pm$ 2.8	97.1 $\pm$ 0.4	+5.9	89.0 $\pm$ 1.9	93.5 $\pm$ 0.9	+4.5
	Texture-w	83.3	85.0 $\pm$ 0.9	90.6 $\pm$ 6.1	+5.7	93.1 $\pm$ 3.0	92.8 $\pm$ 2.3	-0.3
	Texture-s	63.0	69.2 $\pm$ 2.3	72.4 $\pm$ 0.7	+3.2	79.3 $\pm$ 2.4	83.3 $\pm$ 5.1	+4.0
	Noise-w	85.4	83.0 $\pm$ 1.2	91.2 $\pm$ 1.0	+8.2	90.3 $\pm$ 1.1	92.8 $\pm$ 0.9	+2.5
	Noise-s	66.7	72.0 $\pm$ 4.6	73.7 $\pm$ 6.5	+1.7	73.0 $\pm$ 2.7	78.0 $\pm$ 4.3	+5.0
Semantic	Obj.	71.4	78.3 $\pm$ 3.6	85.4 $\pm$ 2.7	+7.1	77.0 $\pm$ 3.7	83.6 $\pm$ 1.7	+6.6

Table 14: Detailed RFT results on the ManiSkill benchmark (OOD tasks).

Method	Steps to 60% SR	Steps to 70% SR	Steps to 80% SR	Steps to 90% SR
OpenVLA + PPO	18	62	133	249
OpenVLA + FAN-PPO	18	37	56	98

Table 15: Training steps required to reach different rollout success rate thresholds on the training subset of ManiSkill with OpenVLA model. Columns report the number of training steps needed to achieve 60%, 70%, 80%, and 90% success rate (SR), respectively. Our method (OpenVLA + FAN-PPO) consistently attains the same success rate with fewer steps, demonstrating improved sample efficiency.

Method	Steps to 55% SR	Steps to 60% SR	Steps to 65% SR	Steps to 70% SR	Steps to 75% SR
OpenVLA + PPO	109	149	209	279	339
OpenVLA + FAN-PPO	29	59	109	129	179

Table 16: Training steps required to reach different evaluation success rate on the test subset of ManiSkill with OpenVLA model. Columns report the number of training steps needed to achieve 55%, 60%, 65%, 70%, and 75% success rate (SR), respectively. Our method (OpenVLA + FAN-PPO) consistently attains the same success rate with fewer steps, demonstrating improved sample efficiency.

Method	Steps to 50% SR	Steps to 60% SR	Steps to 70% SR	Steps to 80% SR	Steps to 90% SR
OpenVLA-OFT + PPO	26	35	46	82	186
OpenVLA-OFT + FAN-PPO	17	24	38	60	151

Table 17: Training steps required to reach different rollout success rate thresholds on the training subset of ManiSkill with OpenVLA-OFT model. Columns report the number of training steps needed to achieve 50%, 60%, 70%, 80%, and 90% success rate (SR), respectively. Our method (OpenVLA-OFT + FAN-PPO) consistently attains the same success rate with fewer steps, demonstrating improved sample efficiency.

Method	Steps to 50% SR	Steps to 60% SR	Steps to 70% SR	Steps to 80% SR	Steps to 90% SR
OpenVLA-OFT + PPO	60	80	120	180	-
OpenVLA-OFT + FAN-PPO	40	50	60	100	240

Table 18: Training steps required to reach different evaluation success rate on the test subset of ManiSkill with OpenVLA-OFT model. Columns report the number of training steps needed to achieve 50%, 60%, 70%, 80%, and 90% success rate (SR), respectively. Our method (OpenVLA-OFT + FAN-PPO) consistently attains the same success rate with fewer steps, demonstrating improved sample efficiency.

Figure 11: Qualitative comparison of VLA task executions on the in-distribution task. Top row: trajectories generated by the baseline OpenVLA + PPO, which fails to accomplish the task. Bottom row: trajectories generated by our OpenVLA + FAN-PPO, which successfully complete the task

Figure 12: Qualitative comparison of VLA task executions on the ’Table’ task under vision OOD conditions. Top row: trajectories generated by the baseline OpenVLA + PPO, which fails to accomplish the task. Bottom row: trajectories generated by our OpenVLA + FAN-PPO, which successfully complete the task

Figure 13: Qualitative comparison of VLA task executions on the ’Texture-w’ task under vision OOD conditions. Top row: trajectories generated by the baseline OpenVLA + PPO, which fails to accomplish the task. Bottom row: trajectories generated by our OpenVLA + FAN-PPO, which successfully complete the task

Figure 14: Qualitative comparison of VLA task executions on the ’Texture-s’ task under vision OOD conditions. Top row: trajectories generated by the baseline OpenVLA + PPO, which fails to accomplish the task. Bottom row: trajectories generated by our OpenVLA + FAN-PPO, which successfully complete the task

Figure 15: Qualitative comparison of VLA task executions on the ’Noise-w’ task under vision OOD conditions. Top row: trajectories generated by the baseline OpenVLA + PPO, which fails to accomplish the task. Bottom row: trajectories generated by our OpenVLA + FAN-PPO, which successfully complete the task

Figure 16: Qualitative comparison of VLA task executions on the ’Noise-s’ task under vision OOD conditions. Top row: trajectories generated by the baseline OpenVLA + PPO, which fails to accomplish the task. Bottom row: trajectories generated by our OpenVLA + FAN-PPO, which successfully complete the task

Figure 17: Qualitative comparison of VLA task executions on the ’obj’ task under semantic OOD conditions. Top row: trajectories generated by the baseline OpenVLA + PPO, which fails to accomplish the task. Bottom row: trajectories generated by our OpenVLA + FAN-PPO, which successfully complete the task

Figure 18: Qualitative comparison of VLA task executions on the ’Recep’ task under semantic OOD conditions. Top row: trajectories generated by the baseline OpenVLA + PPO, which fails to accomplish the task. Bottom row: trajectories generated by our OpenVLA + FAN-PPO, which successfully complete the task

Figure 19: Qualitative comparison of VLA task executions on the ’Instruct’ task under semantic OOD conditions. Top row: trajectories generated by the baseline OpenVLA + PPO, which fails to accomplish the task. Bottom row: trajectories generated by our OpenVLA + FAN-PPO, which successfully complete the task

Figure 20: Qualitative comparison of VLA task executions on the ’M-obj (IND)’ task under semantic OOD conditions. Top row: trajectories generated by the baseline OpenVLA + PPO, which fails to accomplish the task. Bottom row: trajectories generated by our OpenVLA + FAN-PPO, which successfully complete the task

Figure 21: Qualitative comparison of VLA task executions on the ’M-obj (OOD)’ task under semantic OOD conditions. Top row: trajectories generated by the baseline OpenVLA + PPO, which fails to accomplish the task. Bottom row: trajectories generated by our OpenVLA + FAN-PPO, which successfully complete the task

Figure 22: Qualitative comparison of VLA task executions on the ’M-obj (OOD)’ task under semantic OOD conditions. Top row: trajectories generated by the baseline OpenVLA + PPO, which fails to accomplish the task. Bottom row: trajectories generated by our OpenVLA + FAN-PPO, which successfully complete the task

Figure 23: Qualitative comparison of VLA task executions on the ’Obj. Pos.’ task under execution OOD conditions. Top row: trajectories generated by the baseline OpenVLA + PPO, which fails to accomplish the task. Bottom row: trajectories generated by our OpenVLA + FAN-PPO, which successfully complete the task

Figure 24: Qualitative comparison of VLA task executions on the ’Robot Pose’ task under execution OOD conditions. Top row: trajectories generated by the baseline OpenVLA + PPO, which fails to accomplish the task. Bottom row: trajectories generated by our OpenVLA + FAN-PPO, which successfully complete the task

Figure 25: Qualitative comparison of VLA task executions on the ’Obj. Rep.’ task under execution OOD conditions. Top row: trajectories generated by the baseline OpenVLA + PPO, which fails to accomplish the task. Bottom row: trajectories generated by our OpenVLA + FAN-PPO, which successfully complete the task

Figure 26: Sensitivity of OpenVLA + FAN-PPO to the regularization coefficient

\alpha

on the ManiSkill PutOnPlateInScene25Main-v3 environment. Subfigure reports the rollout success rate on the training subset, while subfigure shows the evaluation success rate on the test subset.

Figure 27: Sensitivity of OpenVLA + FAN-PPO to the standard deviation

\sigma

on the ManiSkill PutOnPlateInScene25Main-v3 environment. Subfigure reports the rollout success rate on the training subset, while subfigure shows the evaluation success rate on the test subset.

Figure 28: Training curves during RFT of OpenVLA.

Figure 29: Real-world experiment hardware setup. (a) The RGB-D camera used for perception. (b) The robot arm used for manipulation. (c) The overall workspace configuration, where three types of target objects are placed on the table and the task is to place them into the black box.

Figure 30: Real-world rollouts on four object-in-box manipulation tasks. In each subfigure, the top row shows time-ordered key frames of trajectories generated by the baseline OpenVLA + SFT policy, and the bottom row shows trajectories generated by our proposed OpenVLA + FAN-SFT policy. Task-1 corresponds to the in-distribution setting with matched demonstration conditions, where both methods succeed. Tasks 2–4 introduce different real-world perturbations (object pose, manipulator pose, and box position, respectively), under which the baseline policy fails while our method remains successful.

Boosting Vision-Language-Action Finetuning with Feasible Action Neighborhood Prior

Abstract

Figures & tables

Explore similar work

PriorVLA: Prior-Preserving Adaptation for Vision-Language-Action Models

VOTE: Vision-Language-Action Optimization with Trajectory Ensemble Voting

EXPO-FT: Sample-Efficient Reinforcement Learning Finetuning for Vision-Language-Action Models