LIBERO-Pro2.0: A Benchmark for Evaluating the Robustness of Embodied Foundation Models

Xueyang Zhou; Yangming Xu; Qianjiang Li; Guiyao Tie; Junhao Wu; Ziheng Huang; Guowen Zhang; Zhijie Yuan

cs.ROOct 17, 2026

LIBERO-Pro2.0: A Benchmark for Evaluating the Robustness of Embodied Foundation Models

Authors: Xueyang Zhou, Yangming Xu, Qianjiang Li, Guiyao Tie, Junhao Wu, Ziheng Huang, Guowen Zhang, Zhijie Yuan,

Organizations: Huazhong University of Science and Technology · Tsinghua University · University of Toronto · Wuhan University of Technology ·

Abstract

LIBERO has emerged as a widely adopted benchmark for evaluating robotic manipulation policies. However, strong performance on LIBERO does not necessarily indicate robust embodied capabilities, as models may exploit benchmark-specific regularities rather than generalize to diverse operating conditions. In this work, we introduce LIBERO-Pro2.0, a comprehensive benchmark for systematically evaluating the robustness of embodied foundation models under distribution shifts. LIBERO-Pro2.0 encompasses 22 static perturbations and 20 runtime interventions across six dimensions: Condition, Environment, Observation, Execution, Robot State Configuration, and Language, yielding 1,648 validated shifted task instances. To support robustness-oriented learning, we further collect 39,390 training trajectories through model-generated and human demonstrations. All shifted task instances undergo six rounds of iterative human review and correction to ensure physical plausibility and task validity. We evaluate 17 representative robot policies spanning vision-language-action models, world-action models, and robustness-oriented policies. Our results reveal substantial robustness gaps across all evaluated paradigms: under runtime distribution shifts, the evaluated models exhibit an average performance drop of 26.5 percentage points, with individual drops reaching 32.5 points. These findings highlight the gap between nominal benchmark performance and robustness under distribution shifts, underscoring the need for systematic robustness evaluation of embodied foundation models.

Figures & tables

Figure 1: Overview of LIBERO-Pro2.0. Built upon LIBERO, our benchmark introduces 22 static distribution shifts and 20 runtime interventions across six dimensions: condition, environment, observation, robot, execution, and language. The left and right panels illustrate representative static and runtime perturbations, respectively.

Figure 2: Failure patterns under task variations. Examples for the task “ Pick up the alphabet soup and put it in the basket. ” The policy behaves similarly despite changes to the target object, instruction, or object position.

Figure 4: Perturbation distribution in LIBERO-Pro2.0.

Model	Cond. S01–S05, S07–S09	Env. S06, S10–S14, S22	Obs. S15–S16, S19	Exec. S20	Robot S17–S18	Lang. S21	Avg.	Base	$\Delta$
Vision-Language-Action Policies
OpenVLA ( Kim et al., 2025b )	25.2	37.6	34.4	0.0	54.0	67.5	33.9	64.6	$-30.7$
OpenVLA-OFT ( Kim et al., 2025a )	44.1	82.0	46.3	0.0	66.5	94.6	58.9	97.1	$-38.2$
OpenVLA-OFT w ( Kim et al., 2025a )	40.1	64.9	67.6	0.4	55.4	90.0	53.8	89.2	$-35.4$
OpenVLA-OFT m ( Kim et al., 2025a )	45.9	78.3	49.7	0.0	72.1	92.1	59.3	92.5	$-33.2$
OpenVLA-OFT+ ( Fei et al., 2026 )	47.2	78.5	45.0	0.0	73.8	90.0	59.2	92.1	$-32.9$

Table 1: Leaderboard of robot-policy robustness under static distribution shifts . Category scores and Avg. report mean success rates (%) over valid suite–perturbation entries across four LIBERO suites. Base denotes nominal success, and

\Delta=\mathrm{Avg.}-\mathrm{Base}

is measured in percentage points. All calculations precede rounding. OpenVLA-OFT variants follow LIBERO-Plus ( Fei et al., 2026 ) : w excludes wrist-camera input, m uses joint fine-tuning across the four original suites, and

+

uses mixed fine-tuning on LIBERO-Plus data.

Model	Cond. D01–D06	Env. D07–D09	Obs. D10–D13	Exec. D14–D16	Robot D17–D19	Lang. D20	Avg.	Base	$\Delta$
Vision-Language-Action Policies
OpenVLA ( Kim et al., 2025b )	20.6	41.9	41.3	53.6	29.9	63.8	36.4	64.6	$-28.2$
OpenVLA-OFT ( Kim et al., 2025a )	42.8	63.1	80.4	85.3	57.6	95.4	64.6	97.1	$-32.5$
OpenVLA-OFT w ( Kim et al., 2025a )	34.9	77.8	74.6	81.0	54.0	91.7	61.9	89.2	$-27.3$
OpenVLA-OFT m ( Kim et al., 2025a )	39.6	61.4	80.4	85.7	63.6	93.8	64.3	92.5	$-28.2$
OpenVLA-OFT+ ( Fei et al., 2026 )	41.3	56.9	79.8	84.2	64.0	93.8	63.8	92.1	$-28.3$

Table 2: Leaderboard of robot-policy robustness under runtime distribution shifts .

Figure 5: Comparison between matched static and runtime perturbations. Each pair contains a static shift occurring before execution and its runtime counterpart triggered during execution, including object pose changes (S01/D01, S02/D02), path constraints (S05/D05), occlusion (S12/D04), camera changes (S15/D07), robot-state variations (S17/D19), and language variations (S21/D20).

Model	Cond. D01–D06	Env. D07–D09	Obs. D10–D13	Exec. D14–D16	Robot D17–D19	Lang. D20	Avg.
OpenVLA-OFT	-26.6	-22.8	-15.0	-8.7	-27.3	-1.1	-19.8
$\pi_{0.5}$	-3.5	-19.2	-3.2	-5.3	+7.5	+2.1	-4.1
Anchor-Align	-15.1	-19.5	-12.9	-10.8	-8.9	-7.9	-13.4
FastWAM	-15.6	-9.2	-1.4	-7.5	-6.8	-3.3	-8.7
Cosmos Policy	-11.8	-10.3	-6.4	-13.2	+1.1	-5.4	-8.4

Table 3: Feedback utilization gain

G_{\mathrm{feedback}}

(pp) across dynamic perturbation categories.

Figure 6: Effect of policy execution horizon on robustness to dynamic perturbations. Panels (a)–(e) report task success rates under five representative perturbation types; panel (f) shows the average across these perturbations.

Model	Continue	$\bm{\Delta_{\mathrm{Hold}}}$	$\bm{\Delta_{\mathrm{Retry}}}$	$\bm{\Delta_{\mathrm{Reset}}}$
OpenVLA-OFT	64.6	-15.7	-40.8	-31.6
$\pi_{0.5}$	73.8	+18.9	-24.1	+0.7
Anchor-Align	61.6	+28.1	-16.8	+0.9
FastWAM	70.1	+24.3	-20.9	-0.4
Cosmos Policy	72.2	+26.0	-23.0	+2.5

Table 4: Average performance and intervention gains in RQ1-C.

\Delta

denotes the change in success rate relative to normal continuation.

Figure 7: Weakest-single gap $\Delta_{m}(\mathcal{C})$ under composed perturbations. The heatmap compares static–static, static–dynamic, dynamic–dynamic, and three-way compositions across policies.

Model	S09	S13	D13	S22	P02: S09+S13	P06: S09+D13	T03: S09+S22+D13
OpenVLA-OFT	69.6	94.2	69.6	93.3	12.5	12.5	12.5
$\pi_{0.5}$	70.0	92.5	83.8	94.2	12.5	12.1	12.9
Anchor-Align	72.5	86.7	79.6	87.9	10.0	11.3	11.7
FastWAM	80.3	91.2	83.7	88.3	9.2	11.7	10.0
Cosmos Policy	87.5	99.2	81.7	96.7	12.9	11.7	12.1

Table 5: Key evidence for non-compositional robustness. Although all models perform reasonably well on the constituent perturbations S09, S13, D13, and S22, they all collapse on the composed settings P02, P06, and T03.

Model	Object Pose	Receiver Pose	Path Obstacle	Camera Calibration	Robot State	Total
OpenVLA-OFT	20.8	24.4	14.8	26.9	52.9	28.0
$\pi_{0.5}$	61.9	71.0	19.8	8.1	78.7	47.9
Anchor-Align	27.9	39.4	12.1	6.0	65.4	30.2
FastWAM	28.8	16.0	19.3	8.3	83.1	31.1
Cosmos Policy	42.9	39.2	11.7	57.1	82.9	46.7
Ours	68.8	73.2	96.3	20.8	91.7	70.1

Table 6: Transfer of robustness from training set to tasks held out from additional fine-tuning. Each column reports the average success rate over a static–dynamic perturbation pair. Gain denotes the improvement of Ours over

\pi_{0.5}

Appendix figures & tables26 assets

Supplementary material from the paper’s appendix.

Appendix

Model	S1	S2	S3	S4	S5	S6	S7	S8	S9	S10	S11	S12	S13	S14	S15	S16	S17	S18	S19	S20	S21	S22
OpenVLA	1.7	8.3	0.0	0.0	0.0	48.3	35.0	33.3	43.3	46.7	41.7	0.0	31.7	16.7	1.7	0.0	15.0	51.7	46.7	0.0	63.3	43.3
OpenVLA-OFT	33.3	0.0	0.0	0.0	0.0	86.7	58.3	83.3	81.7	91.7	85.0	55.0	90.0	93.3	0.0	8.3	0.0	81.7	90.0	0.0	88.3	88.3
OpenVLA-OFT w	20.0	1.7	0.0	0.0	0.0	76.7	55.0	86.7	81.7	86.7	90.0	0.0	78.3	75.0	43.3	0.0	40.0	76.7	86.7	0.0	85.0	83.3
OpenVLA-OFT m	40.0	1.7	0.0	0.0	0.0	76.7	55.0	73.3	71.7	80.0	56.7	53.3	85.0	86.7	0.0	1.7	0.0	75.0	86.7	0.0	80.0	85.0
OpenVLA-OFT+	28.3	1.7	0.0	5.0	0.0	60.0	45.0	60.0	56.7	55.0	63.3	65.0	76.7	88.3	0.0	0.0	0.0	83.3	78.3	0.0	80.0	81.7
UniVLA	3.3	5.0	5.0	8.3	0.0	61.7	81.7	76.7	85.0	76.7	76.7	0.0	73.3	78.3	26.7	0.0	40.0	85.0	83.3	0.0	83.3	83.3

Appendix

Table 7: Static and runtime perturbation results on LIBERO-Goal. All values are reported in percentages (%).

Model	S1	S2	S3	S4	S5	S6	S7	S8	S9	S10	S11	S12	S13	S14	S15	S16	S17	S18	S19	S20	S21	S22
OpenVLA	0.0	3.3	6.7	3.3	0.0	0.0	45.0	80.0	81.7	78.3	11.7	0.0	76.7	5.0	10.0	0.0	36.7	88.3	88.3	0.0	81.7	75.0
OpenVLA-OFT	8.3	1.7	0.0	16.7	1.7	0.0	95.0	91.7	60.0	98.3	80.0	70.0	98.3	100.0	10.0	20.0	51.7	100.0	96.7	0.0	96.7	100.0
OpenVLA-OFT w	0.0	0.0	1.7	0.0	0.0	0.0	95.0	98.3	100.0	100.0	61.7	0.0	98.3	86.7	50.0	45.0	96.7	15.0	100.0	0.0	100.0	98.3
OpenVLA-OFT m	43.3	6.7	1.7	3.3	0.0	0.0	76.7	93.3	40.0	98.3	66.7	98.3	86.7	93.3	3.3	5.0	86.7	100.0	98.3	0.0	98.3	100.0
OpenVLA-OFT+	48.3	63.3	0.0	0.0	0.0	0.0	81.7	93.3	83.3	100.0	68.3	100.0	95.0	100.0	0.0	3.3	91.7	100.0	100.0	0.0	100.0	98.3
UniVLA	5.0	3.3	3.3	35.0	0.0	0.0	78.3	91.7	100.0	86.7	26.7	0.0	100.0	51.7	0.0	0.0	66.7	98.3	100.0	0.0	96.7	93.3

Appendix

Table 8: Static and runtime perturbation results on LIBERO-Object. All values are reported in percentages (%).

Model	S1	S2	S3	S4	S5	S6	S7	S8	S9	S10	S11	S12	S13	S14	S15	S16	S17	S18	S19	S20	S21	S22
OpenVLA	15.0	5.0	3.3	N/A	0.0	70.0	88.3	70.0	68.3	36.7	71.7	0.0	73.3	38.3	40.0	28.3	83.3	83.3	88.3	0.0	75.0	80.0
OpenVLA-OFT	73.3	76.7	30.0	N/A	0.0	80.0	100.0	93.3	98.3	88.3	100.0	98.3	93.3	98.3	51.7	81.7	96.7	86.7	100.0	0.0	98.3	100.0
OpenVLA-OFT w	40.0	15.0	5.0	N/A	1.7	75.0	98.3	91.7	98.3	48.3	96.7	6.7	98.3	98.3	93.3	90.0	100.0	10.0	98.3	0.0	98.3	100.0
OpenVLA-OFT m	73.3	96.7	28.3	N/A	0.0	75.0	100.0	95.0	95.0	70.0	95.0	53.3	90.0	98.3	41.7	65.0	100.0	93.3	100.0	0.0	98.3	100.0
OpenVLA-OFT+	66.7	98.3	35.0	N/A	0.0	81.7	96.7	95.0	95.0	86.7	88.3	66.7	98.3	98.3	38.3	56.7	98.3	100.0	98.3	0.0	98.3	98.3
UniVLA	18.3	55.0	10.0	N/A	0.0	75.0	98.3	98.3	98.3	68.3	98.3	1.7	51.7	95.0	73.3	80.0	98.3	98.3	98.3	0.0	98.3	95.0

Appendix

Table 9: Static and runtime perturbation results on LIBERO-Spatial. All values are reported in percentages (%).

Model	S1	S2	S3	S4	S5	S6	S7	S8	S9	S10	S11	S12	S13	S14	S15	S16	S17	S18	S19	S20	S21	S22
OpenVLA	18.3	5.0	26.7	0.0	13.3	1.7	55.0	53.3	18.3	36.7	43.3	0.0	46.7	38.3	28.3	31.7	23.3	50.0	50.0	0.0	50.0	41.7
OpenVLA-OFT	30.0	26.7	36.7	0.0	46.7	33.3	91.7	95.0	38.3	80.0	96.7	41.7	95.0	68.3	0.0	6.7	23.3	91.7	90.0	0.0	95.0	85.0
OpenVLA-OFT w	35.0	8.3	33.3	3.3	61.7	11.7	78.3	85.0	46.7	70.0	66.7	0.0	75.0	70.0	73.3	60.0	41.7	63.3	71.7	1.7	76.7	66.7
OpenVLA-OFT m	31.7	30.0	56.7	0.0	66.7	15.0	95.0	100.0	46.7	90.0	95.0	90.0	91.7	73.3	10.0	88.3	43.3	78.3	96.7	0.0	91.7	88.3
OpenVLA-OFT+	36.7	35.0	48.3	0.0	68.3	10.0	90.0	86.7	43.3	83.3	85.0	91.7	85.0	88.3	0.0	75.0	41.7	75.0	90.0	0.0	81.7	85.0
UniVLA	36.7	28.3	36.7	6.7	30.0	35.0	63.3	68.3	5.0	66.7	75.0	0.0	86.7	26.7	73.3	78.3	61.7	88.3	83.3	0.0	88.3	81.7

Appendix

Table 10: Static and runtime perturbation results on LIBERO-10. All values are reported in percentages (%).

(a): D01–D10
Metric	D01	D02	D03	D04	D05	D06	D07	D08	D09	D10
OpenVLA-OFT
$S_{0,\mathrm{live}}$	97.1	97.1	97.1	97.1	97.1	97.1	97.1	97.1	97.1	97.1
$S_{0,\mathrm{freeze}}$	45.6	43.8	33.8	50.0	42.5	55.0	48.1	48.1	48.1	48.1
$S_{1,\mathrm{live}}$	5.4	24.3	17.6	95.8	16.8	95.3	38.3	56.7	94.2	87.9
$S_{1,\mathrm{freeze}}$	0.0	3.8	0.0	41.9	13.8	43.6	21.3	41.3	48.1	47.5

Appendix

Table 11: Complete feedback-utilization results. Success rates are percentages;

0/1

denotes absence/presence of a perturbation, and live/freeze denotes the visual-observation condition. Live allows the policy to receive updated observations and infer actions normally, whereas Freeze holds the visual observation fixed at its pre-event value for a specified interval. For each model, the freeze duration is set to one-eighth of the number of inference steps required to complete unperturbed tasks, with freezing initiated immediately before the perturbation event. The feedback gain

G_{\mathrm{feedback}}=(S_{1,\mathrm{live}}-S_{1,\mathrm{freeze}})-(S_{0,\mathrm{live}}-S_{0,\mathrm{freeze}})

is measured in percentage points. The two panels cover D01–D10 and D11–D20, respectively.

(b): D11–D20
Metric	D11	D12	D13	D14	D15	D16	D17	D18	D19	D20
OpenVLA-OFT
$S_{0,\mathrm{live}}$	97.1	97.1	97.1	97.1	97.1	97.1	97.1	97.1	97.1	97.1
$S_{0,\mathrm{freeze}}$	48.1	48.1	48.1	48.1	41.3	48.1	48.1	41.9	32.5	48.1
$S_{1,\mathrm{live}}$	84.6	79.6	70.1	70.4	91.7	93.8	93.3	18.3	62.9	95.4
$S_{1,\mathrm{freeze}}$	45.0	47.5	46.3	43.8	43.1	41.3	42.5	1.3	43.8	47.5

Appendix

Table 11: Complete feedback-utilization results (continued). Panel (b): D11–D20.

Figure 8: Recovery-intervention gains, $\Delta_{I}=S_{I}-S_{\mathrm{continue}}$ (percentage points), for Hold, Retry, and Restore (labeled Reset in the figure). Positive values indicate improvement over Continue, while negative values indicate degradation.

(a) D01–D10
Model	D01	D02	D03	D04	D05	D06	D07	D08	D09	D10
Hold
OpenVLA-OFT	$48.1$	$48.8$	$46.3$	$47.5$	$49.4$	$55.7$	$48.8$	$48.8$	$48.8$	$48.8$
$\pi_{0.5}$	$93.8$	$92.5$	$92.5$	$90.6$	$91.3$	$97.9$	$93.1$	$93.1$	$93.1$	$93.1$
Anchor-Align	$89.4$	$90.0$	$89.4$	$91.9$	$88.8$	$92.1$	$89.4$	$89.4$	$89.4$	$89.4$
FastWAM	$92.5$	$93.1$	$90.0$	$92.5$	$92.5$	$96.4$	$95.6$	$95.6$	$95.6$	$95.6$

Appendix

Table 12: Success rates (%) under Hold, Retry, and Restore (Reset) interventions across all dynamic perturbations. Both panels retain the same intervention and model order.

Case	Composition	OpenVLA-OFT	$\pi_{0.5}$	Anchor-Align	FastWAM	Cosmos Policy
Static + Static
P01	S11 + S15	30.4	0.0	0.0	5.0	50.8
P02	S09 + S13	12.5	12.5	10.0	9.2	12.9
P03	S19 + S21	82.9	86.3	82.5	5.8	85.4
P04	S08 + S22	82.5	80.8	73.8	76.3	80.4
Static + Dynamic

Appendix

Table 13: Success rates (%) under pairwise and three-way perturbation compositions. Compositions are grouped by perturbation type.

Panel of Figure 9 — Figure 9: Static perturbations S01–S05. Each row contrasts BASE and the perturbed scene using the external and wrist views.

Panel of Figure 10 — Figure 10: Static perturbations S06–S10.

Panel of Figure 11 — Figure 11: Static perturbations S11–S14.

Panel of Figure 12 — Figure 12: Static perturbations S15–S18.

Panel of Figure 13 — Figure 13: Static perturbations S19–S22.

Panel of Figure 14 — Figure 14: Dynamic perturbations D01–D05. Each row shows the temporal sequence around an intervention, using the views indicated in the panels.

Panel of Figure 15 — Figure 15: Dynamic perturbations D06–D10.

Panel of Figure 16 — Figure 16: Dynamic perturbations D11–D15.

Panel of Figure 17 — Figure 17: Dynamic perturbations D16–D20.

Figure 18: Representative failure rollouts under perturbations; each row contains two five-frame sequences, read from left to right.

Category	Static perturbations	Runtime perturbations
Condition	S01–S05, S07–S09	D01–D06
Environment	S06, S10–S14, S22	D07–D09
Observation	S15–S16, S19	D10–D13
Execution	S20	D14–D16
Robot	S17–S18	D17–D19
Language	S21	D20

Appendix

Table 14: Mapping of static and runtime perturbations to evaluation categories.

ID	Perturbation	Intervention	Constraint
S01	Target planar pose	Translate the target in the horizontal plane and rotate it in yaw.	Preserve the target identity and intended goal.
S02	Receiver planar pose	Translate or rotate the receiver to change the placement destination and approach.	Preserve receiver identity and task goal.
S03	Receiver or support height	Change receiver or support elevation using a support platform or a coupled layout adjustment.	Retain a collision-free initial layout and feasible placement.
S04	Object position swap	Swap the planar positions of two objects; use a safe layout offset when a direct swap is unsuitable.	Keep object identities, task roles, and goal unchanged.
S05	Restricted approach or path	Add geometry that constrains approach or transport.	Retain valid inverse kinematics and a complete collision-free route.
S06	Instance replacement	Replace the target with a same-class or unseen asset specified by the task configuration.	Preserve semantic class and graspability.

Appendix

Table 15: Static perturbation definitions. Interventions are introduced at reset. Modified geometry or fault settings persist; initial-state changes do not prevent normal subsequent motion. Alternatives within a row are selected by the task configuration.

ID	Perturbation	Intervention	Trigger and lifetime
D01	Target pose update	Change target horizontal position or yaw, requiring updated localization and approach.	Approach/pre-grasp; retain the update as configured.
D02	Receiver pose update	Move or rotate the receiver without changing the task goal.	Post-grasp/pre-place; retain the update as configured.
D03	Receiver occupation	Introduce an occupying or blocking object near the receiving region.	Pre-place; occupation/release follows the case configuration.
D04	Temporary occlusion	Insert an occluder between camera and target, then remove it.	Task phase; enter, hold for a window, then exit.
D05	Path obstacle	Place an obstacle on the upcoming transport route while preserving an alternative collision-free path.	Transport; persistence follows the case configuration.
D06	Distractor rearrangement	Move irrelevant objects or rearrange local clutter; exclude the target and receiver.	Task phase; apply the configured scene update.

Appendix

Table 16: Dynamic perturbation definitions. Trigger phase and event lifetime are part of each case configuration. A restored sensor or control channel does not imply rollback of the complete scene. IDs match the paper visualizations.

Suite	Evaluation IDs	Training-pool IDs
Spatial	0, 8	1, 2, 3, 4, 5, 6, 7, 9
Object	1, 8	0, 2, 3, 4, 5, 6, 7, 9
Goal	3, 6	0, 1, 2, 4, 5, 7, 8, 9
Long	5, 8	0, 1, 2, 3, 4, 6, 7, 9

Appendix

Table 17: Task partition for the controlled eight-task studies. IDs are zero-based indices in the pinned release manifest. The training pool is the complement of the evaluation tasks within each suite; it does not specify which demonstrations were actually used in a training run.

Suite	ID	BASE task instruction
Spatial	0	Pick the akita black bowl between the plate and the ramekin and place it on the plate
Spatial	8	Pick the akita black bowl next to the plate and place it on the plate
Object	1	Pick the cream cheese and place it in the basket
Object	8	Pick the chocolate pudding and place it in the basket
Goal	3	Open the top layer of the drawer and put the bowl inside
Goal	6	Put the cream cheese on the bowl

Appendix

Table 18: The eight evaluation tasks. Instructions are taken directly from the BASE scene descriptions, with whitespace normalized.

Suite	ID	BASE task instruction
Spatial	1	Pick the akita black bowl next to the ramekin and place it on the plate
Spatial	2	Pick the akita black bowl from table center and place it on the plate
Spatial	3	Pick the akita black bowl on the cookies box and place it on the plate
Spatial	4	Pick the akita black bowl in the top layer of the wooden cabinet and place it on the plate
Spatial	5	Pick the akita black bowl on the ramekin and place it on the plate
Spatial	6	Pick the akita black bowl next to the cookies box and place it on the plate

Appendix

Table 19: The 32 training tasks. Instructions are taken directly from the BASE scene descriptions, with whitespace normalized.

Model	$\bm{H}$	$\bm{1/8H}$	$\bm{1/4H}$	$\bm{1/2H}$	$\bm{1H}$
OpenVLA-OFT	8	1	2	4	8
$\pi_{0.5}$	10	1	3	5	10
Anchor-Align	8	1	2	4	8
FastWAM	32	4	8	16	32
Cosmos Policy	16	2	4	8	16

Appendix

Table 20: Executed action lengths

K

for different policies and nominal execution horizon ratios in RQ1-B. The prediction horizon

H

remains fixed for each policy.

Distribution Shift Robustness

Distribution shift robustness studies how to keep machine-learning predictions reliable when deployment data differ from training data.

See all 274 papers

LIBERO-RIG: A Hardware-Setup Robustness Benchmark for Vision-Language-Action Models
Oct 13, 2026Andrea Yaoyun Cui, Hanyu Wang, Xinyi Wang +6Robustness of VLA Models Robot Manipulation Benchmarks
Luma AI
LIBERO-VPro: Benchmarking Closed-Loop Visual Robustness of Robotic Foundation Models
Sep 21, 2026Huiqiong Li, Zhiting Mei, Anirudha Majumdar +3World Models for Robotics Robot Foundation Models
Singapore Management University · Princeton University · Fudan University
LIBERO-RECOVER: Beyond Task Success Towards Failure Recovery in Robotic Manipulation Models
Sep 4, 2026Lin Liu, Zhicheng Bao, Lu Zhang +7Robot Failure Recovery Robotic Manipulation
Beta Infinity · Beijing Jiaotong University · School of Information and Communication Engineering Dalian University of Technology +1

Oct 13, 2026J/K move · Enter open · S save

LIBERO-RIG: A Hardware-Setup Robustness Benchmark for Vision-Language-Action Models

Andrea Yaoyun Cui, Hanyu Wang, Xinyi Wang, Bangzheng Li+5

Luma AI

Robustness of VLA Models Robot Manipulation Benchmarks

Open paper AskPDF arXiv

Robot policies are expected to generalize across deployment environments, including variations in the robot itself: cameras are bumped and remounted, calibrations drift, controllers run at different rates, and end-effector pads wear down. We present LIBERO-RIG, a benchmark that perturbs only the robot's hardware and setup while holding every LIBERO task fixed. The benchmark spans eleven perturbation axes, including camera mounting and optics, camera-to-arm calibration, actuation gain, control rate, proprioceptive noise, base pose, table height, and gripper pads. Each axis has five severity levels, yielding 210 conditions across the four LIBERO suites. We evaluate nine recent vision-language-action models using 500 episodes per condition and substantiate each finding with rigorous statistical tests. Pooled success falls from 0.932 to 0.767 at the middle severity and 0.509 at the highest severity. Performance degradation varies substantially across perturbation axes and action interfaces, and training on visual perturbations does not mitigate faults that leave camera images unchanged. We release the benchmark together with manifests specifying the fixed perturbation parameters for each episode.

Abstract

Distribution Shift Robustness

Explore similar work

LIBERO-RIG: A Hardware-Setup Robustness Benchmark for Vision-Language-Action Models

LIBERO-VPro: Benchmarking Closed-Loop Visual Robustness of Robotic Foundation Models

LIBERO-RECOVER: Beyond Task Success Towards Failure Recovery in Robotic Manipulation Models