ManiPhysicsBench: Physics-Based Assessment of Object Preservation in VLA Manipulation

Sangwu Park; Yeonjun In; Wonjoong Kim; Sungwon Kim; Sein Kim; Chanyoung Park

cs.ROOct 2, 2026

ManiPhysicsBench: Physics-Based Assessment of Object Preservation in VLA Manipulation

Authors: Sangwu Park, Yeonjun In, Wonjoong Kim, Sungwon Kim, Sein Kim, Chanyoung Park

Organizations: KAIST

Abstract

Vision-language-action (VLA) models aim to perform diverse manipulation tasks, but task success in existing rigid-body benchmarks does not indicate whether they preserve objects. We introduce ManiPhysicsZoo, which consolidates literature-supported material properties, 3D meshes, and supporting references into reusable object assets. Using these assets, a solver-based assessment computes grasp-specific damage thresholds from object geometry, material properties, and recorded grasp conditions and compares them with recorded contact forces to assess potential deformation and fracture. Building on these components, ManiPhysicsBench evaluates object preservation in LIBERO and SimplerEnv across three physics axes and three difficulty levels. Public VLA checkpoints show a substantial gap between task success and safe success, defined as task completion while preserving the object. Their gripper commands concentrate near full opening and closure, with largely similar aggregate distributions across objects, consistent with binary gripper supervision. We examine how object-specific continuous gripper labels change model behavior by retraining a VLA model. The retrained model shows more object-dependent gripping and higher safe success, but lower task success and limited generalization of object-preserving behavior.

Figures & tables

	Evaluation	Verdict		Assets
	Simulation	Preservation	Physics	Library
LIBERO, SimplerEnv	✓	✗	✗	✗
PhysBench	✗	✗	✗	✗
DefGraspSim	✗	$\triangle$	$\triangle$	$\triangle$
SafeVLA-Bench	✓	✗	✗	✗
OopsieVerse	✓	✓	✗	✗

Table 1: Comparison with representative related work. Simulation : policies are evaluated on manipulation tasks in simulation. Preservation : damage to the object is judged. Physics : the verdict rests on the objects’ own physical response, either computed from its material properties with a physics model or observed in real tests. Library : object assets carry material properties and their sources. ✓ full,

\triangle

partial, ✗ none.

Figure 1: Overview of the framework. Upper left: ManiPhysicsZoo turns each object into an asset that carries its geometry, mass, material properties, and the sources of those properties. Upper right: solver-based assessment builds a mechanical model from the asset and the grasp recorded during a rollout, computes a damage threshold at that grasp with the solver chosen for the object’s response, and compares it with the recorded contact force. Bottom: ManiPhysicsBench instantiates tasks on SimplerEnv and LIBERO, and reports whether each episode preserves the object.

	A: Brittle fracture	B: Soft deformation	C: Thin-walled deformation
High	Egg	Boiled potato	Paper cup
Medium	Cherry tomato	Strawberry	Portion cup
Low	Walnut	Plum	Yakult bottle

Table 2: Objects in ManiPhysicsBench by difficulty. Axes A and B share objects across platforms, while axis C uses different paper cups due to different gripper opening ranges.

	A (Brittle fracture)			B (Soft deformation)			C (Thin-walled deformation)
	High	Mid	Low †	High	Mid	Low †	High	Mid	Low
Model	Egg	Cherry tomato	Walnut	Boiled potato	Strawberry	Plum	Paper cup	Portion cup	Yakult bottle	Average
CogACT	91.7 / 0.0 / 0.0	91.7 / 83.3 / 90.9	33.3	50.0 / 0.0 / 0.0	58.3 / 16.7 / 28.6	91.7	33.3 / 0.0 / 0.0	16.7 / 0.0 / 0.0	91.7 / 83.3 / 90.9	56.9 / 16.7 / 29.3
GR00T N1.6	0.0 / 0.0 / –	75.0 / 58.3 / 77.8	25.0	41.7 / 0.0 / 0.0	0.0 / 0.0 / –	50.0	25.0 / 0.0 / 0.0	0.0 / 0.0 / –	41.7 / 41.7 / 100.0	23.6 / 9.7 / 41.2
GR00T N1.7	75.0 / 0.0 / 0.0	75.0 / 66.7 / 88.9	75.0	66.7 / 0.0 / 0.0	75.0 / 16.7 / 22.2	75.0	50.0 / 0.0 / 0.0	33.3 / 8.3 / 25.0	66.7 / 66.7 / 100.0	62.5 / 15.3 / 24.4
InternVLA-M1	16.7 / 0.0 / 0.0	100.0 / 100.0 / 100.0	91.7	91.7 / 0.0 / 0.0	91.7 / 16.7 / 18.2	100.0	25.0 / 8.3 / 33.3	41.7 / 8.3 / 20.0	66.7 / 66.7 / 100.0	61.1 / 22.2 / 36.4

Table 3: Results on SimplerEnv (WidowX), reported as SR / Safe SR / PR (%). A dash indicates undefined PR when SR is zero. For objects marked

\dagger

, Safe SR equals SR, so only SR is shown. Average covers the High and Mid objects across all three axes, since the Low objects are rarely damaged. Overall aggregates public checkpoints. Relabel denotes object-specific gripper supervision (Sec. 6.3 ).

	A (Brittle fracture)			B (Soft deformation)			C (Thin-walled deformation)
	High	Mid	Low †	High	Mid	Low †	High	Mid	Low
Model	Egg	Cherry tomato	Walnut	Boiled potato	Strawberry	Plum	Paper cup	Portion cup	Yakult bottle	Average
ABot-M0	50.0 / 0.0 / 0.0	16.7 / 0.0 / 0.0	0.0	41.7 / 0.0 / 0.0	8.3 / 0.0 / 0.0	58.3	58.3 / 0.0 / 0.0	0.0 / 0.0 / –	25.0 / 0.0 / 0.0	29.2 / 0.0 / 0.0
DreamVLA	0.0 / 0.0 / –	0.0 / 0.0 / –	0.0	33.3 / 0.0 / 0.0	0.0 / 0.0 / –	8.3	0.0 / 0.0 / –	0.0 / 0.0 / –	0.0 / 0.0 / –	5.6 / 0.0 / 0.0
GR00T N1.7	33.3 / 0.0 / 0.0	0.0 / 0.0 / –	0.0	0.0 / 0.0 / –	0.0 / 0.0 / –	0.0	0.0 / 0.0 / –	0.0 / 0.0 / –	0.0 / 0.0 / –	5.6 / 0.0 / 0.0
MolmoAct2	50.0 / 0.0 / 0.0	100.0 / 33.3 / 33.3	50.0	41.7 / 0.0 / 0.0	0.0 / 0.0 / –	83.3	16.7 / 0.0 / 0.0	16.7 / 8.3 / 50.0	41.7 / 33.3 / 80.0	37.5 / 6.9 / 18.5

Table 4: Results on LIBERO (Panda). Same format as Table 3 .

Figure 2: Distribution of executed gripper closure for axis A on both platforms, including all rollout steps. Histograms are averaged across episodes and models. Zero and one denote fully open and fully closed targets.

Figure 3: QwenGR00T closure during object contact on SimplerEnv: public and object-specific models. Width indicates frequency; white points are medians and thick/thin lines span the 25–75/10–90 percentiles.

Model	SR (%)	Mean closure
QwenGR00T	50.0	0.96
+ Relabel	58.3	0.94

Table 5: Low-friction ice cube task (

\mu=0.12

Appendix figures & tables19 assets

Supplementary material from the paper’s appendix.

Appendix

Household (50)
Almond	Apple	Avocado
Baguette	Banana	Bell
Blueberry	Boiled potato	Broccoli
Carrot	Cashew nut	Cauliflower
Cherry tomato	Chocolate	Cooking pot
Corn	Croissant	Cucumber

Appendix

Table 6: Complete object list of ManiPhysicsZoo , grouped by domain. Objects used in ManiPhysicsBench are shown in bold; the two platform-specific paper cups are listed separately.

Figure 4: The interactive ManiPhysicsZoo catalog.

Figure 5: An object card, illustrated with the egg. The visible portion contains the name, mesh and geometry, material properties with provenance, and the supporting-reference record. The card also links to the mesh and an interactive 3D view.

Object	Mesh / geometry	Material properties	Supporting references / evidence path
Egg (A)	Shell; $44.6\!\times\!44.6\!\times\!56.4$ ; $t=0.36$	Linear elastic; $E=30$ GPa; $\sigma_{t}=19.9$ MPa	Liu et al. (2024) : shell properties and tensile strength $\to$ half-eggshell tests at three loading positions $\to$ breaking-load comparison (L1, predicted load).
Cherry tomato (A)	Skin + core; $30.0\!\times\!30.0\!\times\!40.0$ ; $t=0.20$	Skin/core $E=18.1/0.66$ MPa; $\sigma_{t}=2.06/0.074$ MPa	Li et al. (2021) tissue properties $\to$ intact-fruit compression ( Zheng et al., 2023 ) : reaction difference $+1.7$ – $3.9\%$ (L1, imposed displacement).
Walnut (A)	Shell; $24.8\!\times\!33.0\!\times\!23.5$ ; $t=1.00$	Linear elastic; $E=3.88$ GPa; $\sigma_{t}=19.70$ MPa	Reported rupture loads ( GHARIBZAHEDI et al., 2011 ) vs. nominal grip force $\to$ high-strength control; legacy shell parameters retain L3 provenance.
Boiled potato (B)	Effective solid; $40.0\!\times\!31.6\!\times\!39.0$	Perfect plasticity; $E_{\rm eff}=493.83$ kPa; $\sigma_{y}=56.75$ kPa	Sadowska et al. (2005) : modulus + compression point $\to$ effective-law fit $\to$ other-year stress comparison at reported strains (L1, conditional).
Strawberry (B)	Effective cortex solid; $27.0\!\times\!20.6\!\times\!39.6$	Elastoplastic; $E_{\rm eff}=306.62$ kPa; $\sigma_{y}=52.81$ kPa; hardening branch	An et al. (2022) : cortex compression $\to$ effective-law fit $\to$ bioyield-point reproduction (L1, calibration response).
Plum (B)	Skin + core; $45.0\!\times\!45.0\!\times\!42.3$ ; $t=0.10$	Skin $E=19.02$ MPa, rupture resultant $249.68$ N/m; core $E=1.93$ MPa, $\sigma_{y}=0.28$ MPa	Skin: bulge-test fit $\to$ other-cultivar comparisons ( Brüggenwirth et al., 2014 ) (L1). Core: loquat constitutive transfer ( Liu et al., 2025 ) (L2).

Appendix

Table 7: Benchmark asset records following the Zoo schema. Geometry lists rounded reference Zoo mesh bounds (

x\!\times\!y\!\times\!z

) and modeled wall/layer thickness

t

, in mm; these are distinct from the literature test specimens.

E

\sigma_{y}

, and

\sigma_{t}

denote elastic modulus, constitutive yield stress, and tensile strength. L1–L3 indicate the scoped evidence levels defined in App. A.3 .

Position	Outer crack	Source FE	Measured	Difference	Inner crack
Sharp end	57.6	$\approx$ 56	$\approx$ 50	$+15\%$	1.7–6.3
Blunt end	55.8	$\approx$ 55	$\approx$ 46	$+21\%$	1.6–5.3
Equator	37.8	$\approx$ 38	$\approx$ 35	$+8\%$	1.5–4.4

Appendix

Table 8: Half-eggshell compression at three loading positions. Shell properties and tensile strength are the benchmark values, and none of these loads was used to set them. Outer crack: load at which the outer-surface tensile stress away from the load point reaches

\sigma_{t}

, the crack that Liu et al. (2024) associate with breakage (patch radius 0.5 mm). Inner crack: load at which the inner surface beneath the load reaches

\sigma_{t}

, over patch radii of 0.2–1.0 mm. Source values are approximate, read from the supplementary Fig. S5 of Liu et al. (2024) . Forces in newtons.

Model	Checkpoint
CogACT	CogACT-Base.pt
GR00T N1.6	GR00T-N1.6-bridge
GR00T N1.7	SimplerEnv: GR00T-N1.7-bridge ; LIBERO: GR00T-N1.7-LIBERO/libero_object
InternVLA-M1	InternVLA-M1-Pretrain-RT-1-Bridge
LangForce	LangForce_SimplerEnv_Better
Octo	SimplerEnv: Octo-Base ; LIBERO: 2025-06-19_octo_base_1p5_libero_finetune

Appendix

Table 9: Checkpoint identifiers and weight files used for the public baselines. Environment names within a cell distinguish checkpoints of the same model.

Setting	SimplerEnv / WidowX	LIBERO / Panda
Position gain $k_{p}$	$1000$ N/m	$1000$ N/m
Actuator force range	$[-60,60]$ N	$[-20,20]$ N
Damping	$200$ N s/m (drive)	$100$ N s/m (joint)
Jaw target range	$[14,38]$ mm	$[0,40]$ mm and $[-40,0]$ mm

Appendix

Table 10: Host gripper settings. Force limits apply to each jaw actuator; damping is implemented in the drive for WidowX and at the joint for Panda.

Gain	SimplerEnv	LIBERO
$0.5k_{p}$	48.3 / 24.9 / 51.4	12.3 / 3.4 / 27.4
$k_{p}$ (default)	48.3 / 13.5 / 27.9	12.3 / 1.9 / 15.1
$2k_{p}$	48.3 / 3.6 / 7.5	12.3 / 0.9 / 7.5

Appendix

Table 11: Sensitivity to the position gain. Recorded contact forces are scaled by the gain factor and capped at the actuator limit, with trajectories and FEM damage loads fixed. Each entry is SR / Safe SR / PR (%) over the High and Mid objects, as in the Average column of Tables 3 and 4 . At

0.5k_{p}

, Safe SR is an upper bound because saturated forces are underestimated.

Figure 6: Margins of the damage verdicts. Each point is one successful episode, placed at the largest ratio of recorded contact force to damage threshold over its grasps; ratios above one indicate damage. The shaded band marks ratios within 25% of the threshold, and the dashed line marks five times the threshold. Numbers in parentheses give the episode counts.

Figure 7: Safe closure commands under three position gains. For each object, bars show the median interval of closure commands that keeps the contact force between the lifting requirement and the damage threshold, estimated per grasp from the recorded command-to-force relation. Arrows mark intervals that extend to full closure. The gray band and the dark line show the 10th to 90th percentiles and the median of the policies’ commands during contact. The dotted line marks the median contact onset.

Setting	CalculiX 2.23	FEBio 4.13	OpenRadioss
Integration	Implicit, large deformation	Implicit dynamic ramp	Explicit dynamic
Object elements	S6 shells; C3D10 solids	Quadratic tetrahedra	DKT18 shells
Shell thickness points	3 (3 in-plane points)	Not used for the listed soft-solid cases	5
Pad elements	Quadratic tetrahedra	Native deformable pad	TETRA10, LAW42
Contact	Surface-to-surface penalty	Auto-scaled penalty contact	TYPE24 finite sliding
Contact parameter	$2\times 10^{11}$ Pa/m; friction 0.5	Penalty scale 0.03; gap tolerance 5/10 $\mu$ m	Native contact stiffness; friction inherited from input

Appendix

Table 12: Numerical configurations of the implemented solver routes. FEBio values separated by a slash refer to strawberry/potato. Load durations and numerical increments are solver settings, not damage tolerances.

Figure 8: Object-extraction and Damage-risk annotation instructions used for Bridge demonstrations.

Stage	Operation
Standardization	Lowercase, normalize whitespace, and replace underscores/hyphens with spaces.
Alias mapping	Merge registered variants, e.g., brinjal $\rightarrow$ eggplant and maize/corncob $\rightarrow$ corn.
Modifier cleanup	Remove supported leading modifiers and explicit spatial/state suffixes before alias matching.
Extraction cleanup	Remove pronouns, relation fragments, and malformed labels; preserve unresolved object-like names separately.
Aggregation	Combine identical canonical categories using the saved mapping.
Episode filtering	Require a policy-covered object to be the directly manipulated target of a grasp interaction.

Appendix

Table 13: Rule-based consolidation and target-object filtering. The saved mapping records each name’s disposition and its reason.

Stage	Count	Unit
Full Bridge dataset	53,192	Episodes
Nonempty instructions	38,660	Episodes
Distinct nonempty instructions	19,973	Instruction groups
Raw canonical names	679	Strings
Consolidated concepts	170	Categories, including unresolved
Force-policy categories	74	Object categories

Appendix

Table 14: Data selection for object-specific relabeling.

Setting	Value
Initialization	Qwen3-VL-4B-Instruct + new action model
Training data	Bridge; 51,019 eligible episodes
Trainable modules	Language LoRA + full action model
LoRA rank / scale / dropout	$32/64/0.05$
LoRA targets	q/k/v/o and gate/up/down projections
Trainable parameters	227.533 M

Appendix

Table 15: QwenGR00T object-specific supervision configuration. Sample exposure counts sampled training instances, including repetitions, rather than distinct demonstrations.

Figure 9: Gripper command distributions for all evaluated public checkpoints on SimplerEnv. Histograms and near-binary markers use all rollout steps; arrows and approximate-equality markers use mean closure during bilateral object contact. Each histogram has a common vertical range of 0–1 probability mass per bin. Marker definitions and aggregation are given in App. F.1 and App. F.2 .

Figure 10: Gripper command distributions for all evaluated public checkpoints on LIBERO, with the same histogram scales and marker definitions as Fig. 9 . An absent ordering marker can indicate either that the display conditions are not met or that an object has no recorded bilateral contact; missing contact is not counted as an ordering failure.

Axis	Object	$F_{\mathrm{lift}}$	Median $R_{\mathrm{dmg}}$	$n$	$D$	Level
A	Egg	0.425	5.05	285	0.357	High
A	Cherry tomato	0.568	11.01	236	0.185	Medium
A	Walnut	0.098	63.50	45	0.000	Low
B	Boiled potato	0.627	8.15	212	0.242	High
B	Strawberry	0.152	6.40	72	0.147	Medium
B	Plum	0.519	32.34	11	0.001	Low

Appendix

Table 16: Reference lifting force, median pad–object damage reaction, and difficulty over

n

bracketed grasps. Forces are per jaw, in newtons. S and L denote the SimplerEnv and LIBERO paper-cup variants. Other rows pool available grasps from both platforms.

Figure 11: Initial rollout view of the ice cube object.

Robotic Manipulation

Robotic manipulation studies how robots perceive, plan, and control physical interactions to grasp, move, assemble, and use objects.

See all 1,685 papers

Is Success All You Need? Investigating the Impact of Input Perturbations on VLA Behaviour in Tabletop Manipulation Tasks
Oct 1, 2026Sophie Higham, Riccardo Andrea Izzo, Matteo Matteucci +1Vision-Language-Action Models Robot Manipulation Benchmarks
School of Informatics, University of Edinburgh, UK · Department of Electronics, Informatics and Bioengineering, Politecnico di Milano, Italy
Position: Vision-Language-Action Models Cannot Be Verified to Perform Physical Reasoning
Jun 28, 2026Taozhao Chen, Ian Manchester, Huaming ChenVLM Evaluation Physical Reasoning
School of Electrical and Computer Engineering The University of Sydney Sydney, NSW, Australia · School of Aerospace, Mechanical and Mechatronic Engineering The University of Sydney Sydney, NSW, Australia
PhysVLA: Towards Physically-Grounded VLA for Embodied Robotic Manipulation
Jun 11, 2026Namai Chandra, Shriram Damodaran, Lin WangRobotic Control Robotic Manipulation
Electronic Systems IIT Madras, India · EmPACT Lab Nanyang Technological University Singapore

Oct 1, 2026J/K move · Enter open · S save

Is Success All You Need? Investigating the Impact of Input Perturbations on VLA Behaviour in Tabletop Manipulation Tasks

Sophie Higham, Riccardo Andrea Izzo, Matteo Matteucci, Alessandro Suglia

School of Informatics, University of Edinburgh, UK · Department of Electronics, Informatics and Bioengineering, Politecnico di Milano, Italy

Vision-Language-Action Models Robot Manipulation Benchmarks Robustness of VLA Models

Open paper AskPDF arXiv

Vision-Language-Action (VLA) models have achieved high task success rates on robot manipulation task benchmarks. More recently, there has been an emphasis on evaluating the robustness of VLA models to perturbations. However, this robustness is still predominantly measured through Task Success Rate (TSR). In this work, we propose a benchmark-agnostic evaluation framework to measure the behavioural robustness of models by characterising how successful trajectories are executed under perturbation. We implement this methodology by extending the widely-used LIBERO and LIBERO-Plus benchmarks. Across three state-of-the-art VLA models, four LIBERO task suites and seven perturbation conditions, we evaluate changes in both typical successful behaviour and its variability, including metrics of motion smoothness, efficiency and gripper behaviour. We find that perturbations can alter the behaviour of successful trajectories, a phenomenon which cannot necessarily be inferred from TSR alone. Across LIBERO suites, we identify cases where state-of-the-art VLA models achieve comparable TSR under the same perturbation condition, yet behaviour on successful trajectories diverges substantially. Therefore, to have a more robust assessment of task performance, we argue that suitable measures of robustness should capture not only whether a task is completed, but also how the robot behaves while completing it. When evaluating the robustness of VLA models, TSR may be complemented by behavioural evaluation metrics that characterise the nature and variability of successful task execution by robots.

Abstract

Robotic Manipulation

Explore similar work

Is Success All You Need? Investigating the Impact of Input Perturbations on VLA Behaviour in Tabletop Manipulation Tasks

Position: Vision-Language-Action Models Cannot Be Verified to Perform Physical Reasoning

PhysVLA: Towards Physically-Grounded VLA for Embodied Robotic Manipulation