PhyDEX: A Benchmark for Physical Reasoning in Vision-Language-Action Models via Dynamic Extrinsic Dexterity
Authors: Hao Vo, Taisei Hanyu, Yuki Ikebe, Pham Tri Quang, Sreevenkata Anjani Tishita Godavarthi, Nghi D. Q. Bui, Daniel Sonntag, Chase Rainwater, Duy Minh Ho Nguyen, Anh Nguyen, Ngan Le
Organizations: AICV Lab, University of Arkansas, USA · VinUniversity, Vietnam · Center for AI Research, VinUniversity, Vietnam · University of Stuttgart, Germany · DFKI, Germany · Max Planck Research School for Intelligent Systems, Germany · University of Liverpool, UK
Vision-Language-Action (VLA) models have advanced rapidly in language-conditioned robotic manipulation, yet existing benchmarks mainly evaluate gripper--object contact and largely ignore how objects interact with the environment through gravity, friction, inertia, and mass distribution. As a result, it remains unclear whether current policies truly understand physics or merely interpolate from their training data. We address this gap through the lens of dynamic extrinsic dexterity and introduce PhyDEX, a throwing benchmark in which a robot must infer an object's mass, center of mass, and friction to send it to a destination beyond its kinematic workspace. PhyDEX contains 35K trajectories (143 hours) across nine tasks and 100 scenes, covering uniform, irregular, and dynamic-mass objects with static, multiple, and moving destinations. All data are generated by a scalable, fully automatic pipeline that combines grasp generation, motion planning, and closed-loop throw optimization under randomized physical properties. We evaluate three representative VLAs, GR00T N1.7, DynamicVLA, and ForceVLA, and find that all achieve success rates below 3%. Our analysis reveals two key bottlenecks: limited robustness of pretrained models, which causes most failures before the throw, and physical blindness, as providing ground-truth physical properties roughly doubles success. Experiments on a humanoid robot and in the real world show the same failure patterns. We hope PhyDEX will drive the development of physically grounded robot policies.
Figures & tables
Figure 2: Task taxonomy and statistics of Phy DEX . Left: distribution of episodes across three object types (inner ring) and three destination types (outer ring). Right: example scenes for each of the nine object–destination combinations. boxes mark the target object and boxes mark the destination.
Figure 3: Overview of the Phy DEX data generation pipeline. (a) Environment construction: given a task template, we retrieve a target object, a destination, and a scene from the asset library to create a new task. (b) Domain randomization: we diversify each task visually (camera views, lighting, scene layout), in its trajectories (grasp pose, object pose, robot state), and in its physical properties (mass, CoM, friction). (c) Trajectory generation: GraspGen-X proposes grasp poses, cuRobo plans a collision-free approach, and an SLSQP-based optimizer computes the throwing trajectory q(t) from the initial joint state (q0,q˙0) , the object properties (m,CoM,μ) , and the target p∗ , refining it in a closed loop until the throw succeeds. (d) Data storage: each successful episode is saved in LeRobot format with multi-view RGB, force/torque signals, and actions.
Destination
GR00T N1.7 [ 2 ]
DynamicVLA [ 54 ]
ForceVLA [ 57 ]
Uniform
Single
6%
0%
10%
Multiple
2%
0%
10%
Dynamic
0%
0%
0%
Irregular
Single
0%
0%
0%
Multiple
0%
2%
0%
Dynamic
0%
0%
0%
Table 1: Performance comparison of three baseline VLA models on Phy DEX under full diversity ( α=1 ).
Figure 4: Effect of ground-truth physical properties. Success rates with and without access to the object’s ground-truth physical properties.
α
Object
GR00T N1.7 [ 2 ]
DynamicVLA [ 54 ]
ForceVLA [ 57 ]
1
Uniform
6%
0%
10%
Irregular
0%
0%
0%
Dynamic
0%
0%
0%
Average
2%
0%
3.3%
0.5
Uniform
6%
2%
16%
Irregular
6%
4%
10%
Table 2: Effect of benchmark diversity. Success rates on static single-destination tasks under different diversity scores α .
Object
GR00T N1.7 [ 2 ]
DynamicVLA [ 54 ]
ForceVLA [ 57 ]
Uniform
10%
2%
14%
Irregular
8%
0%
6%
Dynamic
0%
0%
6%
Average
6%
0.7%
8.7%
Table 3: Cross-embodiment evaluation on a humanoid robot. Top: humanoid scenes. Bottom: success rates by object type.
Figure 5: Real-world experimental setup. A UR10e arm with a Robotiq 2F-85 gripper and a wrist-mounted F/T sensor, observed by a base camera (RealSense D455) and a wrist camera (RealSense D405).
Figure 6: Real-world test objects. Three 3D-printed cubes with the same appearance but different physical properties. Left: a standard cube with uniform mass. Middle: a cube with a metal insert near one corner, which shifts its center of mass. Right: a hollow, lightweight cube, used with low-friction tape on the gripper fingers to reduce friction.
Vision-Language-Action (VLA) models excel at mapping visual inputs and natural language instructions directly to robotic control policies. However, because they are trained primarily to fit behavioural demonstration data, they do not explicitly enforce fundamental physical principles such as rigid-body dynamics or contact constraints. This exposes a critical physics gap: standard temporal smoothing applied on top of single-step or chunked VLAs trades trajectory quality for added failures that short-term memory cannot resolve. To bridge this gap, we introduce PhysVLA (Physics-VLA), a plug-and-play, inference-time framework designed to wrap any frozen VLA backbone without retraining, fine-tuning, or weight access, with less than 1 ms of overhead per control step. PhysVLA intercepts the predicted control action, captures only the simulator or system state, and applies a dual-layered correction: (i) a phase-aware finite-state machine that structures discrete task segments (approach, grasp, transport, and place), and (ii) a selective Euler-Lagrange gate that activates only when a dynamics oracle detects kinodynamic inconsistency. Evaluated across OpenVLA, OpenVLA-OFT, Force-VLA, and Generalist-VLA on LIBERO-Spatial with a 7-DoF Franka Panda, the framework delivers absolute success rate increases of up to 17% and stability increases of up to 19% with no per-task regressions, improves trajectory efficiency by up to 15% across all four backbones, and shows up to a 10x improvement in trajectory jerk robustness on a Robosuite Lift cross-simulator sweep. We further validate the framework on a real Agilex Piper arm with a pick-and-place task, confirming that PhysVLA transfers to physical hardware without retraining, with success-rate improvements of up to 50%, establishing physical awareness as a composable, backbone-agnostic runtime module.
Namai Chandra, Shriram Damodaran, Lin Wang
Electronic Systems IIT Madras, India · EmPACT Lab Nanyang Technological University Singapore
Vision-Language-Action (VLA) systems, built on pretrained vision-language models (VLMs), have shown rapidly improving performance on robot manipulation benchmarks. These gains are commonly interpreted as evidence that semantic representations learned from internet-scale data transfer to physical execution generalization. This position paper argues that the assumption underlying this interpretation -- that semantic generalization is sufficient to support physical action decisions -- has not been independently verified and cannot be tested under current evaluation protocols. We support this claim by decomposing VLA policies into semantic mapping and physical action decision, and showing that task success rate -- the dominant evaluation metric -- cannot distinguish between these two sources of capability. As a result, improvements in benchmark performance are consistent with multiple competing explanations, including semantic matching, distributional overlap, and genuine physical generalization. We further argue that this identifiability gap has been reinforced through narrative drift, whereby successive systems inherit and strengthen prior interpretations of performance gains without isolating the underlying causal mechanism. To address this limitation, we propose a research direction based on evaluation designs that introduce controlled variation to separately measure semantic and physical generalization. Such designs make it possible to causally attribute performance without requiring access to model internals, and to empirically assess the role of VLM backbones as semantic interfaces rather than implicit sources of physical competence. Our goal is not to refute the role of VLMs in robotics, but to clarify the conditions under which claims of physical generalization can be meaningfully evaluated.
Taozhao Chen, Ian Manchester, Huaming Chen
School of Electrical and Computer Engineering The University of Sydney Sydney, NSW, Australia · School of Aerospace, Mechanical and Mechatronic Engineering The University of Sydney Sydney, NSW, Australia
Vision-Language-Action (VLA) models have emerged as a generalist robotic agent. However, existing VLAs are hindered by excessive parameter scales, prohibitive pre-training requirements, and limited applicability to diverse embodiments. To improve the practicality of VLAs, we propose a comprehensive benchmark and an improved baseline. First, we propose CEBench, a new benchmark spanning diverse embodiments in both simulation and the real world with consideration of domain randomization. We collect 14.4k simulated trajectories and 1.6k real-world expert-curated trajectories to support training on CEBench. Second, using CEBench as our testbed, we study three critical aspects of VLAs' practicality and offer several key findings. Informed by these findings, we introduce LLaVA-VLA, a lightweight yet powerful VLA designed for practical deployment on consumer-grade GPUs. Architecturally, it integrates a compact VLM backbone with multi-view perception, proprioceptive tokenization, and action chunking. To eliminate reliance on costly pre-training, LLaVA-VLA adopts a two-stage training paradigm including post-training and fine-tuning. Furthermore, LLaVA-VLA extends the action space to unify navigation and manipulation. Experiments across embodiments demonstrate the capabilities of generalization and versatility of LLaVA-VLA , while real-world mobile manipulation experiments establish it as the first end-to-end VLA model for mobile manipulation. We will open-source all datasets, codes, and checkpoints upon acceptance to foster reproducibility and future research.
Wenxuan Song, Jiayi Chen, Xiaoquan Sun +12
The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China. · Huazhong University of Science and Technology, Wuhan, China. · Westlake University, Hangzhou, China. +1