ExPhy: A Benchmark for Explicit Physical Property Learning in Multi-Object Trajectory Forecasting

Rui Wang; Yeteng Wu; Xianlin Zhang; Mengshi Qi

cs.AIAug 20, 2026

ExPhy: A Benchmark for Explicit Physical Property Learning in Multi-Object Trajectory Forecasting

Authors: Rui Wang, Yeteng Wu, Xianlin Zhang, Mengshi Qi

Organizations: State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications Beijing, China · School of Digital Media & Design Art, Beijing University of Posts and Telecommunications Beijing, China

Abstract

Understanding object dynamics requires not only predicting future trajectories but also examining whether a model captures the physical properties that govern motion. However, existing benchmarks rarely expose object-level physical properties as explicit evaluation targets alongside trajectory forecasting. To address this gap, we introduce ExPhy, a multi-object trajectory forecasting benchmark containing 24,000 simulated physical scenes with explicit object-level labels for mass, friction, and restitution. ExPhy provides observed and future trajectories together with an in-distribution (ID) split and two out-of-distribution (OOD) splits over physical parameters (OOD-Parameter) and initial states (OOD-Initial) for jointly evaluating trajectory forecasting and physical property estimation. We further instantiate PhyODE, a physics-guided model with an explicit property interface that estimates physical properties from observed trajectories and uses them for differentiable future rollout. On the long-horizon OOD-Initial setting, PhyODE reduces ADE and FDE by 33.1% and 31.0%, respectively, compared with the strongest baseline. Zero-shot evaluation on ComPhy further assesses cross-benchmark transfer. Property-level analyses reveal that accurate trajectory forecasting does not necessarily imply accurate recovery of the underlying physical properties. Code and data are available at https://github.com/Zest86/ExPhy.

Figures & tables

Figure 1: Comparison between existing benchmarks and the proposed ExPhy benchmark. (a) Existing benchmarks primarily supervise future outcomes, while object-level physical properties are often represented implicitly or indirectly and are unavailable as dedicated evaluation targets. (b) ExPhy provides explicit continuous-valued labels for mass, friction, and restitution, enabling joint evaluation of trajectory forecasting and physical property estimation.

Dataset	Phys. Prop. Eval.		Traj. Eval.	OOD Eval.
Dataset	Obj-level	Cont.	Traj. Eval.	OOD Eval.
CLEVRER [ICLR20]	$\times$	$\times$	$\times$	$\times$
Super-CLEVR [CVPR23]	$\times$	$\times$	$\times$	$\checkmark$
Physion++ [NeurIPS23]	$\circ$	$\times$	$\times$	$\times$
ComPhy [TPAMI25]	$\checkmark$	$\times$	$\times$	$\times$
PhysBench [ICLR25]	$\checkmark$	$\times$	$\times$	$\times$

Table 1: Comparison of representative physical reasoning benchmarks.

\circ

denotes property-dependent evaluation without direct property targets. Obj. denotes direct object-level property evaluation; Cont. denotes continuous parameter regression; Traj. denotes trajectory-level forecasting; and OOD denotes a controlled distribution-shift protocol.

Figure 2: Overview of the ExPhy benchmark. (a) Benchmark construction and annotation pipeline. (b) ID, OOD-Parameter, and OOD-Initial evaluation protocols. Blue and red denote ID and OOD variables, respectively; the blue dashed box and red-shaded outer region mark their location ranges, while arrows depict initial velocities.

Variable	ExPhy-A	ExPhy-B	ExPhy-C
Physical properties
Mass $m$	$[0.1,10]$	$[10.01,15]$	Same
Friction $\mu$	$[0.35,0.60]$	$[0.25,0.34]$	Same
Friction $\mu$	$[0.70,0.95]$	$[0.96,1.00]$	Same
Restitution $e$	$[0.15,0.40]$	$[0.05,0.14]$	Same
Restitution $e$	$[0.55,0.85]$	$[0.86,0.95]$	Same

Table 2: Sampling ranges and supports for the ExPhy splits. For friction and restitution, one of the two listed intervals is selected uniformly at random before sampling within it. ExPhy-B shifts the physical-property distributions, whereas ExPhy-C shifts the initial-state distributions. “Same” denotes the corresponding ExPhy-A setting.

Figure 3: Overview of PhyODE . The physical property estimator $\mathcal{F}_{\psi}$ combines raw motion, invariant trajectory, and pairwise relation features to estimate object-level mass, friction, and restitution. Conditioned on these properties, the differentiable trajectory solver $\mathcal{G}_{\phi}$ combines frictional dissipation and discrete collision updates with a residual Neural ODE, and uses RK4 integration to forecast future trajectories.

Methods	ExPhy-A (In-Distribution)			ExPhy-B (OOD-Parameter)			ExPhy-C (OOD-Initial)
Methods	Short	Mid	Long	Short	Mid	Long	Short	Mid	Long
Physical reasoning baselines
VRDP † [NeurIPS21]	0.04 / 0.08	0.28/0.58	0.41/0.83	0.04 / 0.08	0.29/0.61	0.45/0.92	0.12 /0.23	0.93/1.93	2.13/4.23
PHYCINE † [CVPR23]	0.04 / 0.08	0.34/0.66	0.46/0.90	0.04 / 0.08	0.36/0.70	0.51/1.01	0.12 / 0.21	1.12/2.15	1.95/3.86
PCR † [TPAMI25]	0.05/0.10	0.28/0.57	0.48/0.94	0.05/0.10	0.28 /0.58	0.52/1.04	0.14/0.28	0.67 / 1.47	1.58/3.27
Geometric dynamics baselines

Table 3: Quantitative comparison of trajectory forecasting error (ADE/FDE

\downarrow

) on ExPhy-A, ExPhy-B and ExPhy-C. Lower is better. The prediction horizons are explicitly defined based on observation-prediction steps (

T_{\text{obs}}

T_{\text{pred}}

): Short (10-10), Mid (20-40), and Long (30-60).

\dagger

indicates trajectory-only adaptations of physical reasoning baselines, where visual/perceptual frontends are replaced with trajectory encoders while preserving their original reasoning mechanisms. The baselines are grouped according to their primary inductive biases. Bold and underlined indicate the best and second-best results, respectively.

Methods	ComPhy [TPAMI25]
Methods	Short	Mid	Long
Physical reasoning baselines
VRDP † [NeurIPS21]	0.13 /0.23	0.51 / 0.92	0.79/1.32
PHYCINE † [CVPR23]	0.13 /0.24	0.56/0.99	0.84/1.41
PCR † [TPAMI25]	0.19/0.34	0.77/1.41	1.15/2.00
Geometric dynamics baselines

Table 4: Zero-shot transfer results on ComPhy. All models are trained on ExPhy-A and directly evaluated on ComPhy without fine-tuning. We report ADE/FDE (

\downarrow

\dagger

indicates trajectory-only adaptations of physical reasoning baselines. Best and second-best results are shown in bold and underlined , respectively.

Method	Mass $\downarrow$	Fric. $\downarrow$	Rest. $\downarrow$	Avg. $\downarrow$
Method	A/B	A/B	A/B	A/B
Non-learned baselines
Mean	0.25/0.77	0.17/0.34	0.22/0.41	0.21/0.51
Random	0.33/ 0.75	0.31/0.39	0.30/0.41	0.31/0.52
Supervised property predictors
Temporal MLP	0.24/0.77	0.13/0.28	0.17/0.33	0.18/0.46

Table 5: Object-level property estimation under the Long setting (

T_{\mathrm{obs}}=30

). Entries report NMAE on ExPhy-A/ExPhy-B (ID/OOD-Parameter). All learned models are trained on ExPhy-A and evaluated zero-shot on ExPhy-B. “Prop. only”, “Traj. only”, and “Full” use

\mathcal{L}_{\mathrm{prop}}

\mathcal{L}_{\mathrm{traj}}

, and their joint objective, respectively. Lower is better; bold denotes the best results, including ties.

Figure 4: Visualization of long-horizon trajectory forecasting on ExPhy-A, ExPhy-B, and ExPhy-C. Rows show representative methods from different model families, and columns correspond to ID/OOD splits. Red, blue, and green dots denote predicted, observed, and ground-truth trajectories, respectively.

Figure 5: Qualitative property interventions under the Long setting. Red, blue, and green denote predicted, observed, and ground-truth trajectories, respectively.

Variant	ExPhy-A	ExPhy-B	ExPhy-C
Variant	ADE/FDE $\downarrow$	ADE/FDE $\downarrow$	ADE/FDE $\downarrow$
w/o explicit physics	0.42/0.86	0.48/0.99	1.80/3.41
w/o Neural ODE	0.38/0.79	0.41/0.86	1.12/2.30
PhyODE	0.36 / 0.75	0.40 / 0.85	0.97 / 2.00

Table 6: Component ablation of PhyODE under the Long horizon setting (

T_{\text{obs}}

T_{\text{pred}}=30

60

). We report trajectory forecasting errors as ADE/FDE (

\downarrow

) on both ID and OOD splits. Best results are shown in bold .

Trajectory Prediction

Trajectory prediction forecasts how people, vehicles, robots, and other dynamic systems will move from observed histories and context.

See all 187 papers

PhysiCalWorld: In-Context Physics Calibration for World Models
Oct 21, 2026Yunhao Luo, Wentao He, Yinpei Dai +5World Models Action-Conditioned World Models
University of Michigan
PhysiFormer: Learning to Simulate Mechanics in World Space
Jun 25, 2026Yiming Chen, Yushi Lan, Andrea VedaldiWorld Model Learning Physics-Based Simulation
Visual Geometry Group, University of Oxford
JEPA-x: Cross-Predictive Physics Grounding for Forecastable Latent Dynamics
Aug 25, 2026Kehan Wen, Ziming Li, Siyuan Luo +1Latent Dynamics Modeling Joint-Embedding Predictive Architecture
Department of Electrical and Computer Engineering, NUS

Oct 21, 2026J/K move · Enter open · S save

PhysiCalWorld: In-Context Physics Calibration for World Models

Yunhao Luo, Wentao He, Yinpei Dai, Hyelin Nam+4

University of Michigan

World Models Action-Conditioned World Models World Model-Based Planning ICL for Robotics

Open paper AskPDF arXiv

Existing world models typically perceive a scene through current visual frame or a narrow history window. However, these observations may not reveal the physics properties that determine transition dynamics, such as an object's mass distribution, joint damping, or surface friction. Without calibrating these physics factors for world models, predictions tend to regress toward an empirical prior and induce severe drift in novel test scenes. To this end, we propose PhysiCalWorld, which calibrates action-conditioned world models to the instance-specific dynamics via in-context prompting, without fine-tuning or analytical physics modeling. Our method encodes an exploratory interaction clip collected within the test scene into a compact physics descriptor token, which grounds subsequent predictions in the scene's actual physical behavior. PhysiCalWorld is trained end-to-end with the latent prediction loss alone, without physics labels. At inference time, the descriptor token is extracted once and can be reused across rollouts with negligible compute overhead. We evaluate PhysiCalWorld across four environments covering diverse physics variations and show that it substantially improves rollout prediction accuracy and visual planning success rate. The learned descriptors also form a physically structured latent space and generalize beyond the physics ranges seen during training.

Abstract

Trajectory Prediction

Explore similar work

PhysiCalWorld: In-Context Physics Calibration for World Models

PhysiFormer: Learning to Simulate Mechanics in World Space

JEPA-x: Cross-Predictive Physics Grounding for Forecastable Latent Dynamics