PhysConvex: Physics-Informed 3D Dynamic Convex Radiance Fields for Reconstruction and Simulation

Dan Wang; Xinrui Cui; Serge Belongie; Ravi Ramamoorthi

cs.CVFeb 21, 2026

PhysConvex: Physics-Informed 3D Dynamic Convex Radiance Fields for Reconstruction and Simulation

Authors: Dan Wang, Xinrui Cui, Serge Belongie, Ravi Ramamoorthi

Organizations: University of California, San Diego · Univesity of North Texas · Copenhagen University

Abstract

Reconstructing and simulating dynamic 3D scenes with both visual realism and physical consistency remains a fundamental challenge. Existing neural representations, such as NeRFs and 3DGS, excel in appearance reconstruction but struggle to capture complex material deformation and dynamics. We propose PhysConvex, a Physics-informed 3D Dynamic Convex Radiance Field that unifies visual rendering and physical simulation. PhysConvex represents deformable radiance fields using physically grounded convex primitives governed by continuum mechanics. We introduce a boundary-driven dynamic convex representation that models deformation through vertex and surface dynamics, capturing spatially adaptive, non-uniform deformation, and evolving boundaries. To efficiently simulate complex geometries and heterogeneous materials, we further develop a reduced-order convex simulation that advects dynamic convex fields using neural skinning eigenmodes as shape- and material-aware deformation bases with time-varying reduced DOFs under Newtonian dynamics. Convex dynamics also offers compact, gap-free volumetric coverage, enhancing both geometric efficiency and simulation fidelity. Experiments demonstrate that PhysConvex achieves high-fidelity reconstruction of geometry, appearance, and physical properties from videos, outperforming existing methods.

Figures & tables

Figure 2: Center-driven Gaussian dynamics have limited flexibility for non-uniform deformation and evolving boundaries, while boundary-driven convex dynamics enable spatially adaptive vertex advection, explicit surface evolution, and polyhedral structural refinement.

Figure 3: PhysConvex jointly reconstructs the geometry, appearance, and physical properties of dynamic objects by integrating boundary-driven dynamic convex field, differentiable reduced-order convex simulation, and rendering. Training proceeds in two stages: (1) reconstructing an undeformed convex field from the first multi-view frame, and (2) advecting the boundary-driven convex field via reduced-order simulation while jointly optimizing physical properties via differentiable simulation and rendering under single-view video supervision.

	methods	backpack	bell	blocks	bus	cream	elephant	grandpa	leather	lion	mario	sofa	turtle	Mean Error $\downarrow$
$\log(E)$	PAC-NeRF	3.28	1.08	4.02	3.30	3.22	3.05	2.99	1.20	2.34	3.37	0.20	1.94	2.50
	GIC	1.16	2.87	2.12	1.93	2.13	1.53	0.42	3.45	2.85	1.82	0.65	3.18	2.01
	Vid2Sim	0.69	0.15	0.54	0.26	0.95	0.18	1.07	0.73	0.48	0.50	0.18	0.44	0.51
	Ours	0.32	0.34	0.10	0.28	0.34	0.60	0.97	0.18	0.23	0.54	0.01	0.56	0.37
$\nu$	PAC-NeRF	0.21	0.23	0.33	0.16	0.12	0.06	0.36	0.26	0.14	0.33	0.30	0.01	0.21
	GIC	0.11	0.24	0.29	0.18	0.08	0.24	0.26	0.02	0.14	0.22	0.01	0.08	0.16

Table 1: Quantitative Comparison of Mean Absolute Error (MAE) on Physical System Identification.

		backpack	bell	blocks	bus	cream	elephant	grandpa	leather	lion	mario	sofa	turtle	Mean $\uparrow$
PSNR $\uparrow$	PAC-NeRF	19.37	25.00	23.36	20.72	23.24	22.27	21.63	20.85	22.66	21.01	22.49	22.19	22.06
	Spring-Gaus	17.42	20.49	22.78	20.06	23.58	21.30	21.64	18.29	21.95	20.23	21.89	21.23	20.91
	GIC	18.94	19.55	18.78	20.84	23.81	21.86	20.50	21.00	19.33	21.28	24.16	22.09	21.01
	Vid2Sim w/o LGM	26.87	26.59	32.57	26.53	34.82	26.99	24.45	31.21	27.62	24.83	30.01	30.33	28.57
	Ours	28.09	29.67	32.65	27.97	34.62	29.17	27.67	33.50	27.58	27.46	30.15	31.49	30.00
SSIM $\uparrow$	PAC-NeRF	0.887	0.956	0.940	0.908	0.893	0.922	0.939	0.932	0.936	0.921	0.926	0.923	0.924

Table 2: Quantitative Comparison on Dynamic Reconstruction.

Figure 4: Comparison with baseline [ 28 , 7 , 51 , 9 ] on dynamic reconstruction and future state prediction. Our method preserves high-quality geometry and appearance while producing physically plausible dynamics.

Figure 5: Generalization to novel materials. Elastic (stiff): $E{=}10^{7}$ , $\nu{=}0.49$ . Elastic (soft): $E=8000$ , $\nu=0.4$ . Plasticine: $\tau_{Y}{=}500$ . Sand: $\theta_{f}{=}10^{\circ}$ .

Figure 6: Generalization to novel force and complex boundary conditions. Force: A continuous external force pushing the falling object toward the left. Boundary: An object falls and is subjected to sliding motion on a continuously moving floor.

	PAC-NeRF	Spring-Gaus	GIC	Vid2Sim	Ours
PSNR $\uparrow$	20.11	18.32	19.20	25.07	25.71
SSIM $\uparrow$	0.913	0.905	0.916	0.945	0.948
FoVVDP $\uparrow$	5.948	5.443	5.702	7.770	7.683

Table 3: Quantitative Comparison on Future State Prediction.

	Dynamic Reconstruction			Future State Prediction			System Identification		Primitives	Training Time
$K$	PSNR $\uparrow$	SSIM $\uparrow$	FoVVDP $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	FoVVDP $\uparrow$	$\log{(E)}$	$\nu$	Num.	Average
K=3	24.90	0.939	6.957	19.63	0.919	6.139	0.74	0.05	11827	5min 20s
K=4	29.56	0.960	8.151	24.50	0.944	7.500	0.38	0.04	14405	5min 36s
K=5	29.88	0.961	8.235	25.09	0.946	7.688	0.36	0.04	18773	6min 5s
K=6	30.00	0.962	8.247	25.71	0.948	7.683	0.37	0.04	23681	6min 14s
K=7	30.01	0.962	8.235	24.95	0.945	7.664	0.38	0.05	29646	6min 39s

Table 4: Ablation of the Number

K

of Convex Points (

M=10

) ( Best , Second , Third ).

		Dynamic Reconstruction			Future State Prediction			System Identification		Primitives
K	M	PSNR $\uparrow$	SSIM $\uparrow$	FoVVDP $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	FoVVDP $\uparrow$	$\log{(E)}$	$\nu$	Num.
K=4	M=8	29.47	0.959	8.152	24.55	0.944	7.605	0.351	0.053	14392
	M=10	29.56	0.960	8.151	24.50	0.944	7.500	0.376	0.044	14405
	M=12	29.60	0.960	8.206	24.47	0.943	7.558	0.510	0.043	14180
K=6	M=8	29.96	0.962	8.214	25.05	0.946	7.635	0.330	0.048	23832
	M=10	30.00	0.962	8.247	25.71	0.948	7.683	0.371	0.044	23681

Table 5: Ablation of the Number

M

of Reduced DOFs.( Best , Second , Third )

	PAC-NeRF	GIC	Spring-GS	Vid2Sim	Ours
Avg. Training Time	51 min	69 min	32 min	8 min	6 min
Primitives Num.	-	-	22882	31363	23681

Table 6: Comparison on total training time and primitives number.

Dynamic Scene Reconstruction

Dynamic scene reconstruction builds time-varying 3D models of moving objects and environments from video or multi-view images.

See all 288 papers

PhysReal: Learning Real-World Deformable Object Physics via Hybrid Constitutive Modeling
Sep 7, 2026Yinan Deng, Jianqiao Song, Yisi Zhang +3Dynamic Scene Reconstruction Physics-Based Simulation
DySurface: Consistent 4D Surface Reconstruction via Bridging Explicit Gaussians and Implicit Functions
May 11, 2026Minje Kim, Younghyun Noh, Jaesoon Kim +13D Gaussian Splatting Signed Distance Functions
KAIST · KT · Sungkyunkwan University
ReconPhys: Reconstruct Appearance and Physical Attributes from Single Video
Apr 9, 2026Boyuan Wang, Xiaofeng Wang, Yongkang Li +9Deformable Shape Reconstruction Feed-Forward 3D Reconstruction
GigaAI · Institute of Automation, Chinese Academy of Sciences · Tsinghua University

Sep 7, 2026J/K move · Enter open · S save

PhysReal: Learning Real-World Deformable Object Physics via Hybrid Constitutive Modeling

Yinan Deng, Jianqiao Song, Yisi Zhang, Yuhan Wang+2

Dynamic Scene Reconstruction Physics-Based Simulation Deformable Object Simulation

Open paper AskPDF arXiv

Learning physically plausible dynamics from visual observations is essential for interactive world models and embodied agents. However, modeling real-world deformable objects remains challenging because their dynamics often arise from complex, spatially heterogeneous material responses. To address this challenge, we propose PhysReal, a video-driven framework for learning and simulating the underlying physics of real deformable objects. PhysReal integrates a spatially varying hybrid expert-neural constitutive model with a differentiable MPM simulator and 3DGS renderer. Analytical expert models provide interpretable physical priors, while neural constitutive residuals capture material responses beyond predefined formulations. Spatially distributed patches parameterize the constitutive field, enabling a continuous representation of local material variations. To organize the identification of this model from sparse visual observations, we adopt a progressive curriculum that sequentially optimizes global material properties, spatially varying local parameters, and neural constitutive residuals, together with complementary motion and mask supervision. Extensive experiments on diverse deformable-object interactions demonstrate that PhysReal achieves superior performance in dynamic reconstruction and future-state prediction, while showing strong potential for downstream robotic applications.

Abstract

Dynamic Scene Reconstruction

Explore similar work

PhysReal: Learning Real-World Deformable Object Physics via Hybrid Constitutive Modeling

DySurface: Consistent 4D Surface Reconstruction via Bridging Explicit Gaussians and Implicit Functions

ReconPhys: Reconstruct Appearance and Physical Attributes from Single Video