PhysCreator: Creating Physically-Attributed 3D Assets at Will

Shiyan Zheng; Ming Li; Lei Han; Xiangyu Xu

cs.CVOct 25, 2026

PhysCreator: Creating Physically-Attributed 3D Assets at Will

Authors: Shiyan Zheng, Ming Li, Lei Han, Xiangyu Xu

Organizations: Xi’an Jiaotong University · The Chinese University of Hong Kong, Shenzhen · Discover Robotics

Abstract

Significant advancements in 3D asset creation have enabled high-fidelity 3D generation from images. However, most existing representations remain inherently static, lacking the ability to model physically plausible behaviors under dynamic interactions. We present PhysCreator, a unified framework that directly generates simulation-ready 3D representations with spatially varying physical properties from images. Unlike prior approaches that treat physical attributes as simple post-hoc predictions or rely on expensive per-instance optimization, we reformulate physical property estimation within a unified 3D representation learning framework. We introduce a voxel-aligned latent space by lifting image observations into a spatially structured 3D representation, in which geometry, appearance, and physical properties are jointly modeled and inferred in a feed-forward manner. To enable scalable learning without densely annotated physical ground truth, we leverage vision-language distillation to transfer material-aware semantic priors from powerful vision-language models into spatially grounded voxel-level supervision. Experiments demonstrate that our method achieves superior visual quality, physical plausibility, and generalization compared to prior approaches.

Figures & tables

Figure 1: We present PhysCreator, a physics-aware 3D generative model that produces simulation-ready 3D Gaussians with spatially-varying, semantically-consistent physical properties directly from images. Our method enables the joint prediction of geometry and physically meaningful attributes, such as material type, stiffness, and density, with strong generalization across diverse objects. The generated 3D Gaussians can be seamlessly integrated into simulations, yielding realistic and dynamic physical behaviors.

Figure 2: Overview of PhysCreator. Top : Given observations, our framework first constructs a voxel-aligned latent representation (the 3D latent). Built upon this shared representation, we decode simulation-ready 3D Gaussian primitives together with voxel-wise physical attributes, including Young’s modulus, density, Poisson’s ratio, and material category. Bottom : To provide scalable supervision, vision-language distillation generates part-level semantic and physical annotations, which are lifted into voxel-level labels for training the physical-material decoder. The predicted geometry and physical properties are then passed to an MPM simulator to evaluate and visualize dynamic behavior.

	Flower		Metal Crate		Sand		Rubber Toy		Wooden Box		Average		Runtime Cost
Method	$\mathrm{CLIP}_{\mathrm{sim}}\uparrow$	$\mathrm{U}_{\mathrm{score}}\uparrow$	$\mathrm{CLIP}_{\mathrm{sim}}\uparrow$	$\mathrm{U}_{\mathrm{score}}\uparrow$	$\mathrm{CLIP}_{\mathrm{sim}}\uparrow$	$\mathrm{U}_{\mathrm{score}}\uparrow$	$\mathrm{CLIP}_{\mathrm{sim}}\uparrow$	$\mathrm{U}_{\mathrm{score}}\uparrow$	$\mathrm{CLIP}_{\mathrm{sim}}\uparrow$	$\mathrm{U}_{\mathrm{score}}\uparrow$	$\mathrm{CLIP}_{\mathrm{sim}}\uparrow$	$\mathrm{U}_{\mathrm{score}}\uparrow$	Runtime Cost
DreamPhysics	26.92	2.77	25.03	2.35	20.86	2.85	27.39	3.08	20.86	1.62	24.212	2.534	$>1$ h
OmniPhysGS	25.50	1.46	25.21	1.88	22.29	1.88	26.19	2.38	20.38	2.08	23.914	1.936	$>5$ h
PhysGM	24.08	3.00	25.07	3.31	22.66	2.12	27.83	3.19	25.02	2.42	24.932	2.808	$\sim 10$ s
PhysCreator	28.09	4.12	27.62	4.35	25.05	3.69	28.08	4.23	26.08	4.27	26.984	4.132	$\sim 10$ s
PhysCreator ∗	27.13	4.38	27.96	4.35	23.17	3.08	28.65	4.58	28.10	4.46	27.002	4.170	$\sim 15$ s

Table 1: Quantitative comparison across multiple object categories in terms of semantic consistency (

\mathrm{CLIP}_{\mathrm{sim}},\%

), user-study score (

\mathrm{U}_{\mathrm{score}}

), and per-object runtime cost. PhysCreator ∗ denote our model with multi-view input. Best and second-best results are shown in bold and underlined, respectively. The proposed algorithm achieves consistently favorable semantic and human-evaluation scores while maintaining efficient inference.

Figure 3: Qualitative comparison of simulated dynamics under gravity. PhysCreator produces stable and physically consistent motion with realistic contact and deformation. In contrast, OmniPhysGS exhibits structural collapse after impact with the ground, while DreamPhysics and PhysGM show unrealistic local deformation.

Figure 4: Effectiveness of spatially varying physical property modeling. Top : PhysCreator enables the flower to sway while keeping the pot stable, and allows the snow body to deform while preserving the hat; PhysGM produces implausible deformation due to uniform properties. 1D samples across time (space–time slices highlighted by dotted lines) are included to better visualize the temporal dynamics. Bottom : PhysCreator localizes motion to the flower while keeping the pot and chair stable, whereas PhysGM causes all components to move together. The blue dashed axes highlight this unintended coupled motion.

Method	Chamfer Distance $\downarrow$	F-score @ 2% $\uparrow$
PhysGM ( Lv et al., 2026 )	0.0475	59.05
PhysCreator	0.0431	72.72

Table 2: Quantitative comparison of geometry reconstruction quality. PhysCreator achieves a lower Chamfer Distance and a higher F-score, indicating more accurate and complete surface reconstruction.

Physical Property	DreamPhysics	PhysGM	PhysCreator
Young’s modulus	13.40%	16.19%	7.78%
Density	–	–	11.86%
Poisson ratio	–	18.66%	8.12%

Table 3: Mean relative errors (

\downarrow

) of physical-property predictions. Young’s modulus and density are evaluated in logarithmic space because of their large dynamic ranges, whereas Poisson’s ratio is evaluated in its original scale. “–” indicates that the corresponding method does not predict this property, using a manually selected value for simulation instead.

Property Ratio	PhysCreator	DreamPhysics
$E_{\mathrm{hard}}/E_{\mathrm{soft}}$	13.18	1.70
$\rho_{\mathrm{heavy}}/\rho_{\mathrm{light}}$	11.60	1.00

Table 4: Quantitative comparison of text-conditioned physical control. Higher property ratios indicate stronger responsiveness to the corresponding semantic prompts. PhysCreator demonstrates substantially stronger control than the baseline method ( Huang et al., 2025 ) .

Figure 5: Qualitative comparison of text-conditioned physical control under identical wind forces. PhysCreator produces distinct prompt-consistent bending for “soft” and “hard,” whereas DreamPhysics shows limited responsiveness. The pink dashed curves indicate bending trajectories.

3D Asset Generation

3D asset generation creates and edits objects, characters, materials, and scenes from text, images, video, or geometric constraints.

See all 167 papers

PhysX-Omni: Unified Simulation-Ready Physical 3D Generation for Rigid, Deformable, and Articulated Objects
May 20, 2026Ziang Cao, Yinghao Liu, Haitian Li +53D Asset Generation Physics-Based Simulation
S-Lab, Nanyang Technological University · ACE Robotics
PhysX-CoT: Structured Physical Reasoning from a Single Image to Simulation-Ready 3D Assets
Aug 8, 2026Jie Huang, Xiaohe Li, Jiahao Li +6Part-Level 3D Generation Physical Reasoning
the Key Laboratory of Target Cognition and Application Technology Aerospace Information Research Institute Chinese Academy of Sciences · School of Artificial Intelligence Shanghai Jiaotong University · Space Engineering University
PhysForge: Generating Physics-Grounded 3D Assets for Interactive Virtual World
May 6, 2026Yunhan Yang, Chunshi Wang, Junliang Ye +73D Asset Generation Diffusion Models
HKU · Tencent Hunyuan · ZJU +3

May 20, 2026J/K move · Enter open · S save

PhysX-Omni: Unified Simulation-Ready Physical 3D Generation for Rigid, Deformable, and Articulated Objects

Ziang Cao, Yinghao Liu, Haitian Li, Runmao Yao+4

S-Lab, Nanyang Technological University · ACE Robotics

3D Asset Generation Physics-Based Simulation 3D VLMs

Open paper AskPDF arXiv

Simulation-ready physical 3D assets have emerged as a promising direction owing to their broad applicability in downstream tasks. However, most existing 3D generation methods either neglect physical properties or are limited to a single asset category, e.g., rigid, deformable, or articulated objects. To address these limitations, we introduce PhysX-Omni, a unified framework for simulation-ready physical 3D generation across diverse asset types. Specifically, we develop a novel and efficient geometry representation tailored for Vision-Language Models, which directly encodes high-resolution 3D structures without compression, significantly improving generation performance. In addition, we construct the first general simulation-ready 3D dataset, PhysXVerse, covering diverse indoor and outdoor categories. Furthermore, to comprehensively and flexibly evaluate both generative and understanding capabilities in the wild, we propose PhysX-Bench, which encompasses six key attributes: geometry, absolute scale, material, affordance, kinematics, and function description. Extensive experiments with conventional metrics and PhysX-Bench show that PhysX-Omni performs strongly in both generation and understanding. Moreover, additional studies further validate the potential of PhysX-Omni for applications in simulation-ready scene generation and robotic policy learning. We believe PhysX-Omni can significantly advance a wide range of downstream applications, particularly in embodied AI and physics-based simulation.

Abstract

3D Asset Generation

Explore similar work

PhysX-Omni: Unified Simulation-Ready Physical 3D Generation for Rigid, Deformable, and Articulated Objects

PhysX-CoT: Structured Physical Reasoning from a Single Image to Simulation-Ready 3D Assets

PhysForge: Generating Physics-Grounded 3D Assets for Interactive Virtual World