PhysSkin++: Real-Time and Generalizable Physics-Based Animation via Self-Supervised Neural Skinning
Authors: Yuanhang Lei, Baofei Liu, Xingxuan Li, Tao Cheng, Zixuan Wang, Boming Zhao, Yujun Shen, Siyuan Huang, Ruizhen Hu, Peter Yichen Chen, Hujun Bao, Zhaopeng Cui
Organizations: State Key Laboratory of CAD&CG, Zhejiang University · Ant Group, Hangzhou 310000, China · Beijing Institute for General Artificial Intelligence (BIGAI) · Shenzhen University · University of British Columbia
Generalizing real-time physics-based animation across diverse 3D shapes and spatial discretizations remains challenging. In this paper, we introduce PhysSkin++, a physics-informed framework that learns continuous neural skinning fields from static geometries to construct compact deformation subspaces. Specifically, we propose a Neural Skinning Fields Autoencoder that predicts shape-dependent deformation subspaces without per-object network training. To learn these subspaces without simulation trajectories, we develop a Physics-Informed Self-Supervised Learning strategy that combines skinning-field normalization with conflict-aware gradient correction, stabilizing joint optimization under physical and orthogonality constraints. The learned representation supports efficient reduced dynamics with user control and contact interactions, together with direct animation binding to meshes, point clouds, and 3D Gaussians. For prescribed rig animation, we further introduce a self-supervised complementarity objective to learn complementary elastic responses without enforcing complementarity constraints at runtime. Experiments demonstrate cross-shape generalization of the learned skinning fields, real-time reduced simulation, and applications to controllable, collision-aware, and rig-complementary animation.
Figures & tables
Fig. 1: PhysSkin++ is a generalizable physics-informed neural skinning framework for object animation. The framework is learned directly from static 3D geometries via physics-informed self-supervision without any annotated data. Once trained, PhysSkin++ can be applied in a feed-forward manner to perform neural skinning for diverse 3D shapes and discretizations, enabling real-time physics-based animation. It further supports interactive control, collision-aware multi-object interaction, and rig-complementary animation.
Fig. 2: Given a static 3D shape, we first sample volumetric cubature points for animation and surface points for shape encoding. A shape encoder processes the surface points to produce shape latents, from which a set of learnable queries extract handle latents via cross-attention. Subsequently, spatial queries attend to the handle latents through another cross-attention block to derive point-wise features, which are decoded into neural skinning fields. A geometry-only, physics-informed learning strategy optimizes the network in a self-supervised manner under physical and orthogonality constraints, enabling the learned skinning fields to support real-time, physics-based animation.
Fig. 3: Evolution of neural skinning fields during optimization. Starting from disordered initial representations, our physics-informed self-supervised learning progressively organizes them into physically consistent, approximately orthogonal, and spatially smooth skinning results. For visualization, each skinning weight function Wi is divided by max∣Wi∣ to fall within [−1,1] .
Fig. 4: Qualitative results for controllable and collision-aware reduced dynamics. (a) Dirichlet boundary conditions constrain prescribed regions during deformation. (b) Interactive dragging provides localized user control. (c) Boundary-penalty energies support contact with planar environments across diverse shapes. We encourage readers to visit our project page for additional animation videos.
Fig. 5: Qualitative results for collision-aware multi-object animation. (a) Rigid–deformable contact between objects with distinct geometries. (b) Deformable–deformable contact among multiple objects, jointly resolved by the reduced collision formulation. We encourage readers to visit our project page for additional animation videos.
Fig. 6: Self-supervised rig-complementarity training. The rig Jacobian of a prescribed rig provides a complementarity constraint for learning rig-complementary neural skinning fields from static geometry.
Fig. 7: Qualitative results of rig-complementary animation. For each frame, the rig-driven primary motion without complementary elastic deformation is shown semi-transparently as a reference to the corresponding rig-complementary animation. (a) During interactive rig control, user-manipulated poses define the primary animation, and PhysSkin++ produces the corresponding complementary elastic response at runtime. (b) For user-authored rig animation sequences, PhysSkin++ augments the input kinematic trajectory with temporally evolving complementary elastic deformation. We encourage readers to visit our project page for additional animation videos.
Fig. 8: Qualitative comparisons on the RigNet [ 62 ] dataset. We visualize combined skinning fields obtained by blending all skinning weights together, providing an overall view of deformation influences and smoothness compared with baselines [ 62 , 20 , 14 , 52 ] .
Fig. 9: Qualitative results on unseen 3D shapes from the RigNet [ 62 ] test set. We visualize blended skinning fields to demonstrate our method’s generalization to novel 3D shapes. A single PhysSkin++ model produces all results.
Fig. 10: Qualitative comparisons on the ShapeNet [ 7 ] dataset. We compare with Simplicits [ 44 ] , Anymate [ 14 ] , and Puppeteer [ 52 ] . Zoom in for details.
Fig. 12: Skinning results for the deformable Lego shape family, where the deformable geometries are from Mani-GS [ 18 ] .
Fig. 13: Qualitative results of self-supervised rig-complementarity learning. For each character, the four columns show the prescribed rig structure, its LBS skinning weights, and neural skinning fields learned with and without the rig-space complementarity term Lcomp , respectively. The comparison illustrates how this term changes the spatial distribution of the learned skinning fields across diverse characters and rig structures.
Fig. 14: Top: Trained without Lpot . Bottom: Trained with Lpot .
Department of Computer Science, City University of Hong Kong, Hong Kong SAR, China · Central Media Technology Institute, Huawei, China · School of Computer Science and Engineering, Central South University, Changsha, China