MorphTwin: Enriching Deformable Material Responses from Videos for Morphable and Interactive Physical Twins
Organizations: University of Science and Technology of China · City University of Hong Kong · Multimedia Laboratory, The University of Hong Kong · Columbia University
Abstract
Reconstructing interactive physical twins from video requires accurately capturing how objects deform while maintaining efficient simulation. We observe that reproducing motion in a video does not guarantee correct responses to new forces when the physical representation restricts how an object can deform. In spring-based twins, coupling between volume and shape responses creates such a limitation. We present MorphTwin, which enriches local deformation modes to capture and independently control these responses while retaining standard spring forces and explicit simulation. To support interpretable material editing and transfer, we use offline calibration to map shared relative response changes to stiffness updates on each reconstructed graph. Under identical inverse settings on synthetic scenes, MorphTwin reduces the mean absolute error of Poisson's ratio by 75.8% and the mean relative error of Young's modulus by 22.1% compared to PhysTwin. We introduce an egocentric stereo and RGB-D benchmark covering interactions with objects of varying stiffness. Real-world experiments demonstrate improved reconstruction accuracy and up to 14.4% lower future tracking error than PhysTwin at interactive speeds.
Figures & tables
| Views | Method | PSNR | SSIM | LPIPS | CD | Track | |
| PhysTwin [ jiang2025phystwin ] | 28.74 | 0.9451 | 0.0598 | 0.00946 | 0.0172 | ||
| PGND [ zhang2025pgnd ] | 27.59 | 0.9403 | 0.0722 | 0.01448 | 0.0192 | ||
| GIC [ cai2024gic ] | 24.58 | 0.9225 | 0.1302 | 0.03457 | 0.0756 | ||
| All | MorphTwin | 32.36 | 0.9665 | 0.0511 | 0.00925 | 0.0171 | |
| [1pt/1.5pt] | PhysTwin [ jiang2025phystwin ] | 28.06 | 0.9395 | 0.0782 | 0.00952 | 0.0178 | |
| PGND [ zhang2025pgnd ] | 27.18 | 0.9342 | 0.0893 | 0.01416 | 0.0196 |
Digital Twins
Digital twins are virtual models synchronized with real systems to support monitoring, prediction, simulation, and decision-making.
See all 284 papers