RadarDINO: Distilling a Visual Foundation Model into mmWave Radar for Human Mesh Reconstruction
Organizations: Technical University of Munich, School of Computation, Information and Technology, Chair of Media Technology, Munich Institute of Robotics and Machine Intelligence, Munich
Abstract
Millimeter-wave (mmWave) radar offers a privacy-preserving and illumination-robust alternative for human mesh reconstruction (HMR). However, high-fidelity radar HMR remains challenging. Sparse point cloud-based methods suffer from information loss and missed detections, while raw radar tensor methods like RT-Mesh lack fine-grained semantics and explicit local body modeling. In this paper, we propose RadarDINO, a radar HMR framework that distills visual semantics from a DINOv3 teacher into 3D radar representations and introduces explicit geometric supervision. DINOv3 descriptors are transferred to radar voxels via calibrated RGB-radar geometry, providing voxel- and body-part-level supervision for part-aware radar representations. We further incorporate vertex- and joint-losses to directly constrain reconstructed geometry. On the M4Human dataset, our method outperforms the SOTA baseline across all three evaluation splits.
Figures & tables
| Modality | Method | Efficiency | S1 (Random) | S2 (Cross-Subject) | S3 (Cross-Action) | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Param. (M) | GFLOPs | MVE | MJE | MRE | TE | MVE | MJE | MRE | TE | MVE | MJE | MRE | TE | ||
| RPC | mmMesh [ 16 ] | 41.45 | 2.87 | 132.7 | 112.1 | 11.8 | 70.4 | 170.1 | 147.9 | 16.0 | 100.0 | 173.8 | 146.9 | 16.4 | 91.8 |
| P4Trans. [ 6 ] | 129.01 | 11.76 | 89.5 | 76.6 | 9.2 | 48.6 | 140.8 | 125.2 | 15.0 | 86.4 | 147.8 | 126.4 | 16.1 | 78.4 | |
| RT | RT-Pose [ 9 ] | 6.05 | 50.67 | 100.7 | 87.0 | 9.9 | 56.7 | 148.1 | 131.2 | 15.3 | 93.9 | 152.8 | 131.0 | 16.0 | 84.6 |
| RETR [ 19 ] | 52.42 | 3.01 | 97.1 | 81.8 | 9.1 | 50.4 | 169.7 | 148.6 | 16.3 | 100.1 | 163.1 | 138.3 | 16.6 | 85.4 | |
| RT-Mesh [ 7 ] | 63.25 | 2.60 | 90.9 | 77.2 | 9.6 | 47.6 | 135.1 | 120.2 | 14.9 | 86.1 | 143.1 | 122.0 | 15.6 | 76.0 | |
| Radar Encoder Weights | MVE | MJE | MRE | TE |
|---|---|---|---|---|
| Random initialization | 120.65 | 102.27 | 15.38 | 63.31 |
| DINO-distilled, frozen | 114.95 | 98.30 | 15.04 | 62.75 |
| DINO-distilled, fine-tuned | 111.58 | 94.62 | 14.90 | 61.87 |
| RGB DINO teacher | 87.03 | 71.71 | 13.16 | 36.23 |
Human Mesh Recovery
Human mesh recovery estimates articulated 3D body shape and pose from images, video, or other sensors for animation, tracking, and physical analysis.
See all 55 papers