LiDAR Resolution Recovery via Foundation-Model-Guided Diffusion
Organizations: Dept. of Electrical Engineering and Information Technology Munich University of Applied Sciences, 80335 Munich, Germany
Abstract
High-beam-count LiDAR sensors are costly, yet many perception pipelines require dense angular sampling. Using a pretrained Stable Diffusion model as the backbone, we fine-tune a LiDAR-conditioned depth model with pseudo-depth targets from a 2D foundation model. During training, the LiDAR conditioning is randomly decimated at different beam budgets. We then investigate how much of a LiDAR scan can be recovered from heavily decimated input and characterize performance across the input beam budget. We evaluate against physically held-out real beams on nuScenes and report recovery separately from fit accuracy. Our model yields its largest advantage in very sparse regimes, achieving a accuracy of % from -beam input where scattered interpolation reaches only %. A class-stratified error breakdown further reveals that planar surfaces recover first while objects introducing depth discontinuities degrade earliest. Together, these results quantify the recovery/resolution trade-off for foundation-model-guided LiDAR enhancement.
Figures & tables
| Budget | Method | AbsRel | RMSE | |
|---|---|---|---|---|
| Nearest | 0.692 | 12.62 | 0.451 | |
| Ours (no scene) | 0.260 | 8.67 | 0.668 | |
| Ours (scene) | 0.270 | 8.85 | 0.653 | |
| Nearest | 0.311 | 8.14 | 0.670 | |
| Ours (no scene) | 0.169 | 7.34 | 0.787 | |
| Ours (scene) | 0.174 | 7.53 | 0.779 |