Multi-Rate Flow Matching for Point Cloud Upsampling
Organizations: Institute for Artificial Intelligence, University of Stuttgart, Stuttgart, Germany
Abstract
Flow-matching point-cloud upsamplers refine sparse inputs into dense point clouds by integrating a learned velocity field with a fixed temporal discretization. However, local reconstruction requirements vary substantially across a shape, while a shared solver budget applies the same temporal resolution everywhere. We propose Multi-Rate Flow Matching (MRFM), a patch-adaptive framework that selects an integration budget for each local patch at inference time while keeping the learned velocity field frozen. MRFM relaxes the discrete budget into a continuous variable, optimizes it online through differentiable soft rollouts using an input-derived geometric fidelity objective, and projects the result back to a discrete budget for the final solver rollout. Experiments on PU1K and PUGAN show strong reconstruction quality, while matched-budget permutation and multi-factor analyses demonstrate that the benefit depends on patch-specific allocation rather than on the step distribution alone. MRFM is accuracy-oriented and intentionally uses additional test-time optimization to obtain more suitable local temporal resolutions.
Figures & tables
| Upsampling rate | Method | PU1K | PUGAN | ||||||
|---|---|---|---|---|---|---|---|---|---|
| CD | HD | P2F | JSD | CD | HD | P2F | JSD | ||
| PU-GCN | 5.1497 [-1.1pt] 0.0412 | 6.9526 [-1.1pt] 0.0695 | 1.7702 [-1.1pt] 0.0124 | 2.9957 [-1.1pt] 0.0300 | 11.404 [-1.1pt] 0.0912 | 8.4747 [-1.1pt] 0.0848 | 2.3706 [-1.1pt] 0.0166 | 2.8373 [-1.1pt] 0.0284 | |
| Grad-PU | 358.02 [-1.1pt] 5.3703 | 6.5865 [-1.1pt] 0.0659 | 1.5778 [-1.1pt] 0.0158 | 2.8070 [-1.1pt] 0.0337 | 348.18 [-1.1pt] 5.2227 | 6.4313 [-1.1pt] 0.0643 | 1.5865 [-1.1pt] 0.0159 | 2.6951 [-1.1pt] 0.0323 | |
| PUDM | 4.9152 [-1.1pt] 0.0590 | 6.2694 [-1.1pt] 0.0878 | 1.7081 [-1.1pt] 0.0222 | 2.9511 [-1.1pt] 0.0354 | 10.781 [-1.1pt] 0.1294 | 9.7364 [-1.1pt] 0.1363 | 2.4151 [-1.1pt] 0.0314 | 2.6995 [-1.1pt] 0.0324 | |
| PDANS | 4.5811 [-1.1pt] 0.0596 | 6.3328 [-1.1pt] 0.0950 | 1.6712 [-1.1pt] 0.0234 | 2.7838 [-1.1pt] 0.0362 | 10.389 [-1.1pt] 0.1351 | 9.7406 [-1.1pt] 0.1461 | 2.3345 [-1.1pt] 0.0327 | 2.5931 [-1.1pt] 0.0337 | |
| RepKPU | 4.5694 [-1.1pt] 0.0366 | 5.4980 [-1.1pt] 0.0550 | 1.5626 [-1.1pt] 0.0125 | 2.7861 [-1.1pt] 0.0279 | 10.969 [-1.1pt] 0.0878 | 8.6813 [-1.1pt] 0.0868 | 2.2229 [-1.1pt] 0.0178 | 2.7421 [-1.1pt] 0.0274 | |
| Factor | MRFM step | Random perm. |
|---|---|---|
| Curvature | 0.3161 | 0.0191 0.0451 |
| Sparsity | 0.2575 | 0 |
| Surface non-uniformity | 0.0602 | 0.0154 0.0395 |
| CD@8 | 0.4812 | 0.0197 0.0432 |
| P2F@8 | 0.1412 | 0.0134 0.0482 |
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
| Hyperparameter | Value |
|---|---|
| Patch size | 64 |
| Surface KNN neighbors | 12 |
| Mamba layers | 6 |
| Time embedding dimension | 32 |
| Mamba state dimension | 32 |
| Mamba convolution width | 4 |
| Method | NFE | Latency (ms) | Peak Memory (GB) |
|---|---|---|---|
| PU-GCN | 3 | 113.1692 | 0.1401 |
| Grad-PU | 33 | 276.3827 | 0.1444 |
| PUDM | 10 | 2384.2558 | 0.2223 |
| PDANS | 9 | 2275.4462 | 0.7275 |
| RepKPU | 3 | 144.9831 | 0.1444 |
| PUFM | 5 | 871.4086 | 0.2050 |
| Method | CD | HD | P2F | JSD |
|---|---|---|---|---|
| MLP | 4.0863 | 6.2755 | 1.5506 | 2.5730 |
| RK45 | 4.0712 | 6.2502 | 1.5501 | 2.5128 |
| MRFM (Ours) | 4.0434 | 6.2394 | 1.5498 | 2.4754 |
Point Cloud Learning
Point Cloud Learning develops methods for representing and interpreting unordered 3D point sets from LiDAR, depth sensors, radar, and 3D models.
See all 168 papers