Generative AI models for exploring diffusion dynamics from multi-frame particle tracking
Organizations: State Key Lab of Brain-Machine Intelligence, Zhejiang University, Hangzhou, Zhejiang, China. · ZJU-UIUC Institute, International Campus, Zhejiang University, Hangzhou, Zhejiang, China. · Gaoling School of Artificial Intelligence, Renmin University of China, Beijing, China.
Abstract
Particle-based volumetric flow measurement techniques have proven their efficacy in quantitatively assessing the dynamic characteristics of unsteady flows within the realm of fluid mechanics. Despite recent advances in multi-frame particle tracking, an open challenge in this area is maintaining trajectories in temporally under-sampled 3D sequences, where inter-frame displacements are large. In this work, we develop DiffTR-Flow3D, a diffusion-based model that estimates flow motion from spatiotemporal three-dimensional particle recordings via a stochastic diffusion process. DiffTR-Flow3D conditions the diffusion process with both global and local particle dynamics and enhances flow estimation by leveraging historical particle trajectories. Extensive experiments, including numerical simulations and real-world measurements, demonstrate that DiffTR-Flow3D achieves state-of-the-art displacement estimation accuracy compared with learning-based flow estimators, and, when integrated into particle tracking pipelines, substantially improves their long-term tracking performance, while exhibiting robustness and generalization across diverse flow patterns. This work is expected to provide fluid researchers with an effective tool for the consistent and precise analysis of particle trajectories in temporally under-sampled 3D sequences with large inter-frame displacements.
Figures & tables
| Methods | Params | EPE | ACC Strict | ACC Relax | Outliers |
| GMSF Zhang et al. (2024a) | 7.07M | 0.16313 | 20.40% | 27.73% | 72.27% |
| FLOT Puy et al. (2020) | 0.11M | 0.05865 | 24.99% | 45.59% | 54.41% |
| PointPWC-Net Wu et al. (2020) | 7.72M | 0.01723 | 46.75% | 71.61% | 28.39% |
| PV-RAFT Wei et al. (2021) | 0.19M | 0.01651 | 72.98% | 83.69% | 16.31% |
| 3DFlow Wang et al. (2022) | 4.77M | 0.02699 | 84.91% | 88.29% | 11.71% |
| DiffSF Zhang et al. (2024b) | 9.28M | 0.01403 | 95.52% | 96.24% | 3.76% |
| Methods | Params | EPE | ACC Strict | ACC Relax | Outliers |
| GMSF Zhang et al. (2024a) | 7.07M | 0.16993 | 24.78% | 33.15% | 66.85% |
| FLOT Puy et al. (2020) | 0.11M | 0.05291 | 41.43% | 57.15% | 42.85% |
| PointPWC-Net Wu et al. (2020) | 7.72M | 0.05228 | 44.38% | 62.06% | 37.94% |
| PV-RAFT Wei et al. (2021) | 0.19M | 0.05220 | 32.25% | 51.51% | 48.49% |
| 3DFlow Wang et al. (2022) | 4.77M | 0.03441 | 74.65% | 82.31% | 17.69% |
| DiffSF Zhang et al. (2024b) | 9.28M | 0.01526 | 94.90% | 95.98% | 4.02% |
| Methods | Params | EPE | ACC Strict | ACC Relax | Outliers |
| GMSF Zhang et al. (2024a) | 7.07M | 0.18789 | 25.11% | 33.03% | 66.97% |
| FLOT Puy et al. (2020) | 0.11M | 0.06051 | 41.03% | 54.54% | 45.46% |
| PointPWC-Net Wu et al. (2020) | 7.72M | 0.07101 | 50.14% | 63.61% | 36.39% |
| PV-RAFT Wei et al. (2021) | 0.19M | 0.06379 | 32.09% | 49.41% | 50.59% |
| 3DFlow Wang et al. (2022) | 4.77M | 0.06360 | 63.07% | 75.52% | 24.48% |
| GotFlow3D Liang et al. (2023) | 1.44M | 0.02015 | 85.26% | 90.31% | 9.69% |
| Flow field pattern | EPE | ACC Strict | ACC Relax | Outliers |
| Uniform flow | 0.00028 | % | % | % |
| Transitional boundary flow | 0.00078 | 99.99% | % | % |
| Turbulent channel flow | 0.00078 | 99.91% | 99.97% | 0.03% |
| Beltrami flow | 0.00390 | 98.43% | 98.79% | 1.21% |
| Forced MHD turbulence | 0.00148 | 97.55% | 98.66% | 1.34% |
| Forced isotropic turbulence | 0.00803 | 95.68% | 96.24% | 3.76% |
| First | Last | Mean | First | Last | Mean | First | Last | Mean | |
| SerialTrack | 98.77 | 70.69 | 82.84 | 94.80 | 61.35 | 75.18 | 90.17 | 52.95 | 67.67 |
| 99.56 | 90.89 | 94.59 | 98.38 | 89.99 | 93.42 | 96.89 | 87.28 | 91.52 | |
| T-PT | 99.72 | 95.38 | 97.43 | 95.67 | 79.49 | 86.62 | 82.54 | 58.00 | 67.67 |
| 100.00 | 98.16 | 99.26 | 99.92 | 96.85 | 98.63 | 99.58 | 82.20 | 93.40 | |
| Flow field pattern | Observation Volume | Train Num. | Test Num. | ||
| Channel flow | |||||
| Transitional boundary flow | |||||
| Forced MHD turbulence | |||||
| Forced isotropic turbulence | |||||
| Uniform flow | |||||
| Beltrami flow |
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
| Hidden Dimension | EPE | ACC Strict | ACC Relax | Outliers |
| 64 | 0.00313 | 98.15% | 98.45% | 1.55% |
| 128 | 0.00316 | 98.48% | 98.72% | 1.28% |
| 256 | 0.00405 | 98.24% | 98.61% | 1.39% |
| 512 | 0.00570 | 97.41% | 97.93% | 2.07% |
| Training Strategy | Training Steps | Sampling Steps | EPE | ACC Strict | ACC Relax | Outliers |
| w/o diffusion | – | – | 0.00978 | 94.47% | 95.24% | 4.76% |
| Diffusion | 3 | 1 | 0.00553 | 96.74% | 97.21% | 2.79% |
| Diffusion | 5 | 1 | 0.00407 | 97.40% | 98.04% | 1.96% |
| Diffusion | 5 | 2 | 0.00353 | 98.31% | 98.66% | 1.34% |
| Diffusion | 20 | 1 | 0.00324 | 98.44% | 98.71% | 1.29% |
| Diffusion | 20 | 2 | 0.00316 | 98.48% | 98.72% | 1.28% |
| Variant | EPE | ACC Strict | ACC Relax | Outliers |
| w/o refinement stage | 0.10815 | 21.34% | 37.78% | 62.22% |
| w/o coarse stage | 0.02529 | 84.59% | 90.47% | 9.53% |
| w/o local & global | 0.07840 | 23.87% | 42.85% | 57.15% |
| w/o global correlation | 0.08352 | 26.42% | 41.81% | 58.19% |
| w/o local correlation | 0.00385 | 97.74% | 98.09% | 1.91% |
| Full model | 0.00316 | 98.48% | 98.72% | 1.28% |
| Model | Params (M) | Latency (ms) ( ) | Throughput (samples/s) ( ) | EPE | ACC Strict | ACC Relax |
| GMSF | 7.07 | 38.15 | 26.22 | 0.18789 | 25.11% | 33.03% |
| FLOT | 0.11 | 13.45 | 74.34 | 0.06051 | 41.03% | 54.54% |
| PointPWC-Net | 7.72 | 35.65 | 28.05 | 0.07101 | 50.14% | 63.61% |
| PV-RAFT | 0.19 | 291.42 | 3.43 | 0.06379 | 32.09% | 49.41% |
| 3DFlow | 4.77 | 30.37 | 32.93 | 0.06360 | 63.07% | 75.52% |
| GotFlow3D | 1.44 | 74.01 | 13.51 | 0.02015 | 85.26% | 90.31% |
| First | Last | Mean | First | Last | Mean | First | Last | Mean | |
| STB | 92.91 | 11.21 | 55.10 | 89.15 | 14.60 | 57.83 | 82.67 | 15.87 | 55.87 |
| SerialTrack | 98.77 | 70.69 | 82.84 | 94.80 | 61.35 | 75.18 | 90.17 | 52.95 | 67.67 |
| 99.51 | 85.74 | 92.84 | 98.56 | 83.36 | 90.95 | 96.08 | 76.46 | 86.70 | |
| 99.56 | 90.89 | 94.59 | 98.38 | 89.99 | 93.42 | 96.89 | 87.28 | 91.52 | |
| T-PT | 99.72 | 95.38 | 97.43 | 95.67 | 79.49 | 86.62 | 82.54 | 58.00 | 67.67 |
| Model | Flow Field Pattern | EPE | ACC Strict | ACC Relax | Outliers |
| GMSF | Uniform flow | 0.02591 | 87.17% | 89.54% | 10.46% |
| Transitional boundary flow | 0.03255 | 75.93% | 84.91% | 15.09% | |
| Turbulent channel flow | 0.07480 | 19.23% | 52.14% | 47.86% | |
| Beltrami flow | 0.39621 | 0.09% | 0.49% | 99.51% | |
| Forced MHD turbulence | 0.22258 | 0.08% | 0.28% | 99.72% | |
| Forced isotropic turbulence | 0.28831 | 0.11% | 0.58% | 99.42% |
| Comparison | Associations | Median | P90 | Max | |
| SerialTrack / SerialTrack † | Shared | 440 | 1.88 | 3.36 | 4.35 |
| Added | 201 | 1.59 | 3.19 | 4.46 | |
| Removed | 29 | 1.57 | 3.70 | 5.46 | |
| T-PT / T-PT † | Shared | 431 | 1.75 | 3.22 | 4.46 |
| Added | 325 | 1.91 | 3.43 | 5.46 | |
| Removed | 11 | 1.43 | 2.61 | 4.91 |
| # | Operation | Feature Size | Kernel/Dim. | Activation |
| Feature Extraction (0.08M) | ||||
| 1 | Conv2d | LeakyReLU | ||
| 2 | MaxPool | - | - | |
| 3 | Conv2d | LeakyReLU | ||
| 4 | MaxPool | - | - | |
| 5 | Conv2d | LeakyReLU | ||
3D Point Tracking
3D point tracking estimates how scene points move through space and time, even under camera motion, occlusion, and deformation.
See all 35 papers