Turbo-GMM: FlashAttention-Inspired GPU-Accelerated Gaussian Mixture Model
Organizations: IBM Research · Rensselaer Polytechnic Institute
Abstract
Expectation-maximization (EM) for Gaussian mixture models (GMMs) requires scoring data points against mixture components. Prior fused GPU implementations, notably Flash-GMM, eliminate materialization of the responsibility matrix but traverse the component dimension twice to compute normalization terms and accumulate sufficient statistics. We introduce Turbo-GMM, a single-sweep fused EM kernel that completes responsibility normalization and sufficient-statistic accumulation within one traversal of the component dimension. By retaining the complete component axis on chip, Turbo-GMM avoids the second score computation required by the released two-sweep Flash-GMM implementation. Its applicability is limited by register pressure, occupancy, and the component and feature dimensions, making it particularly suitable for large datasets with relatively small numbers of components. For larger configurations, Turbo-GMM employs a two-pass fallback following the established fused design. We establish algebraic equivalence to standard EM in real arithmetic and evaluate sixteen configurations on an NVIDIA A100. Under the reported benchmarking protocol, Turbo-GMM achieves a median speedup over the evaluated Flash-GMM implementation and up to and over PyTorch and scikit-learn, respectively. The results demonstrate the potential of single-sweep component traversal while identifying on-chip resource constraints as the principal limitation of this approach.
Figures & tables
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
| Shape | Arm | ms/iter | Speedup | Mem (MB) | Gate |
|---|---|---|---|---|---|
| 20K/16/32 | Turbo-GMM | 0.49 | — | 3 | valid |
| Flash-GMM | 0.54 | 1.11 slower | 3 | valid | |
| cuVS | 1.55 | 3.18 slower | — | valid | |
| PyTorch | 0.41 | 1.18 faster | 17 | valid | |
| scikit-learn | 1.16 | 2.39 slower | 18 | valid | |
| gmmx | 0.22 | 2.20 faster | — | valid |
| Arm | Package | Version | Upstream source |
|---|---|---|---|
| Turbo-GMM (ours, Algs. 2 – 3 ) | torch + triton | 2.6.0+cu124 / 3.2.0 | this work |
| Flash-GMM | — (source checkout) | 57c3621 (2026-07-30) | IBM/Flash-GMM |
| cuVS | libcuvs_cu12 | 26.12.0a14 (nightly) | RAPIDS nightly C++ wheel |
| PyTorch | torch | 2.6.0+cu124 | pytorch.org |
| scikit-learn | scikit-learn | 1.9.1 | scikit-learn.org |
| gmmx (JAX) | gmmx | 0.7 | adonath/gmmx |
| Sweeps of | Reads | Score | On-chip | Per-row | |
|---|---|---|---|---|---|
| comp. axis | of | recomputed | state | HBM state | |
| Alg. 2 (ours, one pass) | 1 | 1 | no | none | |
| Flash-GMM ( Bloch et al., 2026 ) | 2 | yes | |||
| Alg. 3 (ours, two pass) | 2 | yes |
Gaussian Mixture Models
Gaussian Mixture Models represent data distributions as weighted combinations of Gaussian components for clustering, density estimation, and multimodal prediction.
See all 66 papers