MatrixSplat: Matrix-Parallel and Feature-Scalable 3D Gaussian Splatting Rendering on GPUs
Organizations: The University of Texas at Arlington · Snap Inc. · The University of Texas at Dallas · University of Georgia ·
Abstract
3D Gaussian Splatting (3DGS) has emerged as a mainstream representation for real-time novel view synthesis and 3D reconstruction. However, existing 3DGS rendering pipelines are poorly aligned with modern GPU architectures: depth-sorted alpha compositing introduces strong per-pixel sequential dependencies, limiting parallelism and resulting in scalar FMA-dominated execution that underutilizes high-throughput matrix units such as Tensor Cores. This mismatch becomes more severe for multimodal and semantically enriched 3DGS, where high-dimensional features must be accumulated from millions of Gaussians, leading to rapidly increasing computation and memory costs. We propose MatrixSplat, a matrix-parallel rendering framework for feature-scalable 3DGS on modern GPUs. We first identify the sequential dependency of alpha compositing as a key obstacle to efficient high-dimensional feature rendering, and adopt an order-independent weighted accumulation formulation that transforms feature rendering into matrix-oriented computation. Our framework features three key designs: (i) a tile-based splatting pipeline that maps feature accumulation to warp-level matrix-multiply-accumulate execution on Tensor Cores; (ii) a warp-granularity Gaussian packing strategy that mitigates visibility irregularity and improves feature reuse; and (iii) a lazy channel-batched feature-fetching pipeline that reduces shared-memory pressure, maintains high warp occupancy, and overlaps memory access with matrix execution for high-dimensional features. On Mip-NeRF 360, LERF, and 3D-OVS, MatrixSplat achieves up to 3.8 speedup over 3DGS and 2 over TC-GS while maintaining rendering quality and semantic accuracy, and further reduces render-kernel time by up to 36.7 for high-dimensional features. Integrated into existing fast renderers, it provides an additional 1.69-2.84 speedup.
Figures & tables
| Method | Stage | Order | Dim. |
|---|---|---|---|
| FlashGS, Speedy-Splat, | Culling, scheduling | Sorted | RGB |
| AdR-Gaussian, HiGS | |||
| TC-GS, GEMM-GS | Alpha (TC) | Sorted | RGB |
| ORANGE | Blending (NPU) | Sorted | RGB |
| Sort-Free, StochasticSplats | Sorting | None | RGB |
| Duplex-GS | Sorting | Proxy | RGB |
| Stage | Kernel | % |
| Rendering | Preprocessing | 7.2 |
| DuplicateWithKeys | 4.1 | |
| Sorting | 14.3 | |
| RenderCUDA | 51.3 | |
| Decoding | — | 19.2 |
| Post-Processing | — | 3.8 |
| Mip-NeRF 360 | LERF | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Method | PSNR | SSIM | LPIPS | mIoU | LocAcc | FPS | PSNR | SSIM | LPIPS | mIoU | LocAcc | FPS |
| 3DGS [ 19 , 39 ] | 30.12 | 0.910 | 0.181 | 0.536 | 0.698 | 156.0 | 23.91 | 0.856 | 0.219 | 0.419 | 0.632 | 237.9 |
| 3DGS(+TC-GS) [ 29 ] | 30.14 | 0.911 | 0.183 | 0.536 | 0.698 | 295.1 | 23.97 | 0.856 | 0.218 | 0.419 | 0.632 | 310.3 |
| MatrixSplat | 30.35 | 0.911 | 0.178 | 0.541 | 0.730 | 599.9 | 23.80 | 0.851 | 0.225 | 0.445 | 0.679 | 702.6 |
| 3D-OVS | ||||||
|---|---|---|---|---|---|---|
| Method | PSNR | SSIM | LPIPS | mIoU | LocAcc | FPS |
| 3DGS [ 19 , 39 ] | 24.71 | 0.757 | 0.234 | 0.802 | 0.937 | 162.0 |
| 3DGS(+TC-GS) [ 29 ] | 24.66 | 0.749 | 0.234 | 0.802 | 0.937 | 294.8 |
| MatrixSplat | 24.89 | 0.752 | 0.239 | 0.804 | 0.985 | 510.9 |
| 3D-OVS | LERF | Mip-NeRF 360 | |||||||
|---|---|---|---|---|---|---|---|---|---|
| Method | FPS | PSNR | LPIPS | FPS | PSNR | LPIPS | FPS | PSNR | LPIPS |
| 3DGS [ 19 ] | 210.2 | 24.78 | 0.239 | 357.0 | 23.91 | 0.217 | 278.9 | 30.13 | 0.183 |
| Sort-Free (WSR) [ 15 ] | 240.3 | 24.55 | 0.239 | 256.1 | 23.90 | 0.227 | 215.7 | 30.35 | 0.178 |
| Duplex-GS [ 31 ] | 319.1 | 25.40 ∗ | 0.171 ∗ | 332.9 | 24.46 | 0.223 | 127.9 | 30.46 | 0.222 |
| MatrixSplat | 521.8 (2.48 ) | 24.88 | 0.234 | 891.9 (2.50 ) | 23.80 | 0.228 | 660.0 (2.37 ) | 30.35 | 0.181 |
| Setting | 16 | 32 | 64 | 128 | 256 | 512 | |
|---|---|---|---|---|---|---|---|
| Vanilla 3DGS | 253.8 | 237.2 | 194.1 | 116.3 | 59.5 | 2.3 | 1.1 |
| + OIEFB | 423.5 | 363.2 | 273.5 | 143.7 | 64.9 | 2.1 | 1.2 |
| + MMA execution | 741.1 | 500.9 | 302.8 | 148.1 | 78.8 | 38.0 | 17.4 |
| + Warp-level packing | 830.5 | 568.2 | 369.0 | 170.8 | 79.3 | 35.7 | 22.4 |
| + Lazy channel-batched fetching | 830.8 | 622.6 | 399.5 | 269.1 | 144.5 | 72.1 | 41.1 |
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
| MMA Shape | FPS |
|---|---|
| mma.m16n8k8 | 534.0 |
| mma.m16n8k16 | 564.4 |
| Counter | Scalar FP32 | Scalar FP16 | Warp-packed MMA |
| Executed warp instr. | 1341 M | 1575 M | 501 M ( 63%) |
| FMA-pipe instr. | 630 M | 763 M | 171 M |
| ALU-pipe instr. | 217 M | 217 M | 175 M |
| Tensor-Core instr. | 0 | 0 | 4.17 M |
| DRAM bytes | 391 MB | 319 MB | 306 MB |
| L2 bytes | 844 MB | 679 MB | 745 MB |
| 3D-OVS | LERF | Mip-NeRF 360 | ||||
|---|---|---|---|---|---|---|
| Method | FPS | PSNR | FPS | PSNR | FPS | PSNR |
| FlashGS [ 11 ] | 794.7 | 24.71 | 1432.2 | 23.99 | 996.4 | 30.06 |
| FlashGS + MatrixSplat | 2122.4 (2.67 ) | 24.81 | 3000.5 (2.09 ) | 23.81 | 2302.5 (2.31 ) | 30.30 |
| GEMM-GS [ 26 ] | 229.2 | 24.63 | 439.8 | 23.97 | 324.1 | 30.10 |
| GEMM-GS + MatrixSplat | 651.0 (2.84 ) | 24.79 | 1248.0 (2.84 ) | 23.80 | 911.4 (2.81 ) | 30.31 |
| Speedy-Splat [ 14 ] | 556.7 | 24.65 | 1000.2 | 23.96 | 704.6 | 30.13 |
| Method | RTX 4090 | RTX 6000 Ada | H100 80GB |
|---|---|---|---|
| 3DGS | 278.9 | 252.1 | 252.5 |
| 3DGS + MatrixSplat | 660.0 (2.37 ) | 595.4 (2.36 ) | 587.5 (2.33 ) |
| FlashGS | 996.4 | 932.7 | 769.7 |
| FlashGS + MatrixSplat | 2302.5 (2.31 ) | 2190.3 (2.35 ) | 1904.7 (2.47 ) |
3DGS Acceleration
3D Gaussian Splatting acceleration reduces the time, memory, and compute needed to train and render Gaussian scene representations.
See all 71 papers