Federated 3D Gaussian Splatting for Large-Scale Scene Reconstruction at Wireless Edge
Authors: Guanlin Wu, Chao Hu, Pu Chen, Juyong Zhang, Han Hu, Shuguang Cui, Jie Xu
Organizations: School of Science and Engineering, the Shenzhen Future Network of Intelligence Institute, and the Guangdong Provincial Key Laboratory of Future Networks of Intelligence, The Chinese University of Hong Kong (Shenzhen), Longgang, Shenzhen 518172, China · Department of Computer Science, University of Liverpool, Liverpool, L697ZX, UK · School of Mathematical Science, University of Science and Technology of China, Hefei 230026, China · School of Information and Electronics, Beijing Institute of Technology, Beijing 100081, China
Three-dimensional (3D) Gaussian splatting (3D-GS) has emerged as a promising technique for large-scale scene reconstruction due to its high rendering efficiency and fidelity. However, the training of large-scale 3D-GS models at wireless edge faces various technical challenges including the limited communication, computation, and graphics processing unit (GPU) memory resources at edge devices, the structural inconsistency issue across local models hindering their effective aggregation, as well as privacy leakage risks associated with raw visual content and camera parameters. To address these challenges, this paper proposes a novel resource-efficient federated learning framework for efficiently training 3D-GS models of large scenes under severe resource constraints. First, we propose an on-device model lightweighting mechanism that adaptively selects and prunes Gaussian points to balance the rendering quality and training efficiency. In this mechanism, we quantitatively evaluate the importance of different Gaussian points at each device to facilitate the pruning, and use a novel importance-to-latency ratio criterion to determine the number of pruned Gaussian points under GPU memory and computation/communication latency constraints. Furthermore, we develop a 3D-GS model recovery mechanism that restores structural consistency across local 3D-GS models without accessing private camera parameters, enabling their effective aggregation towards a global model. Finally, extensive experiments show that our approach significantly accelerates convergence, maintains high rendering quality, and reduces training latency compared to state-of-the-art federated 3D-GS baselines.
Figures & tables
Fig. 1: Illustration of the proposed federated 3D-GS learning framework. At each communication round, the edge server broadcasts the global 3D-GS model to distributed devices; each device then performs importance-to-latency ratio-based point selection, local lightweighted training, and uploads the updated local model, position gradients, and binary mask to the server for recovery and aggregation.
Fig. 2: Distribution and range of importance scores for Gaussian points in an example from experiments with 5 devices and a total of 3,222,794 Gaussian points.
Uk ( ×106 )
Dk
Predicted [GB]
Measured [GB]
Rel. err. [%]
0.756472
320
5.309
5.265
0.84
0.934897
320
5.392
5.329
1.18
1.132375
320
5.484
5.426
1.07
TABLE I: Validation of the Analytical Peak-Memory Model in ( 19 ) Against the Measured Peak GPU Memory During Local 3D-GS Training.
Tsplat(s)
Trender(s)
Measured
Estimated
Measured
Estimated
RTX 3090
1.5530
2.4728
41.3355
43.8591
RTX 3060
4.1076
5.9211
72.3471
75.9602
TABLE II: Estimation and Measurement of Latency for Two Stages (Splatting and Rendering) for 100 Local Epochs.
Fig. 3: Comparison of training loss convergence versus latency on (a) the Building scene, (b) the Rubble scene, and (c) the Polytech scene.
Fig. 4: Comparison of different training designs in terms of PSNR, SSIM, and LPIPS w.r.t. total latency in the Building scene.
Fig. 5: Comparison of different training designs in terms of PSNR, SSIM, and LPIPS w.r.t. total latency in the Rubble scene.
Fig. 6: Comparison of different training designs in terms of PSNR, SSIM, and LPIPS w.r.t. total latency in the Polytech scene.
Scenes
Building
Rubble
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
Fed3D-GS
17.21
0.567
0.375
19.18
0.552
0.429
Proposed
17.91
0.554
0.363
20.74
0.628
0.438
TABLE III: Comparison of Reconstruction Quality With Fed3D-GS.
Method
Building
Rubble
Latency
Points
Memory
Latency
Points
Memory
3D-GS
9.27
4330717
7.18
7.71
3154812
6.59
Fed3D-GS
1.83
1967079
5.93
1.69
1001575
4.68
Proposed
1.51
1358773
5.69
1.39
1260379
4.81
TABLE IV: Comparison of Total Latency (in Hours), Number of Gaussian Points in local model, and Peak GPU Memory Consumption (in GB) for One Device after 30,000 Training Epochs.
Fig. 7: Rendering quality and training loss w.r.t. total latency on the Building scene under moderate/severe camera heterogeneity.
Setting
PSNR ↑
SSIM ↑
LPIPS ↓
Homogeneous (ours)
17.91
0.554
0.363
Moderate heterogeneity
16.92
0.525
0.466
Severe heterogeneity
16.88
0.518
0.471
TABLE V: Reconstruction Quality Under Device Heterogeneity on the Building Scene.
Communication format
PSNR ↑
SSIM ↑
LPIPS ↓
FP32 (ours)
17.91
0.554
0.363
FP16
17.19
0.536
0.448
INT8
11.03
0.290
0.766
TABLE VI: Round-Wise Communication-Precision Ablation on the Building Scene.
Fig. 8: Comparison of total latency required to first reach a PSNR of 17 dB in the Building scene and 19 dB in the Rubble scene. Each bar is decomposed into communication latency, local training latency, and retraining latency (if applicable). Schemes (a)–(d) correspond to benchmark methods: (a) Fed3D-GS, (b) full model training design, (c) full memory load with recovery design, and (d) lightweighting only design, while (e) denotes the proposed design.
Fig. 9: Comparison of the proposed method and Fed3D-GS (with different numbers of local training epochs) in terms of total latency w.r.t. varying server computation resource.
Fig. 10: Qualitative comparison of novel view rendering among 3D-GS, Fed3D-GS, the lightweighting only design, and the proposed method. Our proposed design efficiently suppresses visual artifacts and achieves better reconstruction quality compared with the other benchmark schemes. The first row shows the results on the building scene, while the second and third rows show the results on the rubble scene.
3D Gaussian Splatting (3DGS) enables high-fidelity and real-time 3D scene reconstruction, but scaling training to large-scale scenes requires optimizing hundreds of millions of Gaussians across multiple GPUs. Existing distributed approaches either partition scenes into isolated regions, causing global inconsistency, or rely on global Gaussian-level exchanges, which lead to substantial growth in inter-GPU communication and quickly dominate iteration time. We propose Splaxel, a communication-efficient distributed 3DGS training framework based on pixel-level local rendering and global composition. Instead of synchronizing Gaussians, each GPU renders its local subset and exchanges only partial pixel values, maintaining mathematical consistency while keeping communication cost stable as the scene size increases. Splaxel further reduces pixel-level redundancy through geometric and transmittance visibility prediction and improves GPU utilization via conflict-free camera-view consolidation. Evaluated on large-scale datasets with up to 120M Gaussians, Splaxel achieves up to 7.6× speedup over the state-of-the-art distributed 3DGS framework while preserving high reconstruction quality.
Wenqi Jia, Zhewen Hu, Ying Huang +10
UT Arlington · Texas A&M University · Independent Researcher +5
Training 3D Gaussian Splatting (3DGS) at billion-primitive scale is fundamentally memory-bound: each Gaussian primitive carries a large attribute vector, and the aggregate parameter table quickly exceeds GPU capacity, limiting prior systems to tens of millions of Gaussians on commodity single-GPU hardware. We observe that 3DGS training is inherently sparse and trajectory-conditioned: each iteration activates only the Gaussians visible from the current camera batch, so GPU memory can serve as a working-set cache rather than a persistent parameter store. Building on this insight, we introduce TideGS, an out-of-core training framework that manages parameters across an SSD-CPU-GPU hierarchy via three synergistic techniques: block-virtualized geometry for SSD-aligned spatial locality, a hierarchical asynchronous pipeline to overlap I/O with computation, and trajectory-adaptive differential streaming that transfers only incremental working-set deltas between iterations. Experiments show that TideGS enables training with over one billion Gaussians on a single 24 GB GPU while achieving the best reconstruction quality among evaluated single-GPU baselines on large-scale scenes, scaling beyond prior out-of-core baselines (e.g., approximately 100M Gaussians) and standard in-memory training (e.g., approximately 11M Gaussians).
Chonghao Zhong, Linfeng Shi, Hua Chen +4
Hong Kong University of Science and Technology · Great Wall Motor · Tsinghua University +1
We present F3DGS, a federated 3D Gaussian Splatting framework for decentralized multi-agent 3D reconstruction. Existing 3DGS pipelines assume centralized access to all observations, which limits their applicability in distributed robotic settings where agents operate independently, and centralized data aggregation may be restricted. Directly extending centralized training to multi-agent systems introduces communication overhead and geometric inconsistency. F3DGS first constructs a shared geometric scaffold by registering locally merged LiDAR point clouds from multiple clients to initialize a global 3DGS model. During federated optimization, Gaussian positions are fixed to preserve geometric alignment, while each client updates only appearance-related attributes, including covariance, opacity, and spherical harmonic coefficients. The server aggregates these updates using visibility-aware aggregation, weighting each client's contribution by how frequently it observed each Gaussian, resolving the partial-observability challenge inherent to multi-agent exploration. To evaluate decentralized reconstruction, we collect a multi-sequence indoor dataset with synchronized LiDAR, RGB, and IMU measurements. Experiments show that F3DGS achieves reconstruction quality comparable to centralized training while enabling distributed optimization across agents. The dataset, development kit, and source code will be publicly released.
Morui Zhu, Mohammad Dehghani Tezerjani, Mátyás Szántó +3
University of North Texas · Budapest University of Technology and Economics