Organizations: School of SE, Huazhong University of Science and Technology, Wuhan, Hubei, China · School of CS, Huazhong University of Science and Technology, Wuhan, Hubei, China · School of EIC, Huazhong University of Science and Technology, Wuhan, Hubei, China
Reconstructing high-quality, renderable 3D scenes from long videos is essential for robotic perception and embodied intelligence. Many feed-forward Gaussian reconstruction methods rapidly predict 3D scenes from a small set of images, but extending them to longer sequences presents two bottlenecks: jointly processing all views increases GPU memory requirements, while accumulating pixel-aligned predictions redundantly represents the same surfaces and increases storage and rendering costs. We present LonGaussian, a streaming feed-forward framework that reconstructs compact, globally renderable Gaussian scenes from long image sequences with known intrinsics and unknown camera poses. Built on a pretrained streaming 3D reconstruction backbone, LonGaussian decodes long-context geometric features into dense Gaussians. Within each chunk, multi-frame point tracks link repeated observations of the same surface and guide a learned merger to consolidate the corresponding Gaussian predictions. Rendered-depth consistency filtering further suppresses Gaussians from other chunks that incorrectly occlude locally reconstructed surfaces. On DL3DV and RealEstate10K benchmarks, LonGaussian improves novel-view rendering quality while using substantially fewer Gaussians than the evaluated feed-forward baselines.
Figures & tables
Figure 1: Ultra-long reconstruction with known intrinsics and unknown camera poses. LonGaussian accumulates Gaussians and camera poses in a shared coordinate system to support novel-view rendering.
Figure 2: Gaussian redundancy from repeated observations. Pixel-aligned predictions accumulate overlapping primitives for the same surface, increasing storage and rendering costs; matching colors denote corresponding observations and Gaussians.
Figure 3: Overview of LonGaussian . LoGeR processes overlapping chunks, and AllTracker correspondences guide Gaussian consolidation before cross-chunk filtering and far-field modeling. Snowflakes and flames denote frozen and trainable modules, respectively.
Dataset
Method
64 frames
96 frames
128 frames
PSNR ↑
SSIM ↑
LPIPS ↓
#G ↓
PSNR ↑
SSIM ↑
LPIPS ↓
#G ↓
PSNR ↑
SSIM ↑
LPIPS ↓
#G ↓
DL3DV
WorldMirror
OOM
OOM
OOM
AnySplat
OOM
OOM
OOM
ReCoSplat (official)
13.62
0.234
0.647
8.78
13.67
0.237
0.654
13.17
13.70
0.241
0.660
17.56
ReCoSplat (adapted)
15.95
0.288
0.599
8.78
15.74
0.282
0.606
13.17
15.48
0.276
0.612
17.56
OF3GS
16.29
0.438
0.489
7.10
15.31
0.404
0.545
10.45
14.83
0.316
0.574
13.67
Table 1: Main results on DL3DV and RealEstate10K using complete reconstructed scenes for target-view rendering. #G is the average Gaussian count in millions; pink / yellow mark best/second-best. “OOM” exceeds 24-GB GPU memory.
Figure 4: Representative novel-view comparisons across sequence lengths. Colored boxes show that LonGaussian better preserves structure and detail; complete comparisons appear in the Appendix.
Input N
PSNR ↑
SSIM ↑
LPIPS ↓
#G (M) ↓
Memory
Time
126
20.33±1.87
0.711±0.068
0.404±0.061
7.28
19.99 GB
55 s
250
19.35±2.50
0.681±0.083
0.438±0.073
15.04
20.05 GB
101 s
507
18.40±2.83
0.626±0.118
0.485±0.097
28.16
20.05 GB
197 s
1,021
17.73±2.52
0.624±0.094
0.506±0.091
57.72
20.05 GB
392 s
2,040
18.24±2.31
0.586±0.172
0.530±0.111
111.42
20.05 GB
781 s
3,768
17.17±2.89
0.556±0.125
0.531±0.113
208.29
20.05 GB
1,445 s
Table 2: Single-scene long-sequence scalability. Quality uses seven-chunk rendering (mean ± standard deviation across target views); #G counts the complete filtered representation. Memory is peak allocated GPU memory during streaming construction. Time excludes model loading, metric rendering, and offline prefix filtering.
Configuration
PSNR ↑
SSIM ↑
LPIPS ↓
#G ↓
Time (s) ↓
Dense accumulation
18.48
0.567
0.416
10.02
155.8
Deterministic merging
18.63
0.569
0.432
3.60
138.6
w/o dense complement
17.53
0.550
0.424
0.95
124.8
w/o depth filtering
17.36
0.482
0.486
4.86
121.5
w/o far-field shell
18.59
0.569
0.425
3.71
103.4
Full model
18.88
0.575
0.420
3.54
132.6
Table 3: Component ablation on DL3DV-140 at 96 frames. #G is in millions; time is the per-scene reconstruction and evaluation time.
Table 4: Rendered-depth threshold selection on 20 DL3DV training scenes. Setting tuples denote (relative gap, minimum votes, reliable-view ratio, decision ratio). Metrics are averaged equally across scenes; Δ PSNR is paired against the same reconstruction without filtering.
Hyperparameter
Value
Image resolution
280×490
LoGeR chunk length / overlap (L,O)
(12,3)
Gaussian SH degree
2
Merger hidden dimension
128
AllTracker confidence threshold τtrk
0.1
Minimum track length nmin
2
Appendix
Table 5: Principal architecture, training, and inference hyperparameters.
Figure 6: Single-scene long-sequence scaling. Local rendering quality remains usable as the sequence grows (a–c), while complete representation size and construction time grow approximately linearly and peak memory remains bounded (d–f). Error bars show variation across held-out views.
Figure 7: Long-range revisit failure in the ultra-long sequence. Rendering the complete accumulated scene produces severe ghosting, whereas restricting the same target view to seven neighboring chunks recovers much of the scene structure. The improvement is consistent with interference from distant misregistered chunks; residual local artifacts remain.
Figure 8: Additional novel-view comparisons on DL3DV and RealEstate10K. LonGaussian consistently preserves sharper structure and finer detail across scenes and sequence lengths.
Feed-forward Gaussian reconstruction builds 3D scenes directly from input images for novel view rendering, but learning Gaussian attributes from predicted geometry for high-quality rendering under large viewpoint changes remains challenging. We analyze limitations in Gaussian attribute learning and introduce InfiniSplat v2, which improves rendering quality through architectural and training changes. Our method decouples geometry and attribute prediction with a frozen geometry branch, aligns predicted Gaussians using ground-truth camera poses during training, and adopts large-angle target view supervision. Across six datasets, the feed-forward outputs of the same trained model achieve state-of-the-art mean LPIPS and DISTS among the evaluated methods in both pose-conditioned and pose-free settings. In the pose-conditioned setting, mean LPIPS and DISTS decrease by 25.8% and 32.9%, respectively, compared with the strongest evaluated baselines. We further find that opacity learned with only image reconstruction supervision suppresses geometrically inconsistent points. Project page: https://zju3dv.github.io/InfiniSplat2/
Yongzhen Hu, Hao Yu, Jiawei Wang +7
Zhejiang University · Hello TransTech · Shenzhen University +1
Recent feed-forward Gaussian reconstruction models adopt a pixel-aligned formulation that maps each 2D pixel to a 3D Gaussian, entangling Gaussian representations tightly with the input images. In this paper, we propose AnchorSplat, a novel feed-forward 3DGS framework for scene-level reconstruction that represents the scene directly in 3D space. AnchorSplat introduces an anchor-aligned Gaussian representation guided by 3D geometric priors (e.g., sparse point clouds, voxels, or RGB-D point clouds), enabling a more geometry-aware renderable 3D Gaussians that is independent of image resolution and number of views. This design substantially reduces the number of required Gaussians, improving computational efficiency while enhancing reconstruction fidelity. Beyond the anchor-aligned design, we utilize a Gaussian Refiner to adjust the intermediate Gaussiansy via merely a few forward passes. Experiments on the ScanNet++ v2 NVS benchmark demonstrate the SOTA performance, outperforming previous methods with more view-consistent and substantially fewer Gaussian primitives.
Streaming 3D reconstruction requires more than a sequence of geometric predictions: it requires a persistent scene state that can incorporate new evidence and remain renderable as observations arrive. Latent spatial tokens offer a promising representation for this purpose, but constructing them from an image collection leaves open how to maintain them online, where each observation may both revisit known regions and reveal new content. We introduce S2Tok, a feed-forward framework that maintains a size-adaptive, persistent scene state from uncalibrated image streams. Its central idea is to distinguish updates to the existing representation from selective expansion. A spatially informed transformer integrates each incoming observation with the persistent scene tokens, while a learned admission module selectively expands the representation to limit redundant storage. A hierarchical decoder and Gaussian head convert the evolving state into non-pixel-aligned 3D Gaussians, enabling novel-view rendering without caching previous frames. Experiments across four benchmarks demonstrate competitive streaming rendering quality with compact Gaussian representations. These results support latent spatial tokens as a persistent computational state for online 3D reconstruction, combining learned scene updates with explicit Gaussian rendering.
Fang Li, Jiraphon Yenphraphai, Quentin Herau +5
Applied Intuition · University of Illinois Urbana-Champaign · Purdue University +1