cs.CVOct 16, 2026

Frame-Consistent Feed-Forward 3D Gaussian Splatting from Sparse Unposed Images

Authors: Viruthshaan Vinasirajan, Ranga Rodrigo

Organizations: University of Moratuwa

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Feed-forward Gaussian models have made novel view synthesis almost instantaneous. Feed-forward methods predict Gaussians directly, either in two stages or in a single merged pass. Using two separately trained models is attractive because each can specialize in its own objective: one for accurate pose estimation and the other for high-quality reconstruction, rather than forcing a single jointly trained network to compromise between the two. A natural instantiation pairs a frozen pose estimator, VGGT, with a frozen reconstruction backbone, TokenGS. Prior work, such as VGGT-X and 3R-GS, also adopts a two-stage design, but attributes the resulting quality degradation to noisy pose estimates and introduces learned refinement. We show that this is not the primary source of the problem. Through per-camera analysis across 107 scenes from two datasets, DL3DV and MipNeRF360, using COLMAP poses as the reference, we find that the rotation error after alignment is small (median 1.6 degrees), whereas the translation scale is offset by a large, dataset-consistent factor. Controlled ablations over 140 DL3DV scenes show that the primary obstacles are two integration failures rather than pose error: a coordinate-frame inconsistency and a translation-scale mismatch, both of which can be corrected deterministically. With these fixes, a frozen VGGT driving a frozen TokenGS achieves 22.47 PSNR, within 0.51 dB of the COLMAP reference, with no additional training, when poses are estimated from all scene views. In the sparse setting, where reconstruction uses only four context images under the standard pose-free protocol, the same pipeline matches this all-view quality, yielding a fully COLMAP-free pipeline without any refinement step.

Figures & tables

Figure 1: Our training-free composition pipeline. A frozen VGGT estimates camera poses, and a frozen TokenGS reconstructs 3D Gaussians; the two are connected by three deterministic, training-free operations. All input views (the four context images and the target) pass through VGGT in a single forward pass so that every camera pose lies in one shared coordinate frame. The predicted poses are then corrected by convention conversion, frame-consistent inference, and scale correction before conditioning TokenGS. Reconstruction uses only the four context images (bottom path); the target image is included solely to estimate its pose in the shared frame. No component is trained or fine-tuned.
Figure 2: VGGT vs. COLMAP pose error decomposition. Left: rotation error after convention alignment is small (median 1.60∘1.60^{\circ} on DL3DV, 1.32∘1.32^{\circ} on MipNeRF360). Right: the translation scale ratio is large and consistent (median 7.07.0 on DL3DV), far from the ideal ratio of 1.01.0 . For typical scenes, rotation error is small; the dominant systematic offset is scale.
Pose method#ViewsPSNR ↑\uparrowLPIPS ↓\downarrowSSIM ↑\uparrowPose Est. Time ↓\downarrowTokenGS Time ↓\downarrowEnd-to-end Time ↓\downarrow
COLMAP (ceiling)∼\sim 35022.980.3280.737∼\sim 9 min †0.155 s∼\sim 9 min
VGGT (joint eval views)5022.470.3330.7156.64 s0.155 s6.79 s
VGGT (sparse, per-target)522.440.3300.7140.598 s0.155 s0.753 s
Table 1: Main results on 140 DL3DV scenes (960 ×\times 540, DL3DV images_4 ). Deterministic fixes bring frozen VGGT poses to within 0.51 dB of the COLMAP ceiling with no training. All methods reconstruct Gaussians from the same four context views and the same image resolution; rows differ only in how poses are estimated. COLMAP uses the full capture ( ∼\sim 350 frames). “Joint eval views” runs VGGT once on the 50 TokenGS eval frames (4 context ∪\cup 50 targets). “Sparse, per-target” runs VGGT on five views (4 context + one target) per novel view. End-to-end time is pose estimation + TokenGS.
Figure 3: Qualitative comparison on DL3DV. From left to right: naïve integration, with convention conversion added, with frame-consistent inference added, our full method (with scale correction), the COLMAP-pose reference, and ground truth. Insets show a magnified region for closer comparison. Naïve integration collapses under the coordinate-frame mismatch; each deterministic fix progressively restores structure, and our full pipeline produces renderings visually indistinguishable from the COLMAP reference, using no ground-truth poses.
ConfigurationPSNR ↑\uparrow
Naïve integration (frame mismatch)13.67
+ Convention conversion20.38
+ frame-consistent inference22.24
+ Scale correction ( ×7.0\times 7.0 )22.47
COLMAP ceiling22.98
Table 2: Component ablation on 140 DL3DV scenes. The frame-consistency fix accounts for the majority of the recovery.
ConfigurationPSNR ↑\uparrow
COLMAP intrinsics + COLMAP extrinsics22.98
VGGT intrinsics + COLMAP extrinsics22.94
Table 3: VGGT intrinsics on 140 DL3DV scenes. Replacing COLMAP intrinsics with VGGT’s, keeping COLMAP extrinsics, leaves quality essentially unchanged.
More in this topic

Feed-Forward 3D Reconstruction

Feed-forward 3D reconstruction predicts geometry, cameras, and renderable scene representations directly from images without per-scene optimization.

See all 313 papers

Explore similar work

CardsList
Feb 28, 2026

TokenSplat: Token-aligned 3D Gaussian Splatting for Feed-forward Pose-free Reconstruction

We present TokenSplat, a feed-forward framework for joint 3D Gaussian reconstruction and camera pose estimation from unposed multi-view images. At its core, TokenSplat introduces a Token-aligned Gaussian Prediction module that aligns semantically corresponding information across views directly in the feature space. Guided by coarse token positions and fusion confidence, it aggregates multi-scale contextual features to enable long-range cross-view reasoning and reduce redundancy from overlapping Gaussians. To further enhance pose robustness and disentangle viewpoint cues from scene semantics, TokenSplat employs learnable camera tokens and an Asymmetric Dual-Flow Decoder (ADF-Decoder) that enforces directionally constrained communication between camera and image tokens. This maintains clean factorization within a feed-forward architecture, enabling coherent reconstruction and stable pose estimation without iterative refinement. Extensive experiments demonstrate that TokenSplat achieves higher reconstruction fidelity and novel-view synthesis quality in pose-free settings, and significantly improves pose estimation accuracy compared to prior pose-free methods. Project page: https://kidleyh.github.io/tokensplat/.

Yihui Li, Chengxin Lv, Zichen Tang +2

State Key Laboratory of Complex and Critical Software Environment, Beijing, China · School of Computer Science and Engineering, Beihang University, China · School of Artificial Intelligence, Beihang University, China

3D ReconstructionMulti-View 3D ReconstructionFeed-Forward 3D Reconstruction3D Gaussian SplattingCamera Pose Estimation
AskPDF
Oct 7, 2026

DeltaSplat: Iterative Gaussian Refinement for Pose-Free Feed-Forward 3D Gaussian Splatting

Pose-free feed-forward 3D Gaussian Splatting (3DGS) reconstructs a scene from sparse, unposed images in a single network pass, removing the need for camera calibration and per-scene optimization. However, camera estimation errors propagate into the predicted Gaussians and compound the geometric and photometric inaccuracies of single-pass prediction. To correct these errors, we introduce DeltaSplat, a lightweight Gaussian refinement module for pose-free feed-forward 3DGS. It iteratively renders the current Gaussians at the input context views and predicts per-Gaussian updates from the resulting residuals. A 2D residual alone, however, underdetermines the 3D correction. DeltaSplat therefore conditions each update on per-pixel Plücker rays and rendered depth as a soft geometric prior. A dual-branch convolutional mixer efficiently encodes these inputs, and per-attribute heads decode the fused features into position, opacity, and color updates. The module adds only ~2.2% parameters to the backbone and remains fully feed-forward at inference. On DL3DV, DeltaSplat reaches 26.64 dB PSNR in the pose-free setting, improving its state-of-the-art backbone by 1.75 dB and surpassing even baselines supplied with ground-truth cameras; consistent gains hold across 6-24 views and all camera regimes.

Chanung Park, Seunghyeon Song, Joo Chan Lee +2

Department of Electrical and Computer Engineering, Sungkyunkwan University · Department of Artificial Intelligence, Sungkyunkwan University · Department of Artificial Intelligence, Yonsei University

3D Gaussian Splatting3D ReconstructionFeed-Forward 3D ReconstructionSparse-View 3D Reconstruction
AskPDF
Oct 6, 2026

DensiTok: Making Feed-Forward 3D Gaussian Splatting See More Views Than It Is Given

Feed-forward 3D Gaussian Splatting (3DGS) reconstructs a scene in a single forward pass, replacing per-scene optimization with a network trained across many scenes. Its quality, however, degrades sharply as the number of input images drops. The bottleneck is upstream of the reconstruction heads: from a few unposed views, the internal representation they read carries no evidence for unobserved regions, leaving holes, floaters, and blur. The common remedy supplies that evidence as pixels, synthesizing extra views with an image or video generator and re-encoding them, which is costly and not 3D-consistent by construction. We instead densify the evidence itself. We present DensiTok, a plug-in module for pretrained feed-forward 3DGS models that densifies their internal geometry tokens directly, making a frozen backbone behave as though it had observed many more views than it was given. DensiTok compresses those tokens into a compact latent space, completes the latents of the unobserved viewpoints in a single flow-matching step conditioned on camera geometry, and decodes them back into tokens that the original reconstruction heads. The same module design can be integrated into different pretrained predictors while keeping each backbone and its reconstruction heads frozen. Completion in a low-dimensional latent space requires no image synthesis or additional encoder passes. Across three pretrained backbones and two benchmarks, DensiTok consistently improves sparse-view reconstruction and recovers much of the gap to dense-view reconstruction.

Minhyeok Lee, Jungho Lee, Minseok Kang +3

Yonsei University · NAVER AI Lab · Korea Institute of Science and Technology (KIST)

3D Gaussian SplattingSparse-View 3D ReconstructionFeed-Forward 3D Reconstruction
AskPDF