GradSplat: Gradients Inside the Forward Pass for Feed-Forward 3D Gaussian Splatting

Jingong Chen; Linlian Jiang; Guangyuan Li; Xinxin Zuo; Yang Wang; Chulwoo Pack; Kyle Gao; Kwanghee Won

cs.CVOct 28, 2026

GradSplat: Gradients Inside the Forward Pass for Feed-Forward 3D Gaussian Splatting

Authors: Jingong Chen, Linlian Jiang, Guangyuan Li, Xinxin Zuo, Yang Wang, Chulwoo Pack, Kyle Gao, Kwanghee Won

Organizations: South Dakota State University · Aalto University · Concordia University · Mila – Quebec AI Institute ·

Abstract

Feed-forward 3D Gaussian Splatting predicts a scene from a few images, but our predictor is less accurate than its own Gaussians optimized per scene on those images. This gap lies mostly in the shape of the Gaussians, their scale and rotation, since optimizing the shape alone recovers almost all of it and opacity and color recover less. Gradient steps on the input views correct this shape, and the model holds these views at inference. Log-anisotropy correction changes the axis ratios of each Gaussian in log space at constant volume, and in-forward gradient refinement renders the Gaussians into the input views and applies the gradient of the photometric loss with a learned recurrent update, first to opacity and color and then to the shape. GradSplat is a feed-forward model that refines its own Gaussians by learned gradient steps inside its forward pass, with no target view. On both complete six-view RealEstate10K protocols, it has the best PSNR and SSIM among nine methods and keeps its accuracy from $6$ to $42$ input views. Without fine-tuning, it is the most accurate on DL3DV and has the best PSNR and SSIM on ScanNet++. The code will be released at https://github.com/Jingong-Chen/GradSplat/.

Figures & tables

Figure 2: Motivation. A feed-forward model predicts the Gaussians in one pass, but rendered back into an input view they leave a large error (a). Per-scene optimization of their scale and rotation removes most of it, but takes $1500$ gradient steps on the input views (b). GradSplat takes gradient steps with a learned update inside the model and keeps the single pass (c). The maps show the mean absolute error against the input view on one scale. The small pictures show the marked window of the rendering and a window of a novel view. The left column shows three input views and the same two windows in the input view and in the ground truth, and the box on the upper input view marks where the second window lies in the scene. The model of (a) is our base predictor, and the scene is chosen for a visible difference.

Figure 3: Overview. A backbone feeds a camera head, a dense head and a shape head. Each refinement step renders the Gaussians into the input views and applies the learned update $\mathcal{U}_{\psi}$ to the gradient $\bm{g}_{k}$ of $\mathcal{L}_{\mathrm{in}}$ . The four marks in $\mathcal{U}_{\psi}$ are its transformer blocks, with attention within a view (green) and over all views (blue).

Panel of Figure 4 — Figure 4: Qualitative comparison on RealEstate10K. The upper three rows are scenes of the narrow protocol and the lower two scenes of the wide protocol. Each row shows the six input views, one target view rendered by every method and the ground truth. The boxed region is magnified in the opposite corner, and AnySplat, which renders from its own estimated cameras, is not shown.

	Narrow protocol			Wide protocol			PSNR $\uparrow$ for $V$ input views
Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	$V{=}6$	$12$	$24$	$32$	$42$
pixelSplat [ 6 ]	29.143	0.9141	0.1773	19.748	0.6239	0.4594	28.968	$\times$	$\times$	$\times$	$\times$
MVSplat [ 9 ]	30.140	0.9418	0.0774	20.983	0.7566	0.2542	29.954	28.059	26.666	26.379	26.009
TranSplat [ 53 ]	30.464	0.9296	0.1454	19.849	0.6489	0.4265	30.289	$\times$	$\times$	$\times$	$\times$
DepthSplat [ 47 ]	31.418	0.9524	0.0656	23.018	0.8270	0.1937	31.241	29.318	26.978	26.533	25.594
VolSplat [ 39 ]	31.243	0.9524	0.0656	23.042	0.8217	0.2069	31.120	29.285	26.936	26.676	25.829

Table 1: Novel-view synthesis on RealEstate10K. PSNR

\uparrow

, SSIM

\uparrow

and LPIPS

\downarrow

on the narrow and wide protocols with six input views, and PSNR on the view-count index for

V

input views. The upper block gives every method the input poses and the lower block gives none. In the lower block AnySplat and GradSplat also estimate the intrinsics and the target cameras, GradSplat with the constant

\kappa

shared by all scenes. YoNoSplat † keeps the ground-truth intrinsics and target cameras. Best and second best are shaded per block, and a block of two rows shades the best only.

\lx@sectionsign

marks two-view weights,

\ddagger

a training mixture that contains DL3DV and not RealEstate10K, and

\times

weights whose view embedding holds six views and cannot take more.

	DL3DV			ScanNet++ (zero-shot)
Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
pixelSplat [ 6 ]	17.309	0.4671	0.5580	20.889	0.7038	0.4163
MVSplat [ 9 ]	17.390	0.5362	0.4256	18.511	0.6804	0.3411
TranSplat [ 53 ]	16.964	0.4783	0.5388	21.458	0.7225	0.3919
DepthSplat [ 47 ]	20.023	0.6740	0.3248	17.791	0.6803	0.3249
VolSplat [ 39 ]	19.813	0.6579	0.3425	18.745	0.7033	0.3295

Table 2: Transfer to DL3DV and ScanNet++. PSNR

\uparrow

, SSIM

\uparrow

and LPIPS

\downarrow

on the target views of DL3DV and of the validation set of ScanNet++. The upper block applies the weights of Tab. 1 without fine-tuning, and the lower block the weights that each method trains on DL3DV. YoNoSplat is not evaluated on ScanNet++, since its input views rendered back score

13.733

dB there, and our setting requires

20

dB.

\lx@sectionsign

marks two-view weights,

\ddagger

a training mixture that contains DL3DV and ScanNet++ and not RealEstate10K, and – a value that is not available.

Panel of Figure 5 — Figure 5: Qualitative comparison on DL3DV. Each row shows one target view rendered by every method and the ground truth, and the boxed region is magnified as in Fig. 4 . $\star$ marks weights trained on DL3DV, all other columns use RealEstate10K weights, and TranSplat and AnySplat are not shown.

Panel of Figure 7 — Figure 7: Qualitative comparison on ScanNet++ (zero-shot). Every method uses its RealEstate10K weights without fine-tuning. Each row shows one target view of the validation set rendered by every method and the ground truth, and the boxed region is magnified as in Fig. 4 . YoNoSplat and AnySplat are not shown.

		Narrow protocol		Wide protocol
Model	Steps	PSNR $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	LPIPS $\downarrow$
Base predictor	100	28.156	0.0957	23.501	0.1676
+ shape head, $K_{s}{=}3$	130	30.802	0.0783	25.439	0.1436
+ $K_{a}{=}3$ , $K_{s}{=}5$	154	31.844	0.0713	26.008	0.1360
+ D-SSIM, attention	178	33.683	0.0640	26.755	0.1301

Table 3: Training stages. PSNR

\uparrow

and LPIPS

\downarrow

on the complete narrow and wide indices after each of the four training stages. Each row continues training from the row above and adds the listed parts, the last row is GradSplat, and Steps is the cumulative number of training steps in thousands.

			Narrow protocol		Wide protocol
$K_{s}$	$K_{a}$	First	PSNR $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	LPIPS $\downarrow$
0	0	–	28.583	0.0927	23.959	0.1583
3	0	–	31.031	0.0772	25.643	0.1417
0	3	–	30.254	0.0797	24.797	0.1458
1	1	shape	30.260	0.0798	25.131	0.1441
1	1	appearance	30.855	0.0778	25.384	0.1420

Table 4: Step count and order. PSNR

\uparrow

and LPIPS

\downarrow

on the complete indices when the third stage of Tab. 3 is repeated with

K_{s}

shape steps and

K_{a}

appearance steps, from the same weights and for the same number of training steps. First names the kind of step that is taken first, and the last row is the third stage itself.

	RealEstate10K narrow			RealEstate10K wide			DL3DV			ScanNet++
Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
pixelSplat	29.143	0.9141	0.1773	19.748	0.6239	0.4594	17.309	0.4671	0.5580	20.889	0.7038	0.4163
MVSplat	30.140	0.9418	0.0774	20.983	0.7566	0.2542	17.390	0.5362	0.4256	18.511	0.6804	0.3411
TranSplat	30.464	0.9296	0.1454	19.849	0.6489	0.4265	16.964	0.4783	0.5388	21.458	0.7225	0.3919
DepthSplat	31.418	0.9524	0.0656	23.018	0.8270	0.1937	20.023	0.6740	0.3248	17.791	0.6803	0.3249
VolSplat	31.243	0.9524	0.0656	23.042	0.8217	0.2069	19.813	0.6579	0.3425	18.745	0.7033	0.3295

Table A1 : Complete comparison with six input views. PSNR

\uparrow

, SSIM

\uparrow

and LPIPS

\downarrow

on the complete narrow and wide protocols of RealEstate10K, on

139

scenes of DL3DV and on the validation set of ScanNet++, with the weights of Tab. 1 and no fine-tuning. YoNoSplat has no ScanNet++ entry, since its self-reconstruction there is

13.733

dB and our setting requires

20

dB.

\lx@sectionsign

marks two-view weights,

\ddagger

a training mixture that contains DL3DV and ScanNet++ and not RealEstate10K, and – a setting without a reported value.

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 11 — Figure A2: Further scenes of the narrow protocol, continued. Layout as in Fig. A1 .

Panel of Figure 12 — Figure A3: Further scenes of the narrow protocol, continued. Layout as in Fig. A1 .

Panel of Figure 13 — Figure A4: Qualitative comparison on the narrow protocol, continued. Layout as in Fig. A1 .

Panel of Figure 14 — Figure A5: Qualitative comparison on the wide protocol. Layout as in Fig. 4 , with the PSNR of the scene beneath each rendering.

Panel of Figure 15 — Figure A6: Qualitative comparison on the wide protocol, continued. Layout as in Fig. A5 .

Panel of Figure 16 — Figure A7: Renderings for an increasing number of input views. One target view of two scenes of the view-count index, rendered by DepthSplat, YoNoSplat and GradSplat from $V$ input views, with the PSNR of the scene beneath each rendering. The target is the same in every column of a scene, and a larger $V$ only adds input views.

Method	RealEstate10K	ACID	DTU
ReSplat § [ 46 ]	30.613	31.285	13.958
YoNoSplat [ 48 ]	26.574	26.783	12.572
AnySplat ‡ [ 17 ]	16.579	19.924	11.457
GradSplat	27.301	27.898	10.754

Appendix

Table A2 : Two input views. PSNR

\uparrow

on the complete two-view indices of RealEstate10K, ACID and DTU, with RealEstate10K weights and no fine-tuning. ReSplat and GradSplat are evaluated at

256\times 256

pixels and YoNoSplat and AnySplat at

224\times 224

. At

256\times 256

pixels the position embeddings of the transformer blocks of GradSplat are interpolated bicubically from

14\times 14

16\times 16

tokens.

\lx@sectionsign

marks two-view weights and

\ddagger

a training mixture that contains DL3DV and ScanNet++ and not RealEstate10K.

	Narrow protocol			Wide protocol
Setting	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
AnySplat ‡	19.629	0.6160	0.2949	15.525	0.4250	0.4414
YoNoSplat †	27.816	0.8982	0.0848	21.861	0.7310	0.1868
GradSplat, $\kappa{=}4.04$	26.914	0.8863	0.1053	20.901	0.6945	0.2426
GradSplat, $\kappa{=}1$	26.538	0.8724	0.1711	20.669	0.6638	0.3805
GradSplat, GT targets	19.794	0.6501	0.2345	17.782	0.5524	0.3193

Appendix

Table A3 : Pose-free settings. PSNR

\uparrow

, SSIM

\uparrow

and LPIPS

\downarrow

on the complete narrow and wide protocols without input poses. The upper block repeats the lower block of Tab. 1 , and the lower block is the final model with

\kappa=1

and with ground-truth target cameras and aligned scale.

\dagger

marks ground-truth intrinsics and target cameras and

\ddagger

a training mixture that contains DL3DV and ScanNet++ and not RealEstate10K.

Panel of Figure 19 — Figure A8: Renderings after every step. One target view of three scenes of the narrow protocol before any step, after each of the $K_{a}$ appearance steps and each of the $K_{s}$ shape steps of the final model, and in the ground truth. The PSNR of this view is beneath each rendering.

Noise	$0.00$	$0.25$	$0.50$	$0.593$	$0.794$	$1.00$
PSNR $\uparrow$	33.683	30.289	27.001	26.184	24.864	23.888

Appendix

Table A4 : Rotation noise on the input poses. PSNR

\uparrow

of the final model on the complete narrow protocol when every ground-truth input pose but the first is rotated by the given angle about a random axis. The columns at

0.593^{\circ}

and

0.794^{\circ}

are the mean rotation errors of the poses that YoNoSplat and GradSplat predict, and the column at

0^{\circ}

equals the entry of Tab. 1 .

	Narrow protocol		Wide protocol
Gaussians after	Target	Input	Target	Input
No step	29.731	29.693	24.450	25.512
Appearance step 1	31.501	31.667	25.213	26.668
Appearance step 2	31.969	32.237	25.379	26.931
Appearance step 3	31.813	32.082	25.260	26.738
Shape step 1	32.807	33.392	26.081	28.346

Appendix

Table A5 : Refinement steps. PSNR

\uparrow

of the Gaussians of the final model after every refinement step at inference, on the target views and on the input views rendered back, over the complete narrow and wide indices. The first row is the prediction before any step, and the last row is the output of GradSplat.

$\bm{\mu}$	$\bm{s}$	$\bm{R}$	$\alpha$	$\bm{c}$	DoF	PSNR $\uparrow$
			✓		1	31.401
				✓	12	32.852
			✓	✓	13	32.767
		✓			4	28.849
	✓				3	32.650
	✓	✓			7	34.207

Appendix

Table A6 : Per-scene optimization by parameter group. PSNR

\uparrow

on the target views after

1500

Adam [ 20 ] steps on the input views that update the ticked groups and keep the others as predicted. Mean over

8

scenes of the narrow protocol, starting from a base predictor of an earlier training run, our model without the shape head and the refinement, which scores

27.278

. The groups are centers

\bm{\mu}

, scale

\bm{s}

, rotation

\bm{R}

, opacity

\alpha

and color

\bm{c}

, and DoF counts the optimized degrees of freedom per Gaussian. The learning rate is

3\times 10^{-3}

, and

10^{-1}

for color alone.

Optimized groups	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	HF
None	29.745	0.9357	0.0852	0.857
Opacity and color	32.975	0.9590	0.0860	1.048
Scale and rotation	34.275	0.9694	0.0668	0.902
All but the centers	34.497	0.9710	0.0542	0.991

Appendix

Table A7 : Per-scene optimization by group on an intermediate model. PSNR

\uparrow

, SSIM

\uparrow

, LPIPS

\downarrow

and the high-frequency ratio HF on the target views after

1500

Adam steps on the input views, as a mean over the eight scenes of Tab. A6 . The model already has the shape head and the shape steps, and every row uses a learning rate of

3\times 10^{-3}

Term	Weight	Applied to
Mean squared error	$1$	target views, every iterate
LPIPS	$0.05$	target views, every iterate
D-SSIM	$0.8$ in stage 4	target views, every iterate
Opacity regularizer	$0.01$	output $\theta_{M}$
Depth smoothness	$0.05$	predicted depth
Pose	$0.1$	camera head

Appendix

Table A8 : Terms of the training loss. Weight of every term and the quantity it is applied to. The two distillation weights are the initial values, which Eq. A1 then adjusts.

Stage	Added	Steps	GPUs	Hours	Rate	Params.
1	base predictor	$100{,}000$	$16$	$15.8$	$2.8\times 10^{-4}$	$1148.6$
2	shape head, steps	$30{,}000$	$16$	$6.2$	$8\times 10^{-5}$	$1151.0$
3	appearance steps	$24{,}000$	$16$	$7.1$	$8\times 10^{-5}$	$1155.1$
4	D-SSIM, attention	$24{,}000$	$4$	$21.4$	$8\times 10^{-5}$	$1224.8$
	Total	$178{,}000$		$50.5$

Appendix

Table A9 : Training schedule. Steps, number of GH200 GPUs, duration, learning rate of the heads and updates, and trainable parameters in millions. Stage

2

adds the shape head with three shape steps, stage

3

runs

K_{s}

shape steps and adds the

K_{a}

appearance steps, and stage

4

adds D-SSIM and the transformer blocks of the two updates.

Method	Setting	Printed	Measured
pixelSplat	RealEstate10K, $V=6$	$28.95$	$28.396$
MVSplat	RealEstate10K, $V=6$	$29.13$	$29.294$
TranSplat	RealEstate10K, $V=6$	$29.62$	$29.616$
DepthSplat	RealEstate10K, $V=6$	$30.52$	$30.784$
ReSplat	RealEstate10K, $V=2$	$29.75$	$30.613$
ReSplat	ACID, $V=2$	$29.87$	$31.285$

Appendix

Table A10 : Calibration of the setting. PSNR

\uparrow

printed in the six-view column of the VolSplat paper for the upper block and in the ReSplat paper for the lower block, and the value re-measured in our setting. The upper block uses six input views on

200

scenes of the narrow protocol at

256\times 256

pixels, and the lower block the two-view indices of Tab. A2 .

3D Gaussian Splatting

3D Gaussian Splatting represents scenes as optimizable 3D Gaussian primitives for fast, photorealistic rendering from new viewpoints.

See all 1,027 papers

LoCoSplat: Real-Time Feed-Forward 3D Gaussian Splatting with Minimal 3D Reasoning
Oct 3, 2026Sinan Wang, Jinjin He, Yuchen Sun +33D Gaussian Splatting Multi-View 3D Reconstruction
Georgia Institute of Technology
ZipSplat: Fewer Gaussians, Better Splats
Jun 3, 2026Alexander Veicht, Sunghwan Hong, Dániel Baráth +13D Gaussian Splatting Multi-View 3D Reconstruction
ETH Zürich · Microsoft
ForeSplat: Optimization-Aware Foresight for Feed-Forward 3D Gaussian Splatting
May 21, 2026Yuke Li, Weihang Liu, Cheng Zhang +83D Gaussian Splatting Meta-Learning
ShanghaiTech University · GGU Technology Co., Ltd · Stereye

Oct 3, 2026J/K move · Enter open · S save

LoCoSplat: Real-Time Feed-Forward 3D Gaussian Splatting with Minimal 3D Reasoning

Sinan Wang, Jinjin He, Yuchen Sun, Duowen Chen+2

Georgia Institute of Technology

3D Gaussian Splatting Multi-View 3D Reconstruction Feed-Forward 3D Reconstruction 3DGS Acceleration

Open paper AskPDF arXiv

Feed-forward 3D Gaussian Splatting (3DGS) increasingly aggregates multi-view evidence with heavy learned 3D networks. We propose LoCoSplat (Local-Context Splatting), motivated by the observation that a Gaussian is a local primitive: once depth is predicted, what the 3D stage must add (scale, rotation, opacity) depends on the point cloud around each anchor, and a fixed local average of that neighbourhood is enough to supply it, no heavy network required. LoCoSplat realises exactly this average: it splats a 16-d linear projection of the point features into a fine and a coarse grid and reads both back at each anchor with a 0.14M-parameter pointwise MLP; with no learned 3D network and no dynamic sparse computation, its whole encoder runs as one fp16 CUDA graph. On RealEstate10K, LoCoSplat outperforms every prior feed-forward method on PSNR, SSIM, and LPIPS at 6, 12, and 24 views, with a margin that widens as views densify (+3.3 PSNR over VolSplat, the prior voxel-aligned state of the art, at 24 views) and grows further under zero-shot transfer to ACID and fine-tuning on ScanNet. It reconstructs a 6-view scene in 33 ms on one NVIDIA RTX PRO 6000 GPU, the fastest of seven feed-forward methods and

4.2\times

faster than the previous state of the art, trains

2.7\times

faster (

5.9\times

at 24 views), and uses

6.7\times

less inference memory.

Abstract

3D Gaussian Splatting

Explore similar work

LoCoSplat: Real-Time Feed-Forward 3D Gaussian Splatting with Minimal 3D Reasoning

ZipSplat: Fewer Gaussians, Better Splats

ForeSplat: Optimization-Aware Foresight for Feed-Forward 3D Gaussian Splatting