RADIANT: Object-Centric Feed-Forward Global Illumination

Sarah Zhang; Zhiyang Dou; Minghao Guo; Benjamin Tod Jones; Wojciech Matusik

cs.CVOct 23, 2026

RADIANT: Object-Centric Feed-Forward Global Illumination

Authors: Sarah Zhang, Zhiyang Dou, Minghao Guo, Benjamin Tod Jones, Wojciech Matusik

Organizations: MIT CSAIL

Abstract

Feed-forward neural renderers synthesize global illumination across novel 3D scenes without per-scene optimization or runtime ray tracing. Existing transformer-based renderers tokenize individual triangles, so self-attention scales quadratically with mesh resolution. Yet macroscopic radiative transport is governed by the underlying surface geometry and object arrangements rather than a particular mesh discretization; finer tessellation can thus inflate token counts without changing the underlying transport physics, motivating a discretization-agnostic and object-centric representation. We introduce RADIANT (RADiance-based Inter-Actor Neural Transformer), an object-centric architecture for feed-forward global illumination from point inputs. By representing each object with a compact budget of 32 surface-patch tokens, RADIANT decouples attention sequence length from polygon counts. In a single network, an inter-object scene transformer models 3D radiative exchange and a view decoder renders images at target camera poses. To capture light transport without materializing O(N^2) bias matrices, invariant form-factor attention embeds geometric distances and surface orientations directly into query-key dot products as SE(3)-invariant inner products weighted by learned area and material gains. For primary visibility without mesh connectivity, camera rays cross-attend scene tokens guided by depth-tested point splats in a shared spherical frame. Across a 1.03M-scene benchmark under matched model capacity, RADIANT reaches 25 dB tone-mapped PSNR in 5x fewer training steps than RenderFormer, and achieves 32.2 dB at 600k steps (+3.3 dB over RenderFormer and +7.6 dB over RenderFormer-V2). We further demonstrate the scalability of RADIANT, which completes a 512x512 forward pass of a 500-object scene in 0.83 s, with computational cost scaling with the number of objects rather than polygons.

Figures & tables

Method	Input	Triangle attention	tokens	attention	Primary visibility	Per-scene optimization	Steps to 25 dB
			Transport
Neural Radiosity	mesh †	—	—	—	ray tracing	required	—
RenderFormer	triangles	all pairs	$T$	$O(T^{2})$	learned	none	80k
RenderFormer-V2	triangles	windowed	$T$	$O(T^{2})$	learned	none	n/r
RenderFormer++	triangles	per object	$8N$	$O(N^{2})$	G-buffer	none	—
RADIANT (ours)	point inputs	none	$\bm{32N}$	$\bm{O(N^{2})}$	point splat	none	16k ( $5\times$ )

Table 1: Architectural and computational comparison. Input : input geometric modality. Triangle attention : extent of self-attention over individual triangles. Transport tokens and Transport attention : sequence length and computational complexity of global light exchange (

T

triangles,

N

scene entities including walls and lights). Primary visibility : mechanism resolving camera-ray intersections. Steps to 25 dB : optimizer steps required to attain 25 dB tone-mapped PSNR on the 1.03M-scene benchmark under our matched protocol.

Figure 1: Overview of RADIANT , a single feed-forward network. View-independent part (computed once per scene): the point inputs are pooled into 32 patch tokens per object, and a scene transformer resolves 3D light transport with invariant form-factor attention. View-dependent part (computed per query viewpoint): the view decoder (predictor transformer), guided by rasterized geometry channels and spherical RoPE, cross-attends camera rays to scene tokens to decode the HDR image.

Model	PSNR $\uparrow$ (per-image / pooled)	LPIPS $\downarrow$	$\ell_{1}(\log)$ $\downarrow$
RADIANT (ours)	32.22 / 27.65	0.042	0.012
RenderFormer	28.95 / 25.80	0.077	0.019
RenderFormer-V2	24.60 / 23.05	0.117	0.046

Table 2: Held-out rendering quality under the matched-budget protocol (

512^{2}

, 4,112 images). Each model is evaluated at its best checkpoint, selected over all evaluated checkpoints on the same split (Appendix C ). Bold indicates best performance.

Figure 2: Learning curves of RADIANT (Ours), RenderFormer (RF), and RenderFormer-V2 (RF-V2) on 4,112 held-out images: per-image PSNR (left) and LPIPS (right). Top axis: 8

\times

A100 days at 1.28 s/step, the median step time of our model’s training run. Labels in RADIANT-equivalent time, not per-baseline wall-clock time. Dashed: best values of RF and RF-V2 (Tab. 2 ).

		RADIANT (ours)		RenderFormer		RenderFormer-V2
Entities per scene	Images	PSNR $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	LPIPS $\downarrow$
$\leq 7$	796	32.35	0.037	29.22	0.064	24.46	0.104
8–10	1,376	32.15	0.041	29.00	0.075	24.52	0.116
11–13	1,336	32.44	0.043	29.11	0.079	24.80	0.118
$\geq 14$	604	31.64	0.051	28.12	0.094	24.35	0.136

Table 3: Held-out quality by scene complexity: the 4,112 evaluation images grouped by the number of entities the renderer sees (foreground objects, walls, and lights; the corpus spans 3 to 18). Per-image PSNR (dB) and LPIPS (AlexNet), with all three models at matched step 600k. The three models’ per-image scores are aligned by image index, which identifies the same procedural scene across models (Appendix D ).

Test set	Scenes	Entities	RADIANT (ours)	RenderFormer	RenderFormer-V2
Held-out split	1,028	3–18	32.21 / 0.042	28.95 / 0.077	24.57 / 0.117
A : unseen assets	250	3–17	32.59 / 0.041	29.36 / 0.073	24.90 / 0.110
B : 25–50 entities	280	25–50	25.32 / 0.095	22.50 / 0.171	20.06 / 0.227
C : object sizes of B	250	3–18	34.95 / 0.024	32.35 / 0.044	26.44 / 0.079

Table 4: Unseen assets and larger scenes (Appendix I ): per-image PSNR / LPIPS of the step-600k checkpoints; four views per scene. The held-out row repeats the 600k evaluation of Tab. 8 . Bold: best per row.

Figure 3: Qualitative comparisons on 24 crowded held-out scenes at matched step 600k. Rows in each group of eight scenes: ground truth (GT), RADIANT (Ours), RenderFormer (RF), and RenderFormer-V2 (RF-V2).

Figure 4: Rendering from partial point clouds (Appendix J ) on two held-out scenes (rows), three novel views each, none used for fine-tuning. Each group of three columns: ground truth, the simulated single-view partial point cloud, and RADIANT after 400 fine-tuning steps on four reference views.

		Original views			New views
Point inputs	Fine-tuning	PSNR $\uparrow$	LPIPS $\downarrow$	$\ell_{1}(\log)$ $\downarrow$	PSNR $\uparrow$	LPIPS $\downarrow$	$\ell_{1}(\log)$ $\downarrow$
Complete	none	33.87	0.034	0.004	34.24	0.037	0.004
Partial (single-view scan)	none	26.76	0.085	0.010	26.33	0.100	0.010
Partial (single-view scan)	400 steps	37.51	0.015	0.003	30.30	0.065	0.006
Complete	400 steps	39.02	0.009	0.002	33.58	0.038	0.005

Table 5: Rendering from simulated single-view scans: the two held-out scenes of Fig. 4 , RADIANT at step 600k (per-image PSNR, LPIPS, and

\ell_{1}

in log radiance; the evaluator of Sec. 4.1 ). Original views : the four views of each scene, which supervise the fine-tuning (eight images); new views : eight further views per scene from the training camera distribution, never used for fitting (16 images).

Variant	PSNR $\uparrow$	LPIPS $\downarrow$	$\ell_{1}(\log)$ $\downarrow$	$\Delta$
Full model	$\mathbf{29.96}\pm 0.01$ / 26.16	0.059	0.019	—
No invariant form-factor attention (§ 3.4 )	$29.43\pm 0.07$ / 25.92	0.063	0.021	$-0.52\pm 0.08$
No inter-object scene attention (§ 3.1 ) †	$27.50\pm 0.66$ / 24.89	0.072	0.036	$-2.46\pm 0.66$
No rasterizer channels (hit geometry, routing bias)	$26.84\pm 0.07$ / 24.41	0.092	0.025	$-3.12\pm 0.08$

Table 6: Ablation of key mechanisms (

256^{2}

, 100k steps, mean over seeds 42–44; † seeds 42–47). PSNR per-image (mean

\pm

sample standard deviation) / pooled;

\Delta

: per-image change from the full model, paired within each seed ( † seeds 45–47 against the full model’s three-seed mean). Extended ablations in Tab. 19 .

Appendix figures & tables19 assets

Supplementary material from the paper’s appendix.

Appendix

Hyperparameter	Value	Hyperparameter	Value
Hardware	NVIDIA A100 GPUs	Gradient clipping	Global norm 1.0
Global batch size	24	Mixed precision	bf16 autocast, fp32 master weights
Optimizer	AdamW, $(\beta_{1},\beta_{2})=(0.9,0.999)$	Loss function	MAE in $\log(1{+}L)+0.05\,\text{LPIPS}$
Peak learning rate	$5\times 10^{-5}$	LPIPS backbone	AlexNet
Warm-up	8,000 linear steps	Image resolution	$512\times 512$
Learning rate decay	Cosine to $10^{-6}$ at 100k/200k/300k/600k	Data augmentation	None (1 view per sample)

Appendix

Table 7: Training hyperparameters of the main comparison. The ablations (Sec. 4.3 ) use

256^{2}

images, 100k steps, seeds 42–44 (42–47 without inter-object attention), and 2,948 held-out images.

	RADIANT (ours)		RenderFormer		RenderFormer-V2
Step	PSNR $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	LPIPS $\downarrow$
6k	23.45	0.156	16.15	0.350	—	—
16k	25.91	0.103	19.02	0.244	16.39	0.326
25k	26.97	0.089	20.42	0.199	17.13	0.284
50k	28.63	0.070	23.25	0.154	19.98	0.204
80k	29.70	0.061	25.06	0.133	21.75	0.175

Appendix

Table 8: Evaluation points of Fig. 2 (step 25k and every 50k) and the early steps quoted in Sec. 4.2 (6k, 16k, 80k): per-image PSNR (dB) and LPIPS (AlexNet) on the 4,112 held-out images. —: not evaluated. Bold: best value of each run.

Scenes (after exclusions)	1,027,596
Images per scene / total images	4 / 4,110,384
Resolutions / samples per pixel	$512^{2}$ / 512 and $256^{2}$ / 256
Training / held-out scenes	1,026,568 / 1,028 (4,112 images)
Foreground objects per scene	1–6 (from 18,085 meshes)
Entities per scene (objects, walls, lights)	mean 10.1; p5/p50/p95 = 5/10/15; max 18
Emissive entities per scene	mean 4.5; range 1–8

Appendix

Table 9: Corpus statistics. Entity, light, and triangle statistics are from a random sample of 40,000 training scenes (40 data files of 1,000 scenes); p5/p50/p95 are percentiles.

	RADIANT			RenderFormer
Component	10	50	200	10 (flash)	50 (flash)	200 (flash)	200 (sdpa)
Encoder	7.3	16.0	48.0	0.3	0.9	3.0	3.0
Transport	9.0	16.0	104.0	19.2	116.1	1,203.0	2,959.9
View decoder + DPT	50.2	55.7	75.4	15.6	26.8	71.4	101.5
Other	7.5	7.7	20.6	1.2	1.4	2.6	2.9
Total	74.0	95.4	248.0	36.3	145.3	1,280.0	3,067.3

Appendix

Table 10: Inference time per component (ms, median of 20 forwards, CUDA events around the top-level modules) of the two 206M models at

512^{2}

on one H100 with random weights, batch size 1, and bf16, for 10, 50, and 200 objects (2,048 point inputs or 512 triangles per object). Encoder: point encoder of RADIANT or triangle attribute encodings of RenderFormer; transport: scene transformer or triangle transformer; view decoder + DPT: view-dependent transformer and DPT head; other: rasterization, conditioning, and the remaining operations. RenderFormer with FlashAttention (flash) and, at 200 objects, PyTorch attention (sdpa). Profiled separately from Tab. 11 .

Objects (triangles)	time	memory	sdpa	flash	memory	sdpa	flash
	RADIANT		RenderFormer			Speedup
1 (512)	70.4	4,255	19.4	30.2	1,631	$0.28\times$	$0.43\times$
10 (5,120)	73.3	4,299	33.6	35.9	1,644	$0.46\times$	$0.49\times$
50 (25,600)	95.7	4,480	260.8	145.4	1,954	$2.7\times$	$1.5\times$
100 (51,200)	136.8	5,863	848.1	398.5	2,859	$6.2\times$	$2.9\times$
200 (102,400)	248.5	10,898	3,065.2	1,288.9	4,669	$12.3\times$	$5.2\times$

Appendix

Table 11: Single-forward inference time (ms, median) and peak memory (MiB) vs. number of objects (

512^{2}

, batch size 1, H100), from 2,048 point inputs per object for RADIANT and 512 triangles per object for RenderFormer (surface sampling not timed). RenderFormer runs with PyTorch SDPA or FlashAttention (its memory is that of the SDPA run); speedups divide its time by RADIANT’s.

Figure 5: Inference forward latency (left) and training step latency (right) of the two 206M models versus the number of objects (

512^{2}

, batch size 1, bf16 autocast, one H100 80 GB GPU; 512 triangles per object for RenderFormer, 2,048 point inputs per object for RADIANT). RenderFormer is evaluated with PyTorch SDPA (solid) and FlashAttention (dashed). Training step includes forward pass, loss computation, backward pass, and AdamW update. In complex scenes with 50–500 objects, RADIANT provides up to

22.2\times

faster inference than RenderFormer with PyTorch SDPA and

8.5\times

faster than RenderFormer with FlashAttention. Training steps of RADIANT are

4.3

–

18.0\times

faster than RenderFormer’s with SDPA and

2.3

–

7.5\times

faster than with FlashAttention at 50–200 objects; at 500 objects RADIANT exceeds the 80 GB of GPU memory in training, whereas RenderFormer needs 32.7 GiB. Inference latencies are tabulated in Tab. 11 .

Triangles per object (total)	RADIANT	RF sdpa	RF flash	Speedup (sdpa)	Speedup (flash)
512 (5,120)	72.7	33.5	35.5	$0.46\times$	$0.49\times$
2,048 (20,480)	72.7	182.0	109.8	$2.5\times$	$1.5\times$
8,192 (81,920)	72.7	2,030	872.5	$27.9\times$	$12.0\times$
32,768 (327,680)	72.7	29,965	11,309	$411.9\times$	$155.5\times$

Appendix

Table 12: Single-forward inference time (ms, median) for ten objects as their tessellation grows (

512^{2}

, batch size 1, H100, settings of Tab. 11 ). RADIANT reads 2,048 point inputs per object regardless of tessellation. Speedups divide RenderFormer’s time by RADIANT’s.

Model	Inference (ms)	Training step (s)	Steps to 25 dB	Time to 25 dB (h)
RADIANT	130	1.09	16k	4.8
RenderFormer	42 / 49	0.91 / 0.72	80k	15.9–20.3

Appendix

Table 13: Cost at the scale of the training scenes on one A100 80 GB GPU (corpus scenes). Inference: median time per image on the held-out scenes, batch size 1, bf16 autocast as in Tab. 11 ; RenderFormer with SDPA / FlashAttention. Training step: median time per optimizer step with three training scenes from the corpus (the per-GPU share of the global batch of 24) on a single A100 GPU, each model in its training configuration: RADIANT under bf16 autocast, RenderFormer with / without activation recomputation. Time to 25 dB: steps to 25 dB (Tab. 1 ) multiplied by per-step latency on an 8-GPU node without communication overhead.

		RenderFormer		Speedup
Objects	RADIANT	sdpa	flash	sdpa	flash
$128^{2}$
1	28.6	17.4	28.9	$0.61\times$	$1.01\times$
10	30.3	25.4	33.1	$0.84\times$	$1.09\times$
50	47.6	245.7	137.8	$5.2\times$	$2.9\times$
100	80.8	823.6	384.7	$10.2\times$	$4.8\times$

Appendix

Table 14: Single-forward inference time (ms, median) at

128^{2}

and

256^{2}

for the settings of Tab. 11 . Speedups divide RenderFormer’s time by RADIANT’s.

Patch tokens per object $M$	PSNR $\uparrow$	LPIPS $\downarrow$	$\ell_{1}(\log)$ $\downarrow$	Step time
8	$29.45\pm 0.02$ / 25.93	0.062	0.021	$0.79\times$
16	$\mathbf{29.47}\pm 0.02$ / 25.93	0.062	0.021	$0.83\times$
32 (default)	$29.43\pm 0.07$ / 25.92	0.063	0.021	$1.00\times$
64	$29.44\pm 0.06$ / 25.92	0.063	0.021	$1.09\times$

Appendix

Table 15: Token budget per object, on the ablation subset and held-out split of Tab. 6 under its 100k-step schedule, on the model without invariant form-factor attention (Sec. 3.4 ), mean over seeds 42–44: PSNR per-image (mean

\pm

sample standard deviation) / pooled, LPIPS, and

\ell_{1}

in log radiance. Step time is the relative training step latency normalized to

M{=}32

	$128^{2}$			$256^{2}$			$512^{2}$
Batch size	1	8	32	1	8	32	1	8
RenderFormer (sdpa)	21.5	13.7	12.8	22.5	15.2	14.4	29.2	22.3
RenderFormer (flash)	30.6	12.4	10.7	30.4	13.5	12.1	33.0	19.4
RADIANT (sdpa, ours)	30.3	7.1	5.5	33.1	13.2	11.9	73.2	51.3

Appendix

Table 16: Batched throughput (ms per image, median) of the two 206M models on eight-object scenes (one H100 80 GB GPU, bf16, five warm-up and twenty timed forward passes, host-to-device transfers excluded; 512 triangles per object for RenderFormer, 2,048 points per object for RADIANT).

Resolution	Model	Objects	$t_{\text{scene}}$ (ms)	$t_{\text{view}}$ (ms)	ms/view, $V{=}1$	$V{=}16$	$V{=}64$
$512^{2}$	RADIANT	8	19.5	52.3	71.8	53.5	52.6
	RenderFormer	8	17.7	15.7	33.4	16.8	16.0
	RADIANT	50	37.7	57.9	95.7	60.3	58.5
	RenderFormer	50	117.5	27.8	145.3	35.1	29.6
$256^{2}$	RADIANT	8	19.4	13.9	33.3	15.1	14.2
	RenderFormer	8	17.5	13.3	30.9	14.4	13.6

Appendix

Table 17: Multi-view amortization of the two 206M models (one H100, bf16, batch size 1, random weights, synthetic scenes of 512 triangles per object, FlashAttention for both, three warm-up and ten timed iterations, medians).

t_{\text{scene}}

: view-independent part (RADIANT: encoder and scene transformer; RenderFormer: triangle-token transformer);

t_{\text{view}}

: view-dependent part (RADIANT: rasterization, ray tokens, predictor and DPT head; RenderFormer: camera-space triangle positions, ray generation, view transformer and DPT head). The per-view cost of

V

views is

(t_{\text{scene}}+V\,t_{\text{view}})/V

. RADIANT’s scene transformer computes its float32 attention through PyTorch SDPA under both backends. Under the shared sdpa backend

t_{\text{scene}}

is 20.0/37.7 ms for RADIANT and 15.2/227 ms for RenderFormer at 8/50 objects;

t_{\text{view}}

changes by less than 1 ms for RADIANT and becomes 14.8/34.7 ms at

512^{2}

for RenderFormer.

Module	Configuration	Parameters
Point encoder	16-D inputs, hidden 512/1,024/1,024, backbone 2,048, GroupNorm; 32 FPS patch tokens of 128 neighbors	9,985,024
Scene transformer	12 layers, width 1,024, 8 heads of 128, FFN 1,890, QK-LayerNorm, OBB-RoPE	96,962,716
Register tokens	16	16,384
Receiver/sender encoders	two MLPs $8\to 256\to 1{,}024$	530,944
Invariant-attention gains	two MLPs $8\to 64\to 384$ (Sec. 3.4 )	51,072
Object FiLM	intrinsic-complexity statistics $\to$ scale/shift	2,112,512

Appendix

Table 18: Modules of RADIANT and their parameter counts, totaling 205,942,972 parameters as counted from a trained checkpoint. Widths and layer counts are those of Sec. 3.1 .

Variant	PSNR $\uparrow$	LPIPS $\downarrow$	$\ell_{1}(\log)$ $\downarrow$	$\Delta$
Full model	$\mathbf{29.96}\pm 0.01$ / 26.16	0.059	0.019	—
No invariant form-factor attention (§ 3.4 )	$29.43\pm 0.07$ / 25.92	0.063	0.021	$-0.52\pm 0.08$
No inter-object scene attention (§ 3.1 ) †	$27.50\pm 0.66$ / 24.89	0.072	0.036	$-2.46\pm 0.66$
No FiLM conditioning (§ 3.3 )	$29.81\pm 0.02$ / 26.09	0.060	0.019	$-0.15\pm 0.02$
No register tokens	$29.82\pm 0.02$ / 26.10	0.060	0.019	$-0.13\pm 0.01$
No rasterizer channels (hit geometry, routing bias)	$26.84\pm 0.07$ / 24.41	0.092	0.025	$-3.12\pm 0.08$

Appendix

Table 19: Ablation rows (

256^{2}

, 100k steps, mean over seeds 42–44; † seeds 42–47) with the metrics of Tab. 6 . PSNR per-image (mean

\pm

sample standard deviation) / pooled;

\Delta

: per-image change from the full model, paired within each seed ( † seeds 45–47 against the full model’s three-seed mean). Below the rule: two controls of invariant form-factor attention (Sec. 3.4 ).

Variant	Scene tokens	Rasterizer channels	Splat	PSNR $\uparrow$	LPIPS $\downarrow$	$\ell_{1}(\log)$ $\downarrow$	$\Delta$
Reference (no invariant attention)	✓	✓	✓	$29.43\pm 0.07$ / 25.92	0.063	0.021	—
No rasterized splat	✓	✓	✗	$29.44\pm 0.04$ / 25.92	0.062	0.021	$+0.01\pm 0.07$
No rasterizer channels	✓	✗	✓	$26.31\pm 0.09$ / 24.08	0.099	0.027	$-3.12\pm 0.05$
No rasterizer channels or splat	✓	✗	✗	$26.34\pm 0.13$ / 24.01	0.094	0.027	$-3.09\pm 0.20$
No scene tokens	✗	✓	✓	$23.77\pm 0.01$ / 21.44	0.109	0.062	$-5.66\pm 0.06$

Appendix

Table 20: Rasterizer controls under the 100k-step schedule of Tab. 19 (mean over seeds 42–44), on the model without invariant form-factor attention. Columns indicate whether the view decoder receives the scene tokens, the rasterizer channels (hit channels and routing bias), and the rasterized splat into the output head. PSNR per-image (mean

\pm

sample standard deviation) / pooled;

\Delta

: per-image change from the reference row, paired within each seed.

	Seed
Variant	42	43	44	45	46	47	Mean $\pm$ SD	$\Delta$
Full model	29.97	29.95	29.96	–	–	–	$29.96\pm 0.01$	—
No invariant form-factor attention	29.36	29.49	29.45	–	–	–	$29.43\pm 0.07$	$-0.52\pm 0.08$
No inter-object scene attention †	28.55	26.95	26.93	27.69	26.98	27.89	$27.50\pm 0.66$	$-2.46\pm 0.66$
No FiLM conditioning	29.79	29.81	29.82	–	–	–	$29.81\pm 0.02$	$-0.15\pm 0.02$
No register tokens	29.84	29.81	29.82	–	–	–	$29.82\pm 0.02$	$-0.13\pm 0.01$

Appendix

Table 21: Per-image PSNR (dB) of every row of Tab. 19 at 100k steps (the schedule of Tab. 6 ) for seeds 42, 43, and 44 (42–47 without inter-object attention), with the mean and sample standard deviation.

\Delta

is paired within each seed, against the full model; seeds 45–47 against its three-seed mean. Below the rule: two controls of invariant form-factor attention (Sec. 3.4 ).

Model	PSNR $\uparrow$	LPIPS $\downarrow$	$\ell_{1}(\log)$ $\downarrow$	$\Delta$
RenderFormer	$23.56\pm 0.21$ / 21.84	0.166	0.041	—
RenderFormer, 32 pooled tokens per object	$23.19\pm 0.15$ / 21.59	0.165	0.041	$-0.38\pm 0.09$
RADIANT (full model)	$29.96\pm 0.01$ / 26.16	0.059	0.019	—

Appendix

Table 22: RenderFormer with the triangle tokens of each object pooled into 32 tokens before global transport, trained with the authors’ released training loop under the 100k-step schedule of Tab. 19 (mean over seeds 42–44). PSNR per-image (mean

\pm

sample standard deviation) / pooled;

\Delta

: per-image change from RenderFormer without pooling, paired within each seed. The last row is the full model of Tab. 19 .

Method	Input	Tokens	GI	Amortized
Per-scene fields ( Mildenhall et al., 2020 ; Kerbl et al., 2023 )	images	—	per scene	—
RenderFormer ( Zeng et al., 2025 )	triangles	$O(T)$	✓	✓
RenderFormer-V2 ( Zeng et al., 2026 )	triangles, media, environment maps	$O(T)$ , windowed	✓	✓
RenderFormer++ ( Du et al., 2026 )	triangles	$O(T_{i})$ per object, then $O(NM)$	✓	✓
RADIANT (ours)	point inputs (from meshes or point clouds)	$O(N)$ , 32 per object	✓	✓

Appendix

Table 23: Feed-forward renderers of explicit scenes: geometric input, what the sequence length is proportional to (

T

triangles,

N

objects,

M

learned queries per object), full global illumination without per-scene optimization (GI), and a per-scene representation reused across views (Amortized).

Corruption	Level	Objects	All
Clean input	—	32.1 / 0.043
Point density (samples per entity)	1,024	31.1 / 0.050	28.3 / 0.055
	512	27.1 / 0.079	25.2 / 0.082
	256	23.6 / 0.119	20.0 / 0.194
Position noise ( $\sigma$ , % of box diagonal)	0.5	30.8 / 0.049	23.5 / 0.078
	1	28.5 / 0.062	19.4 / 0.144

Appendix

Table 24: Zero-shot rendering from corrupted point inputs (no fine-tuning; RADIANT at step 600k; first 1,028 held-out images; PSNR per-image / LPIPS). Objects : only the foreground objects are corrupted; All : every entity, including walls, floor, and emitter geometry (emission values unchanged). On clean input the same images score 32.1 (RADIANT), 28.7 (RenderFormer), and 24.5 dB (RenderFormer-V2), all at step 600k.

Neural Rendering

Neural rendering uses learned 3D scene and light representations to synthesize, edit, and relight images from new viewpoints.

See all 120 papers

Jun 29, 2026

RenderFormer++: Scalable and Physically Grounded Feed-Forward Neural Rendering

We present RenderFormer++, a scalable and physically grounded feed-forward neural rendering framework for global illumination in mesh scenes. Existing Transformer-based neural rendering methods such as RenderFormer achieve promising cross-scene generalization, but suffer from limited physical consistency and poor scalability due to the quadratic attention complexity of triangle-level tokenization. To address these issues, we introduce Physics-Informed Transport Guidance (PITG), which embeds rendering-equation inductive biases into the attention mechanism and enforces transport consistency loss, enabling physically consistent light transport modeling. We further propose Hierarchical Object-Centric Tokenization (HOCT), which aggregates triangle-level features into compact object-level tokens via cross-attention with learnable queries, substantially reducing computational and memory costs while preserving geometric and radiometric information. Extensive experiments demonstrate that RenderFormer++ achieves scalable, stable, and generalizable feed-forward global illumination rendering across complex large-scale scenes with improved physical accuracy and efficiency over prior neural rendering methods.

Huangsheng Du, Haoran Zhu, Youcheng Cai +2

University of Science and Technology of China, China

Neural Rendering Efficient ViTs Physics-Informed ML Physically Based Rendering

Ask PDF

Sep 4, 2026

RenderFormer-V2: Neural Rendering with Heterogeneous Scene Primitives

We present 'RenderFormer-V2', a unified learned transformer-based neural rendering model, complementary to modern physics-based rendering systems, that can handle diverse light-transport effects such as caustics, volumetric scattering, environment lighting, textured and displaced surfaces and out-of-distribution materials without per-scene training or specialized code. RenderFormer-V2 models global light transport as a sequence-to-sequence transformation. Following its predecessor, RenderFormer-V2 also employs a two stage process: a view-independent stage that resolves intra-scene primitive to primitive transport, and a view-dependent stage that transforms the internal neural scene representation into image pixels. Different from RenderFormer, our model employs a novel combined windowed-attention and rendering-informed attention sink in the view-independent stage to improve scalability while maintaining render accuracy. To further improve versatility, RenderFormerV2 supports heterogeneous scene primitives, including environment maps and participating media, and it employs a material encoding independent of the underlying surface reflectance model that encodes material appearance via a novel neural embedding. We demonstrate the versatility of RenderFormer-V2 on a variety of scenes and perform an extensive ablation of the improved attention mechanism.

Chong Zeng, Yue Dong, Pieter Peers +2

Stanford University, USA · Microsoft Research, China · College of William & Mary, USA

Neural Rendering 3D Scene Representation

Ask PDF

Aug 6, 2026

Floating Radiance Networks

Recent advances in neural scene representations enable photorealistic novel-view synthesis, yet most methods remain tightly coupled to a single rendering paradigm, limiting their versatility and integration with conventional graphics workflows. We introduce Floating Radiance Networks (FlaRe), a neural scene representation combining explicit ray-traceable geometry with continuous neural radiance functions. A scene is represented by floating planar generalized Gaussian primitives, each carrying a compact latent descriptor of a local radiance field. A lightweight decoder shared across the scene maps this descriptor, local surface coordinates, and viewing direction to color and opacity. This formulation preserves the expressiveness of neural fields while providing an explicitly addressable structure that can be efficiently queried and manipulated. Hardware-accelerated primitive intersections enable interactive rendering and recursive ray-tracing, including reflections, refractions, transparency, and shadows. The same representation further supports primitive-level deformation, mesh extraction, and appearance stylization directly in its learned descriptor space. Experiments across standard reconstruction benchmarks demonstrate competitive rendering quality while using a compact set of primitives. Together, these results establish FlaRe as a versatile representation that brings high-fidelity neural rendering, ray-tracing, geometric manipulation, and appearance editing into a unified scene model. Source code is available online. Source code can be found at: https://github.com/KByrski/FlaRe

Krzysztof Byrski, Rafał Tobiasz, Grzegorz Wilczyński +5

Jagiellonian University, Faculty of Mathematics and Computer Science · IDEAS Research Institute · Poznań University of Technology, Institute of Robotics and Machine Intelligence

Neural Rendering Novel View Synthesis 3D Scene Representation

Ask PDF

Abstract

Neural Rendering

Explore similar work

RenderFormer++: Scalable and Physically Grounded Feed-Forward Neural Rendering

RenderFormer-V2: Neural Rendering with Heterogeneous Scene Primitives

Floating Radiance Networks