Light Field Primitive for Novel View Synthesis

Liang Chen; Jiahui Ning; Xun Jiang; Xing Xu; Jimmy Ren; Fenglei Fan; Heng Tao Shen

cs.CVSep 25, 2026

Light Field Primitive for Novel View Synthesis

Authors: Liang Chen, Jiahui Ning, Xun Jiang, Xing Xu, Jimmy Ren, Fenglei Fan, Heng Tao Shen

Organizations: Tongji University · Hong Kong Metropolitan University · City University of Hong Kong

Abstract

We present Light Field Primitives (LFP), a formulation for novel view synthesis that replaces the dense ray database with a compact set of differentiable primitives in the classical two-plane parameterization. Each primitive condenses a group of rays into one learned record, and its response to a query is governed by how closely that query belongs to the group. Rendering a camera ray then reduces to compositing all responses it elicits, and a scene can be optimized directly from posed images and rendered in real time with rays. Beyond its competitive performance on standard benchmarks, the main advantage of LFP is structural: its primitives reside directly in the 4D ray space, so optical and appearance effects that are already operations on the light field become behaviors of a single shared renderer. With minimal changes to that renderer, LFP supports multi-scale anti-aliasing, defocus deblurring with refocusing, rendering for fisheye cameras, and even transparent object reconstruction with ray refraction, matching specialized frameworks that devote substantial machinery to these effects.

Figures & tables

Panel of Figure 1 — Figure 1: One renderer, five capabilities. Panels in the “Specialized” row are annotated with what each method adds over its base in the row above, and in “LFP” with what LFP adds over its base.

Panel of Figure 2 — Figure 2: Light field primitives for NVS. (a) Two parallel planes index a ray by its crossings at them. (b) A ray group collects the rays that cross a third parallel plane at one point. (c) A primitive responds to a query (blue) by how far the query departs from its ray group, measured against the anchor ray it stores (thick orange), using their crossings at the initial two planes.

Panel of Table 1 — Table 1: Average NVS results. Numbers are cited from ( Kulhanek and Sattler, 2026 ) except those boxed with dash lines, which we ran ourselves. FPS is measured on our device for every method.

Panel of Figure 3 — Figure 3: NVS examples on Barron et al. (2022) ( top ) and Knapitsch et al. (2017) ( bottom ).

		PSNR $\uparrow$					SSIM $\uparrow$					LPIPS $\downarrow$
	Method	$n{=}1$	$n{=}2$	$n{=}4$	$n{=}8$	Avg.	$n{=}1$	$n{=}2$	$n{=}4$	$n{=}8$	Avg.	$n{=}1$	$n{=}2$	$n{=}4$	$n{=}8$	Avg.
Zoom-in	Mip-NeRF	29.26	25.18	24.16	24.10	25.67	0.860	0.727	0.670	0.706	0.741	0.122	0.260	0.370	0.428	0.295
	3DGS	29.19	23.50	20.71	19.59	23.25	0.880	0.740	0.619	0.619	0.715	0.107	0.243	0.394	0.476	0.305
	3DGS + EWA	29.30	25.90	23.70	22.81	25.43	0.880	0.775	0.667	0.643	0.741	0.114	0.236	0.369	0.449	0.292
	Mip-Splatting	29.39	27.39	26.47	26.22	27.37	0.884	0.808	0.754	0.765	0.803	0.108	0.205	0.305	0.392	0.252
	LFP	29.75	27.44	26.48	26.19	27.47	0.879	0.794	0.737	0.753	0.791	0.100	0.213	0.321	0.409	0.261

Table 2: Multi-scale rendering. A scene is trained once, at

n{=}1

, and rendered at four scales: zoom-in renders at

n\times

the training resolution on Mip-NeRF 360, zoom-out at

1/n\times

on Blender. All numbers are cited directly from Yu et al. (2024a) , whose protocol we follow for LFP.

Table 6

Panel of Figure 4 — Figure 4: Qualitative comparisons across different settings.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Field	Stored as	Floats	Role
$\bm{\mu}_{i}$	$(\mu^{u}_{i},\mu^{v}_{i},\mu^{s}_{i},\mu^{t}_{i})$	4	anchor ray; $(\mu^{u}_{i},\mu^{v}_{i})$ frozen after init
$p_{i}$	$\log p_{i}$	1	disparity of the group; coefficient of the coupling
$\Sigma_{i}$	$(r_{i},a_{i},\theta_{i})$	3	extent of the response, via Eq. ( 10 )
$o_{i}$	logit	1	peak contribution in compositing
$\mathbf{c}_{i}$	logit RGB	3	base radiance of the record
$\mathbf{g}^{k}_{i}$	signed RGB	$3K$	amplitude of lobe $k$

Appendix

Table 6: Fields of a primitive

i

. Floats are per primitive, with

K{=}3

lobes.

Panel of Figure 5 — Figure 5: Atlas construction. A single atlas, shown in (a), is the classical two-plane parameterization of LFR ( Levoy and Hanrahan, 1996 ) : a $uv$ plane through the scene centre and an $st$ plane one unit along its normal $\mathbf{n}$ . Only the $uv$ planes are drawn in (b) - (d), each carrying its own $st$ plane one unit away. Every atlas passes through the scene centre, so they differ only in orientation. The orientations are spread evenly here for legibility, and they are the $K$ -means centres of the training camera directions in implementation.

Panel of Figure 6 — Figure 6: Tile assignment without projection. Each cell is a pixel. Green marks the pixels a primitive can affect . (a) Three evaluations of Eq. ( 1 ), one pixel apart, give the rate $\mathbf{J}_{i}$ by Eq. ( 11 ), which together with $\Sigma_{i}+\rho_{i}^{2}\mathbf{I}$ forms the falloff $\mathbf{M}_{i}$ of Eq. ( 14 ); the blue ellipse encloses the pixels satisfying $\Delta\mathbf{x}^{\top}\mathbf{M}_{i}\Delta\mathbf{x}\leq 25$ , and the dashed rectangle bounds it. (b) When $\mathbf{x}^{\mathrm{ref}}$ falls beyond the image margin it is clamped to $\tilde{\mathbf{x}}$ , and one Newton step re-centres the rectangle at $\mathbf{x}^{\mathrm{c}}$ , the zero of the first-order model.The rectangle still reaches the image, so the primitive is kept and assigned to the tiles the rectangle covers.

Panel of Figure 7 — Figure 7: Qualitative examples of anti-aliasing under single-scale training, multi-scale testing.

	Appearance	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	Floats
3DGS	RGB only	26.82	0.804	0.268	3
	SH	27.46	0.815	0.255	48
	ASG	27.69	0.816	0.255	30
LFP	RGB only	26.76	0.798	0.264	3
	SH	27.57	0.808	0.256	48
	ASG	28.02	0.820	0.247	30

Appendix

Table 7: Directional appearance model comparisons, i.e., SH versus ASG, on the 9 scenes of Mip-NeRF 360. The last column counts the floats each primitive spends on appearance. The same three models are attached to 3DGS for reference.

Panel of Figure 8 — Figure 8: Appearance-model ablation. The view-dependent appearance model is replaced by spherical harmonics (SH, degree 3) or by a view-independent colour (RGB only), under an otherwise identical training protocol: (a) inside LFP, (b) inside 3DGS.

Figure 9: Lobe count

K

: quality peaks at

3

, cost grows monotonically.

Number of atlases
	PSNR $\uparrow$			SSIM $\uparrow$			LPIPS $\downarrow$
	all	outdoor	indoor	all	outdoor	indoor	all	outdoor	indoor
1	21.10	17.95	25.03	0.657	0.512	0.838	0.393	0.460	0.310
4	27.72	24.34	31.95	0.811	0.718	0.929	0.257	0.276	0.234
8	27.90	24.64	31.98	0.817	0.728	0.929	0.251	0.264	0.234
16 (ours)	28.02	24.71	32.15	0.820	0.731	0.930	0.247	0.259	0.232

Appendix

Table 8: Atlas design, each metric reported over all scenes and split into the five outdoor and four indoor ones. Top: number of atlases, normals fixed at the cluster centres of the training camera directions. Bottom: the same 16 atlases with the normals left free, at three step sizes, larger learning rate (lr) means more movement for the intial normals.

	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	$\lVert\mathbf{g}\rVert$	$\lVert\mathbf{c}\rVert$	#prim.
with $\lambda_{\mathrm{g}}\lVert\mathbf{g}\rVert^{2}$ (ours)	28.02	0.820	0.247	0.098	1.79	2.00 M
without	27.85	0.815	0.255	0.640	1.85	1.75 M
$\Delta$	$-0.17$	$-0.005$	$+0.008$	$\times 6.57$	$\times 1.03$	$-12\%$

Appendix

Table 9: Highlight regularization, averaged over the nine Mip-NeRF 360 scenes.

\lVert\mathbf{g}\rVert

is the mean amplitude of the penalized lobes and

\lVert\mathbf{c}\rVert

the mean base-colour magnitude. Dropping the term lets the lobes grow by

6.6\times

while the base is left untouched.

Explore similar work

Aug 24, 2026

Neighbor-Aware View Synthesis for Restoring Missing Views in Light-Field Camera Arrays

In light-field (LF) imaging systems, dense spatial sampling from a camera array enables powerful post-capture capabilities such as refocusing and depth estimation. However, real-world LF capture is often affected by hardware malfunctions, where one or more cameras in the array fail, leading to missing sub-aperture images and degraded reconstruction quality. This paper addresses the problem of defective or missing view restoration in light-field camera arrays. We propose a novel generative framework that synthesizes the absent views by exploiting information from a carefully selected subset of neighboring cameras. These selected images, along with a positional encoding map indicating both their locations and the desired target view, are fed into a conditional Generative Adversarial Network (cGAN) trained to generate the missing viewpoint in a geometrically consistent manner. Extensive experiments on synthetic and real-world LF datasets demonstrate that our method produces visually plausible and photometrically accurate reconstructions, outperforming baselines for view interpolation both quantitatively and qualitatively. The proposed framework thus offers a robust and efficient solution for fault-tolerant light-field image acquisition.

Sakshi Goel, Ayush Goyal, K S Venkatesh +1

Indian Institute of Technology Kanpur (IIT Kanpur), India

Novel View Synthesis Image Restoration

Ask PDF

Aug 14, 2026

Fast Implicit Neural Light Field Representation via Geometric Decomposition and Multi-Resolution Low-Rank Features

Implicit neural representations provide a compact and continuous way to reconstruct dense light fields from sampled ray coordinates. However, fast light field reconstruction remains challenging because a light field is a high-dimensional signal with strong spatial-angular redundancy and structured disparity variations. Directly fitting 4D ray coordinates with a neural network often requires considerable optimization time to recover both view appearance and cross-view consistency. To address this issue, this paper proposes a fast implicit light field representation based on geometric decomposition and multi-resolution low-rank features. The proposed method decomposes a 4D light field into a horizontal disparity plane, a spatial texture plane, and a vertical disparity plane. Each plane is represented by a low-rank structure that combines a low-resolution 2D grid with the element-wise product of two high-resolution 1D line features at multiple resolution levels. The fused features are decoded by a lightweight multilayer perceptron to predict RGB values. Experiments on public light field datasets show that the proposed method achieves competitive reconstruction quality while providing a better trade-off among model parameters, training time, and inference efficiency.

Yao Guo, Ligen Shi, Shuchen Sun +2

Institute of Computational Imaging, Beijing Information Science and Technology University, Beijing 102206, China · College of Computer Science (College of Software), Inner Mongolia University, Hohhot 010021, China

Neural Rendering Implicit Neural Representations Image Reconstruction Representation Learning

Ask PDF

Aug 6, 2026

Floating Radiance Networks

Recent advances in neural scene representations enable photorealistic novel-view synthesis, yet most methods remain tightly coupled to a single rendering paradigm, limiting their versatility and integration with conventional graphics workflows. We introduce Floating Radiance Networks (FlaRe), a neural scene representation combining explicit ray-traceable geometry with continuous neural radiance functions. A scene is represented by floating planar generalized Gaussian primitives, each carrying a compact latent descriptor of a local radiance field. A lightweight decoder shared across the scene maps this descriptor, local surface coordinates, and viewing direction to color and opacity. This formulation preserves the expressiveness of neural fields while providing an explicitly addressable structure that can be efficiently queried and manipulated. Hardware-accelerated primitive intersections enable interactive rendering and recursive ray-tracing, including reflections, refractions, transparency, and shadows. The same representation further supports primitive-level deformation, mesh extraction, and appearance stylization directly in its learned descriptor space. Experiments across standard reconstruction benchmarks demonstrate competitive rendering quality while using a compact set of primitives. Together, these results establish FlaRe as a versatile representation that brings high-fidelity neural rendering, ray-tracing, geometric manipulation, and appearance editing into a unified scene model. Source code is available online. Source code can be found at: https://github.com/KByrski/FlaRe

Krzysztof Byrski, Rafał Tobiasz, Grzegorz Wilczyński +5

Jagiellonian University, Faculty of Mathematics and Computer Science · IDEAS Research Institute · Poznań University of Technology, Institute of Robotics and Machine Intelligence

Neural Rendering Novel View Synthesis 3D Scene Representation

Ask PDF