Light Field Primitive for Novel View Synthesis

Liang Chen; Jiahui Ning; Xun Jiang; Xing Xu; Jimmy Ren; Fenglei Fan; Heng Tao Shen

cs.CVSep 25, 2026

Light Field Primitive for Novel View Synthesis

Authors: Liang Chen, Jiahui Ning, Xun Jiang, Xing Xu, Jimmy Ren, Fenglei Fan, Heng Tao Shen

Organizations: Tongji University · Hong Kong Metropolitan University · City University of Hong Kong

Abstract

We present Light Field Primitives (LFP), a formulation for novel view synthesis that replaces the dense ray database with a compact set of differentiable primitives in the classical two-plane parameterization. Each primitive condenses a group of rays into one learned record, and its response to a query is governed by how closely that query belongs to the group. Rendering a camera ray then reduces to compositing all responses it elicits, and a scene can be optimized directly from posed images and rendered in real time with rays. Beyond its competitive performance on standard benchmarks, the main advantage of LFP is structural: its primitives reside directly in the 4D ray space, so optical and appearance effects that are already operations on the light field become behaviors of a single shared renderer. With minimal changes to that renderer, LFP supports multi-scale anti-aliasing, defocus deblurring with refocusing, rendering for fisheye cameras, and even transparent object reconstruction with ray refraction, matching specialized frameworks that devote substantial machinery to these effects.

Figures & tables

Panel of Figure 1 — Figure 1: One renderer, five capabilities. Panels in the “Specialized” row are annotated with what each method adds over its base in the row above, and in “LFP” with what LFP adds over its base.

Panel of Figure 2 — Figure 2: Light field primitives for NVS. (a) Two parallel planes index a ray by its crossings at them. (b) A ray group collects the rays that cross a third parallel plane at one point. (c) A primitive responds to a query (blue) by how far the query departs from its ray group, measured against the anchor ray it stores (thick orange), using their crossings at the initial two planes.

Panel of Table 1 — Table 1: Average NVS results. Numbers are cited from ( Kulhanek and Sattler, 2026 ) except those boxed with dash lines, which we ran ourselves. FPS is measured on our device for every method.

Panel of Figure 3 — Figure 3: NVS examples on Barron et al. (2022) ( top ) and Knapitsch et al. (2017) ( bottom ).

		PSNR $\uparrow$					SSIM $\uparrow$					LPIPS $\downarrow$
	Method	$n{=}1$	$n{=}2$	$n{=}4$	$n{=}8$	Avg.	$n{=}1$	$n{=}2$	$n{=}4$	$n{=}8$	Avg.	$n{=}1$	$n{=}2$	$n{=}4$	$n{=}8$	Avg.
Zoom-in	Mip-NeRF	29.26	25.18	24.16	24.10	25.67	0.860	0.727	0.670	0.706	0.741	0.122	0.260	0.370	0.428	0.295
	3DGS	29.19	23.50	20.71	19.59	23.25	0.880	0.740	0.619	0.619	0.715	0.107	0.243	0.394	0.476	0.305
	3DGS + EWA	29.30	25.90	23.70	22.81	25.43	0.880	0.775	0.667	0.643	0.741	0.114	0.236	0.369	0.449	0.292
	Mip-Splatting	29.39	27.39	26.47	26.22	27.37	0.884	0.808	0.754	0.765	0.803	0.108	0.205	0.305	0.392	0.252
	LFP	29.75	27.44	26.48	26.19	27.47	0.879	0.794	0.737	0.753	0.791	0.100	0.213	0.321	0.409	0.261

Table 2: Multi-scale rendering. A scene is trained once, at

n{=}1

, and rendered at four scales: zoom-in renders at

n\times

the training resolution on Mip-NeRF 360, zoom-out at

1/n\times

on Blender. All numbers are cited directly from Yu et al. (2024a) , whose protocol we follow for LFP.

Table 6

Panel of Figure 4 — Figure 4: Qualitative comparisons across different settings.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Field	Stored as	Floats	Role
$\bm{\mu}_{i}$	$(\mu^{u}_{i},\mu^{v}_{i},\mu^{s}_{i},\mu^{t}_{i})$	4	anchor ray; $(\mu^{u}_{i},\mu^{v}_{i})$ frozen after init
$p_{i}$	$\log p_{i}$	1	disparity of the group; coefficient of the coupling
$\Sigma_{i}$	$(r_{i},a_{i},\theta_{i})$	3	extent of the response, via Eq. ( 10 )
$o_{i}$	logit	1	peak contribution in compositing
$\mathbf{c}_{i}$	logit RGB	3	base radiance of the record
$\mathbf{g}^{k}_{i}$	signed RGB	$3K$	amplitude of lobe $k$

Appendix

Table 6: Fields of a primitive

i

. Floats are per primitive, with

K{=}3

lobes.

Panel of Figure 5 — Figure 5: Atlas construction. A single atlas, shown in (a), is the classical two-plane parameterization of LFR ( Levoy and Hanrahan, 1996 ) : a $uv$ plane through the scene centre and an $st$ plane one unit along its normal $\mathbf{n}$ . Only the $uv$ planes are drawn in (b) - (d), each carrying its own $st$ plane one unit away. Every atlas passes through the scene centre, so they differ only in orientation. The orientations are spread evenly here for legibility, and they are the $K$ -means centres of the training camera directions in implementation.

Panel of Figure 6 — Figure 6: Tile assignment without projection. Each cell is a pixel. Green marks the pixels a primitive can affect . (a) Three evaluations of Eq. ( 1 ), one pixel apart, give the rate $\mathbf{J}_{i}$ by Eq. ( 11 ), which together with $\Sigma_{i}+\rho_{i}^{2}\mathbf{I}$ forms the falloff $\mathbf{M}_{i}$ of Eq. ( 14 ); the blue ellipse encloses the pixels satisfying $\Delta\mathbf{x}^{\top}\mathbf{M}_{i}\Delta\mathbf{x}\leq 25$ , and the dashed rectangle bounds it. (b) When $\mathbf{x}^{\mathrm{ref}}$ falls beyond the image margin it is clamped to $\tilde{\mathbf{x}}$ , and one Newton step re-centres the rectangle at $\mathbf{x}^{\mathrm{c}}$ , the zero of the first-order model.The rectangle still reaches the image, so the primitive is kept and assigned to the tiles the rectangle covers.

Panel of Figure 7 — Figure 7: Qualitative examples of anti-aliasing under single-scale training, multi-scale testing.

	Appearance	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	Floats
3DGS	RGB only	26.82	0.804	0.268	3
	SH	27.46	0.815	0.255	48
	ASG	27.69	0.816	0.255	30
LFP	RGB only	26.76	0.798	0.264	3
	SH	27.57	0.808	0.256	48
	ASG	28.02	0.820	0.247	30

Appendix

Table 7: Directional appearance model comparisons, i.e., SH versus ASG, on the 9 scenes of Mip-NeRF 360. The last column counts the floats each primitive spends on appearance. The same three models are attached to 3DGS for reference.

Panel of Figure 8 — Figure 8: Appearance-model ablation. The view-dependent appearance model is replaced by spherical harmonics (SH, degree 3) or by a view-independent colour (RGB only), under an otherwise identical training protocol: (a) inside LFP, (b) inside 3DGS.

Figure 9: Lobe count

K

: quality peaks at

3

, cost grows monotonically.

Number of atlases
	PSNR $\uparrow$			SSIM $\uparrow$			LPIPS $\downarrow$
	all	outdoor	indoor	all	outdoor	indoor	all	outdoor	indoor
1	21.10	17.95	25.03	0.657	0.512	0.838	0.393	0.460	0.310
4	27.72	24.34	31.95	0.811	0.718	0.929	0.257	0.276	0.234
8	27.90	24.64	31.98	0.817	0.728	0.929	0.251	0.264	0.234
16 (ours)	28.02	24.71	32.15	0.820	0.731	0.930	0.247	0.259	0.232

Appendix

Table 8: Atlas design, each metric reported over all scenes and split into the five outdoor and four indoor ones. Top: number of atlases, normals fixed at the cluster centres of the training camera directions. Bottom: the same 16 atlases with the normals left free, at three step sizes, larger learning rate (lr) means more movement for the intial normals.

	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	$\lVert\mathbf{g}\rVert$	$\lVert\mathbf{c}\rVert$	#prim.
with $\lambda_{\mathrm{g}}\lVert\mathbf{g}\rVert^{2}$ (ours)	28.02	0.820	0.247	0.098	1.79	2.00 M
without	27.85	0.815	0.255	0.640	1.85	1.75 M
$\Delta$	$-0.17$	$-0.005$	$+0.008$	$\times 6.57$	$\times 1.03$	$-12\%$

Appendix

Table 9: Highlight regularization, averaged over the nine Mip-NeRF 360 scenes.

\lVert\mathbf{g}\rVert

is the mean amplitude of the penalized lobes and

\lVert\mathbf{c}\rVert

the mean base-colour magnitude. Dropping the term lets the lobes grow by

6.6\times

while the base is left untouched.

Explore similar work

Neighbor-Aware View Synthesis for Restoring Missing Views in Light-Field Camera Arrays
Aug 24, 2026Sakshi Goel, Ayush Goyal, K S Venkatesh +1Novel View Synthesis Image Restoration
Indian Institute of Technology Kanpur (IIT Kanpur), India
Fast Implicit Neural Light Field Representation via Geometric Decomposition and Multi-Resolution Low-Rank Features
Aug 14, 2026Yao Guo, Ligen Shi, Shuchen Sun +2Neural Rendering Implicit Neural Representations
Institute of Computational Imaging, Beijing Information Science and Technology University, Beijing 102206, China · College of Computer Science (College of Software), Inner Mongolia University, Hohhot 010021, China
Floating Radiance Networks
Aug 6, 2026Krzysztof Byrski, Rafał Tobiasz, Grzegorz Wilczyński +5Neural Rendering Novel View Synthesis
Jagiellonian University, Faculty of Mathematics and Computer Science · IDEAS Research Institute · Poznań University of Technology, Institute of Robotics and Machine Intelligence

Aug 24, 2026J/K move · Enter open · S save

Neighbor-Aware View Synthesis for Restoring Missing Views in Light-Field Camera Arrays

Sakshi Goel, Ayush Goyal, K S Venkatesh, Koteswar Rao Jerripothula

Indian Institute of Technology Kanpur (IIT Kanpur), India

Novel View Synthesis Image Restoration

Open paper AskPDF arXiv

In light-field (LF) imaging systems, dense spatial sampling from a camera array enables powerful post-capture capabilities such as refocusing and depth estimation. However, real-world LF capture is often affected by hardware malfunctions, where one or more cameras in the array fail, leading to missing sub-aperture images and degraded reconstruction quality. This paper addresses the problem of defective or missing view restoration in light-field camera arrays. We propose a novel generative framework that synthesizes the absent views by exploiting information from a carefully selected subset of neighboring cameras. These selected images, along with a positional encoding map indicating both their locations and the desired target view, are fed into a conditional Generative Adversarial Network (cGAN) trained to generate the missing viewpoint in a geometrically consistent manner. Extensive experiments on synthetic and real-world LF datasets demonstrate that our method produces visually plausible and photometrically accurate reconstructions, outperforming baselines for view interpolation both quantitatively and qualitatively. The proposed framework thus offers a robust and efficient solution for fault-tolerant light-field image acquisition.

Abstract

Explore similar work

Neighbor-Aware View Synthesis for Restoring Missing Views in Light-Field Camera Arrays

Fast Implicit Neural Light Field Representation via Geometric Decomposition and Multi-Resolution Low-Rank Features

Floating Radiance Networks