In light-field (LF) imaging systems, dense spatial sampling from a camera array enables powerful post-capture capabilities such as refocusing and depth estimation. However, real-world LF capture is often affected by hardware malfunctions, where one or more cameras in the array fail, leading to missing sub-aperture images and degraded reconstruction quality. This paper addresses the problem of defective or missing view restoration in light-field camera arrays. We propose a novel generative framework that synthesizes the absent views by exploiting information from a carefully selected subset of neighboring cameras. These selected images, along with a positional encoding map indicating both their locations and the desired target view, are fed into a conditional Generative Adversarial Network (cGAN) trained to generate the missing viewpoint in a geometrically consistent manner. Extensive experiments on synthetic and real-world LF datasets demonstrate that our method produces visually plausible and photometrically accurate reconstructions, outperforming baselines for view interpolation both quantitatively and qualitatively. The proposed framework thus offers a robust and efficient solution for fault-tolerant light-field image acquisition.
Figures & tables
Fig. 1 : Light-field missing view synthesis using 4 neighboring views supplied to our novel conditional GAN architecture.
Fig. 2 : Architecture of the proposed U-Net framework (a) U-Net Architecture (b) Model Structure
Fig. 3 : Central Views of scenes in our dataset
Fig. 4 : Visual Comparison for different Light Field Scenes
Model
Our Dataset
MPI Dataset
SSIM ↑
PSNR ↑
LPIPS ↓
SSIM ↑
PSNR ↑
LPIPS ↓
Baseline (Avg 4 Inputs)
0.6519
19.58
0.2582
0.6337
21.12
0.2778
3D U-Net
0.6792
13.73
0.2421
0.6232
17.73
0.2245
3D PatchGAN
0.6833
23.61
0.3298
0.6367
20.72
0.3472
3D PatchGAN + Side-Input
0.7244
22.43
0.1430
0.6712
21.34
0.1323
Proposed
0.7572
24.24
0.1023
0.7006
22.51
0.1184
TABLE I : Quantitative comparison of on our and MPI datasets. The proposed model consistently achieves the best performance across all metrics.
We present Light Field Primitives (LFP), a formulation for novel view synthesis that replaces the dense ray database with a compact set of differentiable primitives in the classical two-plane parameterization. Each primitive condenses a group of rays into one learned record, and its response to a query is governed by how closely that query belongs to the group. Rendering a camera ray then reduces to compositing all responses it elicits, and a scene can be optimized directly from posed images and rendered in real time with rays. Beyond its competitive performance on standard benchmarks, the main advantage of LFP is structural: its primitives reside directly in the 4D ray space, so optical and appearance effects that are already operations on the light field become behaviors of a single shared renderer. With minimal changes to that renderer, LFP supports multi-scale anti-aliasing, defocus deblurring with refocusing, rendering for fisheye cameras, and even transparent object reconstruction with ray refraction, matching specialized frameworks that devote substantial machinery to these effects.
Liang Chen, Jiahui Ning, Xun Jiang +4
Tongji University · Hong Kong Metropolitan University · City University of Hong Kong
Light field (LF) reconstruction from limited and noisy focal stack (FS) measurements is a highly ill-posed inverse problem. Although the LF-to-FS imaging geometry is fixed for a given optical setup, LF spatial-angular structure---including within-view spatial details, cross-view angular dependencies, and disparity across views---varies across scenes. Consequently, a fixed pre-trained prior may not optimally capture the spatial-angular structure of each test LF. We propose Structure-aware Test-time Adaptation for diffusion-based light field Reconstruction (STAR), the first test-time adaptation framework for reconstructing an LF from FS. For each test LF, STAR freezes a pre-trained diffusion prior and fits three lightweight adapters to the observed FS to jointly adapt the three components of the LF's spatial-angular structure. STAR outperforms existing state-of-the-art methods in both two- and three-focal-sheet settings, with shorter inference times than those with test-time parameter updates.
Wontae Choi, Ki Ryum Moon, Jae Young Lee +2
Department of Artificial Intelligence (AI), Sungkyunkwan University (SKKU), Suwon, South Korea · Department of Electrical and Computer Engineering, SKKU, Suwon, South Korea
We present SyncLight, a method to enable consistent, parametric control over light sources across multiple uncalibrated views of a static scene conditioned on a single view. While single-view relighting has advanced significantly, existing generative approaches struggle to maintain the rigorous lighting consistency essential for multi-camera broadcasts, stereoscopic cinema, and virtual production. SyncLight addresses this by enabling precise control over light intensity and color across a multi-view capture of a scene, conditioned on a single reference edit. Our method leverages a multi-view diffusion transformer trained using a latent bridge matching formulation, achieving high-fidelity relighting of the entire image set in a single inference step. To facilitate training, we introduce a large-scale hybrid dataset comprising diverse synthetic environments -- curated from existing sources and newly designed scenes -- alongside high-fidelity, real-world multi-view captures under calibrated illumination. Though trained only on image pairs, SyncLight generalizes zero-shot to an arbitrary number of viewpoints, effectively propagating lighting changes across all views, without requiring camera pose information. SyncLight enables practical relighting workflows for multi-view capture systems.
David Serrano-Lozano, Anand Bhattad, Luis Herranz +2
Computer Vision Center Universitat Autònoma de Barcelona · Johns Hopkins University · Universidad Politécnica de Madrid +1