Neighbor-Aware View Synthesis for Restoring Missing Views in Light-Field Camera Arrays
Organizations: Indian Institute of Technology Kanpur (IIT Kanpur), India
Abstract
In light-field (LF) imaging systems, dense spatial sampling from a camera array enables powerful post-capture capabilities such as refocusing and depth estimation. However, real-world LF capture is often affected by hardware malfunctions, where one or more cameras in the array fail, leading to missing sub-aperture images and degraded reconstruction quality. This paper addresses the problem of defective or missing view restoration in light-field camera arrays. We propose a novel generative framework that synthesizes the absent views by exploiting information from a carefully selected subset of neighboring cameras. These selected images, along with a positional encoding map indicating both their locations and the desired target view, are fed into a conditional Generative Adversarial Network (cGAN) trained to generate the missing viewpoint in a geometrically consistent manner. Extensive experiments on synthetic and real-world LF datasets demonstrate that our method produces visually plausible and photometrically accurate reconstructions, outperforming baselines for view interpolation both quantitatively and qualitatively. The proposed framework thus offers a robust and efficient solution for fault-tolerant light-field image acquisition.
Figures & tables
| Model | Our Dataset | MPI Dataset | ||||
|---|---|---|---|---|---|---|
| SSIM | PSNR | LPIPS | SSIM | PSNR | LPIPS | |
| Baseline (Avg 4 Inputs) | 0.6519 | 19.58 | 0.2582 | 0.6337 | 21.12 | 0.2778 |
| 3D U-Net | 0.6792 | 13.73 | 0.2421 | 0.6232 | 17.73 | 0.2245 |
| 3D PatchGAN | 0.6833 | 23.61 | 0.3298 | 0.6367 | 20.72 | 0.3472 |
| 3D PatchGAN + Side-Input | 0.7244 | 22.43 | 0.1430 | 0.6712 | 21.34 | 0.1323 |
| Proposed | 0.7572 | 24.24 | 0.1023 | 0.7006 | 22.51 | 0.1184 |