Directionally Factorized Illumination with Spherical Harmonic Rotation

Yimeng Dou; Adam M. Saunders; Andre T. S. Hucke; Michael E. Kim; Adam W. Anderson; Eric R. Tkaczyk; Bennett A. Landman; Daniel Moyer

cs.CVOct 17, 2026

Directionally Factorized Illumination with Spherical Harmonic Rotation

Authors: Yimeng Dou, Adam M. Saunders, Andre T. S. Hucke, Michael E. Kim, Adam W. Anderson, Eric R. Tkaczyk, Bennett A. Landman, Daniel Moyer

Organizations: Department of Electrical and Computer Engineering, Vanderbilt University, Nashville, TN, USA · Department of Computer Science, Vanderbilt University, Nashville, TN, USA · Department of Biomedical Engineering, Vanderbilt University, Nashville, TN, USA · Department of Radiology and Radiological Sciences, Vanderbilt University Medical Center, Nashville, TN, USA ·

Abstract

Many real-world capture scenarios involve an active light source rigidly attached to the camera: camera flashes for dark-environment imaging, underwater vehicle lights, and endoscopic/laparoscopic medical procedures. As the camera moves, the co-mounted light moves with it, causing the illumination direction at each surface point to change across views. Neural rendering methods (NeRF, Gaussian Splatting) encode this pose-dependent illumination as surface coloration, producing appearance inconsistencies and making it harder to recover the underlying scene. We present DaFISH, which can be plugged into any novel-view synthesis (NVS) pipeline such as NeRF. It factorizes per-point emitted color into surface albedo and a spherical harmonic (SH)-represented illumination factor. During training, the SH illumination is evaluated directly, enabling stable joint optimization of albedo, lighting, and geometry. DaFISH provides analytical generalization of illumination to arbitrary camera poses without retraining. Building on the closed-form SH-to-irradiance conversion, the training pipeline is fully differentiable from SH evaluation through volume rendering. Averaged over the held-out frames of the five scenes, DaFISH attains the best mean PSNR (38.91,dB), SSIM (0.9613), and LPIPS (0.0582) among all compared methods. It improves over the strongest splatting baseline as well as the closest NeRF-based baseline, while recovering the scene consistently across views compared to prior methods.

Figures & tables

Fig. 1: (a) Robots that operate in dark environments, such as subterranean exploration (SubT challenge) [ 1 ] , deep-water surveys by the remotely operated vehicle Hercules [ 2 ] , and nighttime driving [ 3 ] rely on onboard lights that move with the platform. (b) In standard novel view synthesis (NVS, left), hundreds of images are captured while the camera moves around the scene under fixed illumination field $L$ , and a radiance field is reconstructed with methods such as NeRF, 3DGS, or their derivatives. However, in co-located capture (right), the camera and a light source are rigidly attached, so the surface illumination changes with every pose, as the light source is moving as well. We model this moving illumination through the irradiance $E(\mathbf{n})$ , so the irradiance for a given normal is computed by analytically rotating the coefficients with a Wigner-D matrix of the known camera motion, where the residual spatial variation from the near-field source is absorbed by a learned per-sample MLP (see III for more details). (c) DaFISH can reconstruct the scenes while attributing frame-varying brightness to the illumination branch. The brightness of ground truth and rendered images is increased by 40% for visualization.

Fig. 2: Overview of the proposed method. Given a co-located capture that produces

N

images

I

and the extrinsic camera parameter matrix

c2w

(I. Capture), the method factors appearance into albedo and illumination before volumetric rendering. The albedo branch (IIa) samples a global coordinate position

\mathbf{x}

through a scene MLP to predict per-sample albedo

\mathbf{\rho}(\mathbf{x})

, density

\sigma(\mathbf{x})

, and a global normal

\mathbf{n}

from the density gradient. The illumination branch (IIb) starts from a single trainable

l{=}4

SH code

\mathbf{E}_{sh}\in\mathbb{R}^{25\times 3}

shared across all frames, and rotates it analytically into each frame via the Wigner-D matrix

\mathbf{D}^{(l)}

derived from the known rotation

R_{t}\in\mathrm{SO}(3)

, so that the rotated coefficients become

\widetilde{\mathbf{E}}_{t}

. Albedo

\rho(\bm{x})

and illumination factor

\bm{\alpha}(\bm{x}^{c},\bm{n}^{c})

are multiplied into the factorized radiance

\bm{\rho}^{\prime}=\bm{\alpha}\odot\bm{\rho}

and composited through volumetric rendering, producing a full render

C(\bm{r})

, a clean albedo render

\sum_{i}w_{i}\,\bm{\rho}

(i.e., a render without illumination effects), and an illumination map

\alpha

	Lab1			Lab2			Outdoor1			Outdoor2			FourLights
Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
2DGS [ 7 ]	28.52	0.9363	0.1629	29.70	0.9212	0.1844	26.23	0.9268	0.2330	36.97	0.9352	0.1717	27.28	0.7777	0.1644
3DGS [ 6 ]	30.08	0.9463	0.1017	29.82	0.8626	0.1697	25.17	0.8723	0.2117	37.86	0.9505	0.0920	27.30	0.7637	0.1620
GS-W [ 24 ]	37.03	0.9582	0.0851	29.14	0.8444	0.2393	25.69	0.8685	0.2238	33.47	0.8251	0.1508	27.84	0.8778	0.2043
DarkGS [ 8 ]	38.15	0.9508	0.1249	39.03	0.9587	0.1385	35.25	0.9546	0.1909	41.22	0.9506	0.1708	n/a
SpotlessGS [ 21 ]	35.01	0.9573	0.1107	36.80	0.9642	0.1236	30.44	0.9584	0.1683	40.93	0.9595	0.1578	25.51	0.8182	0.1720

TABLE I: NVS results on five co-located captures of the DarkRobotic dataset [ 8 ] and FourLights [ 10 ] . The relative-illumination baseline ( [ 10 ] ) failed the Outdoor2 reconstruction before reaching the maximum iteration and is reported with n/a there. Best per scene in bold , second-best in italics .

Fig. 3: Qualitative comparison across three co-located captures (rows: Lab1, Outdoor1, and FourLights) and all methods plus ground truth (GT) are split over two stacked panels of four columns (top 4: 2DGS [ 7 ] , 3DGS [ 6 ] , GS-W [ 24 ] , DarkGS [ 8 ] ; bottom 4: SpotlessGS [ 21 ] , relative-illumination [ 10 ] , DaFISH, and GT). The red bounding box highlights the reconstructed detail. Our factorization keeps albedo stable while the directional SH irradiance travels with the camera.

Fig. 4: Clean scene recovery on held-out frames across three DarkRobotic scenes (columns). For each scene, a held-out GT frame (top) is shown next to the clean albedo (no illumination effect render) of the baselines (SpotlessGS [ 21 ] , relative-illumination [ 10 ] ) and DaFISH.

Fig. 5: The row shows the original capture (left) next to the learned illumination field (right) at the same camera orientation.

Variation	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
Full model (Outdoor1)	38.71	0.9687	0.0667
relative-illumination (Outdoor1)	33.67	0.9321	0.1563
SH order $l{=}1$ (FourLights)	25.83	0.9148	0.0954
SH order $l{=}2$ (FourLights)	27.30	0.9197	0.0924
SH order $l{=}4$ (FourLights)	28.46	0.9276	0.0880

TABLE II: Ablation of the DaFISH components on Outdoor1 and FourLights.

Neural Rendering

Neural rendering uses learned 3D scene and light representations to synthesize, edit, and relight images from new viewpoints.

See all 118 papers

Floating Radiance Networks
Aug 6, 2026Krzysztof Byrski, Rafał Tobiasz, Grzegorz Wilczyński +5Neural Rendering Novel View Synthesis
Jagiellonian University, Faculty of Mathematics and Computer Science · IDEAS Research Institute · Poznań University of Technology, Institute of Robotics and Machine Intelligence
RehearsalNeRF: Decoupling Intrinsic Neural Fields of Dynamic Illuminations for Scene Editing
Mar 30, 2026Changyeon Won, Hyunjun Jung, Jungu Cho +3Neural Radiance Fields 3D Scene Editing
Department of AI Convergence, GIST, Gwangju, Korea. · AI R&D Division, CJ Corporation, Seoul, Korea. · Department of Artificial Intelligence, Yonsei University, Seoul, Korea.
RoomLight: A 2.5D Illumination Prior for Indoor Environments
Sep 23, 2026Andreea Ardelean, Bernhard EggerInverse Rendering
Friedrich-Alexander-Universität Erlangen-Nürnberg

Aug 6, 2026J/K move · Enter open · S save

Floating Radiance Networks

Krzysztof Byrski, Rafał Tobiasz, Grzegorz Wilczyński, Mikołaj Zieliński+4

Jagiellonian University, Faculty of Mathematics and Computer Science · IDEAS Research Institute · Poznań University of Technology, Institute of Robotics and Machine Intelligence

Neural Rendering Novel View Synthesis 3D Scene Representation

Open paper AskPDF arXiv

Recent advances in neural scene representations enable photorealistic novel-view synthesis, yet most methods remain tightly coupled to a single rendering paradigm, limiting their versatility and integration with conventional graphics workflows. We introduce Floating Radiance Networks (FlaRe), a neural scene representation combining explicit ray-traceable geometry with continuous neural radiance functions. A scene is represented by floating planar generalized Gaussian primitives, each carrying a compact latent descriptor of a local radiance field. A lightweight decoder shared across the scene maps this descriptor, local surface coordinates, and viewing direction to color and opacity. This formulation preserves the expressiveness of neural fields while providing an explicitly addressable structure that can be efficiently queried and manipulated. Hardware-accelerated primitive intersections enable interactive rendering and recursive ray-tracing, including reflections, refractions, transparency, and shadows. The same representation further supports primitive-level deformation, mesh extraction, and appearance stylization directly in its learned descriptor space. Experiments across standard reconstruction benchmarks demonstrate competitive rendering quality while using a compact set of primitives. Together, these results establish FlaRe as a versatile representation that brings high-fidelity neural rendering, ray-tracing, geometric manipulation, and appearance editing into a unified scene model. Source code is available online. Source code can be found at: https://github.com/KByrski/FlaRe

Abstract

Neural Rendering

Explore similar work

Floating Radiance Networks

RehearsalNeRF: Decoupling Intrinsic Neural Fields of Dynamic Illuminations for Scene Editing

RoomLight: A 2.5D Illumination Prior for Indoor Environments