Estimating scene illumination from images is challenging as lighting is inherently spatially varying, depending on geometry, occlusion, and light sources. Existing methods incorporate geometric cues by deriving local environment maps or probe-based representations, but largely rely on regression or parametric models, resulting in over-smoothed lighting and limited realism. We revisit spatially-varying illumination estimation from a generative perspective and introduce LUMEN, a geometry-conditioned diffusion framework for synthesizing high-dynamic-range (HDR) environment maps at arbitrary spatial locations from single or multi-view images. Specifically, we reconstruct a coarse 3D scene from input images and render partial environment maps at arbitrary locations to encode visibility and spatial structure. These signals are fused with multi-view image features and fed into a diffusion transformer to generate full HDR environment maps. To train the model, we build a synthetic dataset of 700 indoor scenes with dense spatial supervision, including multi-view RGB images, depth, and HDR illumination at multiple sampled locations. This dataset enables the model to learn spatially consistent lighting distributions across diverse scenes. Extensive experiments demonstrate that LUMEN significantly improves visual fidelity and spatial coherence compared to prior approaches.
Figures & tables
Figure 1. Given a set of multi-view images, LUMEN estimates spatially-varying illumination at any location with consistency and achieves seamless virtual object insertion results (the balls and the bunny are rendered using predicted HDRI maps at their positions). teaser
Figure 2. Overview of LUMEN. Given a single-view image or multi-view inputs, we first estimate the camera parameters and reconstruct a coarse point cloud using an off-the-shelf feed-forward 3D reconstruction model. We then render partial environment maps at arbitrary locations in the scene based on the reconstructed geometry, and condition a 2D diffusion model on the input views with these partial environment maps to generate a complete HDRI map.
Dataset
Method
Scale-invariant RMSE ↓
Angular Error ↓
Normalized RMSE ↓
Diffuse
Matte
Mirror
Diffuse
Matte
Mirror
Diffuse
Matte
Mirror
Laval Indoor
StyleLight
0.064
0.152
0.265
3.874
4.606
10.974
0.232
0.411
0.520
DiffusionLight
0.075
0.163
0.282
3.415
4.157
6.483
0.246
0.391
0.457
LuxDiT
0.056
0.140
0.274
2.759
3.875
6.099
0.207
0.388
0.546
Ours
0.046
0.120
0.252
2.700
3.773
6.257
0.180
0.319
0.411
Laval Outdoor
StyleLight
0.131
0.286
0.456
3.524
4.864
7.609
0.293
0.377
0.475
Table 1. Quantitative results on the Laval Indoor and Laval Outdoor datasets, underlined entries indicate the second-best results.
Figure 3. Qualitative comparison on the Laval Indoor and Laval Outdoor datasets. (Please zoom in to see details)
Scene
Method
RMSE ↓
si-RMSE ↓
Angular Error ↓
SSIM ↑
Diffuse
Matte
Mirror
Diffuse
Matte
Mirror
Diffuse
Matte
Mirror
Diffuse
Matte
Mirror
Infinigen (single)
DiffusionLight (CVPR’24)
0.157
0.228
0.460
0.066
0.180
0.428
2.935
4.053
5.394
0.947
0.679
0.471
4DLighting (SIGGRAPH’25)
0.164
0.207
0.430
0.043
0.134
0.381
1.457
2.418
3.595
0.972
0.754
0.533
Ours
0.110
0.149
0.369
0.042
0.108
0.329
2.762
3.382
4.594
0.973
0.832
0.679
In-the-wild (single)
DiffusionLight (CVPR’24)
0.172
0.236
0.351
0.081
0.193
0.310
1.592
2.681
3.856
0.923
0.654
0.447
4DLighting (SIGGRAPH’25)
0.141
0.190
0.289
0.048
0.132
0.238
1.186
1.904
2.865
0.969
0.755
0.519
Table 2. Quantitative comparison for spatially-varying lighting estimation. The results are reported in terms of RMSE, si-RMSE, angular error, and SSIM.
Figure 4. Qualitative comparison of spatially-varying lighting estimation from a single-view image.
Scene
Method
RMSE ↓
si-RMSE ↓
Angular Error ↓
N-RMSE ↓
Diffuse
Matte
Mirror
Diffuse
Matte
Mirror
Diffuse
Matte
Mirror
Diffuse
Matte
Mirror
Laval Indoor
Single-only
0.268
0.287
0.409
0.053
0.130
0.259
3.654
4.359
6.436
0.231
0.396
0.508
Infinigen-only
0.304
0.318
0.458
0.052
0.128
0.264
3.662
4.545
6.826
0.197
0.401
0.557
w/o SigLIP
0.202
0.242
0.362
0.058
0.129
0.249
4.776
5.790
8.743
0.195
0.347
0.450
Ours
0.184
0.218
0.351
0.046
0.120
0.252
2.700
3.773
6.257
0.180
0.319
0.411
In-the-wild
Single-only
0.502
0.457
0.503
0.056
0.123
0.200
2.525
2.975
3.619
0.228
0.543
0.647
Table 3. Ablation results on Laval indoor and in-the-wild scenes. Results are reported in terms of RMSE, si-RMSE, angular error, and normalized RMSE.
Figure 5. Qualitative comparison of ablations.
Figure 6. Qualitative comparison of spatially-varying lighting estimation from multi-view images.
Figure 7. Qualitative comparison of object insertion on the Laval Indoor dataset (top) and the Laval Indoor Spatially Varying HDR dataset (below). (Please zoom in to see details)
Figure 8. Visualizations of object insertion and the estimated HDRI maps from multi-view input images.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Scene
Method
DR-Error ↓
JS ↓
EMD ↓
Laval Indoor
StyleLight
5.541
0.097
0.718
DiffusionLight
3.538
0.094
0.477
LuxDiT
2.960
0.082
0.471
Ours
2.626
0.059
0.379
Laval Outdoor
StyleLight
9.996
0.106
0.663
DiffusionLight
1.688
0.102
0.460
Appendix
Table 4. Quantitative comparison for dynamic range on the Laval Indoor and Laval Outdoor datasets. Lower is better for all metrics.
Dataset
Method
Scale-invariant RMSE ↓
Angular Error ↓
Normalized RMSE ↓
Diffuse
Matte
Mirror
Diffuse
Matte
Mirror
Diffuse
Matte
Mirror
Laval Indoor
StyleLight
0.031
0.049
0.084
3.861
3.908
3.997
0.166
0.259
0.328
DiffusionLight
0.030
0.050
0.094
3.193
3.236
3.318
0.160
0.211
0.233
LuxDiT
0.026
0.043
0.086
3.014
3.066
3.137
0.140
0.227
0.303
Ours
0.025
0.041
0.083
2.957
3.011
3.101
0.126
0.212
0.257
Laval Outdoor
StyleLight
0.099
0.122
0.171
5.922
5.983
6.140
0.269
0.253
0.239
Appendix
Table 5. Quantitative results on the Laval Indoor and Laval Outdoor datasets for the 3-Sphere-Shadow setting with shadow. Underlined entries indicate the second-best results.
Method
si-RMSE ↓
Angular-Error ↓
EverLight
0.091
6.360
StyleLight
0.123
7.090
DiffusionLight
0.090
5.250
LuxDiT
0.089
4.900
Ours
0.087
4.474
Appendix
Table 6. Quantitative results under the array of spheres protocol on the Laval Indoor dataset.
Method
Chess room
Small class
PU21-PSNR ↓
PU21-PSNR ↓
PanoHDR-NeRF
27.61
28.57
GasLight
30.29
30.77
Ours
31.28
34.70
Appendix
Table 7. Quantitative comparison for spatially-varying lighting estimation on PanoHDR-NeRF Dataset.
Figure 9. Qualitative results under the array of spheres protocol on the Laval Indoor dataset.
Figure 10. Visualization of the estimated spatially-varying illumination on in-the-wild scenes. The inserted spheres on the left correspond one-to-one to the rendered results on the right: the spheres are ordered from left to right, matching the results arranged from top to bottom.
Figure 11. Visualization of the object insertion results on the Laval Indoor Spatially Varying dataset.
Figure 12. Image conditioned panorama generation results on real-world captures.
State Key Laboratory of Multimedia Information Processing and National Engineering Research Center of Visual Technology, School of Computer Science, Peking University · Beijing Academy of Artificial Intelligence · School of Software and Microelectronics, Peking University +1