Existing illumination estimation methods primarily recover lighting distributions for rendering, but typically leave the contributions of individual physical light sources at specific scene locations implicit. We address this gap by leveraging the scene understanding capabilities of vision-language models (VLMs) to connect local illumination with its physical sources. We introduce LAMP, a framework for point-conditioned, source-aware illumination reasoning from a single RGB image with marked query points. LAMP constructs a shared inventory of active light sources, ranks contributing sources for each query, and predicts their 3D query-to-source directions. We supervise these relationships using procedurally controlled indoor scenes, reusing independently rendered source responses to derive contribution targets across lighting configurations and obtaining exact directions from source and query positions. On the Infinigen test set, fine-tuning Qwen3-VL-8B increases dominant-source 3D [email protected] from 0% to 70.6%, with 94.5% of directions within 12.5 degrees of ground truth among correctly localized sources. Combined with a lightweight network predicting dense RGB irradiance, LAMP achieves probe-rendering quality comparable to prior lighting-estimation methods while additionally localizing individual physical sources in 3D and explicitly attributing local illumination to them.
Figures & tables
Figure 1: Visualization of source-referenced illumination. Left: predicted 3D source positions and dominant-source attribution, with colored lines connecting each query to its predicted dominant source. Right: probes rendered using the predicted source information.
Figure 2: Pipeline Overview. Given the focal-length-normalized image with query markers and a text prompt, the VLM produces the active source inventory (Stage 1) and the query-conditioned records (Stage 2). During training, two auxiliary objectives, contribution-share attribution and position regression, act on the identifier tokens and are discarded afterwards. A dense network predicts RGB irradiance from the unmarked image. At inference, the ranked sources and the irradiance map are composed to render a probe at any query.
Figure 3: Per-source contributions. The composite render is shown with solo-source responses used to recover each active source’s emission-normalized contribution Cskl . Contributions are overlaid on the grayscale composite, with contours marking the top 3% of values. Overlays are for visualization only; share labels use the continuous linear-RGB maps.
Table 1: Source localization and query-to-source direction. Rank 1 denotes the dominant source. Direction accuracy (%) and median error are conditioned on [email protected] m; “–” denotes unavailable or undefined results. Median angular error here is computed on valid predictions.
Figure 4: Inference cost for 100 queries. Sharing predictions across queries substantially reduces runtime.
Figure 5: Qualitative probe rendering across datasets. Two scenes per dataset and the diffuse spheres are inserted at identical, manually selected query locations for all methods.
Physical ( Nq =110)
Aligned 5∘ ( Nq =59)
Method
Acc.@ 22.5∘ (%)
Med. ( ∘ )
Acc.@ 22.5∘ (%)
Med. ( ∘ )
IllumiDiff (SG)
9.1
63.8
8.5
62.8
DiffusionLight-Turbo (global)
2.7
80.3
1.7
90.8
DiffusionLight-Turbo (point)
3.6
85.4
1.7
92.3
LiMo
26.4
51.1
32.2
45.7
Ours
86.4
4.1
96.6
3.3
Table 2: Query-to-source direction accuracy on source-aligned Infinigen. Our method substantially outperforms lighting-estimation baselines under source-aligned evaluation.
Table 3: Active-source inventory localization. Our method achieves competitive 3D inventory localization without input depth maps. The fine-tuned detector baseline reaches similar performance only when supplied with ground-truth depth.
Dataset
Method
LPIPS ↓
RMSE ↓
si-RMSE ↓
SSIM ↑
RGB Ang. ( ∘ ) ↓
Infinigen
LiMo
0.156
49.138
1.199
0.818
7.79
DiffusionLight-Turbo
0.198
2.518
1.487
0.748
10.60
LuxDiT
0.180
2.014
1.458
0.754
7.92
GaSLight
0.280
2.605
1.467
0.684
14.17
Ours
0.157
1.655
0.963
0.831
6.49
Laval-SV
( Garon et al., 2019 )
0.092
1.004
0.613
0.887
6.95
Table 4: Probe rendering and generalization to external real-world datasets. Across Infinigen and public real-world datasets, our method achieves competitive rendering quality while additionally recovering explicit physical sources in 3D.
Table 5: Training-objective ablation. CE denotes autoregressive cross-entropy. Attribution and position denote auxiliary contribution-share supervision and position regression. Full combines all three objectives.
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Render-level validation of emission-normalized superposition. The left panel shows a composite generated directly by the renderer, the middle panel its reconstruction using Loff+∑k=1KlCskl , and the right panel the absolute pixel-wise difference, amplified by 3× for visibility. The reconstructed and directly rendered composites closely agree in linear RGB radiance space.
Table 6: Full dominant-source results. Conditional direction metrics are restricted to queries whose dominant source is localized within 0.5 m; the all-query block scores every query, counting a missing or unparsable direction as incorrect, with medians over valid outputs, and Valid is the share of queries whose top-ranked entry carries a parsable 3D direction. + GT depth retains predicted source pixels and replaces their depth with ground truth, which leaves the predicted directions unchanged. Dashes denote undefined results.
Table 7: Source-referenced results by number of active sources (Infinigen test set). Metrics as in Table 1 ; the all-query direction block and Valid are defined as in Table 6 .
Set
Dominant source
N (%)
τ =0.25 m
τ =0.5 m
τ =1.0 m
Infinigen
in frame
976 (94.6)
51.7
73.9
84.9
Infinigen
off frame
56 (5.4)
5.4
14.3
33.9
LuxRemix
in frame
5,412 (57.5)
15.9
42.0
60.6
LuxRemix
off frame
4,000 (42.5)
2.2
10.2
26.3
Appendix
Table 8: Dominant-source localization by visibility of the source (Ours). Queries are split by whether the ground-truth dominant source lies inside the image frame. 3D PCK@ τ as in Table 1 .
Table 9: 3D query-to-source direction extraction from ground-truth lighting. Errors are relative to the original query-to-source target on 110 queries from 41 scenes. Parentheses show results on the broader 161-query pool from 52 scenes. Median errors are in degrees and angular accuracies in percent.
Physical (110)
Aligned 5∘ (59)
Aligned 10∘ (60)
Model
Acc.@ 12.5∘ (%)
Acc.@ 22.5∘ (%)
Med. ( ∘ )
Acc.@ 12.5∘ (%)
Acc.@ 22.5∘ (%)
Med. ( ∘ )
Acc.@ 12.5∘ (%)
Acc.@ 22.5∘ (%)
Med. ( ∘ )
Methods included in subset verification
IllumiDiff (SG)
3.6
9.1
63.8
1.7
8.5
62.8
1.7
8.3
62.6
DiffusionLight-Turbo (global reference)
1.8
2.7
80.3
1.7
1.7
90.8
1.7
1.7
90.6
DiffusionLight-Turbo (point)
0.9
3.6
85.4
1.7
1.7
92.3
1.7
1.7
92.3
LiMo
10.9
26.4
51.1
13.6
32.2
45.7
13.3
33.3
45.5
Appendix
Table 10: 3D query-to-source direction on ground-truth-verified Infinigen subsets. Physical includes 110 queries from 41 scenes. Aligned 5∘ and 10∘ include 59 and 60 queries, respectively, each from 29 scenes. Angular accuracies are in percent (higher is better), and median errors in degrees (lower is better).
Method
R off @0.25 m ↑
R off @0.5 m ↑
R off @1.0 m ↑
Qwen3-VL-8B (zero-shot)
0.0 (0/24)
0.0 (0/24)
0.0 (0/24)
YOLOv8n (fine-tuned) + GT depth
4.2 (1/24)
4.2 (1/24)
4.2 (1/24)
YOLOv8n (fine-tuned) + DA depth
0.0 (0/24)
0.0 (0/24)
0.0 (0/24)
Ours
4.2 (1/24)
12.5 (3/24)
25.0 (6/24)
Appendix
Table 11: Recall of off-frame ground-truth sources (Infinigen test set, 24 of 329 sources). Same matching as Table 3 ; counts of recovered sources in parentheses.
Decoded dominant source (%)
Switch failures (%) ↓
Stable
Attribution head
Model
Pre
Post
All
Never
Premature
(%) ↑
Share MAE ↓
Response (%) ↑
Ours
56.7
46.2
51.4
40.9
47.7
61.9
0.220
87.3
Appendix
Table 12: Response to lighting interventions. In each sequence one source (B) is brightened in steps relative to a second source (A) at a fixed query, crossing the point where B becomes dominant. Pre and Post are the accuracies of the decoded dominant source before and after the crossing, Never and Premature are the fractions of sequences in which the decoded source never switches to B after the crossing or switches before it, Stable is the fraction of pre-crossing frame pairs whose decoder margin keeps its sign, and Share MAE and Response are the error of the attribution-head shares against the rendered shares and the fraction of steps whose share change has the correct sign.
Rank 1 ( Nq =1,032)
Rank 2 ( Nq =586)
Rank 3 ( Nq =169)
Training objective
0.25 m
0.5 m
1.0 m
0.25 m
0.5 m
1.0 m
0.25 m
0.5 m
1.0 m
CE only
47.3
69.7
78.2
29.7
45.9
51.0
31.4
37.3
42.0
CE + attribution
48.3
71.3
82.8
29.0
49.5
57.2
24.9
37.9
45.0
CE + position
45.0
68.5
79.3
24.2
41.8
51.4
23.1
32.5
37.9
Full
49.2
70.6
82.2
31.4
49.5
56.8
32.0
40.2
46.2
Appendix
Table 13: Training-objective ablation by source rank (Infinigen test set). 3D PCK (%) of the predicted source at rank r for the models of Table 5 ; Nq counts queries with at least r ground-truth sources.
Readout
Criterion
Result
Verdict
Reference, true source radiometry rendered with our renderer
highlight <10∘
8∘
pass
Linear readout of the final prompt token (fine-tuned / base)
ρ≥0.70
ρ 0.28 / 0.50
fail
Linear readout of the visual tokens around the marker
ρ≥0.70 , highlight ≤18∘
ρ 0.43, 21∘
fail
Convolutional adapter on the image and the structured output (54 scenes)
val ρ≥0.56 , MAE ≤0.65
ρ 0.46, MAE 0.72 (train 0.06)
fail
Scene-count curve, 15 to 84 scenes
MAE ≤0.50 , ρ≥0.56
MAE 0.55, ρ 0.55, bias −0.47
fail
Appendix
Table 14: Reading absolute radiometry from the VLM. ρ is the Spearman correlation between predicted and ground-truth log-irradiance across scenes, MAE is in log units, and the highlight error is the angular error of the rendered probe’s highlight. Criteria were fixed before each test.
Emitters
Shading dir. ( ∘ ) ↓
Single predicted direction
45.1
Share-weighted composition (Ours)
23.7
All ground-truth sources and shares
17.7
Appendix
Table 15: Emitter construction with a fixed irradiance map (Infinigen held-out probes). Median shading-direction error over the 49 diffuse probes of the 17-scene held-out set (fill fraction 0.65).
Ill-posed inverse problems require priors to constrain the solution space toward plausible outcomes. In inverse rendering, learned priors modeling the distribution of natural illumination improve the recovery of scene properties. However, existing models rely on the distant-illumination assumption, representing lighting as a far-field environment map. This limits their applicability to indoor scenes, where illumination is highly spatially varying due to finite-distance emitters, visibility changes, and parallax, all of which are poorly approximated by a single environment map. To address this, we introduce a spatially-aware illumination prior trained on real-world indoor panoramas and their estimated depth. Our variational autoencoder model learns a compact, optimizable latent space that decodes into HDR radiance and depth, parameterizing an area light emitter for direct integration into standard differentiable rendering pipelines. This design bridges the plausibility guarantees of a learned prior with the gradient flow required for downstream optimization. Crucially, by jointly modeling radiance and depth, our prior captures the spatial structure of indoor illumination, instead of treating the light sources as infinitely distant. We demonstrate that this formulation enables spatially-varying illumination modeling and achieves higher-fidelity recovery of indoor lighting compared to existing approaches. Project page: https://andreead-a.github.io/RoomLight
Shadows are observable consequences of latent physical causes, arising from the interaction between illumination, object geometry, and scene structure. A model that understands shadows should be able not only to read a shadow's direction, but also to infer where a shadow should fall when it is absent, and to judge whether a shadow is physically consistent with the scene. We introduce ShadowBench, built from real photographs, which poses these three questions to vision-language models (VLMs). From each scene, we build a correct-shadow, a shadow-removed, and a human-annotated wrong-direction state. Evaluating 19 open VLMs (2B to 78B) and frontier proprietary models, we find that VLMs read a visible shadow's direction well above chance and improve with scale, yet infer the absent direction only weakly and fall to chance at judging physical consistency even under a bias-free two-alternative test that removes the yes-bias of single-image naturalness questions. Only Gemini-3.1-pro exceeds chance on validity, and even then only modestly and far below humans. Chain-of-Thought (CoT) does not help. The missing ingredient is not reasoning but knowledge that image generation models (IGMs) already hold. Routing a VLM through an IGM lifts its inference well above its direct attempt and helps it flag the wrong-direction shadows it otherwise accepts. Cast-shadow direction is a single facet of illumination understanding, but the sharp gap it reveals between VLMs and IGMs suggests image-generation or reconstruction objectives as a route toward illumination-aware VLMs.
Ryugo Morita, Shunsuke Kitada, Stanislav Frolov +5
DFKI, Germany · RPTU Kaiserslautern-Landau, Germany · Hosei University, Japan
We propose a neural formulation for estimating the appearance of complex luminaires. We focus on challenging luminaires with complex light transport (e.g., small emitters enclosed by multiple specular layers) that are difficult for (bidirectional) path tracing. To this end, we use light tracing to construct paths from emitters to the exit surfaces and formulate appearance estimation as a distribution learning problem. Specifically, we model the probability density function (pdf) of outgoing radiance on the exit surfaces using a large normalizing flow network, and recover the outgoing radiance as the product of the estimated pdf and flux. To enable efficient inference, we distill the learned appearance into a lightweight MLP that directly estimates radiance on the exit surfaces. We additionally train a sampling network for effective direct illumination computation from the luminaire, and a blending network to composite the luminaire into the scene. Our formulation makes it feasible to render challenging luminaires using low sample counts in arbitrary scenes.
Pedro Figueiredo, Zixuan Li, Beibei Wang +2
Texas A&M University, USA · Nankai University, China · Nanjing University of Science and Technology, China +1