LAMP: 3D Source-Aware Lighting Estimation at Multiple Points

Yanqiao Wang; Jin Sun

cs.CVOct 13, 2026

LAMP: 3D Source-Aware Lighting Estimation at Multiple Points

Authors: Yanqiao Wang, Jin Sun

Organizations: School of Computing, University of Georgia, Athens, GA 30602, USA

Abstract

Existing illumination estimation methods primarily recover lighting distributions for rendering, but typically leave the contributions of individual physical light sources at specific scene locations implicit. We address this gap by leveraging the scene understanding capabilities of vision-language models (VLMs) to connect local illumination with its physical sources. We introduce LAMP, a framework for point-conditioned, source-aware illumination reasoning from a single RGB image with marked query points. LAMP constructs a shared inventory of active light sources, ranks contributing sources for each query, and predicts their 3D query-to-source directions. We supervise these relationships using procedurally controlled indoor scenes, reusing independently rendered source responses to derive contribution targets across lighting configurations and obtaining exact directions from source and query positions. On the Infinigen test set, fine-tuning Qwen3-VL-8B increases dominant-source 3D [email protected] from 0% to 70.6%, with 94.5% of directions within 12.5 degrees of ground truth among correctly localized sources. Combined with a lightweight network predicting dense RGB irradiance, LAMP achieves probe-rendering quality comparable to prior lighting-estimation methods while additionally localizing individual physical sources in 3D and explicitly attributing local illumination to them.

Figures & tables

Panel of Figure 1 — Figure 1: Visualization of source-referenced illumination. Left: predicted 3D source positions and dominant-source attribution, with colored lines connecting each query to its predicted dominant source. Right: probes rendered using the predicted source information.

Figure 2: Pipeline Overview. Given the focal-length-normalized image with query markers and a text prompt, the VLM produces the active source inventory (Stage 1) and the query-conditioned records (Stage 2). During training, two auxiliary objectives, contribution-share attribution and position regression, act on the identifier tokens and are discarded afterwards. A dense network predicts RGB irradiance from the unmarked image. At inference, the ranked sources and the irradiance map are composed to render a probe at any query.

Figure 3: Per-source contributions. The composite render is shown with solo-source responses used to recover each active source’s emission-normalized contribution $\mathbf{C}_{s_{k}}^{l}$ . Contributions are overlaid on the grayscale composite, with contours marking the top 3% of values. Overlays are for visualization only; share labels use the continuous linear-RGB maps.

Model / Rank	$N_{q}$	3D PCK (%) $\uparrow$			Direction $\mid$ [email protected] m
		0.25 m	0.5 m	1.0 m	$12.5^{\circ}\!\uparrow$	$22.5^{\circ}\!\uparrow$	$30^{\circ}\!\uparrow$	Med. ( ∘ ) $\downarrow$
Infinigen
Qwen3-VL-8B (zero-shot)	1,032	0.0	0.0	0.0	–	–	–	–
Ours, rank 1	1,032	49.2	70.6	82.2	94.5	98.2	98.8	2.7
Ours, rank 2	586	31.4	49.5	56.8	97.6	99.0	99.3	2.1
Ours, rank 3	169	32.0	40.2	46.2	92.6	97.1	98.5	2.6

Table 1: Source localization and query-to-source direction. Rank 1 denotes the dominant source. Direction accuracy (%) and median error are conditioned on [email protected] m; “–” denotes unavailable or undefined results. Median angular error here is computed on valid predictions.

Figure 4: Inference cost for 100 queries. Sharing predictions across queries substantially reduces runtime.

Panel of Figure 5 — Figure 5: Qualitative probe rendering across datasets. Two scenes per dataset and the diffuse spheres are inserted at identical, manually selected query locations for all methods.

Method	Acc.@ $22.5^{\circ}$ (%)	Med. ( ∘ )	Acc.@ $22.5^{\circ}$ (%)	Med. ( ∘ )
	Physical ( $N_{q}$ =110)		Aligned $5^{\circ}$ ( $N_{q}$ =59)
IllumiDiff (SG)	9.1	63.8	8.5	62.8
DiffusionLight-Turbo (global)	2.7	80.3	1.7	90.8
DiffusionLight-Turbo (point)	3.6	85.4	1.7	92.3
LiMo	26.4	51.1	32.2	45.7
Ours	86.4	4.1	96.6	3.3

Table 2: Query-to-source direction accuracy on source-aligned Infinigen. Our method substantially outperforms lighting-estimation baselines under source-aligned evaluation.

Method	[email protected]	[email protected]	[email protected]	[email protected]	[email protected]
Qwen3-VL-8B (zero-shot)	0.0	0.0	0.0	0.0	0.0
YOLOv8n + DAv2 depth	0.7	2.1	1.8	2.0	18.7
YOLOv8n + GT depth	78.4	87.9	75.1	81.0	87.5
Ours	52.0	76.9	76.0	76.5	87.5

Table 3: Active-source inventory localization. Our method achieves competitive 3D inventory localization without input depth maps. The fine-tuned detector baseline reaches similar performance only when supplied with ground-truth depth.

Dataset	Method	LPIPS $\downarrow$	RMSE $\downarrow$	si-RMSE $\downarrow$	SSIM $\uparrow$	RGB Ang. ( ∘ ) $\downarrow$
Infinigen	LiMo	0.156	49.138	1.199	0.818	7.79
	DiffusionLight-Turbo	0.198	2.518	1.487	0.748	10.60
	LuxDiT	0.180	2.014	1.458	0.754	7.92
	GaSLight	0.280	2.605	1.467	0.684	14.17
	Ours	0.157	1.655	0.963	0.831	6.49
Laval-SV	( Garon et al., 2019 )	0.092	1.004	0.613	0.887	6.95

Table 4: Probe rendering and generalization to external real-world datasets. Across Infinigen and public real-world datasets, our method achieves competitive rendering quality while additionally recovering explicit physical sources in 3D.

Training objective	[email protected] $\uparrow$	[email protected] $\uparrow$	[email protected] $\uparrow$	Dir.@ $22.5^{\circ}$ $\uparrow$
CE only	47.3	69.7	78.2	83.4
CE + attribution	48.3	71.3	82.8	83.3
CE + position	45.0	68.5	79.3	83.7
Full	49.2	70.6	82.2	84.6

Table 5: Training-objective ablation. CE denotes autoregressive cross-entropy. Attribution and position denote auxiliary contribution-share supervision and position regression. Full combines all three objectives.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: Render-level validation of emission-normalized superposition. The left panel shows a composite generated directly by the renderer, the middle panel its reconstruction using $\mathbf{L}_{\mathrm{off}}+\sum_{k=1}^{K^{l}}\mathbf{C}_{s_{k}}^{l}$ , and the right panel the absolute pixel-wise difference, amplified by $3\times$ for visibility. The reconstructed and directly rendered composites closely agree in linear RGB radiance space.

Infinigen (all 1,032 test queries)
	3D PCK (%) $\uparrow$			Direction $\mid$ [email protected] m				Direction, all queries
Model	0.25 m	0.5 m	1.0 m	Accuracy (%) $\uparrow$			Med. ( ∘ ) $\downarrow$	Accuracy (%) $\uparrow$			Med. ( ∘ ) $\downarrow$	Valid (%)
				$12.5^{\circ}$	$22.5^{\circ}$	$30^{\circ}$		$12.5^{\circ}$	$22.5^{\circ}$	$30^{\circ}$
Qwen3-VL-8B (zero-shot)	0.0	0.0	0.0	–	–	–	–	2.4	7.2	10.3	99.4	91.7
+ GT depth	0.7	1.6	5.0	0.0	0.0	0.0	85.0	2.4	7.2	10.3	99.4	91.7
Qwen3-VL-32B (zero-shot)	0.0	0.0	0.0	–	–	–	–	0.4	2.1	3.4	107.5	90.8

Appendix

Table 6: Full dominant-source results. Conditional direction metrics are restricted to queries whose dominant source is localized within 0.5 m; the all-query block scores every query, counting a missing or unparsable direction as incorrect, with medians over valid outputs, and Valid is the share of queries whose top-ranked entry carries a parsable 3D direction. + GT depth retains predicted source pixels and replaces their depth with ground truth, which leaves the predicted directions unchanged. Dashes denote undefined results.

Single Source (348 queries, 59 images)
	3D PCK (%) $\uparrow$			Dir $\mid$ [email protected] m (%) $\uparrow$			Med.	Dir, all queries (%) $\uparrow$			Med.	Valid
Model	0.25 m	0.5 m	1.0 m	$12.5^{\circ}$	$22.5^{\circ}$	$30^{\circ}$	( ∘ ) $\downarrow$	$12.5^{\circ}$	$22.5^{\circ}$	$30^{\circ}$	( ∘ ) $\downarrow$	(%)
Qwen3-VL-8B (zero-shot)	0.0	0.0	0.0	–	–	–	–	3.4	8.0	10.9	98.4	86.8
Qwen3-VL-32B (zero-shot)	0.0	0.0	0.0	–	–	–	–	0.6	2.0	3.2	108.5	91.4
Ours	52.6	82.5	94.0	96.5	99.0	99.7	2.4	93.1	96.3	97.4	2.6	100.0
Multi Sources (684 queries, 109 images)

Appendix

Table 7: Source-referenced results by number of active sources (Infinigen test set). Metrics as in Table 1 ; the all-query direction block and Valid are defined as in Table 6 .

Set	Dominant source	$N$ (%)	$\tau$ =0.25 m	$\tau$ =0.5 m	$\tau$ =1.0 m
Infinigen	in frame	976 (94.6)	51.7	73.9	84.9
Infinigen	off frame	56 (5.4)	5.4	14.3	33.9
LuxRemix	in frame	5,412 (57.5)	15.9	42.0	60.6
LuxRemix	off frame	4,000 (42.5)	2.2	10.2	26.3

Appendix

Table 8: Dominant-source localization by visibility of the source (Ours). Queries are split by whether the ground-truth dominant source lies inside the image frame. 3D PCK@

\tau

as in Table 1 .

Representation	Median ( ∘ )	$\leq 5^{\circ}$ (%)	$\leq 10^{\circ}$ (%)
Environment map (DiffusionLight-Turbo)	0.3 (0.3)	92 (94)	93 (95)
12 spherical Gaussians (IllumiDiff)	1.2 (1.2)	69 (68)	70 (68)
Mirror ball (LiMo)	0.5 (0.5)	77 (78)	78 (78)
Order-3 spherical harmonics (UniLight ( Zhang et al., 2026 ) )	9.1 (9.4)	23 (20)	55 (53)
96 sphere anchors (EMLight Zhan et al. (2021) )	8.2 (8.0)	17 (18)	65 (67)

Appendix

Table 9: 3D query-to-source direction extraction from ground-truth lighting. Errors are relative to the original query-to-source target on 110 queries from 41 scenes. Parentheses show results on the broader 161-query pool from 52 scenes. Median errors are in degrees and angular accuracies in percent.

	Physical (110)			Aligned $5^{\circ}$ (59)			Aligned $10^{\circ}$ (60)
Model	Acc.@ $12.5^{\circ}$ (%)	Acc.@ $22.5^{\circ}$ (%)	Med. ( ∘ )	Acc.@ $12.5^{\circ}$ (%)	Acc.@ $22.5^{\circ}$ (%)	Med. ( ∘ )	Acc.@ $12.5^{\circ}$ (%)	Acc.@ $22.5^{\circ}$ (%)	Med. ( ∘ )
Methods included in subset verification
IllumiDiff (SG)	3.6	9.1	63.8	1.7	8.5	62.8	1.7	8.3	62.6
DiffusionLight-Turbo (global reference)	1.8	2.7	80.3	1.7	1.7	90.8	1.7	1.7	90.6
DiffusionLight-Turbo (point)	0.9	3.6	85.4	1.7	1.7	92.3	1.7	1.7	92.3
LiMo	10.9	26.4	51.1	13.6	32.2	45.7	13.3	33.3	45.5

Appendix

Table 10: 3D query-to-source direction on ground-truth-verified Infinigen subsets. Physical includes 110 queries from 41 scenes. Aligned

5^{\circ}

and

10^{\circ}

include 59 and 60 queries, respectively, each from 29 scenes. Angular accuracies are in percent (higher is better), and median errors in degrees (lower is better).

Method	R ${}_{\text{off}}$ @0.25 m $\uparrow$	R ${}_{\text{off}}$ @0.5 m $\uparrow$	R ${}_{\text{off}}$ @1.0 m $\uparrow$
Qwen3-VL-8B (zero-shot)	0.0 (0/24)	0.0 (0/24)	0.0 (0/24)
YOLOv8n (fine-tuned) + GT depth	4.2 (1/24)	4.2 (1/24)	4.2 (1/24)
YOLOv8n (fine-tuned) + DA depth	0.0 (0/24)	0.0 (0/24)	0.0 (0/24)
Ours	4.2 (1/24)	12.5 (3/24)	25.0 (6/24)

Appendix

Table 11: Recall of off-frame ground-truth sources (Infinigen test set, 24 of 329 sources). Same matching as Table 3 ; counts of recovered sources in parentheses.

	Decoded dominant source (%)			Switch failures (%) $\downarrow$		Stable	Attribution head
Model	Pre	Post	All	Never	Premature	(%) $\uparrow$	Share MAE $\downarrow$	Response (%) $\uparrow$
Ours	56.7	46.2	51.4	40.9	47.7	61.9	0.220	87.3

Appendix

Table 12: Response to lighting interventions. In each sequence one source (B) is brightened in steps relative to a second source (A) at a fixed query, crossing the point where B becomes dominant. Pre and Post are the accuracies of the decoded dominant source before and after the crossing, Never and Premature are the fractions of sequences in which the decoded source never switches to B after the crossing or switches before it, Stable is the fraction of pre-crossing frame pairs whose decoder margin keeps its sign, and Share MAE and Response are the error of the attribution-head shares against the rendered shares and the fraction of steps whose share change has the correct sign.

Training objective	0.25 m	0.5 m	1.0 m	0.25 m	0.5 m	1.0 m	0.25 m	0.5 m	1.0 m
	Rank 1 ( $N_{q}$ =1,032)			Rank 2 ( $N_{q}$ =586)			Rank 3 ( $N_{q}$ =169)
CE only	47.3	69.7	78.2	29.7	45.9	51.0	31.4	37.3	42.0
CE + attribution	48.3	71.3	82.8	29.0	49.5	57.2	24.9	37.9	45.0
CE + position	45.0	68.5	79.3	24.2	41.8	51.4	23.1	32.5	37.9
Full	49.2	70.6	82.2	31.4	49.5	56.8	32.0	40.2	46.2

Appendix

Table 13: Training-objective ablation by source rank (Infinigen test set). 3D PCK (%) of the predicted source at rank

r

for the models of Table 5 ;

N_{q}

counts queries with at least

r

ground-truth sources.

Readout	Criterion	Result	Verdict
Reference, true source radiometry rendered with our renderer	highlight $<10^{\circ}$	$8^{\circ}$	pass
Linear readout of the final prompt token (fine-tuned / base)	$\rho\geq 0.70$	$\rho$ 0.28 / 0.50	fail
Linear readout of the visual tokens around the marker	$\rho\geq 0.70$ , highlight $\leq 18^{\circ}$	$\rho$ 0.43, $21^{\circ}$	fail
Convolutional adapter on the image and the structured output (54 scenes)	val $\rho\geq 0.56$ , MAE $\leq 0.65$	$\rho$ 0.46, MAE 0.72 (train 0.06)	fail
Scene-count curve, 15 to 84 scenes	MAE $\leq 0.50$ , $\rho\geq 0.56$	MAE 0.55, $\rho$ 0.55, bias $-0.47$	fail

Appendix

Table 14: Reading absolute radiometry from the VLM.

\rho

is the Spearman correlation between predicted and ground-truth log-irradiance across scenes, MAE is in log units, and the highlight error is the angular error of the rendered probe’s highlight. Criteria were fixed before each test.

Emitters	Shading dir. ( ∘ ) $\downarrow$
Single predicted direction	45.1
Share-weighted composition (Ours)	23.7
All ground-truth sources and shares	17.7

Appendix

Table 15: Emitter construction with a fixed irradiance map (Infinigen held-out probes). Median shading-direction error over the 49 diffuse probes of the 17-scene held-out set (fill fraction 0.65).

Inverse Rendering

Inverse rendering estimates scene geometry, materials, lighting, and other causes of images from observed visual or sensor data.

See all 69 papers

RoomLight: A 2.5D Illumination Prior for Indoor Environments
Sep 23, 2026Andreea Ardelean, Bernhard EggerInverse Rendering
Friedrich-Alexander-Universität Erlangen-Nürnberg
Where Did the Light Come From? Diagnosing Cast-Shadow Reasoning in VLMs and Borrowing the Missing Shadow Prior from Image Generation Models
Oct 22, 2026Ryugo Morita, Shunsuke Kitada, Stanislav Frolov +5VLM Reasoning Physical Reasoning
DFKI, Germany · RPTU Kaiserslautern-Landau, Germany · Hosei University, Japan
PureLight: Learning Complex Luminaires with Light Tracing
Jun 3, 2026Pedro Figueiredo, Zixuan Li, Beibei Wang +2Neural Rendering Physically Based Rendering
Texas A&M University, USA · Nankai University, China · Nanjing University of Science and Technology, China +1

Sep 23, 2026J/K move · Enter open · S save

RoomLight: A 2.5D Illumination Prior for Indoor Environments

Andreea Ardelean, Bernhard Egger

Friedrich-Alexander-Universität Erlangen-Nürnberg

Inverse Rendering

Open paper AskPDF arXiv

Ill-posed inverse problems require priors to constrain the solution space toward plausible outcomes. In inverse rendering, learned priors modeling the distribution of natural illumination improve the recovery of scene properties. However, existing models rely on the distant-illumination assumption, representing lighting as a far-field environment map. This limits their applicability to indoor scenes, where illumination is highly spatially varying due to finite-distance emitters, visibility changes, and parallax, all of which are poorly approximated by a single environment map. To address this, we introduce a spatially-aware illumination prior trained on real-world indoor panoramas and their estimated depth. Our variational autoencoder model learns a compact, optimizable latent space that decodes into HDR radiance and depth, parameterizing an area light emitter for direct integration into standard differentiable rendering pipelines. This design bridges the plausibility guarantees of a learned prior with the gradient flow required for downstream optimization. Crucially, by jointly modeling radiance and depth, our prior captures the spatial structure of indoor illumination, instead of treating the light sources as infinitely distant. We demonstrate that this formulation enables spatially-varying illumination modeling and achieves higher-fidelity recovery of indoor lighting compared to existing approaches. Project page: https://andreead-a.github.io/RoomLight

Abstract

Inverse Rendering

Explore similar work

RoomLight: A 2.5D Illumination Prior for Indoor Environments

Where Did the Light Come From? Diagnosing Cast-Shadow Reasoning in VLMs and Borrowing the Missing Shadow Prior from Image Generation Models

PureLight: Learning Complex Luminaires with Light Tracing