Single-image relighting is highly under-constrained: small illumination changes can produce large, nonlinear variations in shading, shadows, and specularities, while geometry and materials remain unobserved. Existing diffusion-based approaches either rely on intrinsic or G-buffer pipelines that require dense and fragile supervision, or operate purely in latent space without physical grounding, making fine-grained control of direction, intensity, and color unreliable. We observe that a full intrinsic decomposition is unnecessary and redundant for accurate relighting. Instead, sparse but physically meaningful cues, indicating where illumination should change and how materials should respond, are sufficient to guide a diffusion model. Based on this insight, we introduce LightCtrl that integrates physical priors at two levels: a few-shot latent proxy encoder that extracts compact material-geometry cues from limited PBR supervision, and a lighting-aware mask that identifies sensitive illumination regions and steers the denoiser toward shading relevant pixels. To compensate for scarce PBR data, we refine the proxy branch using a DPO-based objective that enforces physical consistency in the predicted cues. We also present ScaLight, a large-scale object-level dataset with systematically varied illumination and complete camera-light metadata, enabling physically consistent and controllable training. Across object and scene level benchmarks, our method achieves photometrically faithful relighting with accurate continuous control, surpassing prior diffusion and intrinsic-based baselines, including gains of up to +2.4 dB PSNR and 35% lower RMSE under controlled lighting shifts.
Figures & tables
Figure 2: Overview of LightCtrl and the object-level rendering pipeline. Our model integrates a few-shot latent proxy, a lighting-aware mask, and a DPO-refined PBR encoder to condition the UNet during denoising. The implicit PBR encoder extracts physically-based latent proxy features tphys , which are concatenated with image tokens timage and light condition tokens tlight to guide the diffusion process. A lighting mask predictor takes the lighting change Δℓ as input to produce a spatially-aware attention mask for the UNet. The output column shows relit results under three different lighting configurations ( Δℓ1 , Δℓ2 , Δℓ3 ), demonstrating the model’s ability to produce diverse and physically plausible relighting effects from a single input image.
Dataset
#Objects
Lighting Control
Pose / Light
OpenIllumi [ 17 ]
64
∼
×
OWL [ 32 ]
8
∼
∼
Rene [ 31 ]
20
∼
×
LightProp [ 11 ]
80K+
✓
×
ScaLight (Ours)
300K+
✓
✓
Table 1: Comparison with other object-level datasets. ScaLight features large-scale scalability, detailed camera and light metadata (recording exact camera and illumination parameters), and fully controllable lighting setups (supporting diverse and flexible lighting effects). Symbols: ✓ full; ∼ limited; × none.
Method
RMSE ↓
SSIM ↑
PSNR ↑
RGB ↔ X [ 40 ]
0.389
0.425
8.25
IC-Light [ 41 ]
0.413
0.337
7.94
LumiNet [ 35 ]
0.139
0.904
17.20
Ours
0.167
0.655
18.30
Table 2: Scene-level relighting comparison on the MIIW testset. Bold indicates the best performance.
Method
Label
Temperature (CCT shift)
Position (yaw/pitch)
Energy (intensity)
RMSE ↓
SSIM ↑
PSNR ↑
RMSE ↓
SSIM ↑
PSNR ↑
RMSE ↓
SSIM ↑
PSNR ↑
IC-Light [ 41 ]
text
0.397
0.346
8.21
0.375
0.363
8.65
0.380
0.382
8.63
DreamLight [ 19 ]
text
0.349
0.559
9.40
0.221
0.662
13.3
0.221
0.671
13.5
x2rgb [ 40 ]
G-buffer
0.472
0.493
7.81
0.487
0.592
8.32
0.484
0.676
8.89
LBM [ 5 ]
image
0.064
0.957
27.8
0.084
0.936
23.1
0.073
0.956
25.3
Luminet [ 35 ]
image
0.172
0.848
15.8
0.146
0.847
17.8
0.164
0.825
16.2
Table 3: Quantitative comparison on relit color output under three lighting variations. All metrics are computed on the normalized RGB range [−1,1] . Temperature : correlated color temperature (CCT) shift; Position : light direction/placement change (yaw/pitch); Energy : light intensity scaling.
Figure 3: Visual comparison of intrinsic decomposition on a chair object. From left to right: ground-truth (GT), our method with DPO, our method without DPO, rgb2x [ 40 ] , and Neural_Lightrig [ 11 ] . The DPO fine-tuning clearly suppresses artifacts and produces more accurate albedo and normal predictions, demonstrating the effectiveness of our DPO strategy.
Figure 4: Qualitative results on in-the-wild images. Our model produces realistic relighting on Internet and product photos (right). Hand-captured examples confirm robust generalization to uncontrolled lighting.
Figure 5: Object-level relighting comparison. Given a single input image (left), we compare our method against IC-Light, Qwen Image Edit, RGB-X, and DiLightNet. Competing methods often introduce strong color shifts, texture distortions, or inconsistent shading, while our approach produces well-relighting results that preserve materials, geometry, and object identity. These examples demonstrate our model’s superior ability to perform physically plausible object-centric relighting.
Method
User Preference Rate ( ↑ )
Scene-Level
Object-Level
LumiNet [ 35 ]
12.83%
4.3%
IC-Light [ 41 ]
9.37%
11.45%
NanoBanana
22.07%
2.8%
Ours
55.73%
81.45%
Table 4: User preference study ( N=35 ). We evaluate human preference separately on complex real-world scenes (MIIW and RWR) and controlled objects (ScaLight). Our method significantly outperforms existing baselines, achieving the highest preference rate in both settings.
Figure 6: Scene-level relighting comparison on indoor environments. From left to right: input, ground-truth relighting, our method, ChatGPT Image, and LumiNet. Our approach produces realistic global illumination and consistent local shading, especially around the reflective light probes highlighted by red boxes, whereas the generative baselines tend to make color shifts or over-smoothed shading.
Figure 7: Targeted real-scene failure analysis on MIIW and RWR [ 39 ] , showing representative errors in shadow recasting, structure preservation, and highlight-heavy regions.
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 8: Illustration of our controllable lighting formulation. A lighting edit is first defined in yaw–pitch space (e.g., Light A → Light B with Δyaw=90∘ , Δpitch=0∘ ). These angular parameters are mapped to unit vectors on the hemisphere, then projected onto a low-order spherical-harmonic (SH) basis. The resulting SH difference ΔsSH forms the directional component of our relative illumination encoding Δℓ .
#PBR Samples
RMSE ↓
SSIM ↑
PSNR ↑
1K
0.468
0.487
12.8
3K
0.421
0.534
14.7
6K
0.331
0.612
19.3
9K
0.194
0.873
23.5
12K
0.105
0.921
28.6
Appendix
Table 5: Effect of PBR supervision scale on relighting quality. Increasing PBR annotations steadily improves performance, yet our experiments show that even sparse supervision provides meaningful benefits for relighting tasks.