LoRA Direction Extraction for Controllable Light Toggling in FLUX.1 Kontext

Petr Golenderov; Dmitry Mazyar; Natalia Sovpel; Alexander Aksenov

cs.CVSep 29, 2026

LoRA Direction Extraction for Controllable Light Toggling in FLUX.1 Kontext

Authors: Petr Golenderov, Dmitry Mazyar, Natalia Sovpel, Alexander Aksenov

Organizations: BURO.io Research Lab

Abstract

We propose a fine-tuning method for flow-matching diffusion models aimed at realistic artificial light modeling without the need for a large training dataset. We address the task of controllable interior image editing, where the goal is to turn artificial light sources on or off while preserving the scene geometry, object placement, materials, and visual identity of the original image. To achieve this, we decompose the task into two independent formulations. We introduce the LoRA Direction Training Method, which extracts the pure direction of the LoRA adapter effect in the diffusion model flow field, and we also introduce specialized loss functions to ensure the realism of the inverse transformation. Additionally, the resulting increment map is used for more precise adjustment of the lighting color and temperature.

Figures & tables

Method	MUSIQ $\uparrow$	ARNIQA $\uparrow$	LIQE $\uparrow$	DirCLIP $\uparrow$	AugCLIP $\uparrow$	State $\uparrow$	Physics $\uparrow$	Preserv. $\uparrow$	Photo $\uparrow$	Cheat $\downarrow$	$S_{\mathrm{total}}\uparrow$
Ours	63.649	0.653	3.976	0.0355	0.7986	1.923	1.692	1.692	1.692	7.7%	0.6076
LightLab [ 8 ]	71.370	0.698	4.481	0.0329	0.7937	2.000	1.400	1.500	1.600	30.0%	0.5907
Qwen	68.247	0.659	3.765	0.0393	0.7826	1.833	1.417	1.333	1.667	16.7%	0.5762
Flux2Klein	69.332	0.663	4.260	0.0362	0.7621	1.700	1.400	1.500	1.600	20.0%	0.5753
LuxRemix [ 10 ]	63.283	0.670	3.596	0.0310	0.7776	1.750	1.000	1.000	1.000	50.0%	0.4969

Table 1: Quantitative assessment of the results of turning on the light. The initial metrics are presented prior to normalization.

Method	MUSIQ $\uparrow$	ARNIQA $\uparrow$	LIQE $\uparrow$	DirCLIP $\uparrow$	AugCLIP $\uparrow$	State $\uparrow$	Physics $\uparrow$	Preserv. $\uparrow$	Photo $\uparrow$	Cheat $\downarrow$	$S_{\mathrm{total}}\uparrow$
Ours	66.716	0.678	4.121	0.0189	0.6509	1.688	1.188	1.188	1.438	31.25%	0.5312
LightLab [ 8 ]	63.350	0.660	3.470	0.0488	0.6111	1.667	1.000	1.333	1.667	31.3%	0.5179
Flux2Klein	70.455	0.673	4.681	0.0263	0.6624	1.688	1.000	1.063	1.313	62.5%	0.5020
LuxRemix [ 10 ]	47.042	0.696	3.340	0.0481	0.4002	2.000	1.000	1.000	1.000	35.0%	0.4742
Qwen	53.304	0.557	2.387	0.0693	0.6211	1.500	1.000	1.000	1.125	93.8%	0.3957

Table 2: Quantitative assessment of the results of turning off the lights. The initial metrics are presented prior to normalization.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 1: Qualitative comparison between our method and LightLab for light activation.

Figure 2: Qualitative comparison between our method and LuxRemix for light activation.

Figure 3: Qualitative comparison between our method and Qwen for light activation.

Figure 4: Qualitative comparison between our method and Flux 2 Klein for light activation.

Figure 5: Qualitative comparison between our method and LightLab for light deactivation.

Figure 6: Qualitative comparison between our method and LuxRemix for light deactivation.

Figure 7: Qualitative comparison between our method and Qwen for light deactivation.

Figure 8: Qualitative comparison between our method and Flux 2 Klein for light deactivation.

Diffusion Model Fine-Tuning

Diffusion model fine-tuning adapts pretrained generative models to new concepts, tasks, domains, and preferences through targeted updates.

See all 181 papers

LightMover: Generative Light Movement with Color and Intensity Controls
Mar 28, 2026Gengze Zhou, Tianyu Wang, Soo Ye Kim +7Image Relighting Controllable Image Generation
AIML, Adelaide University · Adobe Research · University of Hong Kong +1
Learning Illumination Control in Diffusion Models
Apr 27, 2026Nishit Anand, Manan Suri, Christopher Metzler +2Diffusion Model Fine-Tuning Image Relighting
University of Maryland College Park
LGTM: Training-Free Light-Guided Text-to-Image Diffusion Model via Initial Noise Manipulation
Mar 25, 2026Ryugo Morita, Stanislav Frolov, Brian Bernhard Moser +3Controllable Image Generation T2I Diffusion Models
RPTU Kaiserslautern-Landau & DFKI GmbH, Kaiserslautern, Germany · Faculty of Science and Engineering, Hosei University, Tokyo, Japan

Mar 28, 2026J/K move · Enter open · S save

LightMover: Generative Light Movement with Color and Intensity Controls

Gengze Zhou, Tianyu Wang, Soo Ye Kim, Zhixin Shu+6

AIML, Adelaide University · Adobe Research · University of Hong Kong+1

Image Relighting Controllable Image Generation Diffusion-Based Image Editing

Open paper AskPDF arXiv

We present LightMover, a framework for controllable light manipulation in single images that leverages video diffusion priors to produce physically plausible illumination changes without re-rendering the scene. We formulate light editing as a sequence-to-sequence prediction problem in visual token space: given an image and light-control tokens, the model adjusts light position, color, and intensity together with resulting reflections, shadows, and falloff from a single view. This unified treatment of spatial (movement) and appearance (color, intensity) controls improves both manipulation and illumination understanding. We further introduce an adaptive token-pruning mechanism that preserves spatially informative tokens while compactly encoding non-spatial attributes, reducing control sequence length by 41% while maintaining editing fidelity. To train our framework, we construct a scalable rendering pipeline that generates large numbers of image pairs across varied light positions, colors, and intensities while keeping the scene content consistent with the original image. LightMover enables precise, independent control over light position, color, and intensity, and achieves high PSNR and strong semantic consistency (DINO, CLIP) across different tasks.

Abstract

Diffusion Model Fine-Tuning

Explore similar work

LightMover: Generative Light Movement with Color and Intensity Controls

Learning Illumination Control in Diffusion Models

LGTM: Training-Free Light-Guided Text-to-Image Diffusion Model via Initial Noise Manipulation