Portrait video relighting aims to modify the illumination of a portrait video while preserving photorealism and temporal stability. The task becomes particularly challenging under dynamic lighting, where illumination direction, intensity, and color vary over time. To tackle this problem, we introduce FlashRelight, a novel two-stage framework for dynamic portrait video relighting. FlashRelight first suppresses the source illumination through delighting, and then synthesizes the target dynamic lighting on an illumination-neutral portrait video. FlashRelight supports three complementary control modalities: envmap sequences for explicit frame-level lighting control, dynamic lighting text for semantic illumination editing, and lighting reference videos for visual lighting transfer. Training follows a low-cost hybrid strategy that combines static one-light-at-a-time (OLAT) data with generated dynamic-lighting portrait videos, covering both controlled illumination variation and realistic portrait motion. Extensive experiments show that FlashRelight achieves temporally coherent and photorealistic dynamic relighting while enabling flexible cross-video lighting transfer. Code is available at https://github.com/vivoCameraResearch/FlashRelight
Figures & tables
Figure 1: FlashRelight enables portrait video relighting with dynamic lighting through three lighting control modalities: envmap sequences, lighting text, and lighting reference videos.
Figure 2: Data curation pipeline. Hybrid Dynamic Portrait Video Dataset DHybrid includes two data sources: Static OLAT Data DOLAT and Dynamic Gen Data DGen .
Figure 3: Overview of FlashRelight . Our framework consists of a delighting stage and a relighting stage. The delighting model suppresses source illumination and shadows from the input video, producing an illumination-neutral portrait video. The delighted video, together with one active dynamic lighting condition, is then used by the relighting model to synthesize the final relit video.
Figure 4: Training data for the two stages. Both the delighting stage and the relighting stage are trained with hybrid data. However, the Dynamic Gen Data DGen used for training the delighting model and that used for training the relighting model are constructed in different ways.
Figure 5: Qualitative comparison for envmap-sequence-based relighting.
Method
Lighting Acc.
Visual Quality
Video Smoothness
LE ↓
LPIPS ↓
ID ↑
PSNR ↑
tLPIPS ↓
Lite2Relight (SIGGRAPH’24) [ 27 ]
0.1423
0.4154
0.5653
15.6086
0.2721
Relighting-Cai (CVPR’24) [ 2 ]
0.1585
0.4051
0.5193
14.2131
0.2684
DiffusionRenderer (CVPR’25) [ 16 ]
0.1120
0.3781
0.6143
17.3683
0.2458
Ours
0.0584
0.1635
0.8097
23.1099
0.1303
Table 1: Quantitative results for envmap-sequence-based relighting.
Method
FID ↓
ID ↑
tLPIPS ↓
IC-Light [ 43 ]
134.98
0.4332
0.5715
Light-A-Video [ 40 ]
51.78
0.7126
0.1395
Light-X [ 50 ]
66.27
0.6762
0.2224
Ours
31.16
0.7973
0.1417
Table 2: Quantitative results for text-based relighting.
Figure 6: Qualitative comparison for lighting-text-based relighting.
Figure 7: Reference-video-guided lighting transfer. FlashRelight transfers illumination dynamics from a lighting reference video while preserving the identity and motion of the source portrait video.
Figure 8: Ablation on Hybrid Data.
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Stage
Training data
Steps
Learning rate
1
DOLAT
10,000
1×10−4
2
DGen
10,000
5×10−5
3
DOLAT
10,000
1×10−5
Appendix
Table 4: Training schedule of the delighting model.
Stage
Training data
Steps
Learning rate
1
DOLAT
10,000
1×10−4
2
DGen
10,000
5×10−5
3
DHybrid
20,000
1×10−5
Appendix
Table 5: Training schedule of the relighting model.
Method
Component
Steps
Time (s)
Memory (GiB)
FlashRelight
Delighting
50
87.5
15.17
FlashRelight
Relighting
50
98.4
15.23
FlashRelight
Total
50+50
185.9
15.23
DiffusionRenderer
Total
15
503.3
44.22
Appendix
Table 6: Stage-wise inference time and peak reserved memory. Total time sums both stages; peak memory is the maximum of their peaks.
Figure 9: Additional delighting results. The first row shows source videos, and the second row shows the corresponding delighted results.
Figure 10: Additional relighting results. FlashRelight produces temporally coherent relit videos under diverse dynamic lighting conditions.
Figure 11: Cross-domain relighting: generated portrait videos. Four examples are arranged in two columns and two rows. Each example shows seven aligned input and relit frames at t=1,9,17,25,33,41,49 , together with the target environment maps. Target maps are tone-mapped with fixed exposure within each sequence.
Figure 12: Cross-domain relighting (continued): in-the-wild portrait videos. Four examples are arranged in two columns and two rows. Each example shows seven aligned input and relit frames at t=1,9,17,25,33,41,49 , together with the target environment maps. Target maps are tone-mapped with fixed exposure within each sequence.
Figure 13: Cross-domain relighting (continued): mobile-selfie portrait videos. Four examples are arranged in two columns and two rows. Each example shows seven aligned input and relit frames at t=1,9,17,25,33,41,49 , together with the target environment maps. Target maps are tone-mapped with fixed exposure within each sequence.
Figure 14: Cross-domain relighting (continued): digital-human portrait videos. Four examples are arranged in two columns and two rows. Each example shows seven aligned input and relit frames at t=1,9,17,25,33,41,49 , together with the target environment maps. Target maps are tone-mapped with fixed exposure within each sequence.
Figure 15: Relighting under dynamic illumination. From left to right and top to bottom, the examples show flickering overhead light, TV-like lighting changes, uplighting, colored moving directional light, lightning-like illumination, and uplighting. Within each example, rows show target environment maps, input frames, paired ground truth (GT), and FlashRelight outputs. Seven aligned frames are shown at t=1,9,17,25,33,41,49 , except the bottom-left lightning example, which uses t=1,17,33,42,44,46,49 to capture the flash. Target maps are tone-mapped with fixed exposure within each sequence.
We present a diffusion-based method for relighting dynamic portrait videos with photorealism and temporal consistency. Our method is fueled by a hybrid training dataset that consists of real-captured and rendered dynamic portrait videos with diverse subject appearances, facial motions, head poses, and known lighting conditions. Specifically, we construct an LED-based lighting system for realistic lighting emulation and high-speed video relighting data acquisition. By leveraging the image priors embedded in pre-trained video diffusion models, and using per-frame high dynamic range (HDR) environment map as lighting control, we train a high-performance generative model for realistic and identity-preserving dynamic portrait video relighting. In addition to the environment map control, our model uses a synthesized background image to enable control on the camera's exposure level and color tone. Our model can produce temporally consistent relit portrait video that looks realistic and harmonious under a provided new environment and faithfully preserve the subject's expression and fine facial features, including skin tone, wrinkles, and facial hair. Our model generalizes well to unseen data, in terms of the subject appearance, motion, and lighting condition. We perform extensive experiments on relighting in-the-wild videos with various environment maps and demonstrate practical applications on portrait photography. Results show that our method achieves state-of-the-art performance in photorealism, lighting harmony, and temporal consistency.
Yufan Zhang, Yu Ji, Ayo Ajiboye +4
George Mason University, Fairfax, USA · LightThought LLC, Fairfax, USA · Columbia University, New York, USA
Recent diffusion models have achieved remarkable success in image relighting, and this success has quickly been extended to video relighting. However, existing methods offer limited explicit control over illumination in the relighted output. We present LightCtrl, the first controllable video relighting method that enables explicit control of video illumination through a user-supplied light trajectory in a training-free manner. Our approach combines pre-trained diffusion models: an image relighting model processes each frame individually, followed by a video diffusion prior to enhance temporal consistency. To achieve explicit control over dynamically varying lighting, we introduce two key components. First, a Light Map Injection module samples light trajectory-specific noise and injects it into the latent representation of the source video, improving illumination coherence with the conditional light trajectory. Second, a Geometry-Aware Relighting module dynamically combines RGB and normal map latents in the frequency domain to suppress the influence of the original lighting, further enhancing adherence to the input light trajectory. Experiments show that LightCtrl produces high-quality videos with diverse illumination changes that closely follow the specified light trajectory, demonstrating improved controllability over baseline methods. Code is available at: https://github.com/GVCLab/LightCtrl.
Yizuo Peng, Xuelin Chen, Kai Zhang +1
Tsinghua University · GVC Lab, Great Bay University · Adobe Research
Recent advances have shown that large-scale video diffusion models can be repurposed as neural renderers by first decomposing videos into intrinsic scene representations and then performing forward rendering under novel illumination. While promising, this paradigm fundamentally relies on accurate intrinsic decomposition, which remains highly unreliable for real-world videos and often leads to distorted appearances, broken materials, and accumulated temporal artifacts during relighting. In this work, we present Relit-LiVE, a novel video relighting framework that produces physically consistent, temporally stable results without requiring prior knowledge of camera pose. Our key insight is to explicitly introduce raw reference images into the rendering process, enabling the model to recover critical scene cues that are inevitably lost or corrupted in intrinsic representations. Furthermore, we propose a novel environment video prediction formulation that simultaneously generates relit videos and per-frame environment maps aligned with each camera viewpoint in a single diffusion process. This joint prediction enforces strong geometric-illumination alignment and naturally supports dynamic lighting and camera motion, significantly improving physical consistency in video relighting while easing the requirement of known per-frame camera pose. Extensive experiments demonstrate that Relit-LiVE consistently outperforms state-of-the-art video relighting and neural rendering methods across synthetic and real-world benchmarks. Beyond relighting, our framework naturally supports a wide range of downstream applications, including scene-level rendering, material editing, object insertion, and streaming video relighting. The Project is available at https://github.com/zhuxing0/Relit-LiVE.
Weiqing Xiao, Hong Li, Xiuyu Yang +7
Nanjing University, China · BAAI, China · BUAA, China +5