Physics Guided Diffusion Model Based Exposure Interpolation for High Dynamic Range Imaging
Authors: Zhengguo Li, Chaobing Zheng
Organizations: Institute of Advanced Intelligence and Computing, A*STAR, Singapore, 138632 · the school of Information Science and Engineering, Wuhan University of Science and Technology, Wuhan 430081, China
Ghosting artifacts are an open problem for popular multi-shot based high dynamic range (HDR) imaging. An effective solution is to capture two large-exposure-ratio (LER) standard dynamic range (SDR) images simultaneously via a single-shot. Brightness order reversal (BOR) artifacts are generated when the two LER images are directly fused together. It is crucial to interpolate an image with an exposure time in between the exposure times of two LER images. In this paper, a novel physics-guided diffusion model based exposure interpolation algorithm is proposed to address the BOR artifacts. An initial image is interpolated by a physics-driven component, and the residual high-frequency details are generated by a new diffusion model based component. The new diffusion model is based on the correlation analysis of generative models and includes a new predicted target to reduce the variation of the Pearson correlation between the noisy data and the target, as well as the amplification factor of the fitting error. Compared with existing neural augmented exposure interpolation algorithms on top of convolution neural networks, the proposed exposure interpolation algorithm has an inherent advantage to generate more high frequency information. Experiment results validate the proposed exposure interpolation algorithm.
Figures & tables
Fig. 1: Overview of the proposed neural augmented exposure interpolation algorithm on top of a new diffusion model. Through reducing the variation of Ψet,ωt as well as ∣Φ1(t′,t)∣ and ∣Φ2(t′,t)∣ , the fitting error of the neural network is reduced in the reverse process. Subsequently, the quality of the interpolated image is improved.
L(θ)=gθ(1)(et,fm(Z1,Z7),t)−ωt(1)22
Algorithm 1 Neural Augmented Exposure Interpolation Diffusion Model (Training)
Fig. 2: ∣Φ1(t′,t)∣ , ∣Φ2(t′,t)∣ and Ψet,ωt of two different choices of ωt in equations ( 31 ) and ( 44 ) with T as 5. ∣Φ2(1)(t′,t)∣ is smaller than ∣Φ2(1)(t′,t)∣ and the variation of Ψet,ωt(1) is smaller than the variation of Ψet,ωt(2) .
Fig. 3: Comparison of different physics-driven exposure interpolation algorithms. (a) are the inputs, (b), (c) and (d) show the results obtained by the physics-driven exposure interpolation algorithms in [ 18 , 19 ] , and our adopted physics-driven component, respectively. (e) represents the ground truth.
Methods
Data
PSNR ↑
SSIM ↑
FSIM ↑
LPIPS ↓
Entropy ↑
CLIPIQA ↑
MUSIQ ↑
HyperIQA ↑
Algorithm in [ 18 ]
[ 19 , 10 ]
26.65
0.9114
0.9761
0.0905
6.4661
0.4834
58.996
65.07
Algorithm in [ 19 ]
[ 19 , 10 ]
28.07
0.9142
0.98
0.0901
6.5903
0.4836
58.438
65.38
Our
[ 19 , 10 ]
29.79
0.919
0.9823
0.0848
6.691
0.4727
59.0875
65.46
Algorithm in [ 18 ]
[ 34 ]
24.4
0.8633
0.9519
0.1268
7.0101
0.4972
61.7405
65.36
Algorithm in [ 19 ]
[ 34 ]
24.93
0.8702
0.957
0.1192
7.0443
0.496
61.8327
65.44
Our
[ 34 ]
25.87
0.8848
0.9631
0.1083
7.0817
0.4998
61.8573
65.53
TABLE I: Objective comparison among the physics-driven exposure interpolation algorithms in [ 18 , 19 ] and our adopted physics-driven exposure interpolation component.
Target
PSNR ↑
SSIM ↑
FSIM ↑
LPIPS ↓
Entropy ↑
CLIPIQA ↑
MUSIQ ↑
HyperIQA ↑
Eq. ( 31 )
33.55
0.9489
0.9900
0.0441
6.8237
0.4702
63.40
67.23
Eq. ( 44 )
32.77
0.9429
0.9883
0.0492
6.8181
0.4704
61.89
66.94
TABLE II: Objective comparison between two different choices of ωt in equations ( 31 )(the proposed one) and ( 44 ) (one in [ 30 ] ) with T as 5 using date sets in [ 19 , 10 ] .
Fig. 4: Comparison of training curves for two different choices of ωt in equations ( 31 ) and ( 44 ) with T as 5.
α
PSNR ↑
SSIM ↑
FSIM ↑
LPIPS ↓
Entropy ↑
CLIPIQA ↑
MUSIQ ↑
HyperIQA ↑
0.02
33.55
0.9489
0.9900
0.0441
6.8237
0.4702
63.40
67.23
0.1
33.44
0.9493
0.9899
0.0442
6.8139
0.4681
63.28
67.13
1.0
33.12
0.9447
0.9893
0.0454
6.8212
0.4775
62.77
66.91
TABLE III: Objective comparison between different choices of α in equations ( 32 ) with T as 5 on [ 19 , 10 ] .
Fig. 5: Comparison of neural augmented exposure interpolation algorithms in [ 19 , 20 ] and the proposed neural augmented exposure interpolation algorithm. (a) Input images with different exposures. (b) Result of [ 19 ] . (c) Result of [ 20 ] . (d) Result of the proposed algorithm. The red circles mark the regions shown enlarged on the right of each result. The proposed algorithm produces sharper results with clearer details, especially in the highlighted foliage region.
Methods
Data
PSNR ↑
SSIM ↑
FSIM ↑
LPIPS ↓
Entropy ↑
CLIPIQA ↑
MUSIQ ↑
HyperIQA ↑
Algorithm in [ 19 ]
[ 19 , 10 ]
32.07
0.9362
0.9859
0.0527
6.7896
0.4541
58.80
66.19
Algorithm in [ 20 ]
[ 19 , 10 ]
33.41
0.9443
0.9894
0.0472
6.8137
0.4648
60.42
66.75
Our
[ 19 , 10 ]
33.55
0.9489
0.9900
0.0441
6.8237
0.4702
63.40
67.23
Algorithm in [ 19 ]
[ 34 ]
25.38
0.8822
0.9582
0.1239
7.1253
0.4455
60.64
65.51
Algorithm in [ 20 ]
[ 34 ]
26.62
0.8830
0.9613
0.1228
7.1333
0.4744
61.39
66.36
Our
[ 34 ]
26.71
0.8922
0.9621
0.1207
7.1406
0.4745
64.26
67.17
TABLE IV: Comparison of neural augmented exposure interpolation algorithms in [ 19 , 20 ] and the proposed neural augmented exposure interpolation algorithm.
Metrics
Data
GGIF [ 41 ]
MSF [ 18 ]
DeepFuse [ 14 ]
MEFNet [ 42 ]
UltraFusion [ 7 ]
DiFuser [ 43 ]
Proposed
MEF-SSIM
[ 19 , 10 ]
0.8717
0.8782
0.8579
0.8662
0.8834
0.7806
0.9172
MEF-SSIM
[ 34 ]
0.9126
0.9277
0.9104
0.9010
0.9409
0.8081
0.9484
TMQI
[ 19 , 10 ]
0.8328
0.8448
0.8219
0.8185
0.8365
0.8507
0.8768
PSNR
[ 34 ]
14.48
18.25
16.23
13.30
20.24
15.56
21.94
SSIM
[ 34 ]
0.7755
0.8119
0.7679
0.7429
0.8335
0.6884
0.8404
TABLE V: Objective comparison among the resultant MEF algorithm with 6 state-of-the-art exposure fusion algorithms with inputs as two LER images Z1 and Z7 ( ↑ : larger is better).
Fig. 6: Subjective comparison of seven exposure fusion algorithms with inputs as two LER images Z1 and Z7 . These two sets of differently exposed images are from the data set in [ 19 , 44 ] . All images in the set are given in the top two rows. BOR and halo artifacts, color distortion, loss of details in highlight/shadow regions, and reduced global contrast are possible issues for exposure fusion algorithms.
Single LDR to HDR reconstruction remains challenging for over-exposed regions where traditional methods often fail due to complete information loss. We present a training-free approach that enhances existing indirect and direct HDR reconstruction methods through diffusion-based inpainting. Our method combines text-guided diffusion models with SDEdit refinement to generate plausible content in over-exposed areas while maintaining consistency across multi-exposure LDR images. Unlike previous approaches requiring extensive training, our method seamlessly integrates with existing HDR reconstruction techniques through an iterative compensation mechanism that ensures luminance coherence across multiple exposures. We demonstrate significant improvements in both perceptual quality and quantitative metrics on standard HDR datasets and in-the-wild captures. Results show that our method effectively recovers natural details in challenging scenarios while preserving the advantages of existing HDR reconstruction pipelines. Project page: https://github.com/EusdenLin/HDR-Reconstruction-Boosting
Yo-Tin Lin, Su-Kai Chen, Hou-Ning Hu +2
National Yang Ming Chiao Tung University · MediaTek Inc.
Block-wise varying exposures are effective for cap- turing appearance of a real-world high dynamic range (HDR) scene via a single-shot. However, brightness order reversal (BOR) artifacts, blocking artifacts and loss of details in highlight/shadow regions are possible issues for block-wise varying exposures. In this paper, a novel neural block-wise auto-exposure algorithm is proposed to address the issues. The proposed algorithm maps a temporary image and its exposure time to the exposure times of all non-overlapping 16 X 16 blocks. The temporary image can be either an sRGB or a raw image. Although the sRGB image is widely adopted in existing auto-exposure algorithms, exposure times of blocks in the highlight and shadow regions might not be accurately learned due to the limited radiance measurement capability of the sRGB image. This paper thus also explores the feasibility of temporary image as a raw image. Due to the better radiance measurement capability of the raw image, the exposure times of blocks are learned more accurately, and more details in the highlight/shadow regions of the real-world HDR scene are captured by the resultant HDR imaging system. Experiment results validate the proposed neural block-wise auto-exposure algorithm.
Zhengguo Li, Chaobing Zheng, Wei Wang
Institute of Advanced Intelligence and Computing (IAIC), Agency for Science, Technology and Research (A*STAR), Singapore, 138632 · the school of Information Science and Engineering, Wuhan University of Science and Technology, Wuhan 430081, China · Advanced Remanufacturing and Technology Centre (ARTC), Agency for Science, Technology and Research (A*STAR)
Recent generative methods for single-shot high dynamic range (HDR) image reconstruction show promising results, but often struggle with preserving fidelity to the input image. They require separate models to handle highlights and shadows, or sacrifice interpretability by directly predicting the final HDR image. We address these limitations by re-casting single-shot HDR reconstruction as conditional video generation and fusing the generated frames into an HDR image. We finetune a video diffusion model to generate an exposure bracket, conditioned on a low dynamic range (LDR) input. We fuse this image bracket using per-pixel weights predicted by a light-weight UNet. This formulation is simple, interpretable, and effective. Rather than directly hallucinating an HDR image, it explicitly reconstructs the intermediate exposure stack and fuses it into the final output. Our method eliminates the need for separate models across exposure regimes and produces HDR reconstructions with high input fidelity. On quantitative benchmarks, we outperform state-of-the-art generative baselines with comparable model capacity on several reconstruction metrics. Human evaluators further prefer our results in 72% of pairwise comparisons against existing methods. Finally, we show that this input-conditioned sequence generation and fusion framework extends beyond HDR to other image reconstruction tasks, such as all-in-focus image recovery from a single defocus-blurred input.
Chinmay Talegaonkar, Jinshi He, Christopher McKenna +1