HDR-NSFF: High Dynamic Range Neural Scene Flow Fields
Authors: Shin Dong-Yeon, Kim Jun-Seong, Kwon Byung-Ki, Tae-Hyun Oh
Organizations: School of Electrical Engineering, KAIST · Dept. of Electrical Engineering, POSTECH · Grad. School of Artificial Intelligence, POSTECH · School of Computing, KAIST
Radiance of real-world scenes typically spans a much wider dynamic range than what standard cameras can capture. While conventional HDR methods merge alternating-exposure frames, these approaches are inherently constrained to 2D pixel-level alignment, often leading to ghosting artifacts and temporal inconsistency in dynamic scenes. To address these limitations, we present HDR-NSFF, a paradigm shift from 2D-based merging to 4D spatio-temporal modeling. Our framework reconstructs dynamic HDR radiance fields from alternating-exposure monocular videos by representing the scene as a continuous function of space and time, and is compatible with both neural radiance field and 4D Gaussian Splatting (4DGS) based dynamic representations. This unified end-to-end pipeline explicitly models HDR radiance, 3D scene flow, geometry, and tone-mapping, ensuring physical plausibility and global coherence. We further enhance robustness by (i) extending semantic-based optical flow with DINO features to achieve exposure-invariant motion estimation, and (ii) incorporating a generative prior as a regularizer to compensate for limited observation in monocular captures and saturation-induced information loss. To evaluate HDR space-time view synthesis, we present the first real-world HDR-GoPro dataset specifically designed for dynamic HDR scenes. Experiments demonstrate that HDR-NSFF recovers fine radiance details and coherent dynamics even under challenging exposure variations, thereby achieving state-of-the-art performance in novel space-time view synthesis. Project page: https://shin-dong-yeon.github.io/HDR-NSFF/
Figures & tables
Figure 1: High Dynamic Range Neural Scene Flow Fields (HDR-NSFF) reconstruct dynamic HDR radiance field from (a) alternating-exposure monocular videos. Our method enables the rendering of (b) HDR novel views across both spatial and temporal domains. Additionally, we can generate (c) novel LDR views along with their corresponding depth maps.
Figure 2: Comparison of HDR video reconstruction on training views. Given alternating-exposure video, HDR video reconstruction baselines, i.e . , LAN-HDR ( Chung and Cho, 2023 ) , HDRFlow ( Xu et al., 2024 ) , and NECHDR ( Cui et al., 2024 ) fail to produce consistent results, while our model ensures temporal coherence and recovers valid information in saturated regions.
Figure 3: Overall pipeline of our proposed method. HDR-NSFF takes an alternating-exposure monocular video as input and estimates 3D scene flow for the sampled points along each ray. Neighboring frames are then warped to render the HDR radiance at the target frame, which is tone-mapped to LDR via a white-balance and camera-response function module. Photometric loss with the ground-truth LDR images, along with optical flow and depth constraints from off-the-shelf models, jointly optimize both the scene flow fields and tone-mapping module in an end-to-end manner.
Figure 4: Visualization of flow estimation under varying exposure conditions. RAFT yields noticeable errors when exposures vary. Even with gamma correction, RAFT (Gamma) still fails to produce accurate results. While fine-tuning on synthetic data RAFT (Finetuned) shows moderate improvement, our proposed semantic-based approach achieves superior accuracy.
Figure 5: Generative prior (GP) pipeline for HDR dynamic radiance fields optimization. Unseen novel views are first rendered and then refined via a GP to restore details in regions with broken correspondences. These enhanced views serve as pseudo-labels for the progressive optimization of HDR dynamic radiance fields. This bootstrapping mechanism stabilizes correspondences, mitigating issues related to sparse viewpoints and saturated pixels in monocular videos with alternating exposures.
Figure 6: Evaluation setup and sampled sequences from our proposed GoPro dataset . To evaluate novel view synthesis, we use nine GoPro Hero 13 Black cameras synchronized to record multi-view video at three exposures (mid, low, high). We construct a alternating-exposure monocular video by selecting one frame per time step across exposures, and use the remaining views for evaluation.
Figure 7: Qualitative results of novel view synthesis on GoPro dataset. The odd-numbered rows show the LDR-rendered novel views along with their corresponding L1 error maps against the ground-truth novel view (leftmost). Our method consistently yields the smallest error across all scenes. The even-numbered rows present tone-mapped HDR novel views. Compared with HDR-HexPlane ( Wu et al., 2024a ) , our approach produces more accurate radiance, geometry, and motion representations.
Table 8
Figure 9: Qualitative results on novel LDR/HDR view synthesis. We visualize LDR rendering results at varying exposure levels (low, mid, and high), tone-mapped HDR rendering by ours and corresponding ground-truth HDR references. We also visualize histograms of our HDR images and ground truth. For better visualization, we plot HDR histogram using smoothed KDE method.
NYUv2
ScanNet
Methods
Original
+2EV
-2EV
Original
+2EV
-2EV
MiDaS
9.08
13.68
9.35
8.66
13.78
10.22
DPT
9.21
12.96
8.95
8.27
13.62
9.57
Marigold
5.81
11.26
6.66
7.24
14.26
8.33
Depth-Anything-V2
4.87
7.63
5.10
4.82
10.57
6.36
Table 4: Depth estimation results under exposure variance. We employ AbsRel as the evaluation metric.
Full
Dynamic only
Methods
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
w/o Tone-mapping
17.79
0.7048
0.0705
15.59
0.5577
0.1339
Fix CRF
25.55
0.8391
0.0487
20.43
0.6904
0.0911
MLP CRF
28.76
0.8861
0.0394
21.48
0.7256
0.0776
Piecewise CRF
31.01
0.9301
0.0233
22.55
0.7714
0.0697
Table 5: Comparison of tone-mapping designs.
Appendix figures & tables25 assets
Supplementary material from the paper’s appendix.
Appendix
Name
Exp. Time [s]
# of frames
Big jump
9601,28801,76801
324
Side walk
9601,28801,76801
324
Jumping jack
7201,19201,76801
324
Pointing walk
7201,19201,76801
324
Tube toss
7201,19201,76801
324
Bear
1201,4801,19201
324
Appendix
Table S1: Parameter settings for real dataset.
Figure S1: Illustration of two experimental settings. We illustrate two experimental settings described in Sec. 4.2 in manuscript: (a) Novel view synthesis (b) Novel view and time synthesis.
Figure S2: DINOv2 feature visualization under varying exposures. Despite large changes in brightness, DINOv2 embeddings remain consistent, showing robust clustering across different exposure levels.
Figure S3: Effect of generative prior (GP). (a) GP provides GP provides additional plausible views from unseen views, (b) leading to more consistent and sharper reconstructions.
Big jump
Side walk
Method
Full
Dynamic only
Full
Dynamic only
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
NSFF
16.23
0.6268
0.194
13.75
0.4476
0.2755
16.62
0.6346
0.1874
14.1
0.5524
0.2139
4DGS
20.02
0.7283
0.1751
14.21
0.3724
0.319
18.46
0.702
0.1724
13.56
0.3927
0.254
MotionGS
11.4
0.1354
0.4549
9.42
0.0862
0.5628
13.58
0.2387
0.3692
8.79
0.1025
0.513
NeRF-WT
27.09
0.9051
0.0738
16.31
0.5023
0.2301
25.29
0.9061
0.0641
13.23
0.4243
0.2143
Appendix
Table S2: Quantitative results of novel view synthesis on GoPro dataset. The green and yellow colors stand for the best and the second best , respectively.
Big jump
Side walk
Method
Full
Dynamic only
Full
Dynamic only
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
NSFF
17.09
0.6856
0.1858
15.05
0.4765
0.3175
17.48
0.7095
0.1621
15.49
0.609
0.2292
HDR-Hexplane
19.01
0.5076
0.2301
15.13
0.4166
0.2604
19.08
0.4914
0.2389
16.33
0.5778
0.1699
Ours
30.13
0.924
0.0662
22.09
0.756
0.1515
30.18
0.927
0.0564
21.03
0.7304
0.1413
Jumping jack
Pointing walk
Appendix
Table S3: Quantitative results of novel time synthesis on real data. The green and yellow colors stand for the best and the second best , respectively.
synthetic dataset-Full
Methods
Lego
Mutant
Standup
Jumping Jack
PSNR
NSFF
15.45
16.97
13.47
15.53
NeRF-WT
29.55
33.06
32.55
29.25
HDR-Hexplane
28.58
30.88
30.83
29.50
Ours
34.64
36.13
35.80
33.72
Appendix
Table S4: Quantitative results of novel view and time synthesis on synthetic dataset. The green and yellow colors stand for the best and the second best , respectively.
Full
Dynamic only
Methods
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
Ours w/ RAFT ( Teed and Deng, 2020 )
30.42
0.9269
0.0246
21.38
0.7369
0.0675
Ours w/ Finetuned
30.68
0.9234
0.0253
21.51
0.7377
0.0689
Ours w/ Dino-Tracker ( Tumanyan et al., 2024 )
31.01
0.9301
0.0233
22.55
0.7714
0.0697
Appendix
Table S5: Ablation study of flow model. To compare the effect of flow regularization, we compare NVS performance of our approach against the baseline optical flow model (RAFT Teed and Deng (2020) ) and a stronger baseline fine-tuned RAFT on a multi-exposure adaptation of the FlyingThings3D dataset.
Figure S4: Qualitative results of MoSca with our proposed modules on GoPro outdoor scenes for HDR novel view synthesis. MoSca with Tone-mapping module (TM) produces HDR-like appearances but suffers from temporal inconsistency and distorted dynamic geometry due to exposure-sensitive motion estimation. In contrast, MoSca with TM and DINO-Tracker (DT) leverages exposure-robust semantic flow, yielding stable geometry and photometrically consistent HDR novel views under challenging exposure alternation.
Full
Dynamic only
Methods
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
MoSca w/ TM
26.67
0.920
0.060
15.96
0.463
0.064
MoSca w/ TM + DT
29.35
0.937
0.040
21.22
0.751
0.064
Appendix
Table S6: Quantitative results of MoSca with our proposed modules on GoPro outdoor scenes for HDR novel view synthesis. MoSca with Tone-mapping (TM) aligns exposure-varied inputs into a consistent HDR radiance but still suffers from unreliable motion due to CoTracker’s exposure sensitivity. Incorporating DINO-Tracker (DT) provides exposure-robust semantic flow, significantly improving both HDR reconstruction and dynamic motion stability.
Figure S5: Qualitative comparison with 2D-to-4D HDR reconstruction. A two-stage baseline reconstructs HDR video using LAN-HDR ( Chung and Cho, 2023 ) , HDRFlow ( Xu et al., 2024 ) , or NECHDR ( Cui et al., 2024 ) , applies a fixed μ -law tone mapping, and performs 4D reconstruction with MoSca ( Lei et al., 2025 ) . For reference, we also include a MoSca-based variant of our method (HDR-MoSca). While 2D-to-4D baselines fail to recover, our approach yields temporally coherent HDR radiance and stable geometry.
Figure S6: Training view and ours reconstrcuted views on GoPro indoor dataset. The odd-numbered rows show sample LDR frames from the input sequence, captured with alternating exposures (low, high, and mid). The even-numbered rows present our tone-mapped HDR reconstruction results for the corresponding input views. Under- and over-exposed regions are reliably recovered across all scenes. Even in areas where severe saturation is expected, such as the smile-emoji picture on the wall or the region directly beneath the light source, our method accurately reconstructs fine details. Even the mid-exposure frames contain locally saturated regions, yet these areas are consistently restored with high fidelity.
Figure S7: Training view and ours reconstrcuted views on GoPro outdoor dataset. The odd-numbered rows show sample LDR frames from the input sequence, captured with alternating exposures (low, high, and mid). The even-numbered rows present our tone-mapped HDR reconstruction results for the corresponding input views. Across the outdoor scenes, our method robustly reconstructs HDR radiance even under strong and diverse motion patterns. It performs reliably in scenarios involving large vertical motion in Big Jump, lateral motion in Pointing Walk, and fast object motion in Tube Toss. Despite these challenging dynamics, the model successfully restores both saturated regions caused by strong sunlight reflections on buildings and under-exposed regions cast in shadow, demonstrating consistent reconstruction quality across a wide range of exposure conditions.
Figure S8: Qualitative results of novel view synthesis on Tumbler data.
Figure S9: Qualitative results of novel view synthesis on Bear thread data.
Figure S10: Qualitative results of novel view synthesis on Dog data.
Figure S11: Qualitative results of novel view synthesis on Big jump data.
Figure S12: Qualitative results of novel view synthesis on Jumping jack data.
Figure S13: Qualitative results of novel view synthesis on Pointing walk data.
Figure S14: Qualitative results of novel view synthesis on Side walk data.
Figure S15: Qualitative results of novel view synthesis on Tube toss data.
Figure S16: Qualitative results of novel view synthesis on Fire extinguisher data.
Figure S17: Qualitative results of novel view synthesis on Laptop data.
Figure S18: Qualitative results of novel view synthesis on Bag data.
Figure S19: Qualitative results of novel view synthesis on Ball data.
High dynamic range (HDR) novel view synthesis (NVS) aims to reconstruct HDR scenes from multi-exposure low dynamic range (LDR) images. Existing HDR pipelines heavily rely on known camera poses, well-initialized dense point clouds, and time-consuming per-scene optimization. Current feed-forward alternatives overlook the HDR problem by assuming exposure-invariant appearance. To bridge this gap, we propose InstantHDR, a feed-forward network that initializes 3D HDR scenes from uncalibrated multi-exposure LDR collections in a fast single forward pass. Specifically, we design a geometry-guided appearance modeling for multi-exposure fusion, and a meta-network for generalizable scene-specific tone mapping. Due to the lack of HDR scene data, we build a pre-training dataset, called HDR-Pretrain, for generalizable feed-forward HDR models, featuring 168 Blender-rendered scenes, diverse lighting types, and multiple camera response functions. Comprehensive experiments show that our InstantHDR delivers a single-forward HDR initialization at approximately 700 times the speed of SoTA optimization-based methods, and reaches comparable quality in real settings after lightweight post-optimization while remaining approximately 20 times faster. All code, models, and datasets: https://github.com/Bugjudger/InstantHDR.
Dingqiang Ye, Jiacong Xu, Jianglu Ping +3
Johns Hopkins University, USA · Shenzhen University, China
High dynamic range novel view synthesis (HDR-NVS) reconstructs scenes with dynamic details by fusing multi-exposure low dynamic range (LDR) views, yet it struggles to capture ambient illumination-dependent appearance. Implicitly supervising HDR content by constraining tone-mapped results fails in correcting abnormal HDR values, and results in limited gradients for Gaussians in under/over-exposed regions. To this end, we introduce PhysHDR-GS, a physically inspired HDR-NVS framework that models scene appearance via intrinsic reflectance and adjustable ambient illumination. PhysHDR-GS employs a complementary image-exposure (IE) branch and Gaussian-illumination (GI) branch to faithfully reproduce standard camera observations and capture illumination-dependent appearance changes, respectively. During training, the proposed cross-branch HDR consistency loss provides explicit supervision for HDR content, while an illumination-guided gradient scaling strategy mitigates exposure-biased gradient starvation and reduces under-densified representations. Experimental results across realistic and synthetic datasets demonstrate our superiority in reconstructing HDR details (e.g., a PSNR gain of 2.04 dB over HDR-GS), while maintaining real-time rendering speed (up to 76 FPS). Code and models are available at https://huimin-zeng.github.io/PhysHDR-GS/.
Huimin Zeng, Yue Bai, Hailing Wang +1
Department of Electrical and Computer Engineering, Northeastern University · Khoury College of Computer Science, Northeastern University
High dynamic range (HDR) video reconstruction from al-ter-na-ting-exposure sequences remains challenging, especially in regions with extreme luminance variation. We propose a novel HDR reconstruction framework based on dual-stream registration and accurate pyramid fusion. Given three consecutive frames, our method computes optical flow directly with the central frame, while introducing a complementary midpoint displacement strategy to handle cases with severe overexposition. A pyramid fusion stage then merges the resulting radiance and LDR images into a final HDR output. Experimental results demonstrate that our approach consistently outperforms state-of-the-art methods.
Onofre Martorell, Ivan Pereira-Sánchez, Antoni Fuentes +1
Dept. of Mathematics and Computer Science, Universitat de les Illes Balears, Cra. de Valldemossa km 7.5, Palma, 07122, Illes Balears, Spain