Full Dynamic Range Sky-Modelling For Image Based Lighting

Ian J. Maquignaz

Figure 9: Collection of environment maps generated by Icarus with R G B -style encoder and

f_{\text{\tiny Robertson}}

, demonstrating IBL rendering performance from sunrise to sunset.

Figure 10: Collection of environment maps generated by Icarus with R N D -style mapper and

f_{\text{\tiny Robertson}}

, demonstrating IBL rendering performance from sunrise to sunset.

Figure 11: Collection of environment maps generated by AllSky [ 38 ] , demonstrating IBL rendering performance from sunrise to sunset.

Figure 12: Side-by-side comparison of

f_{\text{\tiny Robertson}}

and

f_{\text{\tiny DNN}}

, demonstrating their equivalence in terms of visual quality.

Figure 13: Comparison of model illumination (

\oiint_{I}

, EV) and Peak-Luminance (

PL_{\Omega}

) across approximately 6k sequential samples. A shown, Icarus

512^{2}

with R G B -styles offers stable and accurate real-world illumination from sunrise-to-sunset.

Figure 14: Illustration of Icarus’s ability to transfer solar illumination with R G B -styles. A shown, an obfuscated sun (top row) can be un-obfuscated (middle row) or brightened (bottom row) by transferring the style of solar corona and solar disk from a preferable source image.

Figure 15: Illustration of Icarus’s ability to transfer solar illumination. A shown, a sunset (bottom row) can be brightened (middle row) or given a daytime-like appearance (top row) by transferring the styles of a daytime source image. Icarus

512^{2}

with R G B -styles and

f_{\text{\tiny Roberson}}

fusion.

Figure 16: Illustration of Icarus’s ability to transfer cloud formation. A shown, the gloom of dark clouds (middle row) can be alleviated (top and bottom rows) by transferring styles from preferable source images. Icarus

512^{2}

with R G B -styles and

f_{\text{\tiny Roberson}}

fusion.

Figure 17: Illustration of Icarus’s ability to edit cloud formations. As shown, the clouds of a source image can be edited to realistically remove (top row) or add (middle and top row) clouds. Icarus

512^{2}

with R G B -styles and

f_{\text{\tiny Roberson}}

fusion.

Figure 18: Illustration of Icarus’s ability to support artistic liberties in creating environment maps. As shown, the styles of a source image can be used to create surreal skies. Icarus

512^{2}

with R G B -styles and

f_{\text{\tiny Roberson}}

fusion.

Figure 19: Illustration of Icarus’s ability to support artistic liberties in creating environment maps. As shown, the styles of a source image can be used to create surreal skies. Icarus

512^{2}

with R G B -styles and

f_{\text{\tiny Roberson}}

fusion.

Figure 20: Illustration of Icarus’s ability to support artistic liberties in creating environment maps. As shown, the styles of a source image can be used to create surreal skies. Icarus

512^{2}

with R G B -styles and

f_{\text{\tiny Roberson}}

fusion.

Figure 21: Illustration of Icarus’s ability to support mixing of R G B - and R N D -styles in creating environment maps. Icarus

512^{2}

with R G B -styles, R N D -styles and

f_{\text{\tiny Roberson}}

fusion.

Figure 22: Illustration of Icarus’s ability to support weighted mixing of R G B - and R N D -styles to creating customized environment maps. As depicted by

\lambda

, the weights of styles can be user-modulated to emphasize or subdue selected styles. As shown, the weights of solar features were reduced to create a more uniform overcast sky. Icarus

512^{2}

with R G B -styles, R N D -styles and

f_{\text{\tiny Roberson}}

fusion.

Figure 23: (left) Tone mapping operator (

T_{m}

) compression of HDR Intensity (

\hat{I}

) to LDR intensity (

\check{I}

). (right) Respective non-linearity between error (

\delta

=

0.01

) in compressed LDR-space and error (

\Delta

) in uncompressed HDR-space. As target HDR intensity increases, small errors in LDR-space result in exponentially larger errors in reconstructed HDR-space.

Figure 24: 3D surface of skydome luminance, coloured from low to high intensity. Disk above 3D surface illustrates skydome segmentation. The sun ( red spike) is a small subset of pixels whose intensities belittle the remainder of the skydome. HDRDB sample from June 7th, 2016 at 1:54PM [ 27 ] .

Figure 25: Overcast sky : Mitigation of solar modelling in environment maps through substitution of a parametric Hošek-Wilkie sun (HW [ 25 , 56 ] ) and manual parametric boosting of the HDR environment map (Boosted [ 10 ] ;

\gamma

=0.5,

\beta

=2,

\rho

=6). Adding an HW sun ( LDR+HW Sun ) skips atmospheric attenuation and allows the sun to ‘pierce’ through clouds to create strong shadows. If the sun is obstructed, Boosting an HDR image ( Boosted LDR ) is prone to over-exposing and producing unpredictable shadows. Both mitigation strategies alter the perceived tones in IBL renderings (lambertian planar surface and color chart), but AllSky [ 38 ] and our model ( Icarus , right column) accurately model real-world FDR illumination for photorealistic IBL renderings.

Figure 26: Cloudy with unobstructed sun : Mitigation of solar modelling in environment maps through substitution of a parametric Hošek-Wilkie sun (HW [ 25 , 56 ] ) and manual parametric boosting of the HDR environment map (Boosted [ 10 ] ;

\gamma

=0.5,

\beta

=2,

\rho

=6). When the sun is unobstructed, adding an HW sun creates crisp shadows, realistic light-transmission (black glass orb) and only minor alteration of perceived tones (lambertian surface, color chart). Boosting an HDR image is subjective to user-selection of parameters, which drives both the algorithms’ selection of features and enhancement of intensity. Boosting is prone to over-exposing rendered scenes, producing unpredictable shadows, excessive light-transmission (black glass orb), and severe alteration of perceived tones (lambertian planar surface and color chart). While AllSky [ 38 ] mitigates the skewed tones from the parametric sun and over-saturation of boosting, Icarus (right column) offers greater photorealism and accuracy in illumination (EV and

\oiint_{I}

).

Figure 27: Cloudy with obstructed sun : Mitigation of solar modelling in environment maps through substitution of a parametric Hošek-Wilkie sun (HW [ 25 , 56 ] ) and manual parametric boosting of the HDR environment map (Boosted [ 10 ] ;

\gamma

=0.5,

\beta

=2,

\rho

=6). Adding an HW sun skips atmospheric attenuation and allows the sun to ‘pierce’ through clouds to create strong shadows and unexpected light-transmission (black glass orb). If the sun is obstructed, boosting is prone to over-exposing (see skydome in

2^{nd}

row and light-transmission to black-glass orb), producing unpredictable shadows and excessive light-transmission (black glass orb). AllSky [ 38 ] and Icarus both mitigate alteration of perceived tones (lambertian planar surface and color chart), with Icarus (right column) offering greater photorealism and accuracy in illumination (EV and

\oiint_{I}

).

Figure 28: Clear sky : Mitigation of solar modelling in environment maps through substitution of a parametric Hošek-Wilkie sun (HW [ 25 , 56 ] ) and manual parametric boosting of the HDR environment map (Boosted [ 10 ] ;

\gamma

=0.5,

\beta

=2,

\rho

=6). In low-light settings such as dusk/sunset/sunrise, adding a parametric sun can be difficult given minor errors in camera calibration. Though an added HW sun ‘pierces’ through clouds formations with

10.9\!\times

linear intensity of ground truth FDR Clear Sky , the absence of clouds acting as light-sources allows for relatively-indirect HW solar illumination to produce visually appealing renders without skewed tones. Boosting is subjective to user-selection of parameters driving the algorithms’ selection of features, which in low-light settings is shown to be prone to over-exposing the entirety of the skydome. This over-exposure results in unpredictable shadows (lambertian orb), excessive light-transmission (black glass orb) and altered tones (lambertian planar surface and color chart). AllSky [ 38 ] and Icarus both mitigate alteration of perceived tones (lambertian planar surface and color chart), producing renderings with accurate tones, shadows and light transmission.

Figure 30: Visual representation an LDR exposure bracket normalized to HDR-space by Eq. 3 for various tone mapping operators. Each LDR exposure is a ‘candle-stick’ where upper- and lower-limits illustrate min/max HDR intensities for

\check{I}

clipped to

[0,1]

and the body min/max HDR intensities for

\check{I}

clipped to

[\epsilon,1-\epsilon]

for

\epsilon=\underline{\epsilon}=\overline{\epsilon}=\frac{1}{255}

. Markers (

\circ

) indicate the HDR illumination intensity of an LDR exposure value of 0.5. Insufficient overlap and/or gaps between LDR exposures should be avoided during exposure selection.

Figure 31: Decomposition of an FDR image to LDR exposures by modulation of exposure

\Delta t=2^{-x}

, demonstrating atmospheric features are often visually indiscernible in exposures modulated by

2^{-6}

or less. In exposure

2^{-14}

, the features of the solar corona is indiscernible and only the solar disk remains.

Figure 32: LDR exposure bracket

[2^{0},2^{8},2^{15}]

merged with Robertson exposure fusion, as computed by OpenCV LUT [ 7 ] and by our inline implementation Eq. 26 . As shown in Fig. 32b , our implementation mitigates seams for a smoother FDR sun. Error between the ground truth (original, red outline ) and reconstructed (rebuild, green outline ) exposure range is shown in Fig. 32c and Fig. 32d to be significantly reduced.

Figure 33: Robertson weights by LDR index values ranged [0,255], as computed by OpenCV LUT [ 7 ] and by our inline implementation through Eq. 26 . As shown, the weighting matches exactly.

Figure 34: Example of ghosting in HDRDB [ 27 ] , resulting in visual artifacts including discoloration and blurred cloud formations.

Figure 35: HDRDB mean and mean-augmented skydomes [ 27 ] . Labels ( Fig. 35d ) are 1-channel composites of solar, cloud, skydome (clear-sky), and border masks as proposed by [ 38 ] . Cloud segmentation is produced via colour ratio [ 13 ] and eroded to ‘hand-drawn’ representations Figure 35d is colourized for visualization purposes.

Figure 36: Class-locality heat-maps for decomposed ‘hand-drawn’ HDRDB Discrete (

\mathbb{Z}^{+}

, Fig. 35d ) labels. Heat-maps are colored from low- to high-frequency ( red to blue ), illustrating a disproportional volume of samples from summer months (high-solar elevation) and a near-constant ‘ring’ of clouds around the horizon.

Figure 37: Loss of exposure range (EV) and illumination (

\oiint_{I}

) in environment map format conversion and downsampling across 100 augmentations of an HDRI from June 7th, 2016 at 12:52PM [ 27 ] . Figs. 37 and 37 Left columns: Downsampling sky-latlong (sLL) environment maps. Figs. 37 and 37 Right columns: Converting and downsampling sky-latlong (sLL) environment maps to sky-angular (SA) format. The results show format conversions negatively impact exposure range and illumination, with Fig. 37 demonstrating the instability of output characteristics. Though spline ( spline , order 1) interpolation preserves exposure ranges well in Fig. 37 , Fig. 37 demonstrates significant variance in performance. Pooling ( Pooled ) can provide greater stability in downsampling (lower variance), where ours refers Icarus’s implementation of avg pooling while assuring format conversions are performed at

2\!\times

target downsampling resolution. Adaptive average pooling ( adp-avg ) extends to offers support for downsampling arbitrarily (non powers of 2), which Icarus mitigates. Max pooling ( max ) results in steadily increasing environment luminance (

\oiint_{I}

).

Figure 38: Sky-Angular Environment Map

Figure 39: Histogram of Power-Law (

T_{\gamma}

) tone-mapped HDRDB [ 27 ]

Figure 40: Histogram of

\mu

-lawLog 2 (

T_{\mu\log_{2}}

) tone-mapped HDRDB [ 27 ]

Figure 41: SEAN [ 72 ]

Figure 42: Decoder architectures for LDR bracket generation, exploring configurations of SPADE (S) blocks and their output features. In Fig. 42b , SPADE+ is configured such that SPADE outputs n-features per exposure as indicated by

{\Rrightarrow}

instead of

\rightarrow

.

Figure 43: Architectures for LDR bracket Fusion. In Fig. 43a , a weight

W_{n,c,ij}

is generated to be uniformly applied to LDR exposures

\left\{\check{I}_{n}\right\}^{N}

and exposures times

\left\{\Delta t_{n}\right\}^{N}

. In Fig. 43b , a separate weight

W^{e}_{n,c,ij}

is generated to module exposures

\left\{\check{I}_{n}\right\}^{N}

and a

W^{\Delta t}_{n,c,ij}

to modulate exposures times

\left\{\Delta t_{n}\right\}^{N}

. Features output by SPADE (

{\Rrightarrow}

) are configurable to output n-features or n-features per exposure.

Figure 44: Sensitivity of various commonly reported HDR metrics to truncation of the exposure range. As shown, Integrated Illumination (

\oiint_{I}

, Eq. 1 ), Peak-Luminance (

PL_{\Omega}

Eq. 15 , Peak Ill. Int.

I_{\Omega}

), and VIF [ 58 ] offer sensitivity to truncations of HDR-space exposure. Though PSNR offers sensitivity in LDR-space, the metric offers little sensitivity in HDR-space. HDR-VDP3 offers little sensitivity to truncations of exposure in both LDR- and HDR-space.

		System	$T_{\gamma}$ -cLDR			HDR
	$T_{m}$	Time	LPIPS $\downarrow$	FID $\downarrow$	MiFID $\downarrow$	HDR-VDP3 $\uparrow$	PSNR ${}_{\log_{2}}$ $\uparrow$	EV $\leftarrow$	$\oiint_{I}\leftarrow$
Ground Truth $128^{2}$	-	-	-	-	-	-	-	7.34	1.22
SEAN *	$T_{\gamma}$	8:31	0.16	19.6	303	8.29	140	6.94	1.04
SEAN *	$T_{\mu\log_{2}}$	8:22	0.16	21.6	322	7.96	115	7.27	3.38
SEAN	$T_{\gamma}$	9:28	0.16	20.0	306	8.25	138	7.16	1.08
SEAN	$T_{\mu\log_{2}}$	10:25	0.16	18.5	292	8.04	123	6.49	1.6 $e^{6}$

Appendix

Table 2: The results show that training with the Discriminator Feature Matching loss (indicated by * ) negatively impacts cLDR metrics with higher FID and MiFID scores. Though

\mu\text{-lawLog}_{2}

(

T_{\mu\log_{2}}

) tone mapping generally offers better cLDR metric performance, FDR performance is variable and prone to over-exposure (

\oiint_{I}

). Gamma (

T_{\gamma}

) tone mapping offers similar cLDR performance with stable exposure range (EV) and illumination (

\oiint_{I}

). Average training time is reduced by over 50%, from 9hr 15min for SEAN to 4hr 25min for SEAN (Ours) with no loss in model performance. Bold values indicate category best and underlined values indicate overall best.

		$T_{\gamma}$ -cLDR			HDR
	$T_{m}$	LPIPS $\downarrow$	FID $\downarrow$	MiFID $\downarrow$	HDR-VDP3 $\uparrow$	† PSNR ${}_{\log_{2}}$ $\uparrow$	† EV $\leftarrow$	† $\oiint_{I}\leftarrow$
Ground Truth $128^{2}$	-	-	-	-	-	-	7.34	1.22
Icarus * (SPADE n ; $1\!\times\!f_{\text{\tiny{\color[rgb]{1,0,0}R}{\color[rgb]{0,0.5,0.5}G}{\color[rgb]{0,0,1}B}}}$ )	$T_{\gamma}$	0.14	16.0	255	7.7	124	1.57	0.60
Icarus * (SPADE n ; $1\!\times\!f_{\text{\tiny{\color[rgb]{1,0,0}R}{\color[rgb]{0,0.5,0.5}G}{\color[rgb]{0,0,1}B}}}$ )	$T_{\mu\log_{2}}$	0.14	19.1	307	7.8	124	2.91	0.68
Icarus (SPADE n ; $1\!\times\!f_{\text{\tiny{\color[rgb]{1,0,0}R}{\color[rgb]{0,0.5,0.5}G}{\color[rgb]{0,0,1}B}}}$ )	$T_{\gamma}$	0.14	19.0	306	7.8	122	1.72	0.77
Icarus (SPADE n ; $1\!\times\!f_{\text{\tiny{\color[rgb]{1,0,0}R}{\color[rgb]{0,0.5,0.5}G}{\color[rgb]{0,0,1}B}}}$ )	$T_{\mu\log_{2}}$	0.14	15.7	260	7.9	124	3.30	0.69

Appendix

Table 3: We ablate Icarus to ascertain the impact of multiple exposures on visual quality by training for a singular exposure

[2^{0}]

(

1\!\times\!f_{\text{\tiny{\color[rgb]{1,0,0}R}{\color[rgb]{0,0.5,0.5}G}{\color[rgb]{0,0,1}B}}}

) and three exposures

[2^{0},2^{0},2^{0}]

(

3\!\times\!f_{\text{\tiny{\color[rgb]{1,0,0}R}{\color[rgb]{0,0.5,0.5}G}{\color[rgb]{0,0,1}B}}}

). As in Tab. 2 , the results show that training with the Discriminator Feature Matching loss (indicated by * ) negatively impacts cLDR metrics with higher FID and MiFID scores. Aggressive

\mu\text{-lawLog}_{2}

(

T_{\mu\log_{2}}

) tone mapping offers better cLDR metric performance and is shown to retain a larger exposure range in FDR metrics. As denoted by † , exposures are clipped in LDR-space, crippling FDR performance. The results show that overlapping exposures negatively impact cLDR performance by introducing noise. This is reflected in lost performance in repeating model evaluation with R G B - (

3\!\times\!f_{\text{\tiny{\color[rgb]{1,0,0}R}{\color[rgb]{0,0.5,0.5}G}{\color[rgb]{0,0,1}B}}}

) and H S V -fusion (

3\!\times\!f_{\text{\tiny{\color[rgb]{0,1,1}H}{\color[rgb]{1,0,0}S}{\color[rgb]{0.25,0.25,0.25}V}}}

), where H S V -fusion sustains performance by retaining chroma only from the first exposure. Bold values indicate category best and underlined values indicate overall best.

		$T_{\gamma}$ -cLDR			HDR
	$T_{m}$	LPIPS $\downarrow$	FID $\downarrow$	MiFID $\downarrow$	HDR-VDP3 $\uparrow$	PSNR ${}_{\log_{2}}$ $\uparrow$	EV $\leftarrow$	$\oiint_{I}\leftarrow$
Ground Truth $128^{2}$	-	-	-	-	-	-	7.96	1.22
Icarus (SPADE n ; $f_{\text{\tiny{\color[rgb]{1,0,0}R}{\color[rgb]{0,0.5,0.5}G}{\color[rgb]{0,0,1}B}}}$ )	$T_{\gamma}$	0.14	19.8	316	8.4	147	7.37	0.92
Icarus (SPADE+; $f_{\text{\tiny{\color[rgb]{1,0,0}R}{\color[rgb]{0,0.5,0.5}G}{\color[rgb]{0,0,1}B}}}$ )	$T_{\gamma}$	0.14	18.8	306	8.5	147	8.03	1.05
Icarus (SPADE; $f_{\text{\tiny{\color[rgb]{1,0,0}R}{\color[rgb]{0,0.5,0.5}G}{\color[rgb]{0,0,1}B}}}$ )	$T_{\gamma}$	0.15	23.8	351	8.4	139	8.60	1.25
Icarus (SPADE n ; $f_{\text{\tiny{\color[rgb]{1,0,0}R}{\color[rgb]{0,0.5,0.5}G}{\color[rgb]{0,0,1}B}}}$ )	$T_{\mu\log_{2}}$	0.15	22.0	328	8.4	145	7.75	1.05

Appendix

Table 4: We ablate Icarus with SPADE, SPADE+ and SPADE n decoder architectures to ascertain the requirement for cohesion between SPADE features. We train Icarus with

T_{\gamma}

and

T_{\mu\log_{2}}

tone mapping for exposure set

[2^{0},2^{-8},2^{-12}]

and train Icarus without tone mapping (

T_{\varnothing}

) for exposures

[2^{0},2^{-8},2^{-15}]

. Model evaluation is repeated for R G B - and H S V -fusion, demonstrating the absence of a gap between R G B and H S V -fusion performance which suggests limiting exposure overlap reduces noise in fused imagery. The results demonstrate that tone mapping (

T_{\gamma}

or

T_{\mu\log_{2}}

) improves cLDR and FDR performance, with

T_{\varnothing}

under-performing for all configurations. Bold values indicate category best and underlined values indicate overall best. In this regard, the SPADE+ decoder architecture with

T_{\gamma}

tone mapping is shown to comparatively offer better cLDR and FDR performance.

		$T_{\gamma}$ -cLDR			HDR
	$T_{m}$	LPIPS $\downarrow$	FID $\downarrow$	MiFID $\downarrow$	HDR-VDP3 $\uparrow$	PSNR ${}_{\log_{2}}$ $\uparrow$	EV $\leftarrow$	$\oiint_{I}\leftarrow$
Ground Truth $128^{2}$	-	-	-	-	-	-	7.96	1.22
Icarus (SPADE n ; $f_{\text{\tiny{\color[rgb]{1,0,0}R}{\color[rgb]{0,0.5,0.5}G}{\color[rgb]{0,0,1}B}}}$ ) †	$T_{\gamma}$	0.14	25.5	368	8.5	149	7.88	1.07
Icarus (SPADE+; $f_{\text{\tiny{\color[rgb]{1,0,0}R}{\color[rgb]{0,0.5,0.5}G}{\color[rgb]{0,0,1}B}}}$ ) †	$T_{\gamma}$	0.15	22.6	343	8.5	148	7.93	1.16
Icarus (SPADE; $f_{\text{\tiny{\color[rgb]{1,0,0}R}{\color[rgb]{0,0.5,0.5}G}{\color[rgb]{0,0,1}B}}}$ ) †	$T_{\gamma}$	0.14	19.5	311	8.5	149	7.93	1.09
Icarus (SPADE n ; $f_{\text{\tiny{\color[rgb]{1,0,0}R}{\color[rgb]{0,0.5,0.5}G}{\color[rgb]{0,0,1}B}}}$ ) †	$T_{\mu\log_{2}}$	0.15	25.3	384	8.5	150	7.58	1.00

Appendix

Table 5: We ablate Icarus decoder architectures with a singular output convolution layer (denoted by

\dagger

) instead of parallelized per-exposure output convolution layers as done in Tab. 4 . Icarus is trained for SPADE, SPADE+ and SPADE n architecture and exposures

[2^{0},2^{-8},2^{-12}]

with

T_{\gamma}

and

T_{\mu\log_{2}}

tone mapping. Model evaluation is repeated for R G B - and H S V -fusion, with Bold values indicating category best and underlined values indicating overall best. The results demonstrate that

T_{\gamma}

tone mapping improves SPADE performance, while

T_{\mu\log_{2}}

tone mapping improves SPADE+ performance. The SPADE and SPADE+ decoder architecture differ only in the number of generated latent features (n-features vs. n-features-per-exposure) and offer similar performance improvement in comparison to SPADE n . Given overall cLDR and FDR metrics, SPADE offers better performance over SPADE+, demonstrating increasing latent features does not correlate to improved visual quality. In comparison to Tab. 4 , the results demonstrate that parallelized per-exposure output convolution layers offer improved performance.

		$T_{\gamma}$ -cLDR			HDR
	$T_{m}$	LPIPS $\downarrow$	FID $\downarrow$	MiFID $\downarrow$	HDR-VDP3 $\uparrow$	PSNR ${}_{\log_{2}}$ $\uparrow$	EV $\leftarrow$	$\oiint_{I}\leftarrow$
Ground Truth $128^{2}$	-	-	-	-	-	-	7.96	1.22
Icarus (LDR; $f_{\text{\tiny{\color[rgb]{1,0,0}R}{\color[rgb]{0,0.5,0.5}G}{\color[rgb]{0,0,1}B}}}$ )	$T_{\gamma}$	0.14	26.0	388	8.54	149	7.91	1.06
Icarus (HDR; $f_{\text{\tiny{\color[rgb]{1,0,0}R}{\color[rgb]{0,0.5,0.5}G}{\color[rgb]{0,0,1}B}}}$ )	$T_{\gamma}$	0.14	23.9	357	8.57	150	7.79	1.08
Icarus (LDR+HDR; $f_{\text{\tiny{\color[rgb]{1,0,0}R}{\color[rgb]{0,0.5,0.5}G}{\color[rgb]{0,0,1}B}}}$ )	$T_{\gamma}$	0.14	23.9	362	8.46	147	7.85	0.98
Icarus (LDR; $f_{\text{\tiny{\color[rgb]{0,1,1}H}{\color[rgb]{1,0,0}S}{\color[rgb]{0.25,0.25,0.25}V}}}$ )	$T_{\gamma}$	0.14	24.6	390	8.54	149	7.98	1.10

Appendix

Table 6: We ablate our adversarial LDR and HDR discriminators to ascertain their conditioning of Icarus with SPADE+ for exposures

[2^{0},2^{-8},2^{-14}]

and

T_{\gamma}

tone mapping. Evaluation is repeated for R G B -, H S V -, Robertson- (ours) and Debevec-fusion [ 11 ] . Bold values indicate category best and underlined values indicate overall best. Fusion operators offer contrasting performance, with Roberson-fusion significantly improving visual quality (cLDR) metrics and R G B - and H S V -fusion offer improved illumination (FDR metrics). As shown, Debevec-fusion underperforms in terms of both visual quality and illumination metrics. At a resolution of

128^{2}

, our HDR discriminator outperforms our LDR discriminator, and no advantage is gained from training with both (LDR+HDR) discriminators. When resolution is doubled to

256^{2}

, training with both (LDR+HDR) discriminators provides a clear advantage, offering better handling of the (linear) doubling of the exposure range.

	$T_{\gamma}$ -cLDR			HDR
	LPIPS † $\downarrow$	FID $\downarrow$	MiFID $\downarrow$	HDR-VDP3 $\uparrow$	VIF $\uparrow$	EV $\leftarrow$	$\oiint_{I}\leftarrow$	$PL_{\Omega}\leftarrow$
Ground Truth $256^{2}$	-	-	-	-	-	8.58	1.22	0.374
Icarus R N D -style $f_{\text{\tiny{\color[rgb]{1,0,0}R}{\color[rgb]{0,0.5,0.5}G}{\color[rgb]{0,0,1}B}}}$	0.194	20.1	340	8.21	1.59	9.16	1.28	0.425
Icarus R N D -style $f_{\text{\tiny{\color[rgb]{0,1,1}H}{\color[rgb]{1,0,0}S}{\color[rgb]{0.25,0.25,0.25}V}}}$	0.194	23.7	408	8.15	1.62	9.23	1.31	0.444
Icarus R N D -style $f_{\text{\tiny Robertson}}$	0.19	11	204	8.16	1.52	9.08	1.28	0.425
Icarus R N D -style $f_{\text{\tiny DNN}}$	0.19	14.8	264	8.03	0.97	7.50	0.8426	0.1245

Appendix

Table 7: Icarus R N D -style mapper evaluation repeated for R G B , H S V , Robertson and Debevec fusion given training with

T_{\gamma}

tone mapping for exposures

[2^{0},2^{-8},2^{-15}]

using our LDR- and HDR-bracket discriminators (

\check{\mathcal{D}}

and

\hat{\mathcal{D}}

). Bold values indicate category best and underlined values indicate overall best. LPIPS † is included for continuity but Icarus generates random style codes ( R N D -Style). Poor LPIPS performance paired with good FID, MiFID and HDR-VDP performance is indicative of high visual quality with diverse R N D -Styles. Closely matched EV,

\oiint_{I}

and

PL_{\Omega}

demonstrates that despite variability in cloud formations, Icarus sustains global and solar illumination.

Figure 45: Example of chromatic aberration in LDR bracket fusion

		$T_{\gamma}$ -cLDR		HDR
	Loss ( $\mathcal{L}$ )	FID $\downarrow$	MiFID $\downarrow$	HDR-VDP3 $\uparrow$	VIF $\uparrow$	EV $\leftarrow$	$PL_{\Omega}\leftarrow$	$\oiint_{I}\leftarrow$
Ground Truth $128^{2}$	-	-	-	-	-	7.34	0.451	1.22
$f_{\text{\tiny{\color[rgb]{1,0,0}R}{\color[rgb]{0,0.5,0.5}G}{\color[rgb]{0,0,1}B}}}$	-	23.7	359	8.52	1.07	7.64	0.373	1.16
$f_{\text{\tiny Robertson}}$	-	15.7	264	8.41	0.894	7.49	0.368	1.13
$f_{\text{\tiny DNN}}\left(\left\{\check{I}_{n}\right\}^{N},L\right)$	$\mathcal{D}+\check{\mathcal{D}}+\hat{\mathcal{D}}$	20.4	317	8.4	0.56	7.26	0.283	1.04
$f_{\text{\tiny DNN}}\left(\left\{\check{I}_{n}\right\}^{N},L\right)$	$\check{\mathcal{D}}+\hat{\mathcal{D}}$	16.4	269	7.42	0.227	0.94	0.0006	0.65

Appendix

Table 8: Ablation of fusion module

f_{\text{\tiny DNN}}

adversarial losses, with Icarus trained for exposures

[2^{0},2^{-8},2^{-15}]

and

T_{\gamma}

tone mapping. Fusion module performance is shown to be closely tied to loss selection, with no discriminator independently offering both visual quality and accurate illumination. Combinations of discriminators offer improved performance, with combined

\mathcal{D}+\check{\mathcal{D}}+\hat{\mathcal{D}}

offering balanced visual quality and accurate illumination. Performance with latent inputs (

\xi

) closely matches performance with R G B inputs. Bold values indicate category best and underlined values indicate overall best.

		$T_{\gamma}$ -cLDR		HDR
	Loss ( $\mathcal{L}$ )	FID $\downarrow$	MiFID $\downarrow$	HDR-VDP3 $\uparrow$	VIF $\uparrow$	EV $\leftarrow$	$PL_{\Omega}\leftarrow$	$\oiint_{I}\leftarrow$
Ground Truth $256^{2}$	-	-	-	-	-	8.58	0.367	1.22
$f_{\text{\tiny{\color[rgb]{1,0,0}R}{\color[rgb]{0,0.5,0.5}G}{\color[rgb]{0,0,1}B}}}$	-	31.5	465	8.33	0.894	8.96	0.315	1.08
$f_{\text{\tiny Robertson}}$	-	20.5	338	8.21	0.717	8.81	0.314	1.08
$f_{\text{\tiny DNN}}\left(\left\{\check{I}_{n}\right\}^{N},L\right)$	$\mathcal{D}+\check{\mathcal{D}}+\hat{\mathcal{D}}$	20.4	336	7.98	0.239	5.82	0.018	0.66
$f_{\text{\tiny DNN}}\left(\left\{\check{I}_{n}\right\}^{N},L\right)$	$\check{\mathcal{D}}+\hat{\mathcal{D}}$	20.5	337	8.24	0.304	8.31	0.221	0.94

Appendix

Table 9: Ablation of fusion module

f_{\text{\tiny DNN}}

adversarial losses, with Icarus trained for exposures

[2^{0},2^{-8},2^{-15}]

and

T_{\gamma}

tone mapping. Fusion module performance is shown to be closely tied to loss selection, with no singular discriminator or combination of discriminators offering both visual quality and accurate illumination. Performance with latent inputs (

\xi

) closely matches performance with R G B inputs. Bold values indicate category best and underlined values indicate overall best.

Full Dynamic Range Sky-Modelling For Image Based Lighting

Abstract

Figures & tables

Appendix

HDR Imaging

Single-stage joint diffusion of sky images and clear-sky index for ultra-short-term irradiance forecasting: what the frame-to-irradiance stage discards

Vision Transformers for Estimating Irradiance Using Data Scarce Sky Images

Sky sphere representation in language models

		$T_{\gamma}$ -cLDR			HDR
	Method	LPIPS $\downarrow$	FID $\downarrow$	MiFID $\downarrow$	HDR-VDP3 $\uparrow$	VIF $\uparrow$	EV $\leftarrow$	$\oiint_{I}\leftarrow$	$PL_{\Omega}\leftarrow$
Ground Truth $256^{2}$	-	-	-	-	-	-	8.54	1.21	0.481
AllSky	$T_{\mu\log_{2}}$	0.16	16.7	304	8.15	0.88	8.58	2.01	1.268
Icarus R N D -style †	$f_{\text{\tiny{\color[rgb]{1,0,0}R}{\color[rgb]{0,0.5,0.5}G}{\color[rgb]{0,0,1}B}}}$	-	20.1	340	8.21	1.59	9.16	1.28	0.425
Icarus R N D -style †	$f_{\text{\tiny Robertson}}$	-	11.0	204	8.16	1.52	9.08	1.28	0.425
Ground Truth $512^{2}$	-	-	-	-	-	-	9.44	1.22	0.209

Abstract

HDR Imaging

Explore similar work

Single-stage joint diffusion of sky images and clear-sky index for ultra-short-term irradiance forecasting: what the frame-to-irradiance stage discards

Vision Transformers for Estimating Irradiance Using Data Scarce Sky Images

Sky sphere representation in language models