What Does LOL Measure? Hidden Label Shifts in Low-Light Enhancement Benchmarks

Adityan Thirumani Srinivasan

cs.CVOct 28, 2026

What Does LOL Measure? Hidden Label Shifts in Low-Light Enhancement Benchmarks

Authors: Adityan Thirumani Srinivasan

Organizations: University of Wisconsin–Madison

Abstract

LOL-v1 and LOLv2-Real are the standard benchmarks for supervised low-light image enhancement, and several leading methods ship hand-set test-time adjustments with their evaluation scripts. We audit both benchmarks with 18 public checkpoints of eight methods under one metric implementation that reproduces every published number to within 0.09 dB, and find that their test labels differ from their training labels. (i) The 15 LOL-v1 test labels are rendered differently: the same captures appear in LOLv2 with labels of identical luminance but 1.48x lower saturation, and LOL-v1's own training labels match the LOLv2 rendering; we confirm this on the official release. (ii) The LOLv2-Real test set is the contiguous block of images 690-789, whose labels are 14% darker than the training labels (Cliff's delta = -0.45, p < 1e-12) with no comparable shift in the input statistics we examined; methods trained without any labels score up to 5.5 dB higher on test than on train. (iii) All 15 LOL-v1 test inputs reappear in LOLv2-Real, 12 in its training set. The released test-time settings are consistent with compensating for these shifts. Separating what can be calibrated on training data from what only test-set tuning finds, the test-only residual has median 0.00 dB on LOLv2-Syn, which has no shift, but 0.37 dB on LOL-v1 and 0.92 dB on LOLv2-Real (up to 3.9 dB). Under the shift, test-set checkpoint selection inflates a from-scratch model by 0.9-2.0 dB, and removing the released settings changes which method ranks first. We give concrete reporting recommendations.

Figures & tables

Finding or analysis	Pan [20]	Ours
LOLv2-Real test labels identical to LOL-v1 training labels (91/100)	✓	✓
LOL-v1 test scenes with same-scene training twins	✓(14/15)	(✓) (5/15)
Nearest-neighbour lookup baseline; leakage-free split lists	✓	–
LOL-v1 test inputs inside LOLv2-Real, with re-rendered labels	–	✓
LOL-v1 test labels $1.48\times$ more saturated than the training rendering	–	✓
LOLv2-Real contiguous split with darker test labels	–	✓

Table 1: Scope of this work and the concurrent work of Pan [20] . ✓: established; (✓): reported in passing; –: not addressed.

Method	Benchmark	Checkpoint	Metric	Paper	Ours	$\Delta$
HVI-CIDNet	LOL-v1	w/o perc.	GT-mean	28.14	28.23	$+0.09$
HVI-CIDNet	LOL-v1	w/ perc.	GT-mean	27.71	27.73	$+0.02$
Retinexformer	LOL-v1	–	PSNR	25.16	25.15	$-0.01$
IAT	LOL-v1	–	PSNR	23.38	23.38	$+0.00$
HVI-CIDNet	LOLv2-Real	bestPSNR	PSNR	23.90	23.89	$-0.01$
HVI-CIDNet	LOLv2-Real	bestSSIM	GT-mean	28.39	28.39	$-0.00$

Table 2: Released checkpoints re-scored with one metric implementation. “Paper” is the number the authors report for the metric they report (PSNR unless marked GT-mean). All reproduce within 0.09 dB.

Checkpoint	Benchmark	Released setting	removed	released	removed	released	train / test
			PSNR		GT-mean PSNR		Grid optimum
CIDNet w/o perc.	LOL-v1	saturation $\times 1.3$	23.08	23.61	26.95	28.23	$s$ 1.00 / 1.45
CIDNet w/ perc.	LOL-v1	saturation $\times 1.3$	23.22	23.84	26.65	27.73	$s$ 1.00 / 1.50
CIDNet bestPSNR	LOLv2-Real	brightness $\alpha{=}0.80$	20.56	23.89	27.96	27.95	$b$ 1.02 / 0.80
CIDNet bestSSIM	LOLv2-Real	brightness $\alpha{=}0.82$	20.78	23.89	28.39	28.39	$b$ 1.02 / 0.82
PairLIE	LOL-v1	exponent 0.14 (vs. 0.2)	18.50	19.52	23.51	23.16	–

Table 3: Released test-time settings and what they are worth. “Released” is the native protocol; “removed” is the same checkpoint without the setting. The last column gives the optimal saturation (

s

) or brightness (

b

) of our post-hoc grid on training images and on test images, applied to the plain output.

Figure 1: The 15 LOL-v1 test inputs are pixel-identical to 15 LOLv2-Real inputs, but their labels differ. (a) LOL-v1 test labels are more saturated than the LOLv2 labels of the same captures (dotted: median ratio,

1.48\times

; dashed:

y=x

). (b) Luminance is unchanged. (c) LOL-v1’s training labels (grey) match the LOLv2 rendering (blue), not the test labels (orange). Bars are medians. Official releases.

Figure 2: (a) LOLv2-Real’s test set (shaded) is the last 100 images by index; their labels are darker than the training labels (black: rolling median over 41 images). (b) Seven checkpoints trained without normal-light labels, scored on each benchmark’s training (open) and test (filled) images. On LOLv2-Real (orange) every one scores higher on test, by up to 5.5 dB; on LOLv2-Syn (green) train and test agree.

Statistic	ratio	$p$	$\delta$	ratio	$p$	$\delta$	ratio	$p$	$\delta$
	LOL-v1 (485 / 15)			LOLv2-Real (689 / 100)			LOLv2-Syn (900 / 100)
Input luminance	1.15	0.54	0.09	0.84	0.29	$-0.07$	0.85	0.048	$-0.12$
Input saturation	0.94	0.62	$-0.08$	0.93	0.012	$-0.16$	1.02	0.98	0.00
Input noise (MAD of Laplacian)	1.02	0.53	0.10	1.11	0.22	0.08	0.95	0.53	$-0.04$
Required gain (label/input lum.)	0.84	0.48	$-0.11$	0.90	0.039	$-0.13$	1.07	0.09	0.10
Label luminance	1.08	0.95	0.01	0.86	$3\!\times\!10^{-13}$	$\mathbf{-0.45}$	0.96	0.18	$-0.08$

Table 4: Train versus test image statistics. Ratio of medians (test/train), two-sided Mann–Whitney

p

(unadjusted), and Cliff’s

\delta

(test vs. train). Bold: medium or large effect (

|\delta|\geq 0.33

). The label statistics shift; the input statistics we examined do not shift comparably.

Set	Found in	Identical	Note
LOLv2-Real test labels (100)	LOL-v1 train	91	Pan [20] report 91
LOLv2-Real test inputs (100)	LOL-v1 train	86
LOLv2-Real train inputs (689)	LOL-v1 train	399
LOL-v1 test inputs (15)	LOLv2-Real	15	12 in LOLv2 train, 3 in LOLv2 test
LOL-v1 test labels (15)	LOLv2-Real	0	same captures, re-rendered labels (§ 5 )

Table 5: Pixel-identical images between LOL-v1 and LOLv2-Real (MD5 of decoded RGB). Label counts are per image; some labels are shared by several inputs within a benchmark (LOL-v1: 500 label files, 414 distinct; LOLv2-Real: 789, 642). No label is shared between the training and test sets of the same benchmark.

Figure 3: Post-hoc saturation/brightness calibration for 32 checkpoint–benchmark pairs. (a) Gain on test from the setting that is best on training images. (b) Test-only residual: the oracle gain (best setting on test) minus the gain in (a). The residual is zero without a shift (LOLv2-Syn) and largest under the darker-label shift (LOLv2-Real).

Figure 4: A U-Net trained from scratch, scored every 1,000 iterations on a validation split held out from training (dashed) and on the test set (solid); triangles mark the checkpoint test-set selection would keep. On LOLv2-Real, test PSNR peaks at 2,000 iterations, when outputs are still dark, while validation PSNR keeps rising.

		Selected iteration			Calibration gain on test (dB)
Benchmark	Seed	by val	by test	Inflation (dB)	honest (val-fitted)	oracle
LOL-v1	0	9k	6k	0.12	0.10	0.58
LOL-v1	1	10k	14k	0.08	0.10	0.68
LOL-v1	2	11k	8k	0.16	0.09	0.69
LOLv2-Real	0	10k	2k	0.92	0.00	1.53
LOLv2-Real	1	11k	2k	2.03	0.39	2.21

Table 6: Selection inflation (test PSNR of the test-selected checkpoint minus that of the validation-selected checkpoint) and honest calibration (post-hoc setting fitted on the validation split) for five from-scratch runs.

Figure 5: Means and 95% bootstrap intervals with and without the released test-time settings; numbers above bars give the probability of ranking first over 10,000 resamples. (a) On LOLv2-Real, removing HVI-CIDNet’s brightness setting moves the first rank to IAT. (b) On LOL-v1, removing the saturation setting moves first rank in GT-mean PSNR to Retinexformer, with wide intervals over 15 images.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

			PSNR		Saturation ratio		Brightness ratio
Method	Benchmark	Ckpt	train	test	train	test	train	test
HVI-CIDNet	LOL-v1	w/o perc.	31.90	23.08	0.98	0.68	1.00	1.05
HVI-CIDNet	LOL-v1	w/ perc.	28.06	23.22	0.94	0.64	1.00	1.07
Retinexformer	LOL-v1	–	27.07	25.15	0.98	0.71	1.00	0.97
IAT	LOL-v1	–	25.80	23.38	0.97	0.59	0.98	0.99
HVI-CIDNet	LOLv2-Real	bestPSNR	22.47	20.56	0.87	0.68	0.94	1.26

Appendix

Table 7: Supervised checkpoints, plain protocol. Ratios are output-to-label mean saturation and mean luminance, averaged over images. Train PSNR uses all training images.

	LOL-v1			LOLv2-Real			LOLv2-Syn
Checkpoint	train	test	sat. ratio	train	test	bright. ratio	train	test	bright. ratio
	PSNR	PSNR	tr $\to$ te	PSNR	PSNR	tr $\to$ te	PSNR	PSNR	tr $\to$ te
Zero-DCE	14.14	14.86	2.48 $\to$ 1.40	12.74	18.06	0.61 $\to$ 0.73	17.59	17.76	0.86 $\to$ 0.83
Zero-DCE++	14.56	15.36	2.49 $\to$ 1.44	13.01	18.49	0.65 $\to$ 0.75	17.29	17.58	0.88 $\to$ 0.86
SCI-easy	9.50	9.63	2.65 $\to$ 1.51	9.63	12.04	0.31 $\to$ 0.32	14.34	13.96	0.73 $\to$ 0.68
SCI-medium	14.00	14.85	2.61 $\to$ 1.49	12.40	17.36	0.64 $\to$ 0.70	15.12	15.43	1.06 $\to$ 1.02

Appendix

Table 8: Checkpoints trained without normal-light labels. Train and test PSNR and output-to-label ratios on each benchmark.

Label Shift

Label shift studies how to adapt predictions when class proportions change between training and target data.

See all 23 papers

Rethinking Low-Light Image Enhancement: A Log-Domain Intensity--Chromaticity Decoupling Perspective
May 4, 2026Guangrui Bai, Yifan Mei, Yahui Deng +4Low-Light Image Enhancement
Key Laboratory of Precision and Intelligent Chemistry, Department of Precision Machinery and Precision Instrumentation, University of Science and Technology of China, Hefei, Anhui 230026, China. · College of Mechanical and Vehicle Engineering, Chongqing University, Chongqing 400044, China.
MirrorDistill: Illumination-Aware Latent Distillation for Efficient Low-Light Restoration
Sep 21, 2026Farida Mohsen, Tala Zaim, Nurul Izni Rusli +3Low-Light Image Enhancement Image Restoration
College of Science and Engineering, Hamad Bin Khalifa University, Doha, Qatar. · Faculty of Electrical Engineering & Technology, Universiti Malaysia Perlis, Malaysia · ELEC Department, Vrije Universiteit Brussel, Brussels, Belgium.
BlindPSNR: A No-Reference Fidelity Predictor for Low-Light Image Enhancement
Jul 30, 2026Mingzhe Lyu, Jinqiang Cui, Hong ZhangImage Enhancement Low-Light Image Enhancement
Shenzhen Key Laboratory of Robotics and Computer Vision, Southern University of Science and Technology, Shenzhen 518055, China · Pengcheng Laboratory, Shenzhen 518108, China

May 4, 2026J/K move · Enter open · S save

Rethinking Low-Light Image Enhancement: A Log-Domain Intensity--Chromaticity Decoupling Perspective

Guangrui Bai, Yifan Mei, Yahui Deng, Yuhan Chen+3

Key Laboratory of Precision and Intelligent Chemistry, Department of Precision Machinery and Precision Instrumentation, University of Science and Technology of China, Hefei, Anhui 230026, China. · College of Mechanical and Vehicle Engineering, Chongqing University, Chongqing 400044, China.

Low-Light Image Enhancement

Open paper AskPDF arXiv

Explicit reconstruction constraints derived from the decoupled representation are further imposed to suppress abnormal channel amplification and chromatic noise. Experiments on LOLv2-Real, MIT-Adobe FiveK, and LSRW show that the proposed method achieves competitive or superior quantitative and visual performance, reaching 29.71 dB PSNR and 0.89 SSIM on LOLv2-Real. DarkFace experiments further indicate improved downstream face detection under low-light conditions. Code and pretrained models are available at: https://github.com/mubaisam/ICD.

Abstract

Label Shift

Explore similar work

Rethinking Low-Light Image Enhancement: A Log-Domain Intensity--Chromaticity Decoupling Perspective

MirrorDistill: Illumination-Aware Latent Distillation for Efficient Low-Light Restoration

BlindPSNR: A No-Reference Fidelity Predictor for Low-Light Image Enhancement