Organizations: Xiamen University Malaysia · Xiaomi Corporation · Tsinghua University · Institute of Automation, Chinese Academy of Sciences · Centre for Artificial Intelligence and Robotics, HKISI, CAS, Hong Kong
Nighttime images are inherently susceptible to complex, compound degradations, making accurate quality assessment exceptionally challenging. Score-based IQA methods condense quality into a single number, lacking the multi-dimensional, language-grounded reasoning required for nighttime evaluation. Multimodal Large Language Models (MLLMs) offer a more comprehensive alternative, yet remain constrained by a critical data gap: existing real-world IQA datasets are overwhelmingly daytime-centric and often built from outdated captures, while synthetic alternatives lack nighttime-specific degradation modeling, yielding artifacts far removed from authentic nighttime conditions. To bridge this gap, we present NightIQA, a dataset and benchmark suite purpose-built for MLLM-based nighttime IQA. It comprises three core components: NightIQA-Real, a real-world dataset of 1,769 high-resolution images that we captured ourselves with modern equipment, encompassing six canonical distortion types and structured multi-dimensional annotations; NightIQA-Bench, a benchmark of 1,078 visually grounded multiple-choice questions tailored to MLLM-based nighttime IQA, organized around a four-level evaluation hierarchy spanning Perception, Technical Assessment, Localization, and Reasoning; and NightIQA-Syn, an inherently scalable, scene-aware synthesis pipeline that jointly models global-level and instance-level degradations, with an initial release of 80k high-fidelity images alongside the full generation code. Fine-tuned on NightIQA with Supervised Fine-Tuning (SFT) and Group Relative Policy Optimization (GRPO), Qwen3-VL-8B outperforms open-source IQA methods and achieves higher overall accuracy than leading closed-source models, highlighting the value of domain-specific data and structured supervision.
Figures & tables
Figure 1: NightIQA-Bench accuracy of six models (L1–L4 and average). Trained on NightIQA, our 8B model (red) leads on overall and L2 accuracy, ahead of larger VLMs and IQA baselines, indicating that domain-specific data, not model scale, is the bottleneck for nighttime IQA.
Figure 2: NightIQA-Real annotation and NightIQA-Bench generation in three left-to-right stages. Data Collection (left): 1,769 real nighttime images across three scene categories and six distortion types. Pre-labeling & Verification (middle): four frontier MLLMs pre-label each image along four content-to-impact dimensions, kept by majority agreement and corrected by experts. Final NightIQA-Bench (right): verified labels become a four-level MCQ hierarchy, filtered by an MLLM-as-a-judge with domain-expert validation.
Table 1: Distribution of primary distortion types in NightIQA-Real. N/A denotes images with no visually dominant distortion.
Figure 4: Representative NightIQA-Bench questions across four cognitive levels: L1 (scene recognition), L2 (distortion identification and severity), L3 (spatial reasoning), and L4 (causal degradation analysis).
Figure 5: The NightIQA-Syn degradation pipeline. Left : global-level distortions are applied uniformly via dedicated operators. Right : instance-level distortions follow a three-stage process (target identification, spatial localization, and spatially-coherent rendering) to produce spatially coherent artifacts.
Type
Model
L1: Percep.
L2: Tech.
L3: Local.
L4: Reason.
Avg.
Sampled random guess
25.89
25.09
25.19
24.88
25.26
Closed-Source
GPT-4o [ 25 ]
81.82
71.22
71.51
75.95
73.56
GPT-5 Mini [ 31 ]
83.84
66.57
78.20
75.26
74.21
Gemini-2.5-Flash [ 12 ]
71.72
48.26
61.92
61.86
58.44
Gemini-3.1-Flash-Lite [ 15 ]
89.90
73.26
81.10
79.38
78.94
Doubao-Seed-2.0-Mini [ 6 ]
86.87
62.50
75.29
70.79
71.06
Table 2: Performance comparison on NightIQA-Bench (MCQ accuracy, %). L1–L4: Perception, Technical Assessment, Localization, and Reasoning. Best in bold , second best underlined .
Configuration
L1
L2
L3
L4
Avg.
Qwen3-VL-8B (base)
71.72
59.30
64.83
65.29
63.82
+ GRPO (Real)
82.83
75.00
79.65
74.91
77.18
Δ
+11.1
+15.7
+14.8
+9.6
+13.4
+ SFT (Syn) + GRPO (Real)
87.88
78.20
81.69
80.07
80.71
Δ
+5.1
+3.2
+2.0
+5.2
+3.5
Table 3: Ablation on the training pipeline. SFT uses NightIQA-Syn; GRPO uses NightIQA-Real. Δ : improvement over the preceding row.
Method
Epoch
L1
L2
L3
L4
Avg.
GRPO-only
1
79.80
64.53
75.00
71.13
71.06
2
78.79
70.93
75.58
76.29
74.58
3
85.86
70.06
77.91
76.29
75.70
4
82.83
70.06
77.91
77.32
75.70
5
82.83
75.00
79.65
74.91
77.18
SFT+GRPO
1
80.81
78.20
78.49
78.35
78.57
Table 4: Per-epoch accuracy (%) for GRPO-only (from base) and SFT+GRPO. Best per column in bold : SFT+GRPO leads on every level. We adopt it at epoch 4 for its best average and L4 (Reasoning); GRPO-only peaks at epoch 5.
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Dimension
Field
Values
Semantic Desc.
Subject
free text
Scene category
Nature / Indoor / Urban
Content, layout & lighting
free text
Global Quality
Primary distortion
one of the six types, or N/A
Per-type severity
Slight / Moderate / Severe
Local Distortion
Spatial scope
Global / Local
Appendix
Table 5: NightIQA-Real annotation schema. Distortion types, severity levels, spatial scopes, and scene categories follow the definitions used during capture (Section 3.1 ).
Figure 6: Screenshots of the Label Studio annotation interface used for NightIQA-Real. The interface presents each nighttime image alongside pre-filled annotations from the multi-model ensemble, enabling annotators to verify and refine distortion type, severity, spatial scope, and quality narratives.
Subset
Images
GRPO training corpus (small clusters)
953
Benchmark source (remaining clusters)
816
Total
1,769
Appendix
Table 6: Scene-level train/benchmark partition statistics. All splits are performed at the cluster level to prevent scene leakage.
Figure 9: Benchmark examples for cognitive levels L1–L2. Each panel pairs the nighttime image with its multiple-choice question, options (correct answer marked), and the jointly generated chain-of-thought rationale.
Figure 10: Benchmark examples for cognitive levels L3–L4 (continued from Figure 9 ).
Figure 11: Visual examples of global-level degradations in NightIQA-Syn, grouped by distortion type. Within each category, the columns show two representative scenes and the rows correspond to increasing severity.
Distortion
Parameter
Medium
Severe
Noise
Color shift c
[0.07,0.09]
cm×1.4
ISO intensity α
[0.32,0.37]
αm×1.4
Gaussian sigma σg
[4.8,5.3]
σg,m×1.4
Underexposure
Brightness factor β
[0.35,0.55]
βm/2.5
Noise probability p
[0.2,0.4]
pm×2.5
Noise color shift c
[0.01,0.03]
cm×2.5
Appendix
Table 7: Parameter configurations for global-level degradation operators in NightIQA-Syn.
Figure 13: Visual examples of instance-level degradations in NightIQA-Syn, grouped by distortion type. Within each category, the columns show two representative scenes and the rows correspond to increasing severity.
Distortion
Parameter
Medium
Severe
Local Overexposure
Bloom sigma σ (px)
[45,50]
σm×2.6
Bloom intensity α
[1.6,1.7]
αm×2.6
Motion Blur
Displacement intensity α
[0.10,0.15]
αm×3.5
Warp steps N
[30,35]
Nm×3.5
Vertical shift factor v
[0.008,0.012]
vm×3.5
Matched noise level η
[0.01,0.015]
ηm×3.5
Appendix
Table 8: Parameter configurations for instance-level degradation operators in NightIQA-Syn.
Distortion Type
Medium
Severe
Total
Noise
500
500
1,000
Underexposure
500
500
1,000
Global Overexposure
500
500
1,000
Local Overexposure
501
268
769
Camera Shake
500
500
1,000
Motion Blur
500
500
1,000
Appendix
Table 9: SFT training data distribution by distortion type and severity.
Hyperparameter
Value
Base model
Qwen3-VL-8B
Fine-tuning strategy
Full-parameter SFT
Training epochs
3
Learning rate η0
1×10−5
LR schedule
Cosine annealing
Warmup ratio
0.01
Appendix
Table 10: Hyperparameters for the SFT stage.
Hyperparameter
Value
Base model (after SFT)
Qwen3-VL-8B (SFT ckpt)
Framework / Parallelism
ms-swift / DeepSpeed [ 2 ] ZeRO-3
GPUs / Precision
6 / bfloat16
Training epochs
5
Learning rate
1×10−6
Batch size (per device)
1
Appendix
Table 11: Hyperparameters for the GRPO alignment stage.
School of Computer Science, Northwestern Polytechnical University, Xi’an, China · The Hong Kong University of Science and Technology, Hong Kong SAR, China