Organizations: Xiamen University Malaysia · Xiaomi Corporation · Tsinghua University · Institute of Automation, Chinese Academy of Sciences · Centre for Artificial Intelligence and Robotics, HKISI, CAS, Hong Kong
Nighttime images are inherently susceptible to complex, compound degradations, making accurate quality assessment exceptionally challenging. Score-based IQA methods condense quality into a single number, lacking the multi-dimensional, language-grounded reasoning required for nighttime evaluation. Multimodal Large Language Models (MLLMs) offer a more comprehensive alternative, yet remain constrained by a critical data gap: existing real-world IQA datasets are overwhelmingly daytime-centric and often built from outdated captures, while synthetic alternatives lack nighttime-specific degradation modeling, yielding artifacts far removed from authentic nighttime conditions. To bridge this gap, we present NightIQA, a dataset and benchmark suite purpose-built for MLLM-based nighttime IQA. It comprises three core components: NightIQA-Real, a real-world dataset of 1,769 high-resolution images that we captured ourselves with modern equipment, encompassing six canonical distortion types and structured multi-dimensional annotations; NightIQA-Bench, a benchmark of 1,078 visually grounded multiple-choice questions tailored to MLLM-based nighttime IQA, organized around a four-level evaluation hierarchy spanning Perception, Technical Assessment, Localization, and Reasoning; and NightIQA-Syn, an inherently scalable, scene-aware synthesis pipeline that jointly models global-level and instance-level degradations, with an initial release of 80k high-fidelity images alongside the full generation code. Fine-tuned on NightIQA with Supervised Fine-Tuning (SFT) and Group Relative Policy Optimization (GRPO), Qwen3-VL-8B outperforms open-source IQA methods and achieves higher overall accuracy than leading closed-source models, highlighting the value of domain-specific data and structured supervision.
Figures & tables
Figure 1: NightIQA-Bench accuracy of six models (L1–L4 and average). Trained on NightIQA, our 8B model (red) leads on overall and L2 accuracy, ahead of larger VLMs and IQA baselines, indicating that domain-specific data, not model scale, is the bottleneck for nighttime IQA.
Figure 2: NightIQA-Real annotation and NightIQA-Bench generation in three left-to-right stages. Data Collection (left): 1,769 real nighttime images across three scene categories and six distortion types. Pre-labeling & Verification (middle): four frontier MLLMs pre-label each image along four content-to-impact dimensions, kept by majority agreement and corrected by experts. Final NightIQA-Bench (right): verified labels become a four-level MCQ hierarchy, filtered by an MLLM-as-a-judge with domain-expert validation.
Table 1: Distribution of primary distortion types in NightIQA-Real. N/A denotes images with no visually dominant distortion.
Figure 4: Representative NightIQA-Bench questions across four cognitive levels: L1 (scene recognition), L2 (distortion identification and severity), L3 (spatial reasoning), and L4 (causal degradation analysis).
Figure 5: The NightIQA-Syn degradation pipeline. Left : global-level distortions are applied uniformly via dedicated operators. Right : instance-level distortions follow a three-stage process (target identification, spatial localization, and spatially-coherent rendering) to produce spatially coherent artifacts.
Type
Model
L1: Percep.
L2: Tech.
L3: Local.
L4: Reason.
Avg.
Sampled random guess
25.89
25.09
25.19
24.88
25.26
Closed-Source
GPT-4o [ 25 ]
81.82
71.22
71.51
75.95
73.56
GPT-5 Mini [ 31 ]
83.84
66.57
78.20
75.26
74.21
Gemini-2.5-Flash [ 12 ]
71.72
48.26
61.92
61.86
58.44
Gemini-3.1-Flash-Lite [ 15 ]
89.90
73.26
81.10
79.38
78.94
Doubao-Seed-2.0-Mini [ 6 ]
86.87
62.50
75.29
70.79
71.06
Table 2: Performance comparison on NightIQA-Bench (MCQ accuracy, %). L1–L4: Perception, Technical Assessment, Localization, and Reasoning. Best in bold , second best underlined .
Configuration
L1
L2
L3
L4
Avg.
Qwen3-VL-8B (base)
71.72
59.30
64.83
65.29
63.82
+ GRPO (Real)
82.83
75.00
79.65
74.91
77.18
Δ
+11.1
+15.7
+14.8
+9.6
+13.4
+ SFT (Syn) + GRPO (Real)
87.88
78.20
81.69
80.07
80.71
Δ
+5.1
+3.2
+2.0
+5.2
+3.5
Table 3: Ablation on the training pipeline. SFT uses NightIQA-Syn; GRPO uses NightIQA-Real. Δ : improvement over the preceding row.
Method
Epoch
L1
L2
L3
L4
Avg.
GRPO-only
1
79.80
64.53
75.00
71.13
71.06
2
78.79
70.93
75.58
76.29
74.58
3
85.86
70.06
77.91
76.29
75.70
4
82.83
70.06
77.91
77.32
75.70
5
82.83
75.00
79.65
74.91
77.18
SFT+GRPO
1
80.81
78.20
78.49
78.35
78.57
Table 4: Per-epoch accuracy (%) for GRPO-only (from base) and SFT+GRPO. Best per column in bold : SFT+GRPO leads on every level. We adopt it at epoch 4 for its best average and L4 (Reasoning); GRPO-only peaks at epoch 5.
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Dimension
Field
Values
Semantic Desc.
Subject
free text
Scene category
Nature / Indoor / Urban
Content, layout & lighting
free text
Global Quality
Primary distortion
one of the six types, or N/A
Per-type severity
Slight / Moderate / Severe
Local Distortion
Spatial scope
Global / Local
Appendix
Table 5: NightIQA-Real annotation schema. Distortion types, severity levels, spatial scopes, and scene categories follow the definitions used during capture (Section 3.1 ).
Figure 6: Screenshots of the Label Studio annotation interface used for NightIQA-Real. The interface presents each nighttime image alongside pre-filled annotations from the multi-model ensemble, enabling annotators to verify and refine distortion type, severity, spatial scope, and quality narratives.
Subset
Images
GRPO training corpus (small clusters)
953
Benchmark source (remaining clusters)
816
Total
1,769
Appendix
Table 6: Scene-level train/benchmark partition statistics. All splits are performed at the cluster level to prevent scene leakage.
Figure 9: Benchmark examples for cognitive levels L1–L2. Each panel pairs the nighttime image with its multiple-choice question, options (correct answer marked), and the jointly generated chain-of-thought rationale.
Figure 10: Benchmark examples for cognitive levels L3–L4 (continued from Figure 9 ).
Figure 11: Visual examples of global-level degradations in NightIQA-Syn, grouped by distortion type. Within each category, the columns show two representative scenes and the rows correspond to increasing severity.
Distortion
Parameter
Medium
Severe
Noise
Color shift c
[0.07,0.09]
cm×1.4
ISO intensity α
[0.32,0.37]
αm×1.4
Gaussian sigma σg
[4.8,5.3]
σg,m×1.4
Underexposure
Brightness factor β
[0.35,0.55]
βm/2.5
Noise probability p
[0.2,0.4]
pm×2.5
Noise color shift c
[0.01,0.03]
cm×2.5
Appendix
Table 7: Parameter configurations for global-level degradation operators in NightIQA-Syn.
Figure 13: Visual examples of instance-level degradations in NightIQA-Syn, grouped by distortion type. Within each category, the columns show two representative scenes and the rows correspond to increasing severity.
Distortion
Parameter
Medium
Severe
Local Overexposure
Bloom sigma σ (px)
[45,50]
σm×2.6
Bloom intensity α
[1.6,1.7]
αm×2.6
Motion Blur
Displacement intensity α
[0.10,0.15]
αm×3.5
Warp steps N
[30,35]
Nm×3.5
Vertical shift factor v
[0.008,0.012]
vm×3.5
Matched noise level η
[0.01,0.015]
ηm×3.5
Appendix
Table 8: Parameter configurations for instance-level degradation operators in NightIQA-Syn.
Distortion Type
Medium
Severe
Total
Noise
500
500
1,000
Underexposure
500
500
1,000
Global Overexposure
500
500
1,000
Local Overexposure
501
268
769
Camera Shake
500
500
1,000
Motion Blur
500
500
1,000
Appendix
Table 9: SFT training data distribution by distortion type and severity.
Hyperparameter
Value
Base model
Qwen3-VL-8B
Fine-tuning strategy
Full-parameter SFT
Training epochs
3
Learning rate η0
1×10−5
LR schedule
Cosine annealing
Warmup ratio
0.01
Appendix
Table 10: Hyperparameters for the SFT stage.
Hyperparameter
Value
Base model (after SFT)
Qwen3-VL-8B (SFT ckpt)
Framework / Parallelism
ms-swift / DeepSpeed [ 2 ] ZeRO-3
GPUs / Precision
6 / bfloat16
Training epochs
5
Learning rate
1×10−6
Batch size (per device)
1
Appendix
Table 11: Hyperparameters for the GRPO alignment stage.
In this paper, we present an overview of the NTIRE 2026 challenge on the 3rd Restore Any Image Model in the Wild, specifically focusing on Track 1: Professional Image Quality Assessment. Conventional Image Quality Assessment (IQA) typically relies on scalar scores. By compressing complex visual characteristics into a single number, these methods fundamentally struggle to distinguish subtle differences among uniformly high-quality images. Furthermore, they fail to articulate why one image is superior, lacking the reasoning capabilities required to provide guidance for vision tasks. To bridge this gap, recent advancements in Multimodal Large Language Models (MLLMs) offer a promising paradigm. Inspired by this potential, our challenge establishes a novel benchmark exploring the ability of MLLMs to mimic human expert cognition in evaluating high-quality image pairs. Participants were tasked with overcoming critical bottlenecks in professional scenarios, centering on two primary objectives: (1) Comparative Quality Selection: reliably identifying the visually superior image within a high-quality pair; and (2) Interpretative Reasoning: generating grounded, expert-level explanations that detail the rationale behind the selection. In total, the challenge attracted nearly 200 registrations and over 2,500 submissions. The top-performing methods significantly advanced the state of the art in professional IQA. The challenge dataset is available at https://github.com/narthchin/RAIM-PIQA, and the official homepage is accessible at https://www.codabench.org/competitions/12789/.
Image Quality Assessment (IQA) in open-world environments remains challenging due to limited generalization and interpretability. Recent approaches based on multimodal large language models (MLLMs) introduce textual reasoning for quality prediction, yet their judgments rely heavily on semantically biased internal representations, making them insensitive to low-level perceptual degradations. We propose IQA-T1, a tool-based visual evidence reasoning framework that augments MLLM reasoning with explicit perceptual observations. During inference, the model autonomously invokes specialized analysis tools to generate structured visual evidence, such as noise residual maps, gradient statistics, and frequency spectra, which are progressively integrated into the reasoning process. To support this paradigm, we construct Q-Tool, a dataset containing 11k multimodal reasoning chains grounded in tool-generated evidence. Extensive experiments on seven IQA benchmarks show that IQA-T1 achieves the best overall performance across datasets while producing interpretable and evidence-grounded quality assessments. Code and dataset are available at https://github.com/zibuyu-02/IQA-T1.
Jinjian Wu, Jiaqi Tang, Wei Wei +5
School of Computer Science, Northwestern Polytechnical University, Xi’an, China · The Hong Kong University of Science and Technology, Hong Kong SAR, China
Image quality assessment (IQA) aims to evaluate the perceptual visual quality of images. Recently, the rapid development of Multi-modal Large Language Models (MLLMs) has significantly advanced the IQA field. By integrating chain-of-thought (CoT) reasoning and reinforcement learning (RL), these models can generate more accurate and interpretable results while relying on less training data. However, existing MLLM-based methods are predominantly constrained to holistic, image-level evaluations and lack the capability to produce fine-grained, localized assessments, which substantially limits their practical applicability. To address this limitation, we propose VGIQA, a novel IQA model endowed with both local and global quality assessment capabilities. VGIQA is trained via a two-stage paradigm comprising a cold-start phase followed by reinforcement learning. Driven by carefully designed local quality assessment reward functions, our model exhibits exceptional image quality understanding on VGSR-2K, a new dataset we propose specifically for local quality evaluation, significantly outperforming existing open-source MLLMs such as Qwen2.5-VL-72B-Instruct and Qwen3-VL-32B-Instruct. Furthermore, the local quality evaluation task can effectively enhance the model's performance on the degradation perception task.
Weichang Wang, Liyu Chen, Shuyue Guan +4
State Key Laboratory of Integrated Services Networks, Xidian University · Huawei Foundation Model Department