cs.CVApr 13, 2026
SavePanoptic Pairwise Distortion Graph
Organizations: Huawei Technologies, Canada
Abstract
In this work, we introduce a new perspective on comparative image assessment by representing an image pair as a structured composition of its regions. In contrast, existing methods focus on whole image analysis, while implicitly relying on region-level understanding. We extend the intra-image notion of a scene graph to inter-image, and propose a novel task of Distortion Graph (DG). DG treats paired images as a structured topology grounded in regions, and represents dense degradation information such as distortion type, severity, comparison and quality score in a compact interpretable graph structure. To realize the task of learning a distortion graph, we contribute (i) a region-level dataset, PandaSet, (ii) a benchmark suite, PandaBench, with varying region-level difficulty, and (iii) an efficient architecture, Panda, to generate distortion graphs. We demonstrate that PandaBench poses a significant challenge for state-of-the-art multimodal large language models (MLLMs) as they fail to understand region-level degradations even when fed with explicit region cues. We show that training on PandaSet or prompting with DG elicits region-wise distortion understanding, opening a new direction for fine-grained, structured pairwise image assessment.
Figures & tables
Figure 1: DG Task Overview. Top: Given two images, Panda learns the proposed Distortion Graph ( DG ). Bottom: Grounded Subgraphs illustrate how DG grounds regions in terms of distortion information.
Figure 2: Motivation. Current MLLMs (e.g., Co-Instruct ( 2024c ) ) fail at region-level understanding, struggling even when given explicit region details (name, description, bounding box). DG grounds assessment in regions, relating distortions and attributes to provide a structured view. Optionally, the graph can be fed to an MLLM for region-wise language descriptions. Scene relations (yellow) are not predicted. Best viewed zoomed-in.
Figure 3: Architecture Diagram. Illustration of the proposed Panda architecture to learn Distortion Graph ( DG ). A pair of image is fed as input, and for each region in the pair, their comparative relationship (predicates), distortion type, severity type and quality score (attributes) are predicted.
| Benchmark | Region First | Comparative Nature | Diverse Distortions | Severity Levels | Quality Score |
|---|---|---|---|---|---|
| Q-Bench ( Wu et al., 2024a ) | {\color[rgb]{1,0,0}\newcrossmark} | {\color[rgb]{1,0,0}\newcrossmark} | {\color[rgb]{1,0,0}\newcrossmark} | {\color[rgb]{1,0,0}\newcrossmark} | ✓ |
| DQ495K ( You et al., 2024a ) | {\color[rgb]{1,0,0}\newcrossmark} | ✓ | ✓ | ✓ | {\color[rgb]{1,0,0}\newcrossmark} |
| Seagull-100w ( Chen et al., 2024c ) | ✓ | {\color[rgb]{1,0,0}\newcrossmark} | {\color[rgb]{1,0,0}\newcrossmark} | ✓ | ✓ |
| Q-Pathways ( Wu et al., 2023 ) | {\color[rgb]{1,0,0}\newcrossmark} | {\color[rgb]{1,0,0}\newcrossmark} | ✓ | {\color[rgb]{1,0,0}\newcrossmark} | ✓ |
| MICBench ( Wu et al., 2024c ) | {\color[rgb]{1,0,0}\newcrossmark} | ✓ | ✓ | {\color[rgb]{1,0,0}\newcrossmark} | ✓ |
| Q-Ground100K ( Chen et al., 2024b ) | ✓ | {\color[rgb]{1,0,0}\newcrossmark} | {\color[rgb]{1,0,0}\newcrossmark} | {\color[rgb]{1,0,0}\newcrossmark} | ✓ |
Table 1: Benchmark Summary. A comparison of PandaBench with prior distortion benchmarks in literature. Note that, none of these benchmarks are both region-first and comparative by design.
| Methods | Comparison | Distortion | Severity | Scores | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| A | P | R | F1 | A | P | R | F1 | A | P | R | F1 | SR | PL | |
| Q-SiT ( 2025 ) | – | – | – | – | – | – | – | – | – | – | ||||
| Q-Insight ( 2025 ) | – | – | – | – | – | – | ||||||||
| DepictQA ( 2024b ) | – | – | – | – | – | – | ||||||||
| Seagull ( 2024c ) | – | – | – | – | – | – | ||||||||
Table 2: PandaBench Easy. Results of different MLLMs on the Easy set. indicates open source/open-weight, and denotes closed-source MLLMs, stands for baselines. † indicates method is trained on PandaSet . A : Accuracy, P : Precision, R : Recall, SR : SRCC & PL : PLCC.
| Methods | Comparison | Distortion | Severity | Scores | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| A | P | R | F1 | A | P | R | F1 | A | P | R | F1 | SR | PL | |
| Q-SiT ( 2025 ) | – | – | – | – | – | – | – | – | – | – | ||||
| Q-Insight ( 2025 ) | – | – | – | – | – | – | ||||||||
| DepictQA ( 2024b ) | – | – | – | – | – | – | ||||||||
| Seagull ( 2024c ) | – | – | – | – | – | – | ||||||||
Table 3: PandaBench Hard. Results of different MLLMs on the Hard set. indicates open source/open-weight, and denotes closed-source MLLMs, stands for baselines. † indicates method is trained on PandaSet . A : Accuracy, P : Precision, R : Recall, SR : SRCC & PL : PLCC.
Figure 4: Emergent Results. Feeding predicted DG in prompt as chain-of-thought (CoT) results in improvement of (accuracy) in region-wise distortion understanding of GPT-5 Mini.
Figure 5: Distortion Graph as Context. Illustrative figure analyzing showcase application wherein predicted DG is fed as context to GPT-5 Mini. Top: Sample taken from PandaBench Easy, Bottom: Sample taken from PandaBench Hard. GPT-5 Mini indeed overrides the predicted DG when the pixels disagree with DG .
Appendix figures & tables18 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: PandaSet Summary. Left: A word cloud of region names indicating diversity of the objects in images. Right: A region-wise summary of PandaSet in terms of distortions & severity. All of the distortions are uniformly distributed across the regions, and we broadly categorize the distortions in super categories: weather, camera/equipment, digital, light, and clean .
| Methods | Comparison | Distortion | Severity | Scores | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| A | P | R | F1 | A | P | R | F1 | A | P | R | F1 | SR | PL | |
| Q-SiT ( 2025 ) | – | – | – | – | – | – | – | – | – | – | ||||
| Q-Insight ( 2025 ) | – | – | – | – | – | – | ||||||||
| DepictQA ( 2024b ) | – | – | – | – | – | – | ||||||||
| Seagull ( 2024c ) | – | – | – | – | – | – | ||||||||
Table 4: PandaBench Medium. Results of different MLLMs on the Medium set. indicates open source/open-weight, and denotes closed-source MLLMs, stands for baselines. † indicates method is trained on PandaSet . A : Accuracy, P : Precision, R : Recall, SR : SRCC & PL : PLCC.
Figure 7: Design Choice Ablation. Accuracy comparison of different design choices: backbone feature extractors (solid line) and Transformer blocks (dotted line). Panda maintains balance in size, performance, and efficiency.
| Type | Distortion Classification | |||
|---|---|---|---|---|
| Accuracy | Precision | Recall | F1 | |
| Region Wise | ||||
| Whole Image | ||||
Table 5: Region-wise vs. Whole Image. Comparison of DepictQA ( You et al., 2024a ) across two different evaluation setups: whole image and region-wise for distortion classification task.
| Method | LLM | Vision Tower | Compute Cost / Image Pair (secs) | Monetary Value / Image Pair (USD) | Parameters (Billion) |
|---|---|---|---|---|---|
| Q-SiT ( 2025 ) | Qwen2 LLM | SigLIP | N/A | ||
| Q-Insight ( 2025 ) | Qwen2.5 LLM | Qwen2.5 | N/A | ||
| DepictQA ( 2024a ) | Vicuna v1.5 | CLIP | N/A | ||
| Seagull ( 2024c ) | Vicuna v1.1 | CLIP | N/A | ||
| GPT-5 Nano ( 2025 ) | Proprietary | Proprietary | N/A | Proprietary | |
| GPT-5 Mini ( 2025 ) | Proprietary | Proprietary | N/A | Proprietary |
Table 6: Cost Analysis. A summary of compute and monetary costs of different methods computed for a single image pair with regions.
Figure 8: All Distortion Types. We visualize all different distortion types on the same image taken from PandaSet . Each distortion degrades the image differently. Some distortions ruin the perceptual quality of the image more than others (e.g., haze, contrast decrease).
| Method | Ranking Accuracy | Inference Time |
|---|---|---|
| Q-Insight ( Li et al., 2025 ) | hours | |
| GPT-5 Mini ( OpenAI, 2025 ) | N/A | |
| Panda (ZS) Score Based | mins | |
| Panda (ZS) Predicate Based | mins |
Table 7: Whole-Image Instant Ranking. DG (generated by Panda ) aggregates naturally to whole-image assessment.
Figure 9: Prompt Type (a). A template of prompt for open-source MLLMs. The tags for keywords like image, input, user, assistant, output, etc. that each method requires are added as necessary.
Figure 10: Prompt Type (b). A template of prompt for closed-source MLLMs. Frontier methods have superior instruction following ability, and can reason about the regions from the prompt.
| Methods | Accuracy |
|---|---|
| mPLUG-Owl2 ( 2024 ) | |
| LLaVA-1.6 ( 2024b ) | |
| Q-Instruct ( 2024b ) | |
| Panda (Predicate) | |
| Panda (Score) |
Table 8: Generality of DG Representation on TID2013.
Figure 11: Q-Instruct Behavior. An illustration of output from Q-Instruct ( Wu et al., 2023 ) when prompted for multiple instructions. It is insensitive to the order of image, even when explicitly specified, misses degradation, struggles to follow instruction, and repeats irrelevant information.
Figure 12: Co-Instruct Behavior. An illustration of output from Co-Instruct ( Wu et al., 2024c ) when prompted for multiple instructions. It fails to perform comparative assessment, frequently misses regions, and struggles with instruction following.
Figure 13: Open-Source MLLM Prompt/Output. A representative example of prompt type (a) along with output for all open-source MLLMs evaluated in this work.
Figure 14: Closed-Source MLLM Prompt/Output. A representative example of prompt type (b) along with output for all closed-source MLLMs evaluated in this work. Note that, in this example, the image has regions.
| Standard Deviation | GPT-5 Nano | GPT-5 Mini | GPT-4o | Gemini 2.5 Pro | |
|---|---|---|---|---|---|
| PandaBench Hard | Distortion | ||||
| Severity | |||||
| Comparison | |||||
| SRCC | |||||
| PLCC | |||||
| PandaBench Medium | Distortion | ||||
Table 9: Standard Deviation on Accuracy Metric. The standard deviation for accuracy metric on all four tasks of PandaBench computed over three independent runs of each method. We do not report standard deviation for precision, recall and F1 score for brevity, but they follow similar trends.
Figure 15: PandaBench . Representative samples from Easy, Medium, and Hard splits of PandaBench . In Easy split, only one distortion afflicts the entire image (and its regions, but with varied severity), while in Medium, mixed has region-wise distortions (see the person in blue jacket). In Hard split, the distortion varies by region in both images (see ground, bike, trees, etc.). Taken together, they represent a spectrum of difficulty with subtle degradations inherent in the image to complicated region-wise distortions.
Figure 16: Hyperparmeter Sweep. Plot of optimization objective hyperparameter sweep with cross-validation on validation set of PandaSet . Each grey point denotes an experiment that performed noticeably worse, and we label top five settings with colored mark. A denotes anchor, T denotes Target, Dist. denotes distortion, Sev. is short for severity, and Acc. denotes accuracy.
Figure 17: Dense Distortion Graph Sample. An example of an image pair with several regions resulting in a dense distortion graph. Left image is Anchor (purple nodes in graph), Right image is Target (green nodes in graph). Legend is presented, and ’rels’ is short for relations.