cs.CVApr 21, 2026

DistortBench: Benchmarking Vision Language Models on Image Distortion Identification

Authors: Divyanshu GoyalAkhil EppaVanya Bannihatti Kumar

Organizations: Adobe Inc.

Abstract

Vision-language models (VLMs) are increasingly used in settings where sensitivity to low-level image degradations matters, including content moderation, image restoration, and quality monitoring. Yet their ability to recognize distortion type and severity remains poorly understood. We present DistortBench, a diagnostic benchmark for no-reference distortion perception in VLMs. DistortBench contains 13,500 four-choice questions covering 27 distortion types, six perceptual categories, and five severity levels: 25 distortions inherit KADID-10k calibrations, while two added rotation distortions use monotonic angle-based levels. We evaluate 18 VLMs, including 17 open-weight models from five families and one proprietary model. Despite strong performance on high-level vision-language tasks, the best model reaches only 61.9% accuracy, just below the human majority-vote baseline of 65.7% (average individual: 60.2%), indicating that low-level perceptual understanding remains a major weakness of current VLMs. Our analysis further reveals weak and non-monotonic scaling with model size, performance drops in most base--thinking pairs, and distinct severity-response patterns across model families. We hope DistortBench will serve as a useful benchmark for measuring and improving low-level visual perception in VLMs.

Explore similar work

CardsList
  1. Seeing Isn't Believing: Uncovering Blind Spots in Evaluator Vision-Language Models

    Apr 23, 2026Mohammed Safi Ur Rahman Khan, Sanjay Suryanarayanan, Tushar Anand +1Large Vision Language ModelsBlind Spots

  2. Same Answer, Different Representations: Hidden instability in VLMs

    Feb 6, 2026Farooq Ahmad Wani, Alessandro Suglia, Rohit Saxena +6Instability