cs.CVApr 3, 2026

Banana100: Breaking NR-IQA Metrics by 100 Iterative Image Replications with Nano Banana Pro

Authors: Kenan Tang, Praveen Arunshankar, Andong Hua, Anthony Yang, Yao Qin

Organizations: University of California, Santa Barbara

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

The multi-step, iterative image editing capabilities of multi-modal agentic systems have transformed digital content creation. Although latest image editing models faithfully follow instructions and generate high-quality images in single-turn edits, we identify a critical weakness in multi-turn editing, which is the iterative degradation of image quality. As images are repeatedly edited, minor artifacts accumulate, rapidly leading to a severe accumulation of visible noise and a failure to follow simple editing instructions. To systematically study these failures, we introduce Banana100, a comprehensive dataset of 28,000 degraded images generated through 100 iterative editing steps, including diverse textures and image content. Alarmingly, image quality evaluators fail to detect the degradation. Among 21 popular no-reference image quality assessment (NR-IQA) metrics, none of them consistently assign lower scores to heavily degraded images than to clean ones. The dual failures of generators and evaluators may threaten the stability of future model training and the safety of deployed agentic systems, if the low-quality synthetic data generated by multi-turn edits escape quality filters. We release the full code and data to facilitate the development of more robust models, helping to mitigate the fragility of multi-modal agentic systems.

Figures & tables

NameImage ContentChallengesResolution
BuildingA skyscraperPreservation of highly-regular grid patterns and aerial perspective3392 ×\times 5056
DongpoA plate of Chinese potstickersPreservation of multi-scale food structure and texture5504 ×\times 3072
EkphrasisA still life paintingPreservation of diverse textures of the same type of object5632 ×\times 3072
FogA misty forestPreservation of texture details under lowered color contrast by haze5504 ×\times 3072
HoliExploding colorful Holi powderPreservation of high color contrast and particle textures5632 ×\times 3072
LibraryInterior of a libraryPreservation of deep shadows and shafts of light5632 ×\times 3072
Table 1: The initial images cover diverse content and challenges. The top part of the table includes 11 photorealistic images generated by Nano Banana Pro, and the bottom part includes 2 animation-style images generated by SPICE. The resolutions are width ×\times height.
Figure 2: The reasoning summary from Nano Banana Pro appears as clear-cut generation and evaluation sections. The bold text are section titles, copied verbatim from the reasoning summary from the Nano Banana Pro API. In this example, the first two sections are dedicated to image generation, whereas the last two sections are dedicated to the evaluation of a generated image.
Figure 3: A summary of the failure modes of instruction following, categorized into sub-object level (blue), object level (yellow), and image level (green). The images have been cropped and zoomed for visual clarity. As the failures were consistent across different runs and editing steps, we do not report the exact run index and step index for each image here. See Section 3 for details.
MetricTypical RangeHigher is Better?
ARNIQA [ 2 ][0, 1]Yes
BRISQUE [ 42 ][0, 150]No
CLIPIQA [ 55 ][0, 1]Yes
CNNIQA [ 30 ][0, 1]Yes
DBCNN [ 68 ][0, 1]Yes
HyperIQA [ 51 ][0, 1]Yes
Table 2: A summary of all the No-Reference Image Quality Assessment (NR-IQA) metrics we used for evaluation. In the first part of the table, we show all the NR-IQA metrics implemented in the pyiqa Python library [ 12 ] , with the only exception of MACLIP [ 34 ] , which is only a placeholder and raises a non-implemented error. The typical range is obtained from the pyiqa library, which do not necessarily correspond to the actual observed range. In the second part of the table, we include two recent NR-IQA metrics based on latest large vision-language models.
Figure 4: We normalized NR-IQA scores (BRISQUE as an example) and calculated the difference across steps to quantify the score trend. Please see Section 4.1 for details. The three Δ\Delta values can also be found at the intersection of the second row (Dongpo) and the second column in each heatmap of Figure 5 .
Figure 5: All 21 NR-IQA metrics fail on identifying degradation, assigning higher normalized scores to images with worse quality. The heatmap shows the gap between the pair of normalized scores calculated for the image of a later editing (5, 10, or 20) and the image of the first step. The normalization converts each NR-IQA metric to the same scale of [0, 100], with higher scores corresponding to better image quality. Positive gaps indicate failures and are marked by blue colors in the heatmap. Due to the diversity in the texture types of the 12 initial images, each NR-IQA metric fails on a different set of initial images.
Figure 6: Aggregated results show that none of the 21 NR-IQA metrics fully succeed on all images. This heatmap overlays the 3 heatmaps from Figure 5 , and the brightness of the blue colors correspond to the total number of failures. None of the metrics show a fully white column, corresponding to consistent success.
Figure 7: Despite a consistent drop in image quality, the RALI score (higher is better) fluctuates over the steps. The fluctuation shows that RALI is not robust against the semantic change caused by iterative object addition (add-100-fruits).
Figure 8: The LLM-as-a-judge prompt template to identify whether Nano Banana Pro acknowledges the noise during generation. The reasoning summary, such as one shown in Figure 2 , will be inserted to the end of this prompt template.
Figure 9: Different noise accumulates during image replication by 3 more models. Nano Banana 2 Fast (without reasoning) generates wrinkles that align with the contours of the objects. FLUX.2 [dev] generates scatter points on many of the objects. Qwen Image Edit simplifies the texture and erroneously duplicates objects on the right side of the image to the left side of the image.
Figure 10: Similar to the evaluation of Nano-Banana-Pro generated results, NR-IQA metrics also fail for results from 3 more models. No metric succeeds on all initial images and all models. Interestingly, PI and PIQE fully succeed on Qwen Image Edit, but fails for almost all initial images for Nano Banana 2 Fast. The diverse failure patterns across metrics further confirm the difference of the noise patterns from each model ( Figure 9 ).

Explore similar work

CardsList

  1. Can Nano Banana 2 Replace Traditional Image Restoration Models? An Evaluation of Its Performance on Image Restoration Tasks

    Apr 3, 2026Weixiong Sun, Xiang Yin, Chao DongImage RestorationAll-in-One Image Restoration

    Shenzhen University of Advanced Technology · Fudan University · Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences

  2. On-Policy Self-Distillation for Multi-Turn Image Editing

    Sep 28, 2026Liangbing Zhao, Le Zhuo, Mohamed ElhoseinyText-Guided Image EditingDiffusion Model Distillation

    KAUST · Krea AI

  3. WeAgent-MMGenEdit: A Full-Stack Recipe for Multimodal Agentic Image Generation and Editing

    Sep 4, 2026Hui Zhang, Zongkai Liu, Liqiang Niu +6Agentic Image GenerationImage Editing

    Weixin AI, Tencent