The multi-step, iterative image editing capabilities of multi-modal agentic systems have transformed digital content creation. Although latest image editing models faithfully follow instructions and generate high-quality images in single-turn edits, we identify a critical weakness in multi-turn editing, which is the iterative degradation of image quality. As images are repeatedly edited, minor artifacts accumulate, rapidly leading to a severe accumulation of visible noise and a failure to follow simple editing instructions. To systematically study these failures, we introduce Banana100, a comprehensive dataset of 28,000 degraded images generated through 100 iterative editing steps, including diverse textures and image content. Alarmingly, image quality evaluators fail to detect the degradation. Among 21 popular no-reference image quality assessment (NR-IQA) metrics, none of them consistently assign lower scores to heavily degraded images than to clean ones. The dual failures of generators and evaluators may threaten the stability of future model training and the safety of deployed agentic systems, if the low-quality synthetic data generated by multi-turn edits escape quality filters. We release the full code and data to facilitate the development of more robust models, helping to mitigate the fragility of multi-modal agentic systems.
Figures & tables
Name
Image Content
Challenges
Resolution
Building
A skyscraper
Preservation of highly-regular grid patterns and aerial perspective
3392 × 5056
Dongpo
A plate of Chinese potstickers
Preservation of multi-scale food structure and texture
5504 × 3072
Ekphrasis
A still life painting
Preservation of diverse textures of the same type of object
5632 × 3072
Fog
A misty forest
Preservation of texture details under lowered color contrast by haze
5504 × 3072
Holi
Exploding colorful Holi powder
Preservation of high color contrast and particle textures
5632 × 3072
Library
Interior of a library
Preservation of deep shadows and shafts of light
5632 × 3072
Table 1: The initial images cover diverse content and challenges. The top part of the table includes 11 photorealistic images generated by Nano Banana Pro, and the bottom part includes 2 animation-style images generated by SPICE. The resolutions are width × height.
Figure 2: The reasoning summary from Nano Banana Pro appears as clear-cut generation and evaluation sections. The bold text are section titles, copied verbatim from the reasoning summary from the Nano Banana Pro API. In this example, the first two sections are dedicated to image generation, whereas the last two sections are dedicated to the evaluation of a generated image.
Figure 3: A summary of the failure modes of instruction following, categorized into sub-object level (blue), object level (yellow), and image level (green). The images have been cropped and zoomed for visual clarity. As the failures were consistent across different runs and editing steps, we do not report the exact run index and step index for each image here. See Section 3 for details.
Metric
Typical Range
Higher is Better?
ARNIQA [ 2 ]
[0, 1]
Yes
BRISQUE [ 42 ]
[0, 150]
No
CLIPIQA [ 55 ]
[0, 1]
Yes
CNNIQA [ 30 ]
[0, 1]
Yes
DBCNN [ 68 ]
[0, 1]
Yes
HyperIQA [ 51 ]
[0, 1]
Yes
Table 2: A summary of all the No-Reference Image Quality Assessment (NR-IQA) metrics we used for evaluation. In the first part of the table, we show all the NR-IQA metrics implemented in the pyiqa Python library [ 12 ] , with the only exception of MACLIP [ 34 ] , which is only a placeholder and raises a non-implemented error. The typical range is obtained from the pyiqa library, which do not necessarily correspond to the actual observed range. In the second part of the table, we include two recent NR-IQA metrics based on latest large vision-language models.
Figure 4: We normalized NR-IQA scores (BRISQUE as an example) and calculated the difference across steps to quantify the score trend. Please see Section 4.1 for details. The three Δ values can also be found at the intersection of the second row (Dongpo) and the second column in each heatmap of Figure 5 .
Figure 5: All 21 NR-IQA metrics fail on identifying degradation, assigning higher normalized scores to images with worse quality. The heatmap shows the gap between the pair of normalized scores calculated for the image of a later editing (5, 10, or 20) and the image of the first step. The normalization converts each NR-IQA metric to the same scale of [0, 100], with higher scores corresponding to better image quality. Positive gaps indicate failures and are marked by blue colors in the heatmap. Due to the diversity in the texture types of the 12 initial images, each NR-IQA metric fails on a different set of initial images.
Figure 6: Aggregated results show that none of the 21 NR-IQA metrics fully succeed on all images. This heatmap overlays the 3 heatmaps from Figure 5 , and the brightness of the blue colors correspond to the total number of failures. None of the metrics show a fully white column, corresponding to consistent success.
Figure 7: Despite a consistent drop in image quality, the RALI score (higher is better) fluctuates over the steps. The fluctuation shows that RALI is not robust against the semantic change caused by iterative object addition (add-100-fruits).
Figure 8: The LLM-as-a-judge prompt template to identify whether Nano Banana Pro acknowledges the noise during generation. The reasoning summary, such as one shown in Figure 2 , will be inserted to the end of this prompt template.
Figure 9: Different noise accumulates during image replication by 3 more models. Nano Banana 2 Fast (without reasoning) generates wrinkles that align with the contours of the objects. FLUX.2 [dev] generates scatter points on many of the objects. Qwen Image Edit simplifies the texture and erroneously duplicates objects on the right side of the image to the left side of the image.
Figure 10: Similar to the evaluation of Nano-Banana-Pro generated results, NR-IQA metrics also fail for results from 3 more models. No metric succeeds on all initial images and all models. Interestingly, PI and PIQE fully succeed on Qwen Image Edit, but fails for almost all initial images for Nano Banana 2 Fast. The diverse failure patterns across metrics further confirm the difference of the noise patterns from each model ( Figure 9 ).
Recent advances in generative AI raise the question of whether general-purpose image editing models can serve as unified solutions for image restoration. We conduct a systematic evaluation of Nano Banana 2 across diverse scenes and degradations. Our results show that prompt design is critical, with concise prompts and explicit fidelity constraints achieving a better balance between reconstruction and perceptual quality. Nano Banana 2 achieves competitive full-reference performance and is consistently preferred in user studies, while showing strong generalization in challenging scenarios. However, we observe a gap between perceptual quality and restoration fidelity, as the model tends to produce visually rich results with over-enhanced details and inconsistencies. This issue is not well captured by existing IQA metrics or user studies. Overall, general-purpose models show promise as unified IR solvers from a perceptual perspective, but require improved controllability and fidelity-aware evaluation. Further comparisons and detailed analyses are available in our project repository: https://github.com/yxyuanxiao/NanoBanana2TestOnIR.
Weixiong Sun, Xiang Yin, Chao Dong
Shenzhen University of Advanced Technology · Fudan University · Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences
Instruction-based image editing has achieved strong performance in single-turn settings, yet practical editing is often iterative, with each instruction applied to the output of the previous turn. We find that existing editing models degrade rapidly under recursive editing and attribute this failure to a train-test mismatch in the conditioning distribution: models are trained on clean source images but must repeatedly condition on their own imperfect outputs at inference time. To address this, we propose MT-OPSD, an on-policy self-distillation framework that trains the model on self-generated conditioning states with editing supervision from a clean-conditioned teacher, without requiring multi-turn annotations. We further introduce LME-Bench, a benchmark of 100 ten-turn editing sessions for evaluating long-horizon robustness. Experiments across three editing backbones show that MT-OPSD substantially improves long-horizon editing success and reduces multi-turn collapse while largely preserving single-turn editing quality.
Image generation and editing models have advanced rapidly, yet remain unreliable when prompts require external world knowledge. Bounded and long-tail parametric knowledge prevents direct or reason-then-generate approaches from recovering the required facts and visual appearances. Existing agentic generation and editing methods mitigate this limitation with retrieval tools, yet remain constrained by insufficient visual verification, overloaded policy models, and weak integration of retrieved textual and visual evidence. To address these limitations, we present WeAgent-MMGenEdit, a full-stack recipe including a multimodal harness, a scalable data construction pipeline, a comprehensive benchmark, and post-training methods for the agent policy and image backend. We first introduce WeAgent-Harness, a multimodal runtime with persistent evidence management and dedicated verification and integration tools that organize retrieved multimodal evidence into a dense carrier. Upon this, we develop a scalable pipeline for prompt synthesis and agentic trajectory collection, yielding 23K supervised trajectories and 14.7K RL tasks with three-layer verifiable checklists. We further introduce WeBench-MMGenEdit, a bilingual benchmark covering both knowledge-intensive image generation and multi-image editing. Finally, a two-sided post-training recipe based on SFT and RL improves the agent policy and image backend. Together, WeAgent-MMGenEdit enables a 30B-total/3B-active policy to outperform similarly sized policy models and approach the performance of a 1T-parameter agent.