Multi-Image Editing Benchmarks

Follow this topic Ask about this topic

Momentum

1 paper in the last four weeks, down 67% on the four weeks before. 0.0% of all new papers.

Jul 6Week of Sep 21

Latest papers 42

All topics

Do MLLM Judges Judge the Edit? Auditing Bias in Image Editing Evaluation with Verified Quality Preservation
Oct 1, 2026Yuan Huang, Zirui Song, Xiuying ChenLarge Language Model Judges Multi-Image Editing Benchmarks
Northeastern University · Mohamed bin Zayed University of Artificial Intelligence
VIEScore2: Unified Image Evaluation with Spatially Grounded Explanations
Oct 1, 2026Xianda Du, Max Ku, Weiming Ren +5Multi-Image Editing Benchmarks Synthetic Image Detection
University of Waterloo · National Taiwan University · Nanyang Technological University +1
InfoEdit: Probing Global Layout Reasoning in Infographic Editing
Sep 27, 2026Cheng Yang, Chufan Shi, Huijuan Wang +6Multi-Image Editing Benchmarks Layouts
University of California San Diego · University of Southern California · University of Illinois Urbana-Champaign +1
Capacity Confounds and Coverage Guarantees in Adaptive Sub-model Federated Learning
Aug 7, 2026Alireza Moayedikia, Alicia Troncoso LoraFederated Learning Long-Run Coverage Guarantees
Department of Business Technology and Entrepreneurship, Swinburne University of Technology, Melbourne, Australia · Data Science & Big Data Lab, Universidad Pablo de Olavide, Seville, Spain
COMEX: A Composition-Grounded Benchmark and Learning Framework for Explainable Aesthetic Image Cropping
Aug 4, 2026Rui Yang, Wei Zhou, Dingyong Gou +5Cropping Multi-Image Editing Benchmarks
State Key Laboratory of Mobile Network and Mobile Multimedia Technology, ZTE Corporation · School of Data Science and Institute of Artificial Intelligence, Chang’an University
MIEScore: Human-Aligned Evaluation for Multi-Source Image Editing
Aug 3, 2026Zitong Xu, Huiyu Duan, Xinyun Zhang +7Multi-Image Editing Benchmarks Image Editing
Shanghai Jiao Tong University · Vivo Mobile Communication Co., Ltd · University of Electronic and Science Technology of China
EEG-EditBench: Probing Visual Information in EEG-Image Retrieval Models with Controlled Image Edits
Jul 30, 2026Kaifan Zhang, Lihuo He, Yuqi Ji +4Multi-Image Editing Benchmarks Bird
School of Electronic Engineering, Xidian University, Xi’an, China · School of Software, Tsinghua University, Beijing, China · Chongqing University of Posts and Telecommunications, Chongqing, China
DS@GT ARC at ImageCLEFmedical 2026: Architectural Diversity for Concept Detection and Foundation-Model Scaling for Caption Prediction in Medical Image Analysis
Jul 30, 2026Bowen Wang, Youwen Zhang, Ritesh MehtaMedical Images Convnext
Georgia Institute of Technology, North Ave NW, Atlanta, GA 30332
MPIE-Bench: Benchmarking Anatomically Plausible Multi-Person Interaction Editing
Jul 30, 2026Jiajia Lin, Mingxuan Du, Tuowen Zhou +2Multi-Image Editing Benchmarks Human Mesh Recovery
University of Science and Technology of China, 2Metastone Technology, Beijing, China · Work done during the internship at Metastone.
BG-REAL: A Public Real-Data Anchored Benchmark for Background Manipulation Detection and Localization
Jul 28, 2026Bugra Alperen Uluirmak, Rifat KurbanMultimedia Forensics Multi-Image Editing Benchmarks
Erciyes University, Kayseri, Türkiye · Abdullah Gül University, Kayseri, Türkiye
MEDit-Bench: A Dataset for Evaluating Message-Driven Narrative Video Editing
Jul 28, 2026Katsuya Ogata, Zongshang Pang, Mayu Otani +1Video Editing Multi-Image Editing Benchmarks
1The University of Osaka · 2CyberAgent, Inc.
MuViSeg: Multi-View Segment Correspondences from Dense Geometry Priors
Jul 20, 2026Denis Fatykhoph, Timur Akhtyamov, Konstantin Pakulev +2Multi-View Amodal Segmentation
Applied AI Institute, Moscow, Russia
ChartSync: A Benchmark for Visuo-Logical Cascading Chart Editing
Jul 11, 2026Jiakang Yu, Yixuan Chai, Tianci Wang +7Multi-Image Editing Benchmarks Renderer
1Jianghan University · †Work done during an internship at HiThink Research. · 2HiThink Research +3
Does AI Understand Imaging? A Systematic Benchmark of Agentic AI for Computational Imaging Tasks
Jul 8, 2026Ethan Chung, Chuanjun Zheng, Jasper Tan +3Multi-Image Editing Benchmarks Multimodal Benchmarks
University of Hawaii at Manoa · Glass Imaging
Cross-device Collaborative Test-time Adaptation with Zeroth-order Optimization and Model Merging
Jul 3, 2026Yu Mitsuzumi, Akisato Kimura, Yasuhiro Fujiwara +1Test-Time Adaptation Zeroth-Order Optimization
NTT, Inc. · Kyoto University
Beyond Pixel Diffs: Benchmarking Image Change Captioning for Web UI Visual Regression Testing
Jul 2, 2026Licheng Zhang, Bach Le, Pengtao Zhao +1Image Difference Captioning Multi-Image Editing Benchmarks
School of Computing and Information Systems, University of Melbourne, Melbourne, Australia
Accelerating Hierarchical Sparse Predictive Coding with Hybrid Amortized Inference
Jun 26, 2026Kazuhisa FujitaPredictive Coding Multi-Image Editing Benchmarks
Department of Clinical Engineering, Komatsu University, 10-10 Doihara-Machi, Komatsu, Ishikawa, Japan 923-0921
Scaling Multi-Reference Image Generation with Dynamic Reward Optimization
Jun 25, 2026Wenwang Huang, Yusen Fu, Junjie Wang +6Multi-Reference Image Generation Multi-Image Editing Benchmarks
Harbin Institute of Technology, Shenzhen · Independent Researcher · Shenzhen Loop Area Institute
Do Image Editing Models Understand Lighting?
Jun 25, 2026Tim Küchler, Johann-Friedrich Feiden, Matthias Nießner +1Lighting Multi-Image Editing Benchmarks
Heidelberg University · Zuse School ELIZA · Technical University of Munich
Fabric Image Demoiréing Benchmark from Synthesis to Restoration
Jun 23, 2026Pengchao Wei, Xiaojie GuoMulti-Image Editing Benchmarks Perceptual Aliasing
Tianjin University, Tianjin, China
How Fragile Are Training-Free AI-Generated Image Detectors? A Controlled Audit of Score Direction, Preprocessing, and Compression
Jun 18, 2026Jingwen Zhou, Mingzhe WangAi-Generated Image Detection Unsupervised Detection
Xidian University, Xi’an, China
An Extensive Benchmark for Single-round and Multi-round Instruction-based Image Editing
Jun 14, 2026Yiwei Ma, Ke Ye, Weihuang Lin +4Multi-Image Editing Benchmarks Image Editing
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, 361005, Fujian, P.R. China. · National University of Singapore, 119077, Singapore.
Food-R1: A Unified Multi-Task Food Vision-Language Model with Reinforcement Learning
Jun 3, 2026Yu Zhu, Yongkang Li, Wenjie Zhu +5Diet Recent Vision-Language Models
Huazhong University of Science and Technology, Wuhan, China
TECCI: Tricky Edits of Collected and Curated Images
May 31, 2026Aishwarya Agrawal, Roy Hirsch, Yasumasa Onoe +2Multi-Image Editing Benchmarks Image Editing
Google Research · Google DeepMind
PaintBench: Deterministic Evaluation of Precise Visual Editing
May 29, 2026Kai Xu, Ellis Brown, Shrikar Madhu +3Multi-Image Editing Benchmarks Image Editing
New York University
Benchmarking and Enhancing Text-to-Image Models for Generating Visual Representations in Early Arithmetic Education
May 29, 2026Junling Wang, Boqi Chen, Heejin Do +3Text-To-Image Multi-Image Editing Benchmarks
Department of Computer Science, ETH Zurich · ETH AI Center
PROVE: A Perceptual RemOVal cohErence Benchmark for Visual Media
May 14, 2026Fuhao Li, Shaofeng You, Jiagao Hu +6Multi-Image Editing Benchmarks Spatial Coherence
MiLM Plus, Xiaomi Inc.
PhysEditBench: A Protocol-Conditioned Benchmark for Dense Physical-Map Prediction with Image Editors
May 13, 2026Jiaxin Yang, Yu Hou, Muxin Liu +5Multi-Image Editing Benchmarks Image Editing
Southern University of Science and Technology, Shenzhen, China · The University of Hong Kong, Hong Kong, China · East China Normal University, Shanghai, China
Edit-Compass & EditReward-Compass: A Unified Benchmark for Image Editing and Reward Modeling
May 13, 2026Xuehai Bai, Yang Shi, Yi-Fan Zhang +7Multi-Image Editing Benchmarks Image Editing
HDU · PKU · Kling Team +1
ShapeCodeBench: A Renewable Benchmark for Perception-to-Program Reconstruction of Synthetic Shape Scenes
May 12, 2026Shivam KumarSynthetic Benchmark Multi-Image Editing Benchmarks
Independent Researcher
Vision2Code: A Multi-Domain Benchmark for Evaluating Image-to-Code Generation
May 11, 2026Ajay Vikram Periasami, Junlin Wang, Bhuwan DhingraImage-To-Code Generation Multi-Image Editing Benchmarks
Duke University
FraudBench: A Multimodal Benchmark for Detecting AI-Generated Fraudulent Refund Evidence
May 9, 2026Xinyu Yan, Boyang Chen, Jiaming Zhang +12Ai-Generated Image Detection Authenticity
College of Computing and Data Science, Nanyang Technological University · Alibaba-NTU Global e-Sustainability CorpLab (ANGEL) · Alibaba Group
RewardHarness: Self-Evolving Agentic Post-Training
May 9, 2026Yuxuan Zhang, Penghui Du, Bo Li +11Progress Reward Modeling Multi-Image Editing Benchmarks
University of British Columbia · Kolors Team, Kuaishou Technology · University of Waterloo +4
How Far Is Document Parsing from Solved? PureDocBench: A Source-Traceable Benchmark across Clean, Degraded, and Real-World Settings
May 8, 2026Zhiheng Li, Zongyang Ma, Jiaxian Chen +12Document Parsing Multi-Image Editing Benchmarks
CASIA · UCAS · Beijing Key Laboratory of Super Intelligent Security of Multi-Modal Information +6
ReasonEdit: Towards Interpretable Image Editing Evaluation via Reinforcement Learning
May 8, 2026Honghua Chen, Zitong Xu, Huiyu Duan +3Multi-Image Editing Benchmarks Image Editing
University of Electronic Science and Technology of China · 2Shanghai Jiao Tong University
DiffCap-Bench: A Comprehensive, Challenging, Robust Benchmark for Image Difference Captioning
May 6, 2026Yuancheng Wei, Haojie Zhang, Linli Yao +7Image Difference Captioning Multi-Image Editing Benchmarks
South China University of Technology · Peking University · The University of Hong Kong +2
StableI2I: Spotting Unintended Changes in Image-to-Image Transition
May 6, 2026Jiayang Li, Shuo Cao, Xiaohui Li +6Multi-Image Editing Benchmarks Perceptual Quality
Peking University · Shanghai Artificial Intelligence Laboratory · University of Science and Technology of China +1
MMCORE: MultiModal COnnection with Representation Aligned Latent Embeddings
Apr 21, 2026Zijie Li, Yichun Shi, Jingxiang Sun +8Multi-Image Editing Benchmarks Visual Embeddings
ByteDance Seed
VEFX-Bench: A Holistic Benchmark for Generic Video Editing and Visual Effects
Apr 17, 2026Xiangbo Gao, Sicong Jiang, Bangya Liu +12Video Editing Multi-Image Editing Benchmarks
1Texas A&M University · 2Visko Platform · 3Abaka AI
UniEditBench: A Unified and Cost-Effective Benchmark for Image and Video Editing via Distilled MLLMs
Apr 17, 2026Lifan Jiang, Tianrun Wu, Yuhang Pei +3Video Editing Multi-Image Editing Benchmarks
Zhejiang University Hangzhou, China
VTEdit-Bench: A Comprehensive Benchmark for Multi-Reference Image Editing Models in Virtual Try-On
Mar 12, 2026Xiaoye Liang, Zhiyuan Qu, Mingye Zou +4Virtual Try-On Multi-Image Editing Benchmarks
School of Electronic Information Engineering, Beihang University, Beijing, China · Zhongguancun Academy, Beijing, China · Faculty of Computing, Harbin Institute of Technology, Harbin, China +2
TIIF-Bench: How Does Your T2I Model Follow Your Instructions?
Jun 2, 2025Xinyu Wei, Jinrui Zhang, Zeqing Wang +4Modern Text-To-Image Models Text-To-Image
The Hong Kong Polytechnic University · OPPO Research Institute · Tsinghua University

Oct 1, 2026, cs.CVJ/K move · Enter open · S save

Do MLLM Judges Judge the Edit? Auditing Bias in Image Editing Evaluation with Verified Quality Preservation

Yuan Huang, Zirui Song, Xiuying Chen

Northeastern University · Mohamed bin Zayed University of Artificial Intelligence

Large Language Model Judges Multi-Image Editing Benchmarks Multimodal Evaluation Reference-Guided Multimodal In-Context Verification Edit Betting

Open paper AskPDF arXiv

Multimodal large language models (MLLMs) are increasingly used as automated judges for instruction-based image editing and as reward signals for model training. However, systematically auditing whether these judges are influenced by cues irrelevant to editing quality is challenging because visual interventions may themselves alter the quality being evaluated. A judgment shift can therefore be attributed to bias only when the intervention is verified to preserve the underlying editing quality. To address this challenge, we introduce EditJudgeBias, a counterfactual benchmark with verified quality preservation, comprising 1,196 real editing samples and 13 cues injected across four evaluation sites. We verify quality preservation for the requested edit using calibrated multimodal validators, controls, and human inspection. We then audit five MLLM judges along three complementary dimensions: invariance to quality-preserving cues, agreement with human judgments, and stability of pairwise preferences. Importantly, observed shifts are evaluated against each judge's own zero-dose and re-query noise floors rather than against zero. Experiments show that quality-preserving cues move every judge beyond its own noise. Fabricated majority opinions increase ratings, irrelevant visual elements cause larger shifts than whole-image manipulations, and swapping candidate order reverses up to 60.9% of pairwise decisions. Edit-region cues also tend to reduce human agreement. The three measures characterize judges differently, showing that robustness cannot be captured by a single metric.

Momentum

Related topics

Latest papers 42