IMAGAgent: Orchestrating Multi-Turn Image Editing via Constraint-Aware Planning and Reflection

Fei Shen; Chengyu Xie; Lihong Wang; Zhanyi Zhang; Xin Jiang; Xiaoyu Du; Jinhui Tang; Tat-Seng Chua

cs.GRFeb 12, 2026

IMAGAgent: Orchestrating Multi-Turn Image Editing via Constraint-Aware Planning and Reflection

Authors: Fei Shen, Chengyu Xie, Lihong Wang, Zhanyi Zhang, Xin Jiang, Xiaoyu Du, Jinhui Tang, Tat-Seng Chua

Organizations: National University of Singapore · Nanjing University of Science and Technology · Nanjing Forestry University

Abstract

Existing multi-turn image editing paradigms are often confined to isolated single-step execution. Due to a lack of context-awareness and closed-loop feedback mechanisms, they are prone to error accumulation and semantic drift during multi-turn interactions, ultimately resulting in severe structural distortion of the generated images. To address this, we propose IMAGAgent, a multi-turn image editing agent framework based on a ``plan--execute--reflect'' closed-loop mechanism that achieves deep synergy among instruction parsing, tool scheduling, and adaptive correction within a unified pipeline. Specifically, we first present a constraint-aware planning module that leverages a vision-language model (VLM) to precisely decompose complex natural language instructions into a series of executable sub-tasks, governed by target singularity, semantic atomicity, and visual perceptibility. Then, the tool-chain orchestration module dynamically constructs execution paths based on the current image, the current sub-task, and the historical context, enabling adaptive scheduling and collaborative operation among heterogeneous operation models covering image retrieval, segmentation, detection, and editing. Finally, we devise a multi-expert collaborative reflection mechanism in which a central large language model (LLM) synthesizes evaluations from multiple vision-language models into holistic feedback, triggering fine-grained self-correction and recording feedback outcomes to optimize future decisions. Extensive experiments on our constructed MTEditBench and the MagicBrush dataset demonstrate that IMAGAgent achieves performance significantly superior to existing methods in terms of instruction consistency, editing precision, and overall quality. The code is available at https://github.com/hackermmzz/IMAGAgent.

Figures & tables

Fig. 1: (a) Existing methods suffer from error accumulation due to isolated execution. (b) IMAGAgent ensures structural and intentional consistency through closed-loop feedback.

Fig. 2: IMAGAgent Framework. We reformulate multi-turn image editing into a closed-loop process with three modules: (1) constraint-aware planning for VLM-based instruction decomposition; (2) tool-chain orchestration for dynamic scheduling of vision tools; and (3) multi-expert collaborative reflection for fine-grained critique and self-correction.

Fig. 3: Statistics of MTEditBench. The figure displays the distribution of editing operation types and sequence lengths.

Method	Turn-1				Turn-2				Turn-3				Turn-4				Turn-5				Average
Method	DINO $\uparrow$	CLIP-I $\uparrow$	CLIP-T $\uparrow$	AugCLIP $\uparrow$	DINO $\uparrow$	CLIP-I $\uparrow$	CLIP-T $\uparrow$	AugCLIP $\uparrow$	DINO $\uparrow$	CLIP-I $\uparrow$	CLIP-T $\uparrow$	AugCLIP $\uparrow$	DINO $\uparrow$	CLIP-I $\uparrow$	CLIP-T $\uparrow$	AugCLIP $\uparrow$	DINO $\uparrow$	CLIP-I $\uparrow$	CLIP-T $\uparrow$	AugCLIP $\uparrow$	DINO $\uparrow$	CLIP-I $\uparrow$	CLIP-T $\uparrow$	AugCLIP $\uparrow$
ACE++ [ 22 ]	0.529	0.806	0.153	0.287	0.526	0.803	0.151	0.260	0.517	0.793	0.148	0.236	0.501	0.777	0.142	0.211	0.484	0.761	0.136	0.188	0.511	0.788	0.146	0.236
HQEdit [ 36 ]	0.677	0.813	0.236	0.276	0.674	0.802	0.235	0.248	0.669	0.809	0.238	0.224	0.662	0.799	0.235	0.200	0.650	0.793	0.234	0.174	0.666	0.803	0.236	0.224
UltraEdit [ 18 ]	0.762	0.869	0.251	0.293	0.758	0.865	0.247	0.269	0.746	0.855	0.243	0.245	0.722	0.837	0.233	0.222	0.698	0.820	0.223	0.199	0.737	0.849	0.239	0.246
ICEdit [ 3 ]	0.763	0.847	0.250	0.302	0.759	0.843	0.246	0.280	0.747	0.833	0.243	0.259	0.723	0.816	0.233	0.237	0.699	0.799	0.223	0.215	0.738	0.828	0.239	0.259
VAREdit [ 19 ]	0.766	0.863	0.249	0.298	0.762	0.859	0.244	0.274	0.750	0.849	0.242	0.250	0.726	0.831	0.235	0.228	0.698	0.814	0.228	0.206	0.740	0.843	0.240	0.251

TABLE I: Performance comparison on the MTEditBench dataset. Bold numbers indicate the best results among the evaluated open-source baselines and IMAGAgent, and underlined numbers indicate the second-best results. Entries in gray denote proprietary models.

Method	Turn-1			Turn-2			Turn-3			Average
Method	DINO $\uparrow$	CLIP-I $\uparrow$	CLIP-T $\uparrow$	DINO $\uparrow$	CLIP-I $\uparrow$	CLIP-T $\uparrow$	DINO $\uparrow$	CLIP-I $\uparrow$	CLIP-T $\uparrow$	DINO $\uparrow$	CLIP-I $\uparrow$	CLIP-T $\uparrow$
ACE++ [ 22 ]	0.522	0.791	0.176	0.491	0.770	0.169	0.475	0.748	0.163	0.496	0.770	0.169
HQEdit [ 36 ]	0.522	0.696	0.259	0.441	0.659	0.248	0.397	0.637	0.238	0.453	0.664	0.248
UltraEdit [ 18 ]	0.755	0.852	0.289	0.706	0.827	0.278	0.683	0.810	0.266	0.715	0.830	0.278
ICEdit [ 3 ]	0.757	0.833	0.281	0.708	0.806	0.277	0.681	0.786	0.265	0.715	0.808	0.274
VAREdit [ 19 ]	0.757	0.847	0.276	0.712	0.825	0.264	0.689	0.806	0.258	0.719	0.826	0.266

TABLE II: Performance comparison on the MagicBrush benchmark. Bold numbers indicate the best results among the evaluated open-source baselines and IMAGAgent, and underlined numbers indicate the second-best results. Entries in gray denote proprietary models.

Fig. 4: Qualitative results on MTEditBench. Comparisons show IMAGAgent achieves top visual quality.

Fig. 5: Qualitative results on MagicBrush. IMAGAgent outperforms SOTA in complex spatial and attribute constraints.

Method	Turn-3			Turn-4			Turn-5
Method	DINO $\uparrow$	CLIP-I $\uparrow$	CLIP-T $\uparrow$	DINO $\uparrow$	CLIP-I $\uparrow$	CLIP-T $\uparrow$	DINO $\uparrow$	CLIP-I $\uparrow$	CLIP-T $\uparrow$
Doubao-Seedream-4.0 (Standalone)	0.653	0.742	0.156	0.611	0.685	0.121	0.548	0.596	0.107
Qwen-Image-Edit (Standalone)	0.668	0.731	0.149	0.598	0.694	0.128	0.561	0.583	0.099
GPT-4o (Standalone)	0.742	0.849	0.248	0.698	0.817	0.241	0.657	0.786	0.235
IMAGAgent (GPT-4o)	0.791	0.886	0.258	0.772	0.875	0.255	0.753	0.866	0.252

TABLE III: Backbone-controlled comparison on MTEditBench. Standalone editors directly process the original turn-level instruction sequence, using each preceding output as the input to the next turn. Best results are in bold , and second-best results are underlined .

Fig. 6: User study examined from three perspectives: instruction following, visual realism, and overall quality.

Variant	Components					Metrics
Variant	C-Plan	Tool	Hist.	Refl.	Multi.	DINO $\uparrow$	CLIP-I $\uparrow$	CLIP-T $\uparrow$
w/o C-Plan	✗	✓	✓	✓	✓	0.757	0.869	0.247
w/o Tool Orch.	✓	✗	✓	✓	✓	0.727	0.835	0.238
w/o History	✓	✓	✗	✓	✓	0.646	0.772	0.233
w/o Reflection	✓	✓	✓	✗	–	0.689	0.815	0.221
Single-Expert	✓	✓	✓	✓	✗	0.742	0.854	0.237

TABLE IV: Component ablation on MTEditBench. All variants follow the same evaluation protocol and retain the full configuration except for the indicated component. Best results are in bold , and second-best results are underlined .

Fig. 7: Turn-wise diagnostic ablation on MTEditBench. The plots show how representative mechanism-level variants degrade across successive editing turns.

Fig. 8: Feedback-guided refinement. (a) An example of targeted correction guided by aggregated expert feedback. (b) Distribution of direct acceptance, acceptance after refinement, and below-threshold fallback. Acceptance is determined by the consensus-score threshold.

Variant	Score	Text Critique	Multi-Expert	Retry	Avg. Calls $\downarrow$	DINO $\uparrow$	CLIP-I $\uparrow$	CLIP-T $\uparrow$
No Feedback	✗	✗	✗	✗	1.00	0.689	0.815	0.221
Score-Only Retry	✓	✗	✗	✓	1.75	0.705	0.828	0.226
Best-of-3 Retry	✓	✗	✗	✓	3.00	0.720	0.838	0.231
Single-Expert	✓	✓	✗	✓	1.42	0.742	0.854	0.237
Multi w/o Aggregation	✓	✓	✓	✓	1.48	0.750	0.862	0.241
IMAGAgent	✓	✓	✓	✓	1.40	0.766	0.875	0.248

TABLE V: Comparison of different feedback policies under the same tool pool. All variants share the same planning module, tool set, editing-backbone pool, and maximum retry budget, while differing only in how feedback is generated and utilized. This comparison assesses the contribution of structured reflective feedback under a shared maximum editing-attempt budget. Best results are in bold and the second-best are underlined ; the entire IMAGAgent row is boldfaced for emphasis.

Fig. 9: Eight-turn editing with full-instruction planning. Given the complete composite instruction, the planner consolidates redundant and superseded operations and reorders the retained sub-tasks according to their semantic dependencies. Each turn corresponds to one planned sub-task. Left: the editing trajectory from the source image to Turn-8. Right: the execution order produced by the constraint-aware planner.

Notation	Definition
$\mathcal{I}_{0}$	Initial source image
$\mathcal{T}_{0}$	Editing instruction provided to the planner
$\mathcal{I}^{*}$	Final edited image
$\mathcal{C}$	Historical context containing plans, images, and feedback
$\Phi$	Overall mapping function of IMAGAgent
$\mathcal{U}$	Original ordered instruction sequence $(u_{1},\ldots,u_{L})$

TABLE VI: Summary of Notation.

Fig. 10: Additional qualitative comparison on MTEditBench. We provide an additional five-turn example as a supplement to the qualitative results in the main paper. Compared with representative baselines, IMAGAgent better preserves cross-turn consistency, instruction fidelity, and overall structural stability throughout the editing process.

Explore similar work

Mar 9, 2026

ImageEdit-R1: Boosting Multi-Agent Image Editing via Reinforcement Learning

With the rapid advancement of commercial multi-modal models, image editing has garnered significant attention due to its widespread applicability in daily life. Despite impressive progress, existing image editing systems, particularly closed-source or proprietary models, often struggle with complex, indirect, or multi-step user instructions. These limitations hinder their ability to perform nuanced, context-aware edits that align with human intent. In this work, we propose ImageEdit-R1, a multi-agent framework for intelligent image editing that leverages reinforcement learning to coordinate high-level decision-making across a set of specialized, pretrained vision-language and generative agents. Each agent is responsible for distinct capabilities--such as understanding user intent, identifying regions of interest, selecting appropriate editing actions, and synthesizing visual content--while reinforcement learning governs their collaboration to ensure coherent and goal-directed behavior. Unlike existing approaches that rely on monolithic models or hand-crafted pipelines, our method treats image editing as a sequential decision-making problem, enabling dynamic and context-aware editing strategies. Experimental results demonstrate that ImageEdit-R1 consistently outperforms both individual closed-source diffusion models and alternative multi-agent framework baselines across multiple image editing datasets.

Yiran Zhao, Yaoqi Ye, Xiang Liu +2

National University of Singapore · Adobe Research

Image Editing RL for Image Editing Multimodal Agents Sequential MARL

Ask PDF

Jul 1, 2026

GMO-E $^2$ DIT: Grounded Multi-Operation Editing for E-Commerce Images

Real-world e-commerce image editing often requires multiple, localized, and auditable operations rather than global restyling. This compositional nature poses a dual challenge: models must precisely apply all requested edits to the correct regions while preserving unmodified content, even under ambiguous instructions. Existing one-shot editors conflate intent resolution, spatial grounding, and synthesis into a single step, frequently resulting in partial execution failures, which is unacceptable for commercial scenarios. To address this, we introduce GMO-E

^2

DIT, an agentic editing framework that couples a Vision-Language Model (VLM) with a mask-conditioned image editor to tackle structured multi-turn task completion. Given an underspecified instruction, the VLM agent constructs a region-grounded edit agenda, effectively decoupling cognitive reasoning from generative rendering. The framework then executes sub-programs via operation-aware masks and references, utilizing a reflection-driven loop to inspect intermediate results and determine the subsequent state. This iterative mechanism reliably preserves safe partial progress, retries unfinished operations, and recovers from errors. Furthermore, we develop a unified data pipeline providing aligned supervision for planning, execution, and reflection, alongside EComEditBench, a comprehensive benchmark for instruction-driven evaluation. Extensive experiments demonstrate that GMO-E

^2

DIT achieves competitive performance compared to strong closed-source models, yielding superior instruction accuracy and edit fidelity over existing baselines.

Zipeng Guo, Xiaoan Liu, Lichen Ma +9

Wuhan University · Xi’an Jiaotong University · Sun Yat-sen University

Image Editing Evaluation Text-Guided Image Editing Tool-Using Vision-Language Agents Image Editing

Ask PDF

Jun 4, 2026

Edit-R2: Context-Aware Reinforcement Learning for Multi-Turn Image Editing

Text-guided image editing has advanced rapidly with diffusion models and unified multimodal foundation models. However, most existing methods remain confined to single-turn settings, overlooking the more realistic scenario of multi-turn in-context editing, where users iteratively refine an image through a sequence of instructions. In this setting, a model must follow each new instruction while preserving accumulated session-level constraints, challenged by two coupled failure modes: long-context dilution, where sparse textual constraints become difficult to recover from growing interleaved image-text histories, and state contamination, where earlier editing mistakes degrade subsequent generations. We introduce Edit-R2, a novel reinforcement learning post-training framework for unified multimodal models. Edit-R2 reconstructs the operative session intent, which effectively consolidates scattered historical constraints into an explicit reasoning trace before each editing turn. It further enables multi-turn RL over both reasoning and generation through a unified objective that jointly optimizes intent reconstruction generation in discrete text space and flow-matching image generation in continuous latent space, while a trajectory filtering mechanism suppresses corrupted rollouts to stabilize training under state contamination. To support systematic evaluation, we introduce MICE-Bench, a large-scale benchmark for multi-turn in-context editing with automated metrics for instruction following (IF), content consistency (CC), and global awareness (GA) over accumulated session constraints. Experiments show that Edit-R2 substantially improves multi-turn in-context editing and achieves competitive performance compared against strong baselines.

Yuxiao Ye, Haoran He, Fangyuan Kong +4

Hong Kong University of Science and Technology · Kuaishou Technology

Reinforcement Learning Image Editing Evaluation Unified Multimodal Models Diffusion-Based Image Editing Image Editing RL for Image Editing

Ask PDF

Abstract

Explore similar work

ImageEdit-R1: Boosting Multi-Agent Image Editing via Reinforcement Learning

GMO-E2^22DIT: Grounded Multi-Operation Editing for E-Commerce Images

Edit-R2: Context-Aware Reinforcement Learning for Multi-Turn Image Editing

GMO-E $^2$ DIT: Grounded Multi-Operation Editing for E-Commerce Images