IMAGAgent: Orchestrating Multi-Turn Image Editing via Constraint-Aware Planning and Reflection

Fei Shen; Chengyu Xie; Lihong Wang; Zhanyi Zhang; Xin Jiang; Xiaoyu Du; Jinhui Tang; Tat-Seng Chua

cs.GRFeb 12, 2026

IMAGAgent: Orchestrating Multi-Turn Image Editing via Constraint-Aware Planning and Reflection

Authors: Fei Shen, Chengyu Xie, Lihong Wang, Zhanyi Zhang, Xin Jiang, Xiaoyu Du, Jinhui Tang, Tat-Seng Chua

Organizations: National University of Singapore · Nanjing University of Science and Technology · Nanjing Forestry University

Abstract

Existing multi-turn image editing paradigms are often confined to isolated single-step execution. Due to a lack of context-awareness and closed-loop feedback mechanisms, they are prone to error accumulation and semantic drift during multi-turn interactions, ultimately resulting in severe structural distortion of the generated images. To address this, we propose IMAGAgent, a multi-turn image editing agent framework based on a ``plan--execute--reflect'' closed-loop mechanism that achieves deep synergy among instruction parsing, tool scheduling, and adaptive correction within a unified pipeline. Specifically, we first present a constraint-aware planning module that leverages a vision-language model (VLM) to precisely decompose complex natural language instructions into a series of executable sub-tasks, governed by target singularity, semantic atomicity, and visual perceptibility. Then, the tool-chain orchestration module dynamically constructs execution paths based on the current image, the current sub-task, and the historical context, enabling adaptive scheduling and collaborative operation among heterogeneous operation models covering image retrieval, segmentation, detection, and editing. Finally, we devise a multi-expert collaborative reflection mechanism in which a central large language model (LLM) synthesizes evaluations from multiple vision-language models into holistic feedback, triggering fine-grained self-correction and recording feedback outcomes to optimize future decisions. Extensive experiments on our constructed MTEditBench and the MagicBrush dataset demonstrate that IMAGAgent achieves performance significantly superior to existing methods in terms of instruction consistency, editing precision, and overall quality. The code is available at https://github.com/hackermmzz/IMAGAgent.

Figures & tables

Fig. 1: (a) Existing methods suffer from error accumulation due to isolated execution. (b) IMAGAgent ensures structural and intentional consistency through closed-loop feedback.

Fig. 2: IMAGAgent Framework. We reformulate multi-turn image editing into a closed-loop process with three modules: (1) constraint-aware planning for VLM-based instruction decomposition; (2) tool-chain orchestration for dynamic scheduling of vision tools; and (3) multi-expert collaborative reflection for fine-grained critique and self-correction.

Fig. 3: Statistics of MTEditBench. The figure displays the distribution of editing operation types and sequence lengths.

Method	Turn-1				Turn-2				Turn-3				Turn-4				Turn-5				Average
Method	DINO $\uparrow$	CLIP-I $\uparrow$	CLIP-T $\uparrow$	AugCLIP $\uparrow$	DINO $\uparrow$	CLIP-I $\uparrow$	CLIP-T $\uparrow$	AugCLIP $\uparrow$	DINO $\uparrow$	CLIP-I $\uparrow$	CLIP-T $\uparrow$	AugCLIP $\uparrow$	DINO $\uparrow$	CLIP-I $\uparrow$	CLIP-T $\uparrow$	AugCLIP $\uparrow$	DINO $\uparrow$	CLIP-I $\uparrow$	CLIP-T $\uparrow$	AugCLIP $\uparrow$	DINO $\uparrow$	CLIP-I $\uparrow$	CLIP-T $\uparrow$	AugCLIP $\uparrow$
ACE++ [ 22 ]	0.529	0.806	0.153	0.287	0.526	0.803	0.151	0.260	0.517	0.793	0.148	0.236	0.501	0.777	0.142	0.211	0.484	0.761	0.136	0.188	0.511	0.788	0.146	0.236
HQEdit [ 36 ]	0.677	0.813	0.236	0.276	0.674	0.802	0.235	0.248	0.669	0.809	0.238	0.224	0.662	0.799	0.235	0.200	0.650	0.793	0.234	0.174	0.666	0.803	0.236	0.224
UltraEdit [ 18 ]	0.762	0.869	0.251	0.293	0.758	0.865	0.247	0.269	0.746	0.855	0.243	0.245	0.722	0.837	0.233	0.222	0.698	0.820	0.223	0.199	0.737	0.849	0.239	0.246
ICEdit [ 3 ]	0.763	0.847	0.250	0.302	0.759	0.843	0.246	0.280	0.747	0.833	0.243	0.259	0.723	0.816	0.233	0.237	0.699	0.799	0.223	0.215	0.738	0.828	0.239	0.259
VAREdit [ 19 ]	0.766	0.863	0.249	0.298	0.762	0.859	0.244	0.274	0.750	0.849	0.242	0.250	0.726	0.831	0.235	0.228	0.698	0.814	0.228	0.206	0.740	0.843	0.240	0.251

TABLE I: Performance comparison on the MTEditBench dataset. Bold numbers indicate the best results among the evaluated open-source baselines and IMAGAgent, and underlined numbers indicate the second-best results. Entries in gray denote proprietary models.

Method	Turn-1			Turn-2			Turn-3			Average
Method	DINO $\uparrow$	CLIP-I $\uparrow$	CLIP-T $\uparrow$	DINO $\uparrow$	CLIP-I $\uparrow$	CLIP-T $\uparrow$	DINO $\uparrow$	CLIP-I $\uparrow$	CLIP-T $\uparrow$	DINO $\uparrow$	CLIP-I $\uparrow$	CLIP-T $\uparrow$
ACE++ [ 22 ]	0.522	0.791	0.176	0.491	0.770	0.169	0.475	0.748	0.163	0.496	0.770	0.169
HQEdit [ 36 ]	0.522	0.696	0.259	0.441	0.659	0.248	0.397	0.637	0.238	0.453	0.664	0.248
UltraEdit [ 18 ]	0.755	0.852	0.289	0.706	0.827	0.278	0.683	0.810	0.266	0.715	0.830	0.278
ICEdit [ 3 ]	0.757	0.833	0.281	0.708	0.806	0.277	0.681	0.786	0.265	0.715	0.808	0.274
VAREdit [ 19 ]	0.757	0.847	0.276	0.712	0.825	0.264	0.689	0.806	0.258	0.719	0.826	0.266

TABLE II: Performance comparison on the MagicBrush benchmark. Bold numbers indicate the best results among the evaluated open-source baselines and IMAGAgent, and underlined numbers indicate the second-best results. Entries in gray denote proprietary models.

Fig. 4: Qualitative results on MTEditBench. Comparisons show IMAGAgent achieves top visual quality.

Fig. 5: Qualitative results on MagicBrush. IMAGAgent outperforms SOTA in complex spatial and attribute constraints.

Method	Turn-3			Turn-4			Turn-5
Method	DINO $\uparrow$	CLIP-I $\uparrow$	CLIP-T $\uparrow$	DINO $\uparrow$	CLIP-I $\uparrow$	CLIP-T $\uparrow$	DINO $\uparrow$	CLIP-I $\uparrow$	CLIP-T $\uparrow$
Doubao-Seedream-4.0 (Standalone)	0.653	0.742	0.156	0.611	0.685	0.121	0.548	0.596	0.107
Qwen-Image-Edit (Standalone)	0.668	0.731	0.149	0.598	0.694	0.128	0.561	0.583	0.099
GPT-4o (Standalone)	0.742	0.849	0.248	0.698	0.817	0.241	0.657	0.786	0.235
IMAGAgent (GPT-4o)	0.791	0.886	0.258	0.772	0.875	0.255	0.753	0.866	0.252

TABLE III: Backbone-controlled comparison on MTEditBench. Standalone editors directly process the original turn-level instruction sequence, using each preceding output as the input to the next turn. Best results are in bold , and second-best results are underlined .

Fig. 6: User study examined from three perspectives: instruction following, visual realism, and overall quality.

Variant	Components					Metrics
Variant	C-Plan	Tool	Hist.	Refl.	Multi.	DINO $\uparrow$	CLIP-I $\uparrow$	CLIP-T $\uparrow$
w/o C-Plan	✗	✓	✓	✓	✓	0.757	0.869	0.247
w/o Tool Orch.	✓	✗	✓	✓	✓	0.727	0.835	0.238
w/o History	✓	✓	✗	✓	✓	0.646	0.772	0.233
w/o Reflection	✓	✓	✓	✗	–	0.689	0.815	0.221
Single-Expert	✓	✓	✓	✓	✗	0.742	0.854	0.237

TABLE IV: Component ablation on MTEditBench. All variants follow the same evaluation protocol and retain the full configuration except for the indicated component. Best results are in bold , and second-best results are underlined .

Fig. 7: Turn-wise diagnostic ablation on MTEditBench. The plots show how representative mechanism-level variants degrade across successive editing turns.

Fig. 8: Feedback-guided refinement. (a) An example of targeted correction guided by aggregated expert feedback. (b) Distribution of direct acceptance, acceptance after refinement, and below-threshold fallback. Acceptance is determined by the consensus-score threshold.

Variant	Score	Text Critique	Multi-Expert	Retry	Avg. Calls $\downarrow$	DINO $\uparrow$	CLIP-I $\uparrow$	CLIP-T $\uparrow$
No Feedback	✗	✗	✗	✗	1.00	0.689	0.815	0.221
Score-Only Retry	✓	✗	✗	✓	1.75	0.705	0.828	0.226
Best-of-3 Retry	✓	✗	✗	✓	3.00	0.720	0.838	0.231
Single-Expert	✓	✓	✗	✓	1.42	0.742	0.854	0.237
Multi w/o Aggregation	✓	✓	✓	✓	1.48	0.750	0.862	0.241
IMAGAgent	✓	✓	✓	✓	1.40	0.766	0.875	0.248

TABLE V: Comparison of different feedback policies under the same tool pool. All variants share the same planning module, tool set, editing-backbone pool, and maximum retry budget, while differing only in how feedback is generated and utilized. This comparison assesses the contribution of structured reflective feedback under a shared maximum editing-attempt budget. Best results are in bold and the second-best are underlined ; the entire IMAGAgent row is boldfaced for emphasis.

Fig. 9: Eight-turn editing with full-instruction planning. Given the complete composite instruction, the planner consolidates redundant and superseded operations and reorders the retained sub-tasks according to their semantic dependencies. Each turn corresponds to one planned sub-task. Left: the editing trajectory from the source image to Turn-8. Right: the execution order produced by the constraint-aware planner.

Notation	Definition
$\mathcal{I}_{0}$	Initial source image
$\mathcal{T}_{0}$	Editing instruction provided to the planner
$\mathcal{I}^{*}$	Final edited image
$\mathcal{C}$	Historical context containing plans, images, and feedback
$\Phi$	Overall mapping function of IMAGAgent
$\mathcal{U}$	Original ordered instruction sequence $(u_{1},\ldots,u_{L})$

TABLE VI: Summary of Notation.

Fig. 10: Additional qualitative comparison on MTEditBench. We provide an additional five-turn example as a supplement to the qualitative results in the main paper. Compared with representative baselines, IMAGAgent better preserves cross-turn consistency, instruction fidelity, and overall structural stability throughout the editing process.

Explore similar work

ImageEdit-R1: Boosting Multi-Agent Image Editing via Reinforcement Learning
Mar 9, 2026Yiran Zhao, Yaoqi Ye, Xiang Liu +2Image Editing RL for Image Editing
National University of Singapore · Adobe Research
GMO-E $^2$ DIT: Grounded Multi-Operation Editing for E-Commerce Images
Jul 1, 2026Zipeng Guo, Xiaoan Liu, Lichen Ma +9Image Editing Evaluation Text-Guided Image Editing
Wuhan University · Xi’an Jiaotong University · Sun Yat-sen University
Edit-R2: Context-Aware Reinforcement Learning for Multi-Turn Image Editing
Jun 4, 2026Yuxiao Ye, Haoran He, Fangyuan Kong +4Reinforcement Learning Image Editing Evaluation
Hong Kong University of Science and Technology · Kuaishou Technology

Mar 9, 2026J/K move · Enter open · S save

ImageEdit-R1: Boosting Multi-Agent Image Editing via Reinforcement Learning

Yiran Zhao, Yaoqi Ye, Xiang Liu, Michael Qizhe Shieh+1

National University of Singapore · Adobe Research

Image Editing RL for Image Editing Multimodal Agents Sequential MARL

Open paper AskPDF arXiv

With the rapid advancement of commercial multi-modal models, image editing has garnered significant attention due to its widespread applicability in daily life. Despite impressive progress, existing image editing systems, particularly closed-source or proprietary models, often struggle with complex, indirect, or multi-step user instructions. These limitations hinder their ability to perform nuanced, context-aware edits that align with human intent. In this work, we propose ImageEdit-R1, a multi-agent framework for intelligent image editing that leverages reinforcement learning to coordinate high-level decision-making across a set of specialized, pretrained vision-language and generative agents. Each agent is responsible for distinct capabilities--such as understanding user intent, identifying regions of interest, selecting appropriate editing actions, and synthesizing visual content--while reinforcement learning governs their collaboration to ensure coherent and goal-directed behavior. Unlike existing approaches that rely on monolithic models or hand-crafted pipelines, our method treats image editing as a sequential decision-making problem, enabling dynamic and context-aware editing strategies. Experimental results demonstrate that ImageEdit-R1 consistently outperforms both individual closed-source diffusion models and alternative multi-agent framework baselines across multiple image editing datasets.

Abstract

Explore similar work

ImageEdit-R1: Boosting Multi-Agent Image Editing via Reinforcement Learning

GMO-E2^22DIT: Grounded Multi-Operation Editing for E-Commerce Images

Edit-R2: Context-Aware Reinforcement Learning for Multi-Turn Image Editing

GMO-E $^2$ DIT: Grounded Multi-Operation Editing for E-Commerce Images