On-Policy Self-Distillation for Multi-Turn Image Editing

Liangbing Zhao; Le Zhuo; Mohamed Elhoseiny

cs.CVSep 28, 2026

On-Policy Self-Distillation for Multi-Turn Image Editing

Authors: Liangbing Zhao, Le Zhuo, Mohamed Elhoseiny

Organizations: KAUST · Krea AI

Abstract

Instruction-based image editing has achieved strong performance in single-turn settings, yet practical editing is often iterative, with each instruction applied to the output of the previous turn. We find that existing editing models degrade rapidly under recursive editing and attribute this failure to a train-test mismatch in the conditioning distribution: models are trained on clean source images but must repeatedly condition on their own imperfect outputs at inference time. To address this, we propose MT-OPSD, an on-policy self-distillation framework that trains the model on self-generated conditioning states with editing supervision from a clean-conditioned teacher, without requiring multi-turn annotations. We further introduce LME-Bench, a benchmark of 100 ten-turn editing sessions for evaluating long-horizon robustness. Experiments across three editing backbones show that MT-OPSD substantially improves long-horizon editing success and reduces multi-turn collapse while largely preserving single-turn editing quality.

Figures & tables

Figure 1: Drift under identity editing. Repeatedly instructing a model to leave the image unchanged exposes model-induced errors that accumulate across turns, and similar degradation occurs across different editing models.

Figure 2: Overview of MT-OPSD. The student first constructs self-generated rollout states by repeatedly applying the identity instruction. For each rollout state $\tilde{I}^{(k)}$ , the identity branch uses the state itself as the target under $\mathcal{L}_{\mathrm{id}}$ , while the editing branch aligns the student conditioned on $\tilde{I}^{(k)}$ with a frozen teacher conditioned on the clean source $I^{(0)}$ via sparse query-based velocity matching $\mathcal{L}_{\mathrm{edit}}$ . A rollout curriculum progressively increases the rollout depth, and improved student checkpoints replace the teacher through gated promotion.

Figure 3: Edit-type distribution in LME-Bench.

Figure 4: Qualitative comparison on LME-Bench. Repeated editing leads to distinct failure patterns across backbones, while the training-free baselines either inherit these artifacts or weaken the requested edits. MT-OPSD remains stable through 10 turns and continues to follow the editing sequence across all three backbones.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Additional qualitative comparison on LME-Bench. Selected turns of one editing session across three backbones.

Figure 6: Additional qualitative comparison on LME-Bench. Selected turns of one editing session across three backbones.

Figure 7: Effect of the identity branch. Removing the identity branch preserves instruction following but leads to progressive identity and layout drift across turns. MT-OPSD better preserves untargeted content while following the same ten-turn editing sequences.

Explore similar work

Edit-R2: Context-Aware Reinforcement Learning for Multi-Turn Image Editing
Jun 4, 2026Yuxiao Ye, Haoran He, Fangyuan Kong +4Multi-Reference Image Generation Edit
Hong Kong University of Science and Technology · Kuaishou Technology
AnchorEdit: Maintaining Temporal Consistency in Multi-turn Image Editing via Causal Memory
Jun 10, 2026Hang Xu, Xiaoxiao Ma, Guohui Zhang +7Diffusion-Based Image Editing Image Editing
University of Science and Technology of China · JD Explore Academy
MT-EditFlow: Reinforcement Learning for Multi-Turn Image Editing with Flow Matching
Jun 1, 2026Jiahui Huang, Yasi Zhang, Tianyu Chen +6Image Editing Multi-Reference Image Generation
Apple · University of California, Los Angeles · University of Texas at Austin +1

Jun 4, 2026, cs.AIJ/K move · Enter open · S save

Edit-R2: Context-Aware Reinforcement Learning for Multi-Turn Image Editing

Yuxiao Ye, Haoran He, Fangyuan Kong, Xintao Wang+3

Hong Kong University of Science and Technology · Kuaishou Technology

Multi-Reference Image Generation Edit

Open paper AskPDF arXiv

Text-guided image editing has advanced rapidly with diffusion models and unified multimodal foundation models. However, most existing methods remain confined to single-turn settings, overlooking the more realistic scenario of multi-turn in-context editing, where users iteratively refine an image through a sequence of instructions. In this setting, a model must follow each new instruction while preserving accumulated session-level constraints, challenged by two coupled failure modes: long-context dilution, where sparse textual constraints become difficult to recover from growing interleaved image-text histories, and state contamination, where earlier editing mistakes degrade subsequent generations. We introduce Edit-R2, a novel reinforcement learning post-training framework for unified multimodal models. Edit-R2 reconstructs the operative session intent, which effectively consolidates scattered historical constraints into an explicit reasoning trace before each editing turn. It further enables multi-turn RL over both reasoning and generation through a unified objective that jointly optimizes intent reconstruction generation in discrete text space and flow-matching image generation in continuous latent space, while a trajectory filtering mechanism suppresses corrupted rollouts to stabilize training under state contamination. To support systematic evaluation, we introduce MICE-Bench, a large-scale benchmark for multi-turn in-context editing with automated metrics for instruction following (IF), content consistency (CC), and global awareness (GA) over accumulated session constraints. Experiments show that Edit-R2 substantially improves multi-turn in-context editing and achieves competitive performance compared against strong baselines.

Abstract

Explore similar work

Edit-R2: Context-Aware Reinforcement Learning for Multi-Turn Image Editing

AnchorEdit: Maintaining Temporal Consistency in Multi-turn Image Editing via Causal Memory

MT-EditFlow: Reinforcement Learning for Multi-Turn Image Editing with Flow Matching