A$^2$-Edit: Precise Reference-Guided Image Editing of Arbitrary Objects and Ambiguous Masks

Huayu Zheng; Guangzhao Li; Baixuan Zhao; Siqi Luo; Hantao Jiang; Guangtao Zhai; Xiaohong Liu

cs.CVMar 11, 2026

A $^2$ -Edit: Precise Reference-Guided Image Editing of Arbitrary Objects and Ambiguous Masks

Authors: Huayu Zheng, Guangzhao Li, Baixuan Zhao, Siqi Luo, Hantao Jiang, Guangtao Zhai, Xiaohong Liu

Organizations: Shanghai Jiao Tong University · Shanghai Innovation Institute

Abstract

We propose A^2-Edit, a unified inpainting framework for arbitrary object categories, which allows users to replace any target region with a reference object using only a coarse mask. To address the issues of severe homogenization and limited category coverage in existing datasets, we construct a large-scale multi-category dataset, UniEdit-500K, which includes 8 major categories, 209 fine-grained subcategories, and a total of 500,104 image pairs. Such rich category diversity poses new challenges for the model, requiring it to automatically learn semantic relationships and distinctions across categories. To this end, we introduce the Mixture of Transformer module, which performs differentiated modeling of various object categories through dynamic expert selection, and further enhances cross-category semantic transfer and generalization through collaboration among experts. In addition, we propose a Mask Annealing Training Strategy (MATS) that progressively relaxes mask precision during training, reducing the model's reliance on accurate masks and improving robustness across diverse editing tasks. Extensive experiments on benchmarks such as VITON-HD and AnyInsertion demonstrate that A^2-Edit consistently outperforms existing approaches across all metrics, providing a new and efficient solution for arbitrary object editing.

Figures & tables

Figure 1: Visual Results of A 2 -Edit : A Unified Framework for Arbitrary-Object Inpainting. Our method supports a wide spectrum of real-world scenarios, delivering high-quality results across diverse object categories. The examples illustrate the generality and robustness of our method in handling various image editing tasks. More results are provided in the supplementary material.

Figure 2: Overview of the A 2 -Edit framework. Our architecture takes a reference image, target image, and user mask as core inputs, encodes them into a unified feature space, and feeds the fused features into Mixture-of-Transformers ( MoT ) module. The MoT Block ( MoTB ) denotes an expert-specific Transformer block embedded within the attention and feed-forward layers. The model then dynamically routes features to specialized experts for differentiated modeling. Model is trained via Mask Annealing Training Strategy ( MATS ), a three-stage training process. Final outputs are decoded by a VAE decoder for high-fidelity results.

Figure 3: An overview of our UniEdit-500K dataset. The central pie chart illustrates the proportional distribution across the eight major categories. The word cloud below visualizes the diversity of object classes within the dataset. Surrounding the center are representative examples from each category (Portraits, Accessories, Plants, Vehicles, Animals, Garments, Furniture, and Architecture), showcasing the data format which includes a reference image, a source image, and their corresponding segmentation masks.

Figure 4: Qualitative comparison with existing mask-guided image editing methods. Our method consistently produces more coherent structures, sharper details, and better semantic alignment with the target edit compared to existing methods( MimicBrush Chen et al. (2024a) , FLUX.1-Kontext Labs (2025b) , ACE++ Mao et al. (2025) and InsertAnything Song et al. (2025) ).

Method	VTON-HD						AnyInsertion
	Fine Mask			Rough Mask			Fine Mask			Rough Mask
	DINO-I	CLIP-I	LPIPS	DINO-I	CLIP-I	LPIPS	DINO-I	CLIP-I	LPIPS	DINO-I	CLIP-I	LPIPS
FLUX.1-Fill	58.59	72.59	0.1241	46.18	69.04	0.1509	50.37	66.06	0.1962	43.57	62.89	0.2673
AnyDoor	59.18	72.54	0.1270	45.99	69.38	0.1587	48.15	65.01	0.2070	42.47	60.67	0.2854
MimicBrush	58.99	73.01	0.1022	49.13	68.54	0.1479	50.29	68.88	0.1878	47.89	65.13	0.2011
FLUX.1-Kontext	60.09	79.44	0.0800	56.11	77.91	0.0972	59.49	74.70	0.1094	54.15	74.05	0.1384

Table 1: Qualitative comparison with existing image editing methods. Evaluations conducted on VTON-HD and AnyInsertion datasets show that our A 2 -Edit model significantly outperforms existing methods. The best and second-best results are demonstrated in bold and underlined , respectively. Note that higher DINO-I and CLIP-I scores indicate better performance, while lower LPIPS values are preferred.

Method	UniEdit
	Fine Mask					Rough Mask
	DINO-I $\uparrow$	CLIP-I $\uparrow$	LPIPS $\downarrow$	FID $\downarrow$	VLM $\uparrow$	DINO-I $\uparrow$	CLIP-I $\uparrow$	LPIPS $\downarrow$	FID $\downarrow$	VLM $\uparrow$
FLUX.1-Fill	45.61	63.69	0.2519	93.70	42.7	40.09	61.58	0.3739	128.91	26.3
AnyDoor	46.76	64.23	0.2534	92.19	41.0	40.42	61.39	0.3623	120.80	28.3
MimicBrush	50.02	68.56	0.2133	86.23	47.7	45.07	64.36	0.2871	100.54	36.3
FLUX.1-Kontext	53.47	73.99	0.1403	70.54	52.3	48.29	70.56	0.2091	80.94	54.0

Table 2: Quantitative comparison on the UniEdit test set across diverse object categories.

Figure 5: Statistical results of user study.

Figure 6: A 2 -Edit Ablation Study. Removing the framework, training data, or MATS reduces generation quality and cross-category generalization, while progressively adding each component significantly improves detail fidelity and the model’s ability to handle rough masks.

Method	Fine Mask					Rough Mask
Method	DINO-I $\uparrow$	CLIP-I $\uparrow$	LPIPS $\downarrow$	FID $\downarrow$	VLM $\uparrow$	DINO-I $\uparrow$	CLIP-I $\uparrow$	LPIPS $\downarrow$	FID $\downarrow$	VLM $\uparrow$
w/o. AE	56.32	72.45	0.1423	75.37	64.3	56.18	73.21	0.1364	72.63	65.7
w. MoE	56.69	74.01	0.1211	71.90	70.3	56.11	73.69	0.1366	73.04	70.3
w/o. UE	58.64	74.32	0.1182	70.04	68.7	56.27	73.01	0.1347	72.84	66.3
w. FT	60.27	75.83	0.0981	67.55	71.0	55.65	72.92	0.1516	74.77	64.7
w. FT + RT	61.44	76.43	0.0941	63.72	74.7	59.70	74.86	0.1073	60.33	73.0

Table 3: Ablation Result. We evaluate model variants on UniEdit. w/o. AE: without Arbitrary Editing framework; w. MoE: with MoE on FFN only; w/o. UE: without UniEdit-500K; w. FT: training with fine masks; w. FT + RT: training with fine masks and augmented rough mask.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Results on the virtual try-on task. The first row shows the source images, the first column shows the reference images, and the remaining images are generated by our model.

Figure 8: Results on portraits and animals. The first three rows show human edits—including face swapping, portrait replacement, and hairstyle or color changes—where the model preserves stable identity features. The fourth row demonstrates similarly consistent results in animal face-swapping scenarios.

Figure 9: Pet Results. Given different masks under a shared background, the model generates diverse poses of the same reference animal while accurately controlling position and shape.

Figure 10: More results on inpainting tasks across different object categories.

Figure 11: Multi-step editing results. Each row represents one editing sequence, where the images in the middle correspond to the reference inputs provided at each editing step.

Figure 12: Qualitative comparison with text-guided image editing models.

Figure 13: VLM-based evaluation metric test example.

Figure 14: Garment/Accessory category hold-out experiment results.

Figure 15: Portrait category hold-out experiment results.

Image Editing

Image editing changes selected visual content while preserving the parts of an image that should remain unchanged.

See all 315 papers

BindEdit: Taming Attention Leakage for Precise Multi-Object Image Editing
Jun 17, 2026Chaewon Park, Soyoon Lee, Naeun Lee +3Text-Guided Image Editing Attention Control in Diffusion Models
1Sookmyung Women’s University · 2Yonsei University · 3Samsung Research
Masked Generative Transformer Is What You Need for Image Editing
May 11, 2026Wei Chow, Linfeng Li, Xian Sun +14Transformer Text-Guided Image Editing
1ByteDance · 2National University of Singapore · 3Duke University +2
Refinement Is Inherently Editable: Training-Free Prompt-to-Prompt Image Editing with Generative Refinement Network
Sep 17, 2026Yulong Chen, Ziqian Zhang, Haoyu Zhang +4Text-Guided Image Editing Training-Free Image Editing
City University of Hong Kong (Dongguan), Guangdong, China · City University of Hong Kong, Hong Kong, China · Mohamed bin Zayed University of Artificial Intelligence, Masdar, Abu Dhabi

Jun 17, 2026J/K move · Enter open · S save

BindEdit: Taming Attention Leakage for Precise Multi-Object Image Editing

Chaewon Park, Soyoon Lee, Naeun Lee, Minjung Shin+2

1Sookmyung Women’s University · 2Yonsei University · 3Samsung Research

Text-Guided Image Editing Attention Control in Diffusion Models Diffusion-Based Image Editing Image Editing Diffusion Models

Open paper AskPDF arXiv

Real image editing enables precise manipulation of visual content, yet existing methods often fail in complex multi-object scenarios, causing semantic blending, object duplication, or incomplete edits. We attribute these failures to attention leakage, where signals across spatial regions and text tokens become entangled during the denoising process. Specifically, we identify two distinct forms of leakage: Edit-Token Leakage, where ambiguous token-region alignment leads to object blending, and Source Dominance Leakage, where tokens of unchanged source objects overwhelm the attention intended for target entities. To resolve these leakages, we propose BindEdit, which enforces attention-level constraints within a single diffusion trajectory. To suppress Edit-Token Leakage, BindEdit jointly regularizes cross- and self-attention so that each target token group is bound to its corresponding spatial region while maintaining instance-level separation. To suppress Source Dominance Leakage, a cross-attention re-balancing mechanism amplifies target token influence and attenuates residual source semantics within editable regions. Moreover, a region fidelity term ensures that each target concept is expressed coherently across the entire editing mask. Additionally, we propose a comprehensive multi-object benchmark encompassing diverse object counts and categories. Extensive experiments demonstrate that BindEdit consistently outperforms existing methods within a single diffusion trajectory, maintaining robust performance across both single- and multi-object editing scenarios.

Abstract

Image Editing

Explore similar work

BindEdit: Taming Attention Leakage for Precise Multi-Object Image Editing

Masked Generative Transformer Is What You Need for Image Editing

Refinement Is Inherently Editable: Training-Free Prompt-to-Prompt Image Editing with Generative Refinement Network