Rethinking Structure Preservation in Text-Guided Image Editing with Visual Autoregressive Models

Tao Xia; Jiawei Liu; Yukun Zhang; Ting Liu; Wei Wang; Lei Zhang

cs.CVMar 30, 2026

Rethinking Structure Preservation in Text-Guided Image Editing with Visual Autoregressive Models

Authors: Tao Xia, Jiawei Liu, Yukun Zhang, Ting Liu, Wei Wang, Lei Zhang

Organizations: Beijing Institute of Technology, Beijing, China · Shenyang Institute of Automation, CAS, Shenyang, China · Meitu Inc, MTLab, Beijing, China

Abstract

Visual autoregressive (VAR) models have recently emerged as a promising family of generative models, enabling a wide range of downstream vision tasks such as text-guided image editing. By shifting the editing paradigm from noise manipulation in diffusion-based methods to token-level operations, VAR-based approaches achieve better background preservation and significantly faster inference. However, existing VAR-based editing methods still face two key challenges: accurately localizing editable tokens and maintaining structural consistency in the edited results. In this work, we propose a novel text-guided image editing framework rooted in an analysis of intermediate feature distributions within VAR models. First, we introduce a coarse-to-fine token localization strategy that can refine editable regions, balancing editing fidelity and background preservation. Second, we analyze the intermediate representations of VAR models and identify structure-related features, by which we design a simple yet effective feature injection mechanism to enhance structural consistency between the edited and source images. Third, we develop a reinforcement learning-based adaptive feature injection scheme that automatically learns scale- and layer-specific injection ratios to jointly optimize editing fidelity and structure preservation. Extensive experiments demonstrate that our method achieves superior structural consistency and editing quality compared with state-of-the-art approaches, across both local and global editing scenarios.

Figures & tables

Figure 2: Method overview. Given a source image $I_{\text{src}}$ and its corresponding text description $T_{\text{src}}$ , our framework generates the edited image $I_{\text{edit}}$ according to the target text $T_{\text{tgt}}$ . The tokenizer $E$ encodes the input $I_{\text{src}}$ into multi-scale residuals $\{R_{1},\ldots,R_{K}\}$ . For clarity, we illustrate the inference pipeline at the $k$ -th scale. The framework adopts a dual-branch architecture, where the source branch takes $\tilde{F}_{k-1}^{\text{src}}$ and the target branch takes $\tilde{F}_{k-1}^{\text{tgt}}$ , producing probability distributions $P_{k}^{\text{src}}$ and $P_{k}^{\text{tgt}}$ . The intermediate feature maps—illustrated in (b)—are selectively injected from the source branch into the target branch ( Sec. 3.3 ) with learnable weights ( Sec. 3.4 ). The predicted distributions $P_{k}^{\text{tgt}}$ are fed into the CFTL module ( Sec. 3.2 ) to obtain a refined editing mask $\hat{M}_{k}$ , which is then used by token reassembly (TR) to produce the prediction $R_{k}^{\text{tgt}}$ at the $k$ -th scale. Finally, multi-scale predictions $\{R_{1},\ldots,R_{\gamma},R_{\gamma+1}^{\text{tgt}},\ldots,R_{K}^{\text{tgt}}\}$ are jointly decoded to produce the final edited image $I_{\text{edit}}$ . Here, $\gamma$ denotes the number of source scales reused in the target branch.

Figure 3: Effect of CFG on background preservation and editing fidelity. A higher CFG yields better editing fidelity but weaker background preservation, whereas a lower CFG produces the opposite effect. Our hybrid method achieves a good balance. The number in each grid indicates the amount of tokens that need to be replaced at each position.

Figure 4: VAR features and attention maps. PCA visualization of intermediate representations and attention maps in a VAR block. Cross-attention (CA) offers rough object localization akin to diffusion models, but VAR self-attention (SA) does not display the spatial affinity structure typically observed in diffusion-based methods.

Figure 5: Dependency analysis for feature injection. Top: Experimental results of injecting features at different scales and layers, where the numbers denote layer and scale indices. Injecting features at the 0-th layer and at scales 5–8 leads to markedly better structural preservation. Bottom: Feature injection ratios after genetic algorithm optimization.

Figure 6: Qualitative comparison with baselines. Visual results of text-guided image editing from P2P [ 11 ] , MasaCtrl [ 3 ] , RFInversion [ 35 ] , AREdit [ 49 ] and SAVAREdit (Ours). The original images and corresponding source/target prompts are provided. Our method achieves superior detail preservation in non-edited regions for local edits, and better structural consistency in global editing scenarios.

Method	Base Model	Structure	Background Preservation			CLIP Similarity $\uparrow$
		Distance $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	Whole	Edited
P2P [ 11 ]	diffusion	0.0694	17.87	0.7114	0.2088	25.01	22.44
Pix2Pix-Zero [ 30 ]	diffusion	0.0617	20.44	0.7467	0.1722	22.80	20.54
MasaCtrl [ 3 ]	diffusion	0.0284	22.17	0.7967	0.1066	23.96	21.16
PnP [ 48 ]	diffusion	0.0282	22.28	0.7905	0.1134	25.41	22.55
PnP-DirInv. [ 20 ]	diffusion	0.0243	22.46	0.7968	0.1061	25.41	22.62

Table 1: Quantitative comparison with baselines. Our method achieves comparable CLIP similarity to other state-of-the-art methods while demonstrating superior structure preservation.

Figure 7: Ablation study on CFTL. CFTL enables more precise localization of editable regions, thereby achieving better background preservation while maintaining high editing fidelity.

Figure 8: Ablation study on FI. FI exhibits strong capability in preserving image structures even without reusing any tokens ( $\gamma=0$ ). Moreover, compared with the token-reuse strategy ( $\gamma>0$ ), FI provides greater flexibility and achieves a better balance between structural consistency and editing fidelity.

Figure 9: Ablation study on AFI. AFI achieves a better balance between structural consistency and editing fidelity.

Text-Guided Image Editing

Text-guided image editing changes selected image content from natural-language instructions while aiming to preserve everything else.

See all 158 papers

Edit-VAR: Taming Visual Autoregressive Model for Precise Video Editing
Sep 18, 2026Chongbo Zhao, Jiangming Wang, Xilai Wang +5Video Editing Instruction-Based Video Editing
Sun Yat-sen University · South China University of Technology · Tsinghua University +3
Model the Edit, Not the Image: Visual Autoregressive Editing from a Source-Centric Perspective
Aug 10, 2026Hongyi Fang, Chuwen Xie, Benjia Zhou +6Text-Guided Image Editing Visual Autoregressive Models
Beijing Institute of Technology, Zhuhai · Zhejiang University · Tencent +1
Prompt-Guided Image Editing with Masked Logit Nudging in Visual Autoregressive Models
Apr 16, 2026Amir El-Ghoussani, Marc Hölle, Gustavo Carneiro +1Text-Guided Image Editing Visual Autoregressive Models
Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany · University of Surrey, United Kingdom

Sep 18, 2026J/K move · Enter open · S save

Edit-VAR: Taming Visual Autoregressive Model for Precise Video Editing

Chongbo Zhao, Jiangming Wang, Xilai Wang, Xinyu Wang+4

Sun Yat-sen University · South China University of Technology · Tsinghua University+3

Video Editing Instruction-Based Video Editing Visual Autoregressive Models

Open paper AskPDF arXiv

Text-guided video editing modifies target content while preserving the appearance and temporal coherence of unedited regions. Training-based approaches provide strong control but demand substantial data and computation. Training-free methods fall into inversion-free and inversion-based paradigms. Inversion-free approaches avoid trajectory recovery, but their source-preserving guidance can limit editing strength and leave semantic changes incomplete. Inversion-based approaches recover a latent trajectory before regeneration, where approximation errors can accumulate and cause source-content drift and temporal inconsistency. We introduce Edit-VAR, the first training-free and inversion-free framework for text-guided video editing with a pretrained visual autoregressive video model. Edit-VAR directly encodes the source video into multi-scale discrete tokens and performs probability-guided conditional token replacement for source preservation. Attention-guided token-wise and scale-aware modulation selectively relaxes source constraints over edit-relevant positions and generation stages. Scale-Decoupled Generation, implemented as late-scale constraint release, regenerates motion-consistent details and reduces texture fragmentation. Residual-guided token pruning further exploits redundancy at the final two high-resolution scales to reduce inference cost. Extensive experiments and a blind user study demonstrate that Edit-VAR outperforms existing training-free video editing methods overall in editing fidelity, source preservation, temporal coherence, and inference efficiency.

Abstract

Text-Guided Image Editing

Explore similar work

Edit-VAR: Taming Visual Autoregressive Model for Precise Video Editing

Model the Edit, Not the Image: Visual Autoregressive Editing from a Source-Centric Perspective

Prompt-Guided Image Editing with Masked Logit Nudging in Visual Autoregressive Models