Edit-VAR: Taming Visual Autoregressive Model for Precise Video Editing

Chongbo Zhao; Jiangming Wang; Xilai Wang; Xinyu Wang; Jingyi Tang; Chunjie Hao; Pengjie Song; Yue Ma

cs.CVSep 18, 2026

Edit-VAR: Taming Visual Autoregressive Model for Precise Video Editing

Authors: Chongbo Zhao, Jiangming Wang, Xilai Wang, Xinyu Wang, Jingyi Tang, Chunjie Hao, Pengjie Song, Yue Ma

Organizations: Sun Yat-sen University · South China University of Technology · Tsinghua University · Shandong University ·

Abstract

Text-guided video editing modifies target content while preserving the appearance and temporal coherence of unedited regions. Training-based approaches provide strong control but demand substantial data and computation. Training-free methods fall into inversion-free and inversion-based paradigms. Inversion-free approaches avoid trajectory recovery, but their source-preserving guidance can limit editing strength and leave semantic changes incomplete. Inversion-based approaches recover a latent trajectory before regeneration, where approximation errors can accumulate and cause source-content drift and temporal inconsistency. We introduce Edit-VAR, the first training-free and inversion-free framework for text-guided video editing with a pretrained visual autoregressive video model. Edit-VAR directly encodes the source video into multi-scale discrete tokens and performs probability-guided conditional token replacement for source preservation. Attention-guided token-wise and scale-aware modulation selectively relaxes source constraints over edit-relevant positions and generation stages. Scale-Decoupled Generation, implemented as late-scale constraint release, regenerates motion-consistent details and reduces texture fragmentation. Residual-guided token pruning further exploits redundancy at the final two high-resolution scales to reduce inference cost. Extensive experiments and a blind user study demonstrate that Edit-VAR outperforms existing training-free video editing methods overall in editing fidelity, source preservation, temporal coherence, and inference efficiency.

Figures & tables

Figure 2: Motivation. Comparison between continuous inversion-based editing and our discrete VAR formulation. Inversion and subsequent regeneration can introduce source-content drift, whereas direct VAR encoding provides multi-scale discrete source tokens for reconstruction and conditional replacement without iterative trajectory inversion.

Figure 3: Overview of our framework. In Phase 1, the source video is encoded and a forward pass under the source prompt caches per-token probabilities and cross-attention maps. In Phase 2, a forward pass under the edit prompt performs probability-guided token replacement steered by the attention-guided spatial modulation field, which assigns low $\gamma$ to edited regions and high $\gamma$ to preserved regions. Source caching stops at scale $S_{\mathrm{stop}}$ ; beyond that, the editing pass generates freely.

Figure 4: Spatial modulation visualization. Given a source frame and an edit-target word, the cross-attention map localizes the editing region, which is then mapped to a per-token gamma field.

Training-based reference
Method	Target Alignment		Video Quality		Non-edit Preservation			Time $\downarrow$
Method	CLIP-S ${}_{\mathrm{full}}\uparrow$	CLIP-S ${}_{\mathrm{edit}}\uparrow$	Smooth. $\uparrow$	Aesth. $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	Time $\downarrow$
VACE-V2V	0.957	0.970	0.936	0.605	13.51	0.380	0.618	9 min
Training-free, inversion-based
RAVE	0.962	0.975	0.973	0.552	18.81	0.674	0.437	17 min
FADE	0.966	0.972	0.958	0.533	14.75	0.626	0.211	14 min

Table 1: Main quantitative comparison with wall-clock inference time.

Figure 5: Qualitative comparison. Each example shows two frames for the edits “red boat hull” $\rightarrow$ “dark blue boat hull” and “tree frog” $\rightarrow$ “poison dart frog.” Edit-VAR consistently realizes the target appearance across frames while preserving scene geometry and unrelated background content. Competing methods may leave source attributes incompletely edited or introduce unintended changes to the background, lighting, and object appearance.

Variant	Target Alignment		Video Quality		Non-edit Preservation			Time (s)
Variant	CLIP-S ${}_{\mathrm{full}}\uparrow$	CLIP-S ${}_{\mathrm{edit}}\uparrow$	Smooth. $\uparrow$	Aesth. $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	Time (s)
Full method (no pruning)	0.979	0.980	0.989	0.611	22.89	0.833	0.172	85s
w/o attention guidance	0.935	0.931	0.975	0.560	19.56	0.563	0.373	85s
w/o scale envelope	0.958	0.933	0.979	0.514	20.43	0.667	0.287	85s
w/o Scale-Decoupled Generation	0.935	0.951	0.980	0.524	21.43	0.612	0.189	130s
AREdit-style uniform $\gamma$	0.895	0.913	0.975	0.541	16.30	0.498	0.456	85s

Table 2: Ablation study. The uniform-

\gamma

variant is the AREdit-style baseline.

K

denotes the percentage of retained tokens; random pruning controls for the token budget.

Figure 6: Qualitative ablation. Attention guidance and the scale envelope enable localized token replacement, while Scale-Decoupled Generation reduces fragmented source appearance. Residual pruning with $K=50\%$ remains visually close to the full method.

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Parameter	Value
$\gamma_{\mathrm{start}}$	2.00
$\gamma_{\mathrm{end}}^{\mathrm{fg}}$ / $\gamma_{\mathrm{end}}^{\mathrm{bg}}$	1.60 / 1.78
Image / video transition scale	5 / 4
Scale-envelope width $w_{t}$	0.06
Attention center $c_{a}$ / width $w_{a}$	0.50 / 0.10
Directly extracted attention layers	First 5 of 36

Appendix

Table 3: Final implementation configuration. The same settings are used for all editing categories.

Figure 7: Effect of the number of attention layers. We compare localization maps obtained by aggregating the first 1, 3, 5, and 10 layers and all layers. Aggregating the first five layers provides a stable and concentrated localization signal, while using additional layers brings limited qualitative change and incurs greater storage cost. We therefore use the first five layers throughout our experiments.

Pipeline	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
Wan2.1	25.62	0.898	0.124
InfinityStar	29.93	0.920	0.061

Appendix

Table 4: Source-video reconstruction fidelity of the Wan2.1 and InfinityStar pipelines. Wan2.1 uses VAE encoding followed by inversion-based reconstruction and decoding, whereas InfinityStar directly reconstructs the video from its encoded multi-scale tokens.

Figure 8: Qualitative source-video reconstruction comparison. We show representative source frames together with reconstructions from the inversion-based Wan2.1 pipeline and the direct token-based InfinityStar pipeline, highlighting differences in structural preservation, appearance fidelity, and fine visual details.

Parameter	Value	CLIP-S ${}_{\mathrm{edit}}\uparrow$	PSNR $\uparrow$	LPIPS $\downarrow$	Smooth. $\uparrow$
$\gamma_{\mathrm{end}}^{\mathrm{fg}}$	1.40	0.980	19.75	0.189	0.985
	1.50	0.977	21.06	0.183	0.984
	1.55	0.981	22.37	0.178	0.983
	1.60 †	0.980	22.89	0.172	0.989
	1.65	0.971	22.97	0.175	0.981
	1.70	0.954	23.01	0.177	0.980

Appendix

Table 5: Hyperparameter sensitivity analysis. We vary one parameter at a time while keeping the others fixed. † denotes the default configuration.

Figure 9: Additional qualitative comparisons with the baselines evaluated in the main paper. All methods use the same source video and target editing prompt for each case, with matched temporal sampling and output resolution.

Figure 10: User study results. Distributions of the four ordinal ratings ( Fail , Pass , Good , and Excellent ) assigned independently to each method. From left to right, the three panels report editing quality, background preservation, and temporal consistency.

Selection	Kept $K$	Phase 2		Two-Phase Total		Quality
Selection	Kept $K$	Time (s) $\downarrow$	Speedup $\uparrow$	Time (s) $\downarrow$	Speedup $\uparrow$	CLIP-S ${}_{\mathrm{full}}\uparrow$	CLIP-S ${}_{\mathrm{edit}}\uparrow$	Smooth. $\uparrow$	Aesth. $\uparrow$	LPIPS $\downarrow$
No pruning	100%	65.0	1.00 $\times$	84.8	1.00 $\times$	.979	.980	.989	.611	.172
Residual-guided	90%	61.2	1.06 $\times$	81.0	1.05 $\times$	.979	.978	.989	.610	.175
Residual-guided	70%	51.2	1.27 $\times$	70.9	1.19 $\times$	.975	.977	.989	.608	.175
Residual-guided	50%	43.9	1.48 $\times$	63.7	1.33 $\times$	.972	.976	.988	.609	.176
Residual-guided	30%	39.4	1.65 $\times$	59.2	1.43 $\times$	.965	.970	.975	.593	.204

Appendix

Table 6: Quality–efficiency trade-off of residual-guided token pruning. Pruning is applied to the final two generation scales. Runtime is measured on a single NVIDIA A800 GPU with 81 frames. Phase 2 speedup is relative to its 65.0 s unpruned runtime; two-phase speedup is relative to the 84.8 s pipeline with Scale-Decoupled Generation and no pruning.

Edit Type	Method	Target Alignment		Video Quality		Non-edit Preservation
Edit Type	Method	CLIP-S ${}_{\mathrm{full}}\uparrow$	CLIP-S ${}_{\mathrm{edit}}\uparrow$	Smooth. $\uparrow$	Aesth. $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
Object Replacement	Best Baseline	0.941	0.957	0.988	0.611	20.76	0.801	0.190
Object Replacement	Ours	0.957	0.970	0.991	0.613	21.87	0.815	0.176
Object Addition	Best Baseline	0.976	0.984	0.970	0.581	20.07	0.788	0.199
Object Addition	Ours	0.979	0.985	0.988	0.603	22.59	0.823	0.174
Background Replacement	Best Baseline	0.956	0.963	0.977	0.580	19.73	0.781	0.209

Appendix

Table 7: Per-edit-type quantitative comparison. Results broken down by editing category.

Figure 11: Additional results: Object replacement.

Figure 12: Additional results: Object addition.

Figure 13: Additional results: Background replacement.

Figure 14: Additional results: Attribute editing.

Video Editing

Video editing uses AI to alter, compose, relight, or reorganize moving footage while controlling how changes persist across frames.

See all 172 papers

Model the Edit, Not the Image: Visual Autoregressive Editing from a Source-Centric Perspective
Aug 10, 2026Hongyi Fang, Chuwen Xie, Benjia Zhou +6Text-Guided Image Editing Visual Autoregressive Models
Beijing Institute of Technology, Zhuhai · Zhejiang University · Tencent +1
One Editor, Many Edits: A Unified Training-Free Framework for Diverse Video Editing
Sep 3, 2026Adheesh Sunil Juvekar, Onkar Kishor Susladkar, Kiet A. Nguyen +6Video Editing Instruction-Based Video Editing
University of Illinois Urbana-Champaign
Rethinking Structure Preservation in Text-Guided Image Editing with Visual Autoregressive Models
Mar 30, 2026Tao Xia, Jiawei Liu, Yukun Zhang +3Text-Guided Image Editing Image Editing
Beijing Institute of Technology, Beijing, China · Shenyang Institute of Automation, CAS, Shenyang, China · Meitu Inc, MTLab, Beijing, China

Aug 10, 2026J/K move · Enter open · S save

Model the Edit, Not the Image: Visual Autoregressive Editing from a Source-Centric Perspective

Hongyi Fang, Chuwen Xie, Benjia Zhou, Yu-Xuan Qiu+5

Beijing Institute of Technology, Zhuhai · Zhejiang University · Tencent+1

Text-Guided Image Editing Visual Autoregressive Models Image Editing

Open paper AskPDF arXiv

Next-scale visual autoregressive models (VARs) have emerged as a powerful generative paradigm, producing high-quality images through efficient coarse-to-fine prediction. However, their potential for text-guided image editing remains largely underexplored. Existing training-free VAR editing approaches often formulate editing as target-conditioned regeneration guided or constrained by the source image, and may rely on inversion, test-time optimization, attention control, or user-provided masks. This generation-centric formulation does not fully exploit the multiscale source representations provided by VARs and may introduce additional computation or intervention. We instead take a source-centric perspective on VAR editing, in which the encoded source image tokens serve as the primary visual state and the editing process focuses on condition-induced changes. Based on this perspective, we propose EditMod, which compares source- and target-conditioned predictions under a shared autoregressive context, treats their difference as a scale-wise editing direction, and applies it as a residual update to source tokens at selected scales. Experiments show that EditMod achieves leading source-image fidelity while maintaining strong text alignment, and completes end-to-end editing of a 1K image in only 1.57 seconds on a single A100 GPU without per-image preparation.

Abstract

Video Editing

Explore similar work

Model the Edit, Not the Image: Visual Autoregressive Editing from a Source-Centric Perspective

One Editor, Many Edits: A Unified Training-Free Framework for Diverse Video Editing

Rethinking Structure Preservation in Text-Guided Image Editing with Visual Autoregressive Models