ALIVE: Interaction-Aligned Object Insertion for First-Frame-Guided Video Editing

Zhenghong Zhou; Zhe Lin; Jiebo Luo; Yuqian Zhou

cs.CVOct 6, 2026

ALIVE: Interaction-Aligned Object Insertion for First-Frame-Guided Video Editing

Authors: Zhenghong Zhou, Zhe Lin, Jiebo Luo, Yuqian Zhou

Organizations: University of Rochester · Adobe Research

Abstract

Current video editors can insert objects but often struggle to make them participate in interactions such as being picked up or manipulated. We introduce ALIVE, a framework that makes inserted objects "alive" through coherent interactions with the source video's contents, using an edited first frame and an instruction naming only the added object. We curate 35,800 editing pairs combining 3D-rendered, model-generated, and real-world videos with general editing pairs from ROSE. Each pair differs in the target object's presence while preserving the surrounding action, teaching editors coordinated object behavior and source preservation. We further train a vision-language model (VLM) to predict interaction guidance from the same inputs. We introduce the ALIVE-interaction benchmark to assess interaction fidelity, source preservation, and visual coherence using a unified VLM-based protocol, and evaluate on the general video object insertion benchmark. Without VLM guidance, ALIVE improves Overall over the strongest evaluated baseline by 43.9% and 4.4% on the two benchmarks, respectively. VLM-predicted guidance further improves the ALIVE-interaction score by 0.95 points without additional user inputs.

Figures & tables

Figure 1: Bringing inserted objects to life. Each panel shows the source video, edited first frame and object-only instruction, followed by corresponding results from Señorita and ALIVE. ALIVE enables added objects to be picked up, worn, opened, or cut in response to source actions.

Figure 2: ALIVE data construction and prompt annotation. Top: construction and quality filtering. Bottom: editing pairs from four sources (left) and multi-level prompts (right). P0 (no text) and P1 (object identity only) omit interaction descriptions; P2–P4 provide interaction guidance. Gold outlines mark edited first frames; P4 shading marks chunks with inclusive frame ranges.

Figure 3: ALIVE overview. (a) Source and noisy target latents are channel-concatenated (C); the edited first-frame latent stays clean (gold). Flames mark editor and VLM LoRA fine-tuning. (b) A supervised VLM predicts P4 descriptions and intervals from the same visual inputs and P1. P4 texts are independently encoded. Attention maps contrast shared P0–P3 with time-local P4 conditioning.

ALIVE-interaction benchmark (128 cases)
		VLM evaluation			Automatic metrics
Method	Guidance	Overall	Task	Motion	PickScore	Frame CLIP	Video ViCLIP	TC-CLIP	TC-DINO
Señorita	P1	60.24	53.52	36.52	19.95	24.04	19.09	0.9572	0.9494
NovaEdit	P0	36.36	8.59	0.39	18.94	21.15	15.41	0.9744	0.9734
I2VEdit	P0	56.55	48.44	33.79	19.68	23.48	18.61	0.9363	0.9132
AnyV2V	P2 †	42.75	44.53	31.25	19.77	24.12	19.48	0.9141	0.8916

Table 1: Comparison on interaction-focused and general object insertion. Both ALIVE variants share the LTX-2.5 (22B) editor; VLM P4 is predicted from source/

E_{0}

/P1.

\dagger

marks interaction descriptions supplied. VLM scores use a 0–100 scale; automatic metrics retain native scales. Higher is better throughout; bold/underline denote best/second-best scores, including ties.

Figure 4: Interaction-focused object insertion. Book handling (a) and basket carrying (b), each shown at the first frame, 40%, and 90% of the video. Square AnyV2V frames are displayed more narrowly. $\dagger$ marks provided interaction guidance, following the methods’ official prompt formats.

Training source	Pairs	Pooled	ALIVE-interaction benchmark ( $N=128$ )			general video object insertion benchmark ( $N=103$ )
		Overall	Overall	Task	Motion	Overall	Task	Motion
3D-rendered videos	14,362	74.85	73.08	76.37	62.89	77.06	78.16	71.12
Real-world videos	8,542	81.51	80.19	88.48	79.49	83.16	83.50	80.10
Model-generated videos	5,896	81.26	82.81	90.82	82.23	79.32	79.85	71.84
General editing pairs (ROSE)	7,000	70.93	60.12	60.74	45.12	84.36	88.35	81.31
All four sources	35,800	84.39	82.35	89.65	82.03	86.93	88.83	83.01

Table 2: Single-source and mixed-source training with the same 5k-step budget. All scores use our 16-frame evaluation (0–100;

\uparrow

). Pooled averages all 231 cases with equal weight:

(128\,S_{\mathrm{int}}+103\,S_{\mathrm{gen}})/231

, where

S_{\mathrm{int}}

and

S_{\mathrm{gen}}

are the benchmark means. Bold and underline mark the best and second-best scores per column.

Figure 5: Effect of VLM-predicted interaction guidance. P1 and predicted P4 condition the same editor with fixed visual inputs. Predicted prompts and inclusive frame ranges appear below. Red boxes highlight strap artifacts around the eyes (a) and an exposed root ball (b) in the P1 results.

VLM-enhanced P1
Guidance	$N$	Task	ID	Motion	BG	Temp.	Visual	Overall
P0	128	91.41	81.84	83.40	93.36	80.47	75.00	85.38
P1	128	93.75	83.98	86.13	91.80	81.05	75.20	86.71
P2	128	93.16	82.23	86.91	92.38	80.66	77.15	86.68
P3	128	94.73	83.40	88.48	91.60	81.25	75.59	87.37
P4	128	95.12	85.94	88.67	93.55	82.81	78.32	88.69

Table 3: Prompt guidance on the ALIVE-interaction benchmark. All rows use the same editor, visual inputs, and case-specific seeds. Annotated P2–P4 supply interaction information; VLM variants infer it from source/

E_{0}

/P1. P4 uses temporal chunks. Scores are 0–100 (

\uparrow

); bold and underline mark first and second place. ID: identity; BG: background; Temp.: temporal consistency.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Output	Steps	Time (s)	Peak (GiB)
Señorita	$768\!\times\!448\!\times\!33$	30	109.18	28.04
NovaEdit	$832\!\times\!480\!\times\!81$	50	407.70	19.63
I2VEdit	$1024\!\times\!576\!\times\!14$	25	863.62	65.16
AnyV2V	$512\!\times\!512\!\times\!16$	50	172.79	30.31
PropFly	$832\!\times\!480\!\times\!25$	50	71.63	21.14
Wan 14B, P1	$768\!\times\!512\!\times\!81$	30	304.64	64.46

Appendix

Table 4: Measured inference cost on A100 80GB. Output is width

\times

height

\times

frames; only denoising iterations are counted under Steps. Time includes each editor’s required inversion and adaptation. Peak denotes PyTorch peak allocated memory in GiB. VLM prompt prediction is measured separately in the last row; dashes mean not applicable.

Optimizer steps	Pairs	ALIVE-interaction benchmark	general video object insertion benchmark
5k	35,800	82.35	86.93
20k	35,800	86.46	89.20

Appendix

Table 5: Effect of training budget on P1-only LTX-2.5 editors (LoRA rank 128, batch 16). Scores report Overall (

\uparrow

) on both benchmarks.

Backbone	Steps	ALIVE-interaction benchmark	general video object insertion benchmark
Wan 2.2 TI2V (5B)	5k	71.85	81.54
Wan 2.2 I2V (14B)	5k	89.82	89.44
LTX-2.5 (22B)	5k	82.35	86.93

Appendix

Table 6: Backbone comparison after 5k steps of P1-only training. Scores report Overall (

\uparrow

) on both benchmarks; bold and underline mark the best and second-best scores.

Input	Role
Basic instruction	Identify the intended addition without prescribing an action sequence.
SOURCE	Establish non-target scene content, camera motion, and interaction context.
Edited first frame $E_{0}$	Specify the intended object or person and initial edit.
REFERENCE	Provide identity, contact, occlusion, interaction, and natural-effects evidence.
CANDIDATE	Generated result to evaluate at every displayed timepoint.

Appendix

Table 7: Inputs to the evaluation judge. The same roles and presentation are used for all benchmarks and methods. Only CANDIDATE is scored.

Dimension	Criterion	Weight
Task fidelity	Complete addition and coherent participation in interactions supported by SOURCE and $E_{0}$ .	0.25
Object identity	Stable identity, geometry, and attributes consistent with $E_{0}$ .	0.15
Motion / propagation	Plausible motion or appropriate stability, contact, timing, and occlusion.	0.20
Background preservation	Faithful non-target source content and camera motion.	0.15
Temporal consistency	Stable appearance and smooth transitions without flicker or disappearance.	0.15
Visual quality	Realistic boundaries, lighting, and natural effects.	0.10

Appendix

Table 8: Evaluation dimensions. Weights are identical across benchmarks and ablations.

Explore similar work

SimInsert: Seamless Video Object Insertion via Regional Sparse Attention Fusion
May 22, 2026Xinyu Chen, Yuyi Qian, Jiang Lin +9Video Editing Diffusion-Based Image Editing
1State Key Laboratory of Novel Software Technology, Nanjing University, Nanjing, China · School of Intelligence Science and Technology, Nanjing University, Suzhou, China · 4Xi’an Jiaotong-Liverpool University +3
One Editor, Many Edits: A Unified Training-Free Framework for Diverse Video Editing
Sep 3, 2026Adheesh Sunil Juvekar, Onkar Kishor Susladkar, Kiet A. Nguyen +6Video Editing Edit
University of Illinois Urbana-Champaign
LIVE: Leveraging Image Manipulation Priors for Instruction-based Video Editing
Apr 18, 2026Weicheng Wang, Zhicheng Zhang, Zhongqi Zhang +6Video Editing Video Dataset
Nankai University · Kuaishou Technology · Pengcheng Laboratory +1

May 22, 2026, cs.CVJ/K move · Enter open · S save

SimInsert: Seamless Video Object Insertion via Regional Sparse Attention Fusion

Xinyu Chen, Yuyi Qian, Jiang Lin, Shenyi Wang+8

1State Key Laboratory of Novel Software Technology, Nanjing University, Nanjing, China · School of Intelligence Science and Technology, Nanjing University, Suzhou, China · 4Xi’an Jiaotong-Liverpool University+3

Video Editing Diffusion-Based Image Editing Spatial Coherence Objects

Open paper AskPDF arXiv

Video object insertion requires ensuring spatio-temporal coherence and interactive realism, extending far beyond simple content placement. However, current approaches are often hindered by a reliance on explicit motion engineering or resource-intensive retraining, restricting their flexibility and generalization. To bridge this gap, we present \textit{SimInsert}, a training-free paradigm that efficiently decouples the task into intuitive single-frame editing and semantic motion description. By harnessing the robust generative priors of image-to-video diffusion models, SimInsert propagates edits temporally, strictly preserving background invariance while enabling plausible, text-driven interactions between the inserted object and the dynamic environment. Our approach hinges on non-invasive guidance mechanisms that enforce structural consistency, facilitate seamless boundary fusion, and counteract the fidelity drift that typically accumulates during the denoising trajectory. Extensive quantitative experiments validate our efficacy: SimInsert surpasses state-of-the-art methods with an 18.8% gain in PSNR, 20.1% in SSIM, and a 44.1% decrease in LPIPS, offering a streamlined solution for high-fidelity video editing.

Abstract

Explore similar work

SimInsert: Seamless Video Object Insertion via Regional Sparse Attention Fusion

One Editor, Many Edits: A Unified Training-Free Framework for Diverse Video Editing

LIVE: Leveraging Image Manipulation Priors for Instruction-based Video Editing