Organizations: Cornell University, Ithaca, New York, USA · Adobe Research, Cambridge, Massachusetts, USA · Adobe Research, San Francisco, California, USA
In film editing, every frame matters. While macro, content-level edits shape the high-level narrative of a film, small local edits often control timing and direct visual attention. For example, editors can adjust the precise timing of clips to control the length of a meaningful silence, or cut to shots that highlight a specific character's reactions to important moments. These small, detailed edits are slow and tedious to make in standard editing tools because users have to map high-level narrative goals to low-level frame-by-frame adjustments. At the same time, these fine-scale adjustments, which we call micro-edits, can be nuanced and expressive, with different choices serving different creative goals. In this work, we start by conducting a formative study with professional editors to understand the structure of micro-edits and how they are used. We then derive a mathematical formalization for this structure to develop tools that leverage AI and optimization to make exploring micro-edits simpler and more efficient for users. Finally, we validate our tools in multiple user studies and a wide range of video examples.
Figures & tables
Figure 1. After a rough cut via macro-level clip selections yielding straight cuts with coupled audio-video (left), editors refine cuts through micro-edits (center): rhythm edits adjust timing (e.g., pauses), while view edits redirect visual attention. We formalize micro-edits as continuous optimization within audio-visual synchrony-constrained windows, where audio-video can be safely decoupled. Our interactive tools leverage optimization and AI to support different creative objectives (right): automatically balancing character focus, and surfacing perceptual cues for cut-on-action, cut-before-blink, and reaction shots. All video frames shown in this paper are copyrighted and belong to EditStock. Project page: https://microedits.github.io/ A three-panel diagram illustrating the transition from a rough cut to specific AI-assisted micro-edits. The left panel shows a straight cut where Speaker A's teal video and audio tracks transition simultaneously to Speaker B's pink tracks. A purple dotted box denotes the micro-editing window over a pause in dialogue. The center panel shows the video cut shifted left within this window, decoupling it from the audio cut. Below, inset diagrams visually contrast an L-cut (video track transitions before audio) and a J-cut (video track transitions after audio). The right panel shows timeline visualizations for four micro-edits: Character Focus, Cut on Action (aligning dot markers), Cut Before Blink (using eye icons), and Reaction Shots (showing an overlapping yellow video segment).
Figure 2. Edit structure showing how an edited video combines segments from two source media across video and audio tracks. Each cut Xi is defined by its output time ( xouti ), source media ( xmi ), and source time ( xini ). Striped regions indicate unused footage. Colors indicate source origin. Edit structure showing how an edited video combines segments from two source media across video and audio tracks. Three stacked timelines display Input A (teal, top), Output Video (middle), and Input B (pink, bottom). Each features a video track marked by an eye icon and an audio track marked by a speaker icon. Input A and B show the original source media, with solid colored blocks for used segments and gray striped blocks indicating unused footage. Input A provides segments 1 and 3, while Input B provides segments 0 and 2. The Output Video timeline shows these segments spliced together sequentially (0, then 1, 2, and 3), alternating between pink and teal. The vertical cut lines between segments in the Output Video are staggered across the video and audio tracks, visually demonstrating that the video and audio transitions occur at slightly different times.
Figure 4. System overview: From a rough cut, our backend performs analysis (VLM, AV synchrony, reaction-shot indexing, cut-on-action, blinks) and surface cut suggestions on UI. System overview flowchart showing how a rough cut is analyzed to generate cut suggestions for the UI. The diagram is divided into a top "Analysis + Candidate Generation" backend and a bottom "Interactive UI" frontend. In the backend, source clips A and B are processed in parallel by VLM Coarse Analysis and AV Sync Processing. These analyses identify timeline constraints: a "Low-sync region" that generates "Reaction Shots" (using semantic indexing and EmoCLIP), and a "Safe AV synchrony decoupling window" that generates "Split Edits" (including Character Focus, Cut-on-Action, and Blink Detection). Bi-directional arrows connect these generated candidates to the Interactive UI below. The UI section displays the resulting tools, split into "Rhythm Editing" (showing timeline visualizations for Trim and Extend operations) and "View Editing" (showing visualizations for Reaction Shots, Cut Before Blink, Cut on Action, and Character Focus).
Table 1. Median (min–max) ratings (1–7 scale) for Rhythm, Character Focus, Cut-on-Action, Blink, Reaction. Creativity Support: Exploration, Expression, Effort, Immersion, Enjoyment, Efficiency ( Cherry and Latulipe, 2014 ) .
Figure 7. Perceived synchrony ( N=47 ) across four scenes. (A) Perceived lip-sync violation rate (% responses reporting violation), and (B) perceived cut naturalness (1–7). Error bars show 95% CI. Brackets indicate Optimized vs. Out-of-Window comparisons within each scene (* p<.05 , *** p<.001 ). Two side-by-side bar charts comparing perceived lip-sync violation and cut naturalness across four video scenes. Both charts group data by three conditions (Optimizer, Boundary, and Violation) for scenes titled Little Women, Anesthesia, Mental Load, and Soup Spoon. Chart A shows that the Optimizer condition consistently yields the lowest lip-sync violation rates (roughly 20 percent for three of the scenes) compared to the Violation condition (which ranges from 70 to 95 percent). Chart B shows the inverse trend for cut naturalness on a 1 to 7 scale: the Optimizer condition consistently scores the highest (averaging around 5.5 for most scenes), while the Violation condition scores the lowest. Brackets confirm statistically significant differences between the Optimizer and Violation conditions across all scenes.
Figure 8. Operation sequences for all nine participants in Study 2. Each icon represents a timeline operation (rhythm edit, marker apply/reset, reaction shot insert, undo, or preview). Stacked icons indicate consecutive repeated operations. The top group (P1–P5) edited Pumpkin Princess with MicroEditor; the bottom group (P6–P8, P10) edited Anesthesia with MicroEditor (P9’s log was not recorded due to a logging error). Sequence charts mapping the chronological editing operations for nine participants. Nine horizontal rows of icons represent specific tool interactions. A clear phased pattern emerges across users: sessions typically begin with "Rhythm" edits to adjust pacing, shift to view edits like "Cut-on-action" and "L/J optimization," and conclude with "Reaction shot" insertions. Frequent "Preview" and "Undo/Reset" icons throughout all phases highlight a highly iterative, trial-and-error editing workflow.
Baseline (manual)
MicroEditor (assisted)
PID
Ops
Ops/ min ↓
Total (min)
Ops
Ops/ min ↓
Edit (min)
RS (min)
Total (min)
P1
39
4.4
8.8
18
1.3
9.2
4.7
13.9
P2
21
2.4
8.6
17
1.6
8.5
2.3
10.9
P3
37
5.0
7.4
5
1.1
3.5
1.1
4.6
P4
69
4.2
16.4
14
0.9
12.5
3.4
15.9
P5
60
10.0
6.0
25
1.5
12.7
4.4
17.1
Table 2. Per-participant interaction metrics in Study 2 ( N=9 , counterbalanced). RS = time in the reaction shot browser. Bold Ops/min indicates the lower value per participant.
Figure 11. View identification using vision-language model analysis, shown on a product review scene captured by the authors. The VLM distinguishes the human presenter from product demonstration shots and extracts distinctive features for each view, enabling the Character Focus feature to redistribute screen time through automated L/J cut suggestions. Four rows of video frames from a product review. Rows alternate between a presenter speaking to the camera and close-ups of a glove held up to the camera. Beside each row, the vision-language model's view label and extracted visual features are listed.
Stage / Component
Time
Stage 1: VLM Coarse Analysis
Proxy generation (480p)
30s
VLM footage analysis (Gemini)
90s
Stage 2: AV Sync Processing
ASR transcription
45s
Active speaker detection (LR-ASD)
60s
Appendix
Table 3. Preprocessing time breakdown (1-minute scene, 4 source clips, MacBook Pro M2)
ID
Demographics
MicroEditor Feature Ratings
Age
Gender
Experience
Content Types
Rhythm
Character Focus
Action
Blink
P1
22
F
7+ years, Premiere
Narrative, YouTube, TikTok
6
7
5
7
P2
27
M
15 years
Narrative, Documentary
6
6
7
5
P3
26
F
7 years, Premiere
Narrative, Documentary
7
7
6
4
P4
26
F
5 years
Narrative, YouTube
6
7
7
4
P5
18
M
6 years
Narrative, Digital Media
6
7
7
6
Appendix
Table 4. Participant demographics and feature ratings from the MicroEditor user study
Metric
P1
P2
P3
P4
P5
P6
Median
Creativity Support Dimensions
Exploration: Helped explore possibilities
6
7
6
6
6
5
6.0
Exploration: Encouraged different cuts
7
7
7
6
6
4
6.5
Expressiveness: Achieved creative vision
5
5
6
6
6
6
6.0
Expressiveness: Achieved editing style
6
5
6
6
5
5
5.5
Effort/Reward: Quality worth effort
6
6
6
6
6
4
6.0
Appendix
Table 5. Creativity support and user experience metrics. Participants rated agreement on a 7-point scale (1=Strongly Disagree, 7=Strongly Agree) with statements adapted from the Creativity Support Index and task-specific measures.
ID
Demographics
MicroEditor Feature Ratings
Age
Gender
Experience
Content Types
Rhythm
Char. Focus
Action
Blink
Reaction
P1
20
F
10+ years
Narrative, Animation
6
6
7
4
6
P2
20
F
3 years
Dance, Trailers
6
7
6
5
7
P3
20
M
3-4 years
Reels, Product Videos
5
6
4
1
7
P4
24
M
7 years
Narrative, News
7
6
7
5
7
P5
19
F
4 years
Narrative, Commercial
6
7
6
3
7
Appendix
Table 6. Participant demographics and feature ratings from User Study 2 (with Reaction Shot Retrieval)
Metric
P1
P2
P3
P4
P5
P6
P7
P8
P9
P10
Med.
Creativity Support Dimensions
Exploration: Helped explore possibilities
6
6
5
7
7
7
6
6
6
6
6.0
Exploration: Encouraged different cuts
7
5
6
6
7
7
6
6
6
6
6.0
Expressiveness: Achieved creative vision
5
5
6
4
5
7
5
6
5
6
5.0
Expressiveness: Achieved editing style
5
6
6
5
6
7
6
6
5
6
6.0
Effort/Reward: Quality worth effort
6
6
5
4
7
7
6
6
5
7
6.0
Appendix
Table 7. Creativity support and user experience metrics for User Study 2. Participants rated agreement on a 7-point scale (1=Strongly Disagree, 7=Strongly Agree).
Video editing spans diverse editing paradigms, yet achieving high-quality instruction-guided and subject-guided editing within a single unified framework remains challenging. We introduce EditVid, a training-free framework combining sparse causal memory for local coherence, correspondence-based post-attention token injection for long-range identity preservation, and soft latent blending for edit locality. The same framework supports instruction-guided and reference-guided edits, including style transfer, attribute modification, object insertion, part-level editing, and subject replacement. On FiVE, EditVid achieves 78.16 FiVE-Acc, compared with 58.95 for the strongest evaluated training-free baseline, while obtaining competitive results on IVEBench. A user study further shows a 51.8% overall preference for EditVid over 7 competing methods.
Adheesh Sunil Juvekar, Onkar Kishor Susladkar, Kiet A. Nguyen +6
Recent advances in image editing have enabled fine-grained and highly controllable visual modifications. However, achieving similar levels of controllability in video editing remains an open challenge, particularly for precise local edits, dynamic temporal effects, and consistent lighting across frames.In this paper, we propose EditDistill, a framework that distills image-level visual transformations from edits to guide video editing.Specifically, given a source video and an edit prompt, we first generate an edited version of the initial video frame using an off-the-shelf image editing model. We then distill the visual transformation of this original-edited image pair into a compact representation termed an edit descriptor.This descriptor is injected into a video diffusion model through our lightweight fusion module, which steers the denoising process toward the desired edit. To align the video trajectory with the demonstrated image edit while preserving motion and layout, we train with edit-descriptor alignment and DINO structural consistency losses, and derive an equivalent inference-time guidance formulation under flow matching.Extensive experiments demonstrate that our method achieves strong performance on standard benchmarks and produces high-quality video editing results across diverse scenarios. For more details and videos, please refer to the anonymous webpage: https://yu-shaonian.github.io/EditDistill-Web/.
Guojun Lei, Hong Li, Hongbing Yang +3
State Key Lab of CAD&CG, Zhejiang University · Beihang University · Tsinghua University +1
Video editing is fundamentally message-driven: even from the same source footage, the selected shots change depending on the narrative the editor wishes to convey. Benchmarks for a closely related task, video summarization, reduce editorial intent to a single, message-agnostic notion of saliency and thus do not account for this diversity. For evaluating message-driven video editing, we present MEDit-Bench, a dataset and benchmark, which pairs long-form videos with multiple editing messages and multiple professionally produced edits per message, demonstrating that different messages yield substantially different edits from the same source. We define an automatic evaluation protocol based on temporal alignment metrics, and find that an LLM-as-a-judge preference, a natural proxy for narrative quality, is unreliable for this task due to severe position bias. We additionally annotate each message with ambiguity and contextfulness scores, and show that both dimensions negatively correlate with model performance, establishing message difficulty as a meaningful stratification factor. Experiments with state-of-the-art MLLMs and reinforcement fine-tuned baselines show that while strong models approach human temporal alignment at lenient thresholds, all models fall behind humans at stricter criteria. A human perceptual study further confirms a large quality gap, with professional human edits remaining consistently preferred over model outputs.