cs.HCOct 29, 2026
SaveMicro-Edits: Computational Support for Fine-Scale Edits in Film
Organizations: Cornell University, Ithaca, New York, USA · Adobe Research, Cambridge, Massachusetts, USA · Adobe Research, San Francisco, California, USA
Abstract
In film editing, every frame matters. While macro, content-level edits shape the high-level narrative of a film, small local edits often control timing and direct visual attention. For example, editors can adjust the precise timing of clips to control the length of a meaningful silence, or cut to shots that highlight a specific character's reactions to important moments. These small, detailed edits are slow and tedious to make in standard editing tools because users have to map high-level narrative goals to low-level frame-by-frame adjustments. At the same time, these fine-scale adjustments, which we call micro-edits, can be nuanced and expressive, with different choices serving different creative goals. In this work, we start by conducting a formative study with professional editors to understand the structure of micro-edits and how they are used. We then derive a mathematical formalization for this structure to develop tools that leverage AI and optimization to make exploring micro-edits simpler and more efficient for users. Finally, we validate our tools in multiple user studies and a wide range of video examples.
Figures & tables
Figure 1. After a rough cut via macro-level clip selections yielding straight cuts with coupled audio-video (left), editors refine cuts through micro-edits (center): rhythm edits adjust timing (e.g., pauses), while view edits redirect visual attention. We formalize micro-edits as continuous optimization within audio-visual synchrony-constrained windows, where audio-video can be safely decoupled. Our interactive tools leverage optimization and AI to support different creative objectives (right): automatically balancing character focus, and surfacing perceptual cues for cut-on-action, cut-before-blink, and reaction shots. All video frames shown in this paper are copyrighted and belong to EditStock. Project page: https://microedits.github.io/ A three-panel diagram illustrating the transition from a rough cut to specific AI-assisted micro-edits. The left panel shows a straight cut where Speaker A's teal video and audio tracks transition simultaneously to Speaker B's pink tracks. A purple dotted box denotes the micro-editing window over a pause in dialogue. The center panel shows the video cut shifted left within this window, decoupling it from the audio cut. Below, inset diagrams visually contrast an L-cut (video track transitions before audio) and a J-cut (video track transitions after audio). The right panel shows timeline visualizations for four micro-edits: Character Focus, Cut on Action (aligning dot markers), Cut Before Blink (using eye icons), and Reaction Shots (showing an overlapping yellow video segment).
Figure 2. Edit structure showing how an edited video combines segments from two source media across video and audio tracks. Each cut is defined by its output time ( ), source media ( ), and source time ( ). Striped regions indicate unused footage. Colors indicate source origin. Edit structure showing how an edited video combines segments from two source media across video and audio tracks. Three stacked timelines display Input A (teal, top), Output Video (middle), and Input B (pink, bottom). Each features a video track marked by an eye icon and an audio track marked by a speaker icon. Input A and B show the original source media, with solid colored blocks for used segments and gray striped blocks indicating unused footage. Input A provides segments 1 and 3, while Input B provides segments 0 and 2. The Output Video timeline shows these segments spliced together sequentially (0, then 1, 2, and 3), alternating between pink and teal. The vertical cut lines between segments in the Output Video are staggered across the video and audio tracks, visually demonstrating that the video and audio transitions occur at slightly different times.
Figure 3. Our web-based editor provides standard NLE controls alongside specialized micro-editing tools. Track colors correspond to characters, and blue overlays mark silence regions for rhythm edits, which editors adjust with pre-roll and trim/extend hotkeys during playback. For view edits, the Character Focus panel (right) automates L/J-cut placement to emphasize a selected character’s screen time, and applied offsets appear on the timeline (C). Markers at each cut surface perceptual-cue suggestions: cut-before-blink candidates (B) and cut-on-action candidates (D). Hovering over a marker reveals an AI-generated explanation; clicking applies or resets the suggested offset. The Reaction Shot Browser (A) retrieves non-speaking moments, filterable by character, emotion, and shot type; editors insert reactions by selecting transcript text or a timeline position. Video frames © EditStock. A screenshot of the web-based video editor interface showing specialized micro-editing tools and timeline. The image is split into a left section displaying two stacked Reaction Shot Browsers (Label A), each showing a grid of video thumbnails next to a side panel of filtering options. The right section shows the main editor, containing a text transcript, a video player, a right-side settings panel, and a multi-track timeline at the bottom. Dotted lines highlight three timeline features: Label B, "Cut-before-Blink," points to a small eye icon marker at a cut; Label C, "Applied L- and J-cuts," points to visually staggered transitions between the colored video and audio tracks; and Label D, "Cut-on-Action," points to a green marker that opens a pop-up dialog box containing an AI analysis explanation and an "Apply Cut" button.
Figure 4. System overview: From a rough cut, our backend performs analysis (VLM, AV synchrony, reaction-shot indexing, cut-on-action, blinks) and surface cut suggestions on UI. System overview flowchart showing how a rough cut is analyzed to generate cut suggestions for the UI. The diagram is divided into a top "Analysis + Candidate Generation" backend and a bottom "Interactive UI" frontend. In the backend, source clips A and B are processed in parallel by VLM Coarse Analysis and AV Sync Processing. These analyses identify timeline constraints: a "Low-sync region" that generates "Reaction Shots" (using semantic indexing and EmoCLIP), and a "Safe AV synchrony decoupling window" that generates "Split Edits" (including Character Focus, Cut-on-Action, and Blink Detection). Bi-directional arrows connect these generated candidates to the Interactive UI below. The UI section displays the resulting tools, split into "Rhythm Editing" (showing timeline visualizations for Trim and Extend operations) and "View Editing" (showing visualizations for Reaction Shots, Cut Before Blink, Cut on Action, and Character Focus).
Figure 5. Proxy for audio-visual synchrony ( ). Top: active speaker detection for dialogue scenes—high synchrony (green) when speaking, low (red) when silent/off-screen. Bottom: general synchrony network for non-dialogue content (tennis). Both guide safe decoupling for L/J-cuts and reaction shots. Video frames © EditStock . Two line graphs demonstrating proxy models for audio-visual synchrony in dialogue and non-dialogue scenes. The top graph shows active speaker detection. Four video frames of a woman correspond to a synchrony curve plotted against time, with a background audio waveform. A horizontal threshold line at zero separates low synchrony (red line, matching "Not Speaking" frames) from high synchrony (green peaks, matching "Speaking" frames). Spoken words track along the curve: it remains red below the threshold during the phrase "I'm having this party tonight," spikes green above the threshold for "A party?", dips red again, and spikes green for "Sounds great." The bottom graph analyzes non-dialogue content, featuring a tennis player. Five video frames map to a similar curve with a 0.5 threshold. The curve spikes green above the threshold twice, corresponding exactly to the moments the player's racket strikes the ball, and remains red during the movement between hits.
Figure 6. Numbered frame mosaics for VLM-based cut-on-action alignment. J-cut ( ): A (unused post-roll) and B (visible start) extend forward from the straight cut. Equal indices = same moment, so A4 B4 s. L-cut ( ): A (visible end) and B (unused pre-roll) extend backward from the straight cut. A has 3 frames (short clip), B has 22 (longer footage), so matching aligns by counting backward: B21 A2 s. Video frames © EditStock. Diagram illustrating how matching frame numbers between two video clips calculates the timing offset for L-cuts and J-cuts. The left side explains a J-cut for a matching head turn by comparing frames extending forward from the original straight cut point. Since both clips start counting from the same moment, matching frame 4 of the outgoing clip (Clip A) to frame 4 of the incoming clip (Clip B) aligns the action, resulting in a positive 0.67-second offset. The right side explains an L-cut for a matching head tilt, comparing frames extending backward from the cut point. Because the clips have different lengths (Clip A has 3 frames, Clip B has 22), the frame numbers differ. Aligning frame 21 of the incoming clip back to frame 2 of the outgoing clip aligns the action by counting backward, creating a negative 0.22-second offset.
| Features | Creativity Support | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Rhy. | Char. | Act. | Blk. | Rct. | Expl. | Expr. | Effort | Imm. | Enj. | Effi. | |
| Study 1 | 6.0 | 7.0 | 7.0 | 5.5 | — | 6.0 | 6.0 | 6.0 | 5.5 | 7.0 | 7.0 |
| 6–7 | 6–7 | 5–7 | 4–7 | — | 5–7 | 5–6 | 4–6 | 5–7 | 6–7 | 5–7 | |
| Study 2 | 6.0 | 6.0 | 6.5 | 4.5 | 7.0 | 6.0 | 5.0 | 6.0 | 5.5 | 7.0 | 7.0 |
| 5–7 | 5–7 | 4–7 | 1–5 | 6–7 | 5–7 | 4–7 | 4–7 | 3–7 | 5–7 | 5–7 | |
Table 1. Median (min–max) ratings (1–7 scale) for Rhythm, Character Focus, Cut-on-Action, Blink, Reaction. Creativity Support: Exploration, Expression, Effort, Immersion, Enjoyment, Efficiency ( Cherry and Latulipe, 2014 ) .
Figure 7. Perceived synchrony ( ) across four scenes. (A) Perceived lip-sync violation rate (% responses reporting violation), and (B) perceived cut naturalness (1–7). Error bars show 95% CI. Brackets indicate Optimized vs. Out-of-Window comparisons within each scene (* , *** ). Two side-by-side bar charts comparing perceived lip-sync violation and cut naturalness across four video scenes. Both charts group data by three conditions (Optimizer, Boundary, and Violation) for scenes titled Little Women, Anesthesia, Mental Load, and Soup Spoon. Chart A shows that the Optimizer condition consistently yields the lowest lip-sync violation rates (roughly 20 percent for three of the scenes) compared to the Violation condition (which ranges from 70 to 95 percent). Chart B shows the inverse trend for cut naturalness on a 1 to 7 scale: the Optimizer condition consistently scores the highest (averaging around 5.5 for most scenes), while the Violation condition scores the lowest. Brackets confirm statistically significant differences between the Optimizer and Violation conditions across all scenes.
Figure 8. Operation sequences for all nine participants in Study 2. Each icon represents a timeline operation (rhythm edit, marker apply/reset, reaction shot insert, undo, or preview). Stacked icons indicate consecutive repeated operations. The top group (P1–P5) edited Pumpkin Princess with MicroEditor; the bottom group (P6–P8, P10) edited Anesthesia with MicroEditor (P9’s log was not recorded due to a logging error). Sequence charts mapping the chronological editing operations for nine participants. Nine horizontal rows of icons represent specific tool interactions. A clear phased pattern emerges across users: sessions typically begin with "Rhythm" edits to adjust pacing, shift to view edits like "Cut-on-action" and "L/J optimization," and conclude with "Reaction shot" insertions. Frequent "Preview" and "Undo/Reset" icons throughout all phases highlight a highly iterative, trial-and-error editing workflow.
| Baseline (manual) | MicroEditor (assisted) | |||||||
| PID | Ops | Ops/ min | Total (min) | Ops | Ops/ min | Edit (min) | RS (min) | Total (min) |
| P1 | 39 | 4.4 | 8.8 | 18 | 1.3 | 9.2 | 4.7 | 13.9 |
| P2 | 21 | 2.4 | 8.6 | 17 | 1.6 | 8.5 | 2.3 | 10.9 |
| P3 | 37 | 5.0 | 7.4 | 5 | 1.1 | 3.5 | 1.1 | 4.6 |
| P4 | 69 | 4.2 | 16.4 | 14 | 0.9 | 12.5 | 3.4 | 15.9 |
| P5 | 60 | 10.0 | 6.0 | 25 | 1.5 | 12.7 | 4.4 | 17.1 |
Table 2. Per-participant interaction metrics in Study 2 ( , counterbalanced). RS = time in the reaction shot browser. Bold Ops/min indicates the lower value per participant.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 9. Two participants editing the same scene Anesthesia : P6 with MicroEditor (top) and P1 editing manually (bottom). The MicroEditor row labels cut types per user’s decisions (e.g., optimization based on AV synchrony, cut-on-action). When cut-on-action is a strong cue, manual and assisted edits sometimes converge on similar cut timing (0.9 vs 1.1s J-cut offset). Video frames © EditStock. Two timeline diagrams of the same scene, one edited with MicroEditor by participant P6 and one edited manually by participant P1. Each shows reaction shot, video, and audio tracks with colored segments per speaker and dashed lines marking cuts, labeled with L-cut and J-cut offsets in seconds. Thumbnails of the two characters and a reaction shot icon appear on the left.
Figure 10. System-level synchrony analysis for an edit of Anesthesia scene. The visualization shows the continuous synchrony signal ( , purple line), word-level timing (yellow boxes), and the calculated safe decoupling windows used to generate the L/J-cut and Cut-on-Action suggestions surfaced to the user. Video frames © EditStock. Eight stacked plots, one per clip of the scene, each showing a continuous synchrony curve, word-level speech boxes, the audio waveform, and labeled markers for the applied cut type and offset. A small character thumbnail identifies each clip at the left.
Figure 11. View identification using vision-language model analysis, shown on a product review scene captured by the authors. The VLM distinguishes the human presenter from product demonstration shots and extracts distinctive features for each view, enabling the Character Focus feature to redistribute screen time through automated L/J cut suggestions. Four rows of video frames from a product review. Rows alternate between a presenter speaking to the camera and close-ups of a glove held up to the camera. Beside each row, the vision-language model's view label and extracted visual features are listed.
| Stage / Component | Time |
|---|---|
| Stage 1: VLM Coarse Analysis | |
| Proxy generation (480p) | 30s |
| VLM footage analysis (Gemini) | 90s |
| Stage 2: AV Sync Processing | |
| ASR transcription | 45s |
| Active speaker detection (LR-ASD) | 60s |
Table 3. Preprocessing time breakdown (1-minute scene, 4 source clips, MacBook Pro M2)
| ID | Demographics | MicroEditor Feature Ratings | ||||||
| Age | Gender | Experience | Content Types | Rhythm | Character Focus | Action | Blink | |
| P1 | 22 | F | 7+ years, Premiere | Narrative, YouTube, TikTok | 6 | 7 | 5 | 7 |
| P2 | 27 | M | 15 years | Narrative, Documentary | 6 | 6 | 7 | 5 |
| P3 | 26 | F | 7 years, Premiere | Narrative, Documentary | 7 | 7 | 6 | 4 |
| P4 | 26 | F | 5 years | Narrative, YouTube | 6 | 7 | 7 | 4 |
| P5 | 18 | M | 6 years | Narrative, Digital Media | 6 | 7 | 7 | 6 |
Table 4. Participant demographics and feature ratings from the MicroEditor user study
| Metric | P1 | P2 | P3 | P4 | P5 | P6 | Median |
| Creativity Support Dimensions | |||||||
| Exploration: Helped explore possibilities | 6 | 7 | 6 | 6 | 6 | 5 | 6.0 |
| Exploration: Encouraged different cuts | 7 | 7 | 7 | 6 | 6 | 4 | 6.5 |
| Expressiveness: Achieved creative vision | 5 | 5 | 6 | 6 | 6 | 6 | 6.0 |
| Expressiveness: Achieved editing style | 6 | 5 | 6 | 6 | 5 | 5 | 5.5 |
| Effort/Reward: Quality worth effort | 6 | 6 | 6 | 6 | 6 | 4 | 6.0 |
Table 5. Creativity support and user experience metrics. Participants rated agreement on a 7-point scale (1=Strongly Disagree, 7=Strongly Agree) with statements adapted from the Creativity Support Index and task-specific measures.
| ID | Demographics | MicroEditor Feature Ratings | |||||||
| Age | Gender | Experience | Content Types | Rhythm | Char. Focus | Action | Blink | Reaction | |
| P1 | 20 | F | 10+ years | Narrative, Animation | 6 | 6 | 7 | 4 | 6 |
| P2 | 20 | F | 3 years | Dance, Trailers | 6 | 7 | 6 | 5 | 7 |
| P3 | 20 | M | 3-4 years | Reels, Product Videos | 5 | 6 | 4 | 1 | 7 |
| P4 | 24 | M | 7 years | Narrative, News | 7 | 6 | 7 | 5 | 7 |
| P5 | 19 | F | 4 years | Narrative, Commercial | 6 | 7 | 6 | 3 | 7 |
Table 6. Participant demographics and feature ratings from User Study 2 (with Reaction Shot Retrieval)
| Metric | P1 | P2 | P3 | P4 | P5 | P6 | P7 | P8 | P9 | P10 | Med. |
| Creativity Support Dimensions | |||||||||||
| Exploration: Helped explore possibilities | 6 | 6 | 5 | 7 | 7 | 7 | 6 | 6 | 6 | 6 | 6.0 |
| Exploration: Encouraged different cuts | 7 | 5 | 6 | 6 | 7 | 7 | 6 | 6 | 6 | 6 | 6.0 |
| Expressiveness: Achieved creative vision | 5 | 5 | 6 | 4 | 5 | 7 | 5 | 6 | 5 | 6 | 5.0 |
| Expressiveness: Achieved editing style | 5 | 6 | 6 | 5 | 6 | 7 | 6 | 6 | 5 | 6 | 6.0 |
| Effort/Reward: Quality worth effort | 6 | 6 | 5 | 4 | 7 | 7 | 6 | 6 | 5 | 7 | 6.0 |
Table 7. Creativity support and user experience metrics for User Study 2. Participants rated agreement on a 7-point scale (1=Strongly Disagree, 7=Strongly Agree).
More in this topic
Video Editing
Video editing uses AI to alter, compose, relight, or reorganize moving footage while controlling how changes persist across frames.
See all 180 papers