Place-it-R1: Unlocking Environment-aware Reasoning Potential of MLLM for Video Object Insertion
Organizations: The Hong Kong University of Science and Technology · AI Technology Center, Tencent Video, Tencent · Peking University
Abstract
Video object insertion is fundamental to video editing, yet existing diffusion methods often produce visually plausible but physically inconsistent results. We present Place-it-R1, an end-to-end framework for physically plausible video object insertion driven by environment-aware MLLM reasoning. Rather than treating reasoning as a generic text prompt, Place-it-R1 uses the MLLM to analyze the target environment, infer object-scene interactions, and determine where an insertion is physically valid. The resulting reasoning is translated into two complementary forms of guidance for video diffusion: semantic guidance that describes the intended physical interaction and spatial guidance that provides a valid insertion region in each frame. To further align generation with local physical realism, we introduce Spatial Direct Preference Optimization, which leverages an MLLM to rank generated candidates, and introduces a region-aware preference objective that explicitly localizes physical-violation penalties to the inserted-object region. Place-it-R1 further offers flexible and standard modes to trade off environment adaptation and scene preservation. Extensive experiments show that Place-it-R1 produces more physically coherent and visually natural insertions than state-of-the-art methods and achieves competitive results against commercial systems.
Figures & tables
| Benchmark | Method | Identity | Video Quality | Physics | ||||
|---|---|---|---|---|---|---|---|---|
| CLIP-I | DINO-I | Smooth. | Aesth. | PC | PR | PP | ||
| UNIC | UNIC ( Ye et al., 2025 ) | 0.5980 | 0.2450 | 0.9610 | 0.5627 | 4.20 | / | 5.33 |
| Kling (commercial model) ( Keling, 2025 ) | 0.6203 | 0.2509 | 0.9540 | 0.5641 | 4.41 | / | 5.93 | |
| PIKA (commercial model) ( Pika, 2025 ) | 0.6862 | 0.3752 | 0.9944 | 0.6151 | 4.34 | / | 6.11 | |
| Lucy-edit pro (commercial model) ( pro, 2025 ) | 0.6021 | 0.2629 | 0.9865 | 0.5693 | 4.28 | / | 5.79 | |
| \cellcolor blue!8 PhysInsert(standard mode) | \cellcolor blue!80.6043 | \cellcolor blue!8 0.2897 | \cellcolor blue!8 0.9928 | \cellcolor blue!80.5684 | \cellcolor blue!8 4.53 | \cellcolor blue!8/ | \cellcolor blue!8 6.21 | |
| Variant | Identity | Video Quality | Physics | |||
|---|---|---|---|---|---|---|
| CLIP-I | DINO-I | Smooth. | Aesth. | PC | PR | |
| Direct trajectory (without ASR) | 0.7305 | 0.3747 | 0.9923 | 0.5137 | 4.05 | 0.70 |
| ASR: spatial only | 0.7678 | 0.4489 | 0.9862 | 0.4989 | 3.92 | 0.67 |
| ASR: text + spatial | 0.7832 | 0.4492 | 0.9892 | 0.5102 | 4.02 | 0.69 |
| \cellcolor blue!8 Full ASR: continuous + spatial | \cellcolor blue!8 0.7938 | \cellcolor blue!8 0.4925 | \cellcolor blue!80.9906 | \cellcolor blue!8 0.5305 | \cellcolor blue!8 4.17 | \cellcolor blue!8 0.86 |