Recent advances in Video Foundation Models (VFMs) have revolutionized human-centric video synthesis, yet fine-grained and independent editing of subjects and scenes remains a critical challenge. Recent attempts to incorporate richer environment control through rigid 3D geometric compositions often encounter a stark trade-off between precise control and generative flexibility. Furthermore, the heavy 3D pre-processing still limits practical scalability. In this paper, we propose ONE-SHOT, a parameter-efficient framework for compositional human-environment video generation. Our key insight is to factorize the generative process into disentangled signals. Specifically, we introduce a canonical-space injection mechanism that decouples human dynamics from environmental cues via cross-attention. We also propose Dynamic-Grounded-RoPE, a novel positional embedding strategy that establishes spatial correspondences between disparate spatial domains without any heuristic 3D alignments. To support long-horizon synthesis, we introduce a Hybrid Context Integration mechanism to maintain subject and scene consistency across minute-level generations. Experiments demonstrate that our method significantly outperforms state-of-the-art methods, offering superior structural control and creative diversity for video synthesis. Our project has been available on: https://martayang.github.io/ONE-SHOT/.
Figures & tables
Figure 1: Human-Environment Synthesis by ONE-SHOT . Three kinds of conditions are decoupled and factorized within our ONE-SHOT framework to enable compositional generation. Blue ) : human dynamics. Green ) : textual prompt. Orange ) : environment space. The generated videos exhibit high consistency in human subjects and environments w.r.t. given conditions, and further accurately convey the textual interaction of “holding a longsword”.
Figure 2: Model Architecture. Our model is built upon a Pretrained VFM (gray, right) augmented with an additional Conditioning Branch (cyan, left) for concept injection. The environmental condition cenv is encoded from 2D-projected point clouds and depth maps, while identity appearance cid and context memory cmem jointly maintain visual coherence. Human dynamics cmot are disentangled from environmental inputs and injected through the proposed Decoupled Motion Cross-Attention.
Figure 3: Decoupled Motion Cross-attention. The canonical-space human pose is injected into the video through cross-attention, where the proposed Dynamic-Grounded-RoPE bridges the spatial discrepancy between the environment and human spaces.
Self-reconstruction (Traj100)
Cross-composition (Identity/Motion Swap)
Method
FID ↓
FVD ↓
MS ↑
BC ↑
SubSim ↑
FID ↓
FVD ↓
MS ↑
BC ↑
SubSim ↑
Face ↑
WanAnimate [ 5 ]
21.25
224.08
0.893
0.956
0.189
152.07
1435.3
0.892
0.950
0.368
0.196
RealisDance [ 36 ]
22.17
247.54
0.920
0.946
0.237
155.71
1416.5
0.986
0.962
0.230
0.257
Uni3C [ 3 ]
55.47
775.38
0.853
0.928
0.185
162.53
1519.5
0.924
0.950
0.321
0.203
RealisMotion [ 19 ]
18.89
193.89
0.916
0.940
0.356
151.19
1486.8
0.960
0.919
0.300
0.238
ONE-SHOT (ours)
16.88
181.17
0.940
0.959
0.317
150.98
1382.4
0.993
0.963
0.349
0.241
Table 1: Quantitative Results on Traj100 (Self-Reconstruction) and Our Constructed Testset (Cross-Composition). Lower is better for FID/FVD. Higher is better for others.
Figure 4: Self-Reconstruction and Cross-Composition Comparisons on Traj100. (a) Our method better preserves both the scene structure and human appearance, and (b) maintains stable subject placement and motion consistency when swapping identity.
Figure 5: Text-guided editing with compositional controls . Beyond composing scene, identity, and motion, our method enables instruction-based editing via text prompts, indicating strong compatibility with the pretrained VFM and minimal loss of its native text-conditioned editing ability.
Ablation
FID ↓
FVD ↓
MS ↑
ONE-SHOT (ours)
16.88
181.17
0.9404
w/o RoPE grounding
22.34
229.76
0.9268
label-only RoPE
21.36
221.66
0.9256
w/o face reference
21.37
215.43
0.9378
w/o subject-cropped reference
19.43
200.23
0.9351
w/o context memory
18.17
182.98
0.9330
Table 2: Ablations on Traj100 (Self-Reconstruction).
Figure 6: Ablations of Dynamic-Grounded-RoPE and Face Appearance Integration on Long-Horizon Generation. Removing RoPE grounding weakens motion controllability, leading to pose drift and reduced adherence to the provided motion control. Removing face reference weakens identity anchoring, causing increased identity drift over time. Our full model achieves both better motion following and identity preservation.
Figure 7: Effect of context memory on viewpoint revisit consistency . Our model maintains structural and temporal consistency even after camera loops, whereas memoryless baselines suffer from appearance drift.
Self-reconstruction (Traj100)
Cross-composition
Method
FID ↓
FVD ↓
MS ↑
BC ↑
SubSim ↑
FID ↓
FVD ↓
MS ↑
BC ↑
SubSim ↑
Face ↑
WAN-VACE (fine-tuned)
20.96
185.09
0.935
0.955
0.311
151.68
1465.3
0.912
0.947
0.182
0.099
ONE-SHOT (ours)
16.88
181.17
0.940
0.959
0.317
150.98
1382.4
0.993
0.963
0.349
0.241
Table 3: Comparison with fine-tuned WAN-VACE on self-reconstruction and cross-composition. WAN-VACE is adapted to our setting by fine-tuning on the same training data as ONE-SHOT, with SMPL-X used as compositional control input. Lower is better for FID/FVD. Higher is better for others.