cs.CVAug 27, 2026

LiveVVT: High-Fidelity Video Virtual Try-On in Real Time

Authors: Yushe Cao, Shikun Feng, Ruxiang Duan, Liyong Wang, Dianxi Shi, Chun Yu, Junliang Xing

Organizations: Tsinghua University · Zhongguancun Academy · South China University of Technology · Beijing Jiaotong University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Diffusion-based Video Virtual Try-On (VVT) achieves high visual fidelity through bidirectional spatio-temporal modeling, but complete-clip dependence incurs prohibitive latency and computational overhead in practical continuous deployment. Naively enforcing causality disrupts pretrained bidirectional priors and substantially degrades synthesis quality. We introduce LiveVVT, a rolling streaming diffusion framework that preserves bounded bidirectional modeling within causal recurrent generation. Within a fixed-size window, LiveVVT jointly denoises multiple video chunks under bounded look-ahead, preserving local bidirectional interactions while emitting one clean chunk per iteration. Beyond the window, two complementary memories sustain long-term consistency: a bounded temporal memory propagates recent dynamics and occlusion context, whereas a persistent global appearance memory, constructed once from the target garment and a frontal try-on keyframe, anchors garment details and dressed appearance throughout the stream. We further introduce a progressive distillation framework integrating bidirectional VVT learning, teacher-trajectory regression for causal few-step adaptation, and Collaborative Matching Distillation, which couples teacher-distribution matching with rolling flow matching on real videos to align optimization with recurrent inference. Experiments on paired and unpaired long-sequence benchmarks demonstrate superior generation quality over similarly sized models, with 26×26\times lower latency and 11×11\times higher throughput, enabling high-fidelity real-time streaming VVT.

Figures & tables

Figure 1: Comparison of latency and throughput between LiveVVT and state-of-the-art methods.
Figure 2: Overview of LiveVVT. (a) Rolling streaming try-on emits one clean chunk per update from a staggered-noise window with persistent global appearance memory A=(Ag,Af)\mathcal{A}=(\mathcal{A}_{g},\mathcal{A}_{f}) and temporal memory Hk\mathcal{H}_{k} . (b) Bidirectional VVT learning, teacher-trajectory regression, and CoMD progressively transfer offline bidirectional priors to few-step streaming generation.
 PairedUnpaired
MethodVFID I ↓\downarrowVFID R ↓\downarrowSSIM ↑\uparrowLPIPS ↓\downarrowVFID I ↓\downarrowVFID R ↓\downarrow
OOTDiff+AM46.66624.4500.7200.23847.80225.857
CatVTON+AM45.75420.7070.7390.23146.12520.728
VACE18.0860.4180.7620.14927.5190.871
ViViD20.8870.3370.7920.13027.9630.462
CatV2TON20.6940.2320.8150.13228.3430.356
Table 1: Quantitative comparison of LiveVVT with competing methods on the long-sequence ViViD-SL benchmark.
 PairedUnpaired
MethodVFID I ↓\downarrowVFID R ↓\downarrowSSIM ↑\uparrowLPIPS ↓\downarrowVFID I ↓\downarrowVFID R ↓\downarrow
OOTDiff+AM33.2252.3110.7560.24639.1392.172
CatVTON+AM33.3342.5550.7690.24636.8631.547
VACE20.3730.4050.7870.17331.1521.648
ViViD24.2440.6110.7850.14528.3561.037
CatV2TON22.0180.3760.8110.15630.6531.133
Table 2: Quantitative comparison of LiveVVT with competing methods on the long-sequence ViT-HDL benchmark.
Figure 3: Long-Term and Multi-View Try-On Comparison.
Figure 4: Long-stream scalability: (a) per-update latency, (b) total inference time, and (c) peak GPU memory usage.
Figure 5: Qualitative comparison across progressive distillation stages under rolling inference.
ConfigurationVFID I ↓\downarrowVFID R ↓\downarrowSSIM ↑\uparrowLPIPS ↓\downarrowFPS ↑\uparrow
Ag\mathcal{A}_{g} only14.4150.1310.8240.09426.74
+ Af+\,\mathcal{A}_{f}13.9600.1150.8260.09325.85
+ H+\,\mathcal{H}13.8710.1120.8320.08822.60
+ H,Af+\,\mathcal{H},\mathcal{A}_{f} (Ours)13.1940.0900.8380.09922.39
Table 3: Ablation of the frontal try-on and temporal memory components on ViViD-SL under paired setting.
Training stageVFID I ↓\downarrowVFID R ↓\downarrowSSIM ↑\uparrowLPIPS ↓\downarrow
Stage I17.2020.2810.7860.144
Stage II14.3710.1560.8030.116
Stage III13.1940.0900.8380.099
Table 4: Ablation of the three progressive distillation stages under rolling inference on ViViD-SL under paired setting.
Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

 PairedUnpaired
MethodVFID I ↓\downarrowVFID R ↓\downarrowSSIM ↑\uparrowLPIPS ↓\downarrowVFID I ↓\downarrowVFID R ↓\downarrow
OOTDiff+AM42.9551.8160.6560.25947.0012.622
CatVTON+AM33.0431.4220.7140.21339.7261.192
VACE23.5890.8750.8350.12832.2922.378
ViViD28.7120.8120.7910.14935.9541.316
CatV2TON29.8881.6330.7960.16337.3851.420
Appendix
Table 5: Full-sequence quantitative comparison on the long-sequence TikTokDress-L benchmark under paired and unpaired settings.
Figure 6: Average single-image try-on inference time of LiveVVT and state-of-the-art image try-on methods, evaluated at a resolution of 512×384512\times 384 .
 PairedUnpaired
MethodFID ↓\downarrowKID ↓\downarrowSSIM ↑\uparrowLPIPS ↓\downarrowFID ↓\downarrowKID ↓\downarrow
IDM-VTON6.3381.3220.8810.0799.6111.639
OOTDiff9.3054.0860.8190.08812.4084.689
CatVTON6.1390.9640.8690.0979.1431.267
UniFit8.7990.7020.8830.065––
Ours5.9100.4190.8730.0549.2120.889
Appendix
Table 6: Quantitative comparison of LiveVVT with publicly available image try-on methods on the VITON-HD test set.
λ\lambdaVFID I ↓\downarrowVFID R ↓\downarrowSSIM ↑\uparrowLPIPS ↓\downarrow
014.5150.2050.8260.094
0.112.8070.1230.8340.096
0.213.1940.0900.8380.099
0.413.4350.1460.8380.104
0.814.4460.1460.8430.104
Appendix
Table 7: Ablation of the RFM weight λ\lambda in CoMD on ViViD-SL under the paired setting. The highlighted row denotes the default configuration used in the main experiments.
ConditioningVFID I ↓\downarrowVFID R ↓\downarrowSSIM ↑\uparrowLPIPS ↓\downarrowFPS ↑\uparrow
Cached13.1940.0900.8380.09922.39
Repeated Encoding13.2800.1030.8400.08919.73
Appendix
Table 8: Quality and throughput comparison of garment-conditioning strategies under the paired setting. Cached stores garment KV features in the global appearance memory, whereas Repeated Encoding recomputes garment conditioning at every rolling update.
Figure 7: Long-sequence comparison for a white blouse under substantial pose and scale changes.
Figure 8: Long-sequence comparison for denim shorts across large viewpoint changes.
Figure 9: Long-sequence comparison for a dark knit top across front and rear views.
Figure 10: Long-sequence comparison for black shorts. MagicTryOn exhibits an appearance shift in later frames, whereas LiveVVT maintains consistent garment color and texture.
Figure 11: Unpaired long-sequence try-on examples across diverse garment categories and viewpoint changes. LiveVVT preserves garment appearance through full-body rotations and extended rolling generation.
Figure 12: Additional unpaired long-sequence examples with substantial viewpoint and camera-scale changes. LiveVVT maintains a coherent dressed appearance after rear views, self-occlusion, and abrupt zooming.
Figure 13: Challenging in-the-wild try-on examples with complex backgrounds, rapid motion, and severe self-occlusion. LiveVVT retains robust garment appearance and temporal continuity across these unconstrained sequences.
More in this topic

Virtual Try-On

Virtual try-on generates images or videos showing people wearing selected garments while preserving appearance, fit, pose, and identity.

See all 55 papers

Explore similar work

CardsList
Aug 6, 2026

UniVVT: A Unified End-to-End Framework for High-Fidelity Video Virtual Try-on

Video Virtual Try-On (VVT) synthesizes a video of a person wearing a target garment while preserving identity, motion, and scene dynamics. Dominant approaches cast VVT as mask-conditioned video inpainting and rely on separate modules for human parsing, pose estimation, and garment warping. This multi-stage design complicates deployment and, more critically, allows errors in explicit geometric priors to propagate irreversibly into the generated video. We present UniVVT, a unified end-to-end framework that reframes VVT as semantically conditioned video generation, eliminating mask, pose, and warping modules at inference. At its core, a scene-task perceiver built on a Multimodal Large Language Model jointly encodes the source video, target garment, and task instruction into compact, task-aware latent tokens, implicitly capturing what to transfer and where and how to transfer it. A lightweight semantic bridge then aligns these tokens with the conditioning space of a diffusion-based video generator, enabling coherent garment transfer. To robustly couple the heterogeneous components, we devise a three-stage progressive training strategy comprising semantic alignment, joint task adaptation, and flexible-resolution refinement. Extensive experiments demonstrate that UniVVT achieves state-of-the-art performance across multiple benchmarks, validating implicit semantic guidance as a simple and effective alternative to fragile geometric preprocessing for end-to-end virtual try-on.

Yushe Cao, Shikun Feng, Fei Shen +5

Tsinghua University · Zhongguancun Academy · National University of Singapore +2

Virtual Try-OnIdentity-Preserving Video GenerationControllable Video Generation
AskPDF
Aug 14, 2026

InstructVVT: Instruction-Driven Video Virtual Try-On without Auxiliary Spatial Priors

Video virtual try-on is a highly constrained editing task requiring the precise replacement of a target person's clothing while strictly preserving the original video's spatial structure and temporal dynamics. Existing methods heavily rely on auxiliary handcrafted spatial priors (e.g., masks, poses) for editing control. However, these priors are prone to failure in unconstrained real-world videos and often compress rich visual context into incomplete structural signals. Furthermore, standard reconstruction objectives fail to fully capture try-on-specific human preferences. To address these challenges, we propose InstructVVT, an instruction-driven and reference-guided video virtual try-on framework based on a Diffusion Transformer (DiT) that operates without inference-time spatial priors. Our core insight is to recover fine-grained control directly from the input triplet (source video, reference garment, and instruction) via a dual-level reference conditioning scheme. Specifically, an MLLM infers semantic edit tokens for target disambiguation and structural preservation, while a lightweight conditioning pathway explicitly injects fine-grained visual garment details. Finally, we design a try-on-specific reward and utilize the DiffusionNFT algorithm to align the model with human preferences. Extensive experiments on ViViD-S and TripVVT-Bench demonstrate that InstructVVT outperforms state-of-the-art open-source methods in garment fidelity, structural preservation, and temporal consistency, despite requiring fewer inference-time controls.

Dingbao Shao, Song Wu, Xinyu Chen +18

Virtual Try-OnInstruction-Based Video EditingVideo Diffusion ModelsTemporal Consistency in Video Generation
AskPDF
Aug 31, 2026

FlowVVTON: Flow-Guided Mask-Free Video Virtual Try-On

Video virtual try-on aims to transfer a target garment onto a moving person across video frames. Current methods rely on human parsing masks or pose keypoints that frequently fail under large motions and occlusions, causing boundary artifacts and temporal inconsistency. A further limitation is that most approaches rely solely on attention mechanisms for temporal modeling, providing no explicit motion supervision. We propose FlowVVTON, a mask-free framework that eliminates parsing mask dependency entirely. Optical flow is used solely as a training-time supervision signal: a flow-warped latent loss, applied across all layers of the generation model, enforces multi-scale temporal consistency by aligning adjacent-frame features under explicit physical motion constraints. A two-stage training strategy establishes mask-free spatial alignment before introducing flow-guided temporal supervision. Experiments on TikTokDress show that FlowVVTON outperforms baselines by substantial margins, particularly in temporal consistency (5.7×\times VFID-R improvement over SwiftTry), while requiring no segmentation masks, pose keypoints, or region annotations at any stage.

Shengyao Chen, Xianbing Sun, Liqing Zhang +1

Shanghai Jiao Tong University, Shanghai, China

Virtual Try-OnTemporal Consistency in Video Generation
AskPDF