cs.CVNov 10, 2026

TaoMate-R1: Temporal Role Specialization for Streaming Multi-Reference Video Generation

Authors: Qijun Gan, Chenwei Zhang, Xinyang Li, Meiguang Jin, Junfeng Ma

Organizations: Alibaba Group

Abstract

Streaming multi-reference video generation establishes a scene from specified people and objects, then continues it under changing instructions. We introduce TaoMate-R1, a streaming multi-reference generator that organizes adaptation parameters through temporal role specialization. It assigns separate low-rank adapters to scene establishment and continuation on a frozen backbone. Each adapter performs both denoising and context writing for its role; Self Gradient Forcing jointly trains the roles through the history they construct. This allocation contrasts with operation-based specialization, which separates denoising from context writing across all chunks. A fixed-replay gradient analysis characterizes the update directions admitted by the two sharing schemes. On 32 multi-reference cases, we compare the schemes at the same two-adapter budget. Across ten approximately two-minute streams with changing instructions and shared teacher-audio conditioning, TaoMate-R1 reduces the mean boundary luminance jump from 1.96 to 0.613, with higher boundary feature similarity. It also achieves higher mean CLIP and DINO temporal similarity across short interactions, long streams, and public reference compositions. These results support grouping adaptation parameters by scene establishment and continuation for streaming multi-reference generation. Our code and model are released at https://github.com/TaoLiveAIGC/TaoMate-H3.

Figures & tables

Explore similar work

CardsList
  1. ReferStream: Interactive Subject Injection for Streaming Video Generation

    Oct 27, 2026Shuolin Xu, Guanyuan Li, Boyu Li +5Subject-Driven Video GenerationStreaming Video Generation

  2. TokenStream: Multi-Subject Personalized Video Generation with Token-wise LoRA and Guided Attention

    Nov 12, 2026Hung Nguyen, Quang Qui-Vinh Nguyen, Khoi NguyenIdentity-Preserving Video GenerationSubject-Driven Video Generation

  3. Stream-T1: Test-Time Scaling for Streaming Video Generation

    May 6, 2026Yijing Tu, Shaojin Wu, Mengqi Huang +4Streaming Video GenerationTest-Time Scaling