We introduce TokenStream, an optimization-based framework that composes independently learned subject identities in text-to-video generation without a reference-video training dataset or joint optimization of each new subject combination. Learning from a single reference image makes subject appearance difficult to separate from static-image bias, while merging independently trained adapters can mix identities. Token-wise LoRA (ToRA) restricts low-rank updates to the cross-attention value projections of subject-associated text tokens, providing a separate adaptation path for each subject. A T2V-guided stream copies queries and keys from the frozen base model during early denoising to reuse its layout and motion priors while retaining personalized values. We also introduce Multi-DB, a 30-subject, 122-prompt benchmark with appearance-free prompts and varied interactions. On Multi-DB at 1.3B, TokenStream improves identity preservation from 0.422 for DreamBooth to 0.467 and textual alignment from 0.652 to 0.688.
Figures & tables
Figure 1: Multi-subject video personalization with TokenStream. TokenStream uses a single reference image per subject and composes independently trained subject adapters at inference time. Compared with Textual Inversion and DreamBooth-style adaptation, it better preserves subject appearance while retaining the motion and prompt-following behavior of the pretrained T2V model.
Figure 2: Overall framework of TokenStream. Subject adapters are learned independently and applied only to their text-token spans in the value projection. The frozen T2V-guided stream transfers Q,K attention features during the first TW denoising steps; the personalized stream retains its values and continues for all T steps. Each model evaluation traverses L self-attention–cross-attention–FFN blocks. Both streams share frozen weights but incur additional guidance computation.
Figure 3: What Multi-DB adds. Appearance-free prompts test reference identity; richer interactions and four motion-difficulty levels stress composition and dynamics. The examples and table compare prompt design with OpenS2V-Eval.
Table 4
Figure 4: Qualitative comparisons on Multi-DB. Selected examples illustrate subject binding and prompt-conditioned composition. They do not establish reliable articulated motion or success on every scene; omission, identity mixing, and same-class failures are shown in the supplement.
Setting
MS
MA
TA ↑
IP ↑
Components
TI, no guidance
0.996
0.039
0.664
0.390
ToRA, no guidance
0.978
0.022
0.641
0.420
TI + guidance
0.898
0.185
0.455
0.446
ToRA + guidance
0.878
0.199
0.689
0.469
Guided attention layers
Table 3: Ablation studies. Each block is an independent sweep. MS/MA are descriptive; higher TA/IP are better. Without ToRA, TI is used.
Method
VQ
MQ
TA
IP
Textual Inv.
0.8
0.8
0.8
0.0
DreamBooth
5.7
10.6
10.6
4.1
Phantom
10.6
11.4
10.6
12.2
VACE
12.2
13.0
10.6
8.1
TokenStream
70.7
64.2
65.9
75.6
Table 4: User-study preferences (%). Ten participants.
Figure 5: Guidance-window trade-off. Textual alignment improves up to 20 guided steps, then declines. Identity preservation decreases overall with longer guidance, while sampling latency increases.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Phase
Time
Optimization (ToRA, per subject)
∼ 4 min
Inference (T2V-guided stream)
∼ 1 min 50 s
Appendix
Table 6: Runtime of ToRA optimization and T2V-guided inference on a single H100.
Figure 6: Success case under moderate shape mismatch. TokenStream preserves the long legs of a Flamingo despite mismatch with the generic “bird” prior.
Figure 7: Failure cases under stronger prior mismatch. When the subject geometry or expected dynamics strongly differs from the generic T2V prior, the guided stream can bias the coarse layout or motion. These examples illustrate the limitation discussed in the main paper.
Method
Ref.-video dataset
Cost per new subj.
Store /subj.
Latency
Peak mem.
Phantom
required
feed-fwd
—
≈ 2m10s
≈ 24 GiB
VACE
required
feed-fwd
—
≈ 2m15s
≈ 24 GiB
Textual Inv.
none
∼ 4 min
8 KB
≈ 2m10s
≈ 24 GiB
DreamBooth
none
∼ 4 min + merge
38 MB
≈ 2m10s
≈ 24 GiB
Ours
none
∼ 4 min
5.6 MB
2m11s
23.74 GiB
Appendix
Table 7: Cost comparison on one H100 with Wan2.1-T2V-1.3B, 50 steps, 832×480 , 81 frames ( TW=10 for TokenStream). Entries marked ≈ are estimated, not measured; the TokenStream row is the mean of three timed runs.
Setting
TA ↑
IP ↑
Missing ↓
Latency
Peak mem.
N=2
0.620
0.487
0.189
130.95 s
23.742 GiB
N=3
0.611
0.328
0.405
128.81 s
23.745 GiB
N=4
0.494
0.282
0.425
131.52 s
23.747 GiB
N=6
0.574
0.218
0.560
132.33 s
23.753 GiB
Same class, N=2
0.675
0.515
0.150
128.69 s
23.742 GiB
with role phrases
0.652
0.431
0.234
130.20 s
23.742 GiB
Appendix
Table 8: Composing N independently trained ToRAs, and the same-class case. “Missing” is the mean fraction of undetected requested subject–frame pairs.
Figure 8: Composing many subjects, four frames from the median-IP clip of each group. Requested subjects are missing and the remaining ones fuse together, while the main benchmarks target two to three non-human subjects.
Figure 9: Two subjects of the same class (“ dog1 and dog2 play together in a yard”). The two instances are rendered as one fused animal rather than two identities. We do not claim composability in this setting.
Method
Prompt
TA ↑
IP ↑
Textual Inv.
appearance-free
0.726
0.423
appearance-revealing
0.777
0.536
DreamBooth
appearance-free
0.683
0.651
appearance-revealing
0.819
0.702
Ours
appearance-free
0.646
0.668
appearance-revealing
0.858
0.634
Appendix
Table 9: Prompt-sensitivity diagnostic on two-subject scenes. One video per cell, so this illustrates prompt sensitivity rather than a ranking.
Method
Bkg. sim. to ref. ↓
Subject detection rate
Wan2.1 (base)
0.121
0.758
DreamBooth
0.261
0.867
Ours
0.142
0.708
Appendix
Table 10: Background similarity between generated non-subject regions and the reference background. The base model does not receive the reference and provides an empirical comparison level.
Subject-driven and multi-element video generation are central to controllable video synthesis, but existing methods still struggle to preserve identity consistency and model complex relationships among multiple subjects. In this paper, we propose Aura, a unified framework for high-fidelity and identity-consistent video generation. To better capture scene dynamics and subject interactions, we introduce AI director-level captions that provide dense and structured descriptions of video content. We further leverage a vision-language model (VLM) with learnable queries to extract multimodal semantic features from textual and visual references, covering both global semantics and fine-grained visual cues. To bridge the representational gap between the VLM and the Diffusion Transformer (DiT), we design a two-stage alignment strategy that progressively maps VLM features into the DiT feature space. For visual conditioning, we adopt token concatenation to inject reference information directly into the generation process. To distinguish heterogeneous subject types and reduce common copy-paste artifacts, we develop a subject-aware RoPE-Shift mechanism. To further differentiate reference images of different categories, we introduce subject-aware learnable tokens. In addition, we introduce Memory Tokens to balance the training signal across examples with different numbers of reference subjects. During inference, Progressive-APG (Adaptive Prompt Guidance) further alleviates oversaturation and improves semantic alignment with user prompts. Finally, we build a high-quality video-subject image dataset through a dedicated data construction pipeline. Extensive experiments show that our method achieves state-of-the-art performance on both single-subject generation and more challenging multi-element scenarios.
Recent advances in diffusion models have significantly improved text-to-video generation, enabling personalized content creation with fine-grained control over both foreground and background elements. However, precise face-attribute alignment across subjects remains challenging, as existing methods lack explicit mechanisms to ensure intra-group consistency. Addressing this gap requires both explicit modeling strategies and face-attribute-aware data resources. We therefore propose LumosX, a framework that advances both data and model design. On the data side, a tailored collection pipeline orchestrates captions and visual cues from independent videos, while multimodal large language models (MLLMs) infer and assign subject-specific dependencies. These extracted relational priors impose a finer-grained structure that amplifies the expressive control of personalized video generation and enables the construction of a comprehensive benchmark. On the modeling side, Relational Self-Attention and Relational Cross-Attention intertwine position-aware embeddings with refined attention dynamics to inscribe explicit subject-attribute dependencies, enforcing disciplined intra-group cohesion and amplifying the separation between distinct subject clusters. Comprehensive evaluations on our benchmark demonstrate that LumosX achieves state-of-the-art performance in fine-grained, identity-consistent, and semantically aligned personalized multi-subject video generation. Code and models are available at https://jiazheng-xing.github.io/lumosx-home/.
Jiazheng Xing, Fei Du, Hangjie Yuan +7
Zhejiang University · National University of Singapore · DAMO Academy, Alibaba Group +1
Autoregressive video models can generate minute-long videos in real time, but they produce generic subjects from text rather than specific subjects from user-provided images. Existing customization methods either require costly per-subject optimization or use pretrained conditioning networks that jointly process all video frames with bidirectional attention. Neither approach is designed for causal streaming. We present Custom Forcing, a training-free method that stores reference-based anchor frames in the persistent KV cache of a frozen autoregressive video model. However, fixed anchors face two limitations: simple conditioning allows identity to drift, and the text prompt continues to favor a generic subject. To address these problems, drift-adaptive value amplification (DVA) scales reference influence with the degree of identity drift, while anchor contrast guidance (ACG) steers generation away from the generic class prior. Over two-minute rollouts, fixed anchors fall from 0.58 to 0.42 in DINO-I, while Custom Forcing keeps it between 0.58 and 0.62 without reducing motion. Custom Forcing also achieves higher subject similarity than bidirectional customization methods and better preserves identity over 30s than causal image-to-video and reference-to-video models, while generating each frame 9.5-28.5 times faster than these long-video baselines.
Yunseung Ok, Hyunsoo Kim, Minseo Kim +1
Kyung Hee University · The University of Texas at Austin