MSR: Multiple Subject Reference for Video Generation

Guannan Li; Jiaji Chen; Jingyuan Liao; Yu Geng; Baolan Qiu

cs.CVSep 16, 2026

MSR: Multiple Subject Reference for Video Generation

Authors: Guannan Li, Jiaji Chen, Jingyuan Liao, Yu Geng, Baolan Qiu

Organizations: Licon Studio

Abstract

Conditioning a video generator on multiple images requires preserving appearance while associating each reference with its intended role. We present MSR (Multiple Subject Reference), a slot-aware conditioning scheme for LTX-based video generation. Each reference image is independently encoded as a static clip and represented by a separate latent-token group. A compact Fourier-feature multilayer perceptron adds a numeric slot embedding, while slot-dependent temporal offsets modify the group's rotary coordinates. The reference groups are prepended to noisy target tokens and serve as clean context during target-only flow-matching training. We implement this scheme through low-rank adaptation and release the resulting weights and inference workflows. Qualitative examples demonstrate compositions containing distinct characters and referenced environments in realistic and stylized scenes. Development observations suggest reduced reference confusion relative to an earlier continuous-reference baseline, while similar clothing, complex garments, and viewpoint changes remain challenging. We describe the conditioning mechanism, the retained training configuration, and the observed strengths and limitations of the released system. A supplementary audio-reference experiment adds voice conditioning while keeping the visual parameters frozen.

Figures & tables

Panel of Figure 1 — Figure 1: MSR visual conditioning. Each reference is encoded separately, tagged in latent-token space, and assigned a slot-dependent positional offset. Clean reference tokens precede noisy target tokens. Only the target suffix contributes to the training loss. The diagram abstracts the existing transformer and its text-conditioning path.

Property	Earlier MSR V1/V2	Released visual MSR
Training backbone	LTX	LTX
Reference encoding	Continuous pseudo-video of input images	Independent static clip per image
Learned image-slot MLP	Absent	Present
Slot-dependent negative offset	Absent	Present

Table 1: Version definitions. Both variants use the same LTX training backbone. Comparisons in development used the same inference backbone; this table describes the combined version changes, not isolated ablations.

Reference images per example	2	3	4	5	6	7	8
Share of examples (%)	20.35	25.99	29.53	23.89	0.12	0.08	0.04

Table 2: Distribution of reference-image counts in the visual-stage manifest, expressed as percentages of examples. Reference counts refer to image slots, not the number of subjects or views inside an image. Percentages are rounded to two decimal places and may not sum to 100%. Unique file paths do not establish content-level deduplication.

Setting	Retained visual-stage configuration
Backbone	LTX (22B)
Text encoder	Gemma 3 12B IT
LoRA rank / scaling $\alpha$ / dropout	128 / 128 / 0
Slot MLP	$33\rightarrow 256\rightarrow 128$ , SiLU, 41,600 parameters
Optimizer / learning rate	AdamW / $10^{-4}$
Schedule / precision	Cosine / BF16

Table 3: Settings recorded for the stage that produced the released visual adapter. The exact public checkpoint step and a full historical schedule are not reconstructed from a later resume configuration.

Panel of Figure 2 — Figure 2: Public visual MSR case 03. Top: two character references and a scene reference. Bottom: full generated frames sampled at approximately 1.83, 6.04, and 10.25 seconds. The images show distinguishable appearance cues and the referenced setting across changes in framing. Frames are not cropped; the composite views in each character reference were already present in the input image.

Condition	Observed behavior
Simple clothing, props, scenes	Reference appearance was often maintained in tested examples.
Similar character appearance or clothing	Subject blending and clothing exchange could occur.
Turning or returning after a cut	Appearance could become inconsistent after the viewpoint change.
Complex overlapping garment structures	Sleeve and skirt structures could merge or be misinterpreted.

Table 4: Qualitative development observations. No event counts or failure probabilities were retained, so the rows do not imply measured rates.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 4 — Figure 4: Additional public visual MSR case 07. Top: reference images. Bottom: three full frames from the published clip. Framing changes emphasize different characters, while several salient appearance cues remain visible. The subtitles are part of the supplied output. The figure does not evaluate the later audio-reference extension.

Video Generation

Video generation studies models that create and edit moving images from text, images, audio, and structured controls.

See all 363 papers

Rethinking Position Embedding as a Context Controller for Multi-Reference and Multi-Shot Video Generation
Apr 4, 2026Binyuan Huang, Yuning Lu, Weinan Jia +3Multi-Shot Video Generation Identity-Preserving Video Generation
Wuhan University · University of Science and Technology of China · Hong Kong University of Science and Technology +1
Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment
Jul 5, 2026Zixiang Zhou, Zhentao Yu, Yifeng Ma +9Video Diffusion Models Identity-Preserving Video Generation
Tencent Hunyuan
UniCustom: Unified Visual Conditioning for Multi-Reference Image Generation
May 12, 2026Yiyan Xu, Qiulin Wang, Wenjie Wang +5Subject-Driven Image Generation Image Generation
University of Science and Technology of China · Kling Team, Kuaishou Technology

Apr 4, 2026J/K move · Enter open · S save

Rethinking Position Embedding as a Context Controller for Multi-Reference and Multi-Shot Video Generation

Binyuan Huang, Yuning Lu, Weinan Jia, Hualiang Wang+2

Wuhan University · University of Science and Technology of China · Hong Kong University of Science and Technology+1

Multi-Shot Video Generation Identity-Preserving Video Generation Positional Encoding Temporal Consistency in Video Generation

Open paper AskPDF arXiv

Recent proprietary models such as Sora2 demonstrate promising progress in generating multi-shot videos conditioned on multiple reference characters. However, academic research on this problem remains limited. We study this task and identify a core challenge: when reference images exhibit highly similar appearances, the model often suffers from reference confusion, where semantically similar tokens degrade the model's ability to retrieve the correct context. To address this, we introduce PoCo (Position Embedding as a Context Controller), which incorporates position encoding as additional context control beyond semantic retrieval. By employing side information of tokens, PoCo enables precise token-level matching while preserving implicit semantic consistency modeling. Building on PoCo, we develop a multi-reference and multi-shot video generation model capable of reliably controlling characters with extremely similar visual traits. Extensive experiments demonstrate that PoCo improves cross-shot consistency and reference fidelity compared with various baselines.

Abstract

Video Generation

Explore similar work

Rethinking Position Embedding as a Context Controller for Multi-Reference and Multi-Shot Video Generation

Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment

UniCustom: Unified Visual Conditioning for Multi-Reference Image Generation