Omni reference-to-video (R2V) generation is emerging as a general paradigm that enables flexible video customization from diverse and compositional visual references. However, existing benchmarks fall short of these emerging capabilities: their test cases cover only a limited range of reference types and compositions, and their evaluation protocols largely assess holistic reference consistency, overlooking whether reference factors are properly preserved, disentangled, and routed. Meanwhile, the high cost of constructing omni R2V training data makes suitable training resources scarce. To address these gaps, we introduce OmniVBench and the Omni-R2V Dataset, establishing a shared foundation for evaluating and training omni R2V models. OmniVBench expands R2V evaluation across broader reference types, richer reference compositions, and more flexible instruction-driven customization, covering 7 task families and 18 fine-grained tasks. Complementing this task design, we introduce factor-grounded evaluation with 12,172 case-specific checklist items, explicitly assessing whether intended reference factors are faithfully preserved, correctly disentangled and bound to their targets, and realized according to the instruction. We further introduce the Omni-R2V Dataset, bringing industrial-grade training resources for diverse R2V tasks to the broader research community. Drawing primarily on a large-scale corpus of professional video footage, it comprises 340K processed training samples spanning diverse reference types and multi-reference compositions. Specifically, we develop task-specific pipelines for reference-target pair construction, offering a practical and scalable recipe for omni R2V data construction. Extensive evaluation of advanced open- and closed-source R2V models reveals substantial performance variation across diverse tasks and distinct capability patterns across evaluation dimensions.
Figures & tables
Figure 1: Overview of OmniVBench and the Omni-R2V Dataset .
Benchmark
Content Ref.
Motion Ref.
Style Ref.
Structure Ref.
Narrative Ref.
Multiple Refs.
Object
Character
Scene
Action
Camera Motion
Style
Greybox
Line Art
Rough Storyboard
Multi-Panel Storyboard
Story
Preceding- Shot
Multi- Content
Cross- Aspect
OpenS2V-Eval Yuan et al. [2025]
✓
✓
✓
VACE-Bench Jiang et al. [2025]
✓
✓
✓
✓
✓
UniVBench Wei et al. [2026b]
✓
✓
✓
✓
IntelligentVBench Pan et al. [2026]
✓
✓
✓
✓
FashionVideoBench Wu et al. [2026]
✓
✓
✓
✓
✓
Table 1: Comparison of reference task coverage between OmniVBench and existing reference-based video generation benchmarks.
Dataset
# Size
Reference Modality
Reference Task Coverage
Processed Data Provided
Image
Video
Content
Motion
Style
Structure
Narrative
Multi- Content
Cross- Aspect
OpenS2V-5M [ Yuan et al., 2025 ]
5.4M
✓
✓
✓
✓
Phantom-Data Chen et al. [2025c]
1M
✓
✓
✓
MuSS Zhang et al. [2026a]
30K
✓
✓
Omni-R2V (Ours)
340K
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
Table 2: Comparison of Omni-R2V Dataset with existing R2V datasets in terms of scale, reference modalities, reference task coverage, and processed data provision (i.e., whether processed samples are directly provided without requiring users to download and process raw source videos).
Figure 2: Representative reference–target pairs from the Omni-R2V Dataset.
Figure 3: Representative examples of OmniVBench. Each category encompasses fine-grained generation tasks defined by distinct reference types and control objectives, with examples illustrating the reference inputs and corresponding generated outputs.
Figure 4: Statistics of OmniVBench. We present the distribution of benchmark samples from multiple perspectives, including (a) the word cloud of textual prompts, (b) prompt length distributions across task categories, (c) reference video length distribution, and (d) the number and modality composition of reference inputs per sample.
Figure 5: Overview of the Omni-R2V Dataset construction pipeline.
Model
Content
Motion
Style
Structure
Narrative
Multi- Content
Cross- Aspect
Overall
Open-Source Models
UniVideo [ Wei et al., 2026a ]
66.01
44.52
47.40
48.84
36.97
58.00
46.09
49.69
OmniWeaving [ Pan et al., 2026 ]
62.23
50.58
41.40
63.19
38.03
47.49
39.25
48.88
LoomVideo [ Wu et al., 2026 ]
61.24
52.09
42.03
62.53
46.98
49.02
44.86
51.25
Bernini [ Liu et al., 2026 ]
69.27
55.92
50.62
69.44
42.02
60.32
51.03
56.95
MiniMax H3 [ MiniMax, 2026 ]
78.86
65.32
66.24
72.94
73.90
77.22
72.36
72.41
Table 3: Fine-grained comparison of different models on OmniVBench. Each task score is averaged over Reference Fidelity (RF), Instruction Realization (IR), and Visual Quality (VQ). The first-place and second-place results are highlighted accordingly.
Figure 6: Model performance across RF, IR, and VQ.
Figure 7: Fine-grained comparison of Reference Fidelity (RF) across R2V task families.
Model
Content
Motion
Style
Structure
Narrative
Multi-Content
Cross-Aspect
IRDR
IRTC
IRDR
IRTC
IRDR
IRTC
IRDR
IRTC
IRDR
IRTC
IRDR
IRTC
IRDR
IRTC
UniVideo Wei et al. [2026a]
62.32
73.47
57.27
70.14
15.39
50.78
71.13
82.80
26.03
33.62
57.48
54.68
53.23
53.92
OmniWeaving Pan et al. [2026]
78.36
85.87
39.95
51.05
59.33
67.13
67.23
92.09
23.78
36.35
48.55
67.66
40.55
54.20
LoomVideo Wu et al. [2026]
87.91
88.65
57.28
68.38
89.64
68.42
76.62
93.95
58.88
37.48
65.00
74.09
55.83
63.27
Bernini Liu et al. [2026]
83.19
91.90
72.65
61.46
68.15
72.58
77.54
97.11
28.57
37.81
65.77
78.01
55.94
64.08
MiniMax H3 MiniMax [2026]
98.21
97.28
94.14
94.99
91.94
85.65
95.42
98.14
95.05
89.47
95.93
97.80
93.61
90.71
Table 4: Fine-grained comparison of Instruction Realization (IR) on OmniVBench. IRDR measures reference-factor disentanglement and routing, while IRTC measures target compliance.
Table 12
Appendix figures & tables14 assets
Supplementary material from the paper’s appendix.
Appendix
Figure A.1: Prompt statistics of Omni-R2V Dataset .
Task Family
Sub-Task
# Cases
Content
Object
33
Character
54
Scene
25
Subtotal
112
Motion
Action
47
Camera Motion
48
Appendix
Table A.1: Detailed composition of OmniVBench. We report the number of evaluation cases for each sub-task.
L2 Sub-dimensions
Evaluation criteria
Content Fidelity
Entity/Scene Identity Fidelity; Surface Fidelity; Part Fidelity
Table A.3: Instruction Realization (IR) L2 sub-dimensions and evaluation criteria.
L2 Sub-dimensions
Evaluation component
Technical Quality
Technical branch of DOVER++ Wu et al. [2023]
Aesthetic Quality
Aesthetic Predictor V2.5 discus0434 [2024]
Physical Plausibility
Coherence/Physics dimension of UnifiedReward 2.0 Wang et al. [2025b]
Appendix
Table A.4: Video Quality (VQ) L2 sub-dimensions and automatic evaluation components.
Score
Instruction Realization
Reference Fidelity
1
The instruction is not executed, or the output is unrelated to the requested task.
The required reference information is absent, or the output has no meaningful correspondence with the reference.
2
The output shows an attempt to follow the instruction but largely fails or introduces incorrect entities.
The required reference information is only weakly preserved, with substantial deviations that make the correspondence difficult to recognize.
3
The instruction is partially fulfilled, but key requirements are missing or incorrectly realized.
The required reference information is recognizable but only partially preserved, with clear inconsistencies or errors.
4
The main requirements are fulfilled, with minor deviations in details.
The required reference information is largely preserved, with minor local or temporal inconsistencies.
5
All applicable instruction requirements are fulfilled, including specified entities, attributes, positions, durations, and quantities.
The required reference information is faithfully preserved throughout the relevant portions of the video, with no substantive inconsistencies.
Appendix
Table A.5: Five-point rating criteria for human evaluation of Instruction Realization (IR) and Reference Fidelity (RF). Only requirements applicable to each sample are assessed.
Criterion
Content
Motion
Style
Structure
Narrative
Multi-Content
Cross-Aspect
Avg.
Reference Usability
97.5
90.0
100.0
97.0
92.5
96.0
94.5
95.4
Reference–Target Consistency
100.0
92.5
98.0
98.5
98.5
96.0
95.5
97.0
Instruction Correctness
94.5
86.5
96.5
94.5
99.0
94.0
94.5
94.2
Overall
97.3
89.7
98.2
96.7
96.7
95.3
94.8
95.5
Appendix
Table A.6: Human evaluation of Omni-R2V Dataset quality across the seven task families. Each entry reports the average Pass rate (%) of two annotators.
Figure A.2: Example of factor-grounded evaluation on a multi-reference case, showing the case-specific RF/IR checklists and corresponding model scores.
Figure A.3: Qualitative comparison on the content reference task.
Figure A.4: Qualitative comparison on the structure reference task.
Figure A.5: Qualitative comparison on the narrative reference task.
Figure A.6: Qualitative comparison on the narrative reference task.
Figure A.7: Qualitative comparison on the style reference task.
Figure A.8: Qualitative comparison on the cross-aspect reference task.
Audio-video generation using heterogeneous multimodal references has emerged as a new challenge, requiring both compositional control over generation and grounded understanding of multimodal context. In this paper, we introduce ORAV Bench for Omni Reference Audio-Video Generation, comprising 380 task instances with 2-10 references, 9 semantic roles, and 30 role compositions. Instructions specify the relationships among references; the media supply the identities, dynamics, and audio characteristics to be realized. To evaluate these open-ended outputs, we develop a reference-aware pairwise protocol that prepares visual and auditory evidence, compares the intended contribution of each reference, and checks the overall verdict in both presentation orders. On held-out instances, it achieves 86.08% effective agreement with human judgments. Across 5 frontier systems, overall rankings conceal distinct strengths across reference compositions. A recurring failure is to reproduce unintended source content in place of the requested result, despite closely resembling a reference. Reproducible pointwise diagnostics of quality, reference affinity, and speech reveal distinct dimensions of model behavior. ORAV thus offers a benchmark for tracking progress toward controllable, compositional, and reference-faithful audio-video generation.
Creative intent in video editing can be expressed through text instructions (e.g., "remove the people"), image references (e.g., subject appearance), video references (e.g., camera movement), or a combination of the above. We introduce OmniXRef, a unified framework for multimodal multi-reference video editing that allows users to combine variable numbers of text, image, and video references to control diverse attributes of a source video. We first devise a unified input design with a reference-bank interface and introduce R2-RoPE to support generalization to more references. Then our model, OmniXRef-Model, uses a single-stream diffusion transformer to jointly process the source video and multimodal context from an MLLM in an interleaved sequence. To address the scarcity of training data for video-reference editing, we construct OmniXRef-Data using synthesis pipelines for different editing tasks and combinations of references. We use curriculum learning to progressively increase task complexity and resolution. To support systematic evaluation of video as a control modality, we introduce OmniXRef-Bench, a benchmark for video-reference attribute transfer across Camera, Motion, Content, Style, and Visual Effects (VFX). Comprehensive evaluations show that our OmniXRef-Model outperforms task-specific baselines and commercial models.
Multi-reference-to-audio-video (MR2AV) generation aims to generate coherent audio-video content conditioned on multiple references and textual instructions. Existing benchmarks mainly focus on text-driven generation, single-reference subject preservation, or isolated audio-video alignment, leaving the emerging MR2AV setting largely unexplored. Compared with these settings, MR2AV requires models to jointly reason over multiple references while generating synchronized visual and audio content. Models must not only preserve each reference faithfully but also correctly bind and compose multiple referenced entities into coherent audio-visual events. To address this gap, we introduce MultiRef-Compass, a unified benchmark for MR2AV generation. It comprises 350 carefully curated samples constructed through a scalable and controllable asset-composition pipeline, covering multi-view subject preservation, multi-entity binding, and human-object-scene composition. To provide interpretable assessment, MultiRef-Compass defines an evaluation protocol with four dimensions: Basic Quality, Reference Consistency, Audio-Visual Consistency, and Instruction Following, using 14 sub-metrics. MultiRef-Compass integrates automatic metrics with a rejudging-enhanced MLLM-as-a-Judge framework, enabling scalable and auditable evaluation of both perceptual fidelity and reference-conditioned composition. Extensive experiments on eight representative MR2AV systems reveal substantial room for improvement across multiple evaluation dimensions, underscoring the need for a comprehensive benchmark and positioning MultiRef-Compass as a foundation for future MR2AV research.
Xiaohan Zhang, Yuqing Wen, Junlin Chen +9
Nanjing University · Kling Team · National University of Singapore +3