Current audio-driven video diffusion models can synthesize realistic talking-head videos but generally require dozens of diffusion denoising steps over multi-second video chunks, making them unsuitable for streaming applications such as voice assistants and virtual avatars, where low-latency and online causal generation are required properties. AURA generates video chunk by chunk with a rolling KV cache, requiring only the audio of the current chunk to start generating, rather than a multi-second audio look-ahead like bidirectional models. It comes in two sizes: AURA 1.3B streams 576p video in real time on a single GPU (20.7 fps for a 16 Hz playback rate), making it suited to interactive avatar applications, and AURA 5B streams 720p video at 9.7 fps. Our two-stage distillation pipeline first aligns the causal student with the teacher, by training it to match the teacher's trajectories with fewer diffusion steps and a chunk-wise sliding window causal attention mask, then applies Distribution Matching Distillation (DMD) with backward simulation to close the remaining quality gap. On standard talking-head benchmarks, AURA 1.3B runs 23x faster than its own bidirectional teacher and reaches a quality comparable to TalkVerse at a lower resolution, and AURA 5B matches its TalkVerse teacher at 720p while running 17x faster. To support reproducibility and further research on streaming avatar generation, we release the weights of both models along with a dataset of 30K synthetic videos and their precomputed teacher trajectories.
Figures & tables
Figure 1: AURA streams talking-head video chunk by chunk on a single GPU : AURA 5B at 720 p and 9.7 fps, AURA 1.3B at 576 p in real time ( 20.7 fps for 16 Hz playback). (Left) Two reference identities and a frame generated by each model at t=4 s, from the same reference image and audio. (Right) Quality–speed trade-off: FVD averaged over the Hallo3 test and CelebV-HQ test sets versus end-to-end time to generate 10 s of video on one H100 (log scale, lower is better). Circle areas are proportional to parameter counts, and the shaded band marks real-time generation. Video results can be found at kyutai.org/aura .
Figure 2: (Left) Sliding window chunk-wise causal attention mask used during training by the causal student. (Right) Per-chunk mixed denoising timestep training allows the use of a single shared KV cache across all denoising steps at inference.
Figure 3: Two-stage distillation pipeline . Stage 1 (top, ODE teacher forcing) : the frozen teacher generates a multi-step ODE trajectory from noise to clean latents, conditioned on a reference image, speech audio, and text prompt. A per-chunk mixed timestep state is sampled from this trajectory and fed to the causal student Gθ , which is trained to predict the corresponding clean latents under an MSE loss. Stage 2 (bottom, DMD with backward simulation) : the student generates a video in inference mode without gradients (backward simulation, left), and a per-chunk mixed-timestep state is sampled from its own denoising trajectory. The student is then re-run on this state with gradients enabled, and a DMD loss is computed by comparing the score predicted by a frozen teacher and an online fake score network (right). Gradients flow back through the gradient-enabled student pass only.
Method
Params
Res
Steps
DiT / VAE (fps) ↑
Total (fps) ↑
FID ↓
FVD ↓
JEDi ↓
CSIM ↑
Sync-C ↑
Sync-D ↓
WE ↓
Hallo3
14.5B
480p
50
0.24/3.2
0.22
42.8
183.4
0.180
0.74
5.06
9.9
0.0036
Wan-S2V
14B
720p
40
0.08/6.7
0.08
57.7
196.4
0.257
0.71
5.42
9.0
0.0032
TalkVerse
5B
720p
50
0.69/7.4
0.58
58.5
210.8
0.398
0.74
3.82
10.5
0.0019
Live Avatar
14B
480p
4
8.4/4.3
1.9
52.0
181.1
0.442
0.77
4.90
9.3
0.0039
AURA 5B (ours)
5B
720p
4
19.7/22.7
9.7
56.9
195.5
0.313
0.79
4.30
9.87
0.0032
AURA 1.3B (ours)
1.3B
480p
4
28.8/96.9
20.7
56.8
177.1
0.403
0.84
4.07
9.93
0.0036
Table 1: Main results on Hallo3 test . FVD and JEDi are averaged over the three temporal offsets ( @0 , @101 , @109 ); per-offset breakdowns are in the supplementary material (Tab. 8 ). WE : warping error. DiT / VAE : throughput of the diffusion transformer and VAE decoder alone, in frames per second on a single H 100 GPU. Total (fps): end-to-end throughput, including model offload and overhead. Bold : best; underlined : second best. AURA 1.3B outputs 480 p at 16 Hz (like Wan-S2V and Live Avatar), the other AURA and TalkVerse rows 25 Hz.
Method
Params
Res
Steps
DiT / VAE (fps) ↑
Total (fps) ↑
FID ↓
FVD ↓
JEDi ↓
CSIM ↑
Sync-C ↑
Sync-D ↓
WE ↓
Hallo3
14.5B
480p
50
0.24/3.2
0.22
39.1
202.5
0.243
0.71
6.23
9.4
0.0038
Wan-S2V
14B
720p
40
0.08/6.7
0.08
50.4
289.6
0.577
0.62
6.24
8.9
0.0061
TalkVerse
5B
720p
50
0.69/7.4
0.58
53.2
255.9
0.582
0.59
4.78
10.5
0.0059
Live Avatar
14B
480p
4
8.4/4.3
1.9
52.4
267.5
1.010
0.67
5.90
9.2
0.0057
AURA 5B (ours)
5B
720p
4
19.7/22.7
9.7
54.0
208.1
0.320
0.72
4.94
10.0
0.0043
AURA 1.3B (ours)
1.3B
480p
4
28.8/96.9
20.7
51.5
248.8
0.574
0.77
4.76
10.1
0.0046
Table 2: Main results on CelebV-HQ test . FVD and JEDi are averaged over the three temporal offsets ( @0 , @101 , @109 ); per-offset breakdowns are in supplementary material. Columns and naming conventions follow the same as in Table 1 .
Figure 4: Human preference study. Per-axis Bayesian Elo ( ↑ ) from 400 pairwise comparisons by 27 raters between AURA 5B, AURA 1.3B, TalkVerse and Hallo3 on the Hallo3 test clips; error bars are 95% CIs.
#Steps
FID ↓
FVD ↓
JEDi ↓
CSIM ↑
Sync-C ↑
Sync-D ↓
WE ↓
Sharp. ↑
DiT (fps) ↑
4
56.9
195.5
0.31
0.79
4.30
9.87
0.0032
50.4
19.7
3
55.4
198.8
0.33
0.77
4.39
9.77
0.0031
37.5
26.0
2
55.5
198.2
0.37
0.79
4.39
9.81
0.0031
36.5
38.5
1
58.6
229.2
0.66
0.65
4.26
9.71
0.0030
27.0
75.8
Table 3: Effect of the number of diffusion steps per chunk on quality and speed. DiT throughput is reported in frames per second on a single H 100 . FVD and JEDi are averaged over three temporal offsets ( @0 , @101 , @109 ). Other experiments use 4 steps. Best in bold .
Variant
FID ↓
FVD ↓
JEDi ↓
CSIM ↑
Sync-C ↑
Sync-D ↓
WE ↓
Sharp. ↑
Stage 1 only
68.3
264.8
0.58
0.55
3.55
10.07
0.0023
18.3
Stage 1 + Stage 2
56.9
195.5
0.31
0.79
4.30
9.87
0.0032
50.4
Table 4: Effect of Stage 2 on generation quality . Adding the DMD distillation stage substantially improves perceptual quality (FID, FVD, JEDi), identity preservation (CSIM), and lip-sync. Warp reaches lower values with Stage 1 only, but this is the same metric artifact on blurrier outputs as observed in Tab. 3 (see Sec. 4.4 ). Best value per column in bold .
Figure 5: Qualitative results from AURA on two diverse test sequences with different audios, generated by AURA 5B (top) and AURA 1.3B (bottom). Each row shows the reference identity image (left) and three frames sampled from the generated video at approximately t=1,3,5 s. The samples illustrate the diversity of inputs AURA handles, including varied lighting, head poses, and identities. Figure 1 shows both models on the same inputs; full videos, more results and comparisons with the baselines are on the project page .
Variant
FID ↓
FVD ↓
JEDi ↓
CSIM ↑
Sync-C ↑
WE ↓
Δ Col. ↓
DiT (fps) ↑
4 steps (default)
56.9
195.5
0.31
0.79
4.30
0.0032
0.4
19.7
5-step ( t=0 refresh)
57.7
196.7
0.33
0.78
4.26
0.0032
0.4
15.6
5-step ( t=50 refresh)
54.5
210.0
0.38
0.74
4.29
0.0038
3.1
15.6
5-step ( t=50 , used)
54.5
209.8
0.37
0.74
4.31
0.0038
3.1
15.6
Table 5: KV cache refresh strategy . Effect of an additional inference step to refresh it. Best value per column in bold .
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
Stage 1
Stage 2 (student)
Stage 2 (critic)
Training steps
120 K
≈833
≈4,167
Global batch size
256
256
256
Learning rate
1e-5
3.5e-7
3.5e-7
Optimizer
AdamW
AdamW
AdamW
Optimizer (β1,β2)
(0.9, 0.999)
(0.9, 0.999)
(0.9, 0.999)
Weight decay
0.01
0.01
0.01
Appendix
Table 6: AURA 5B training hyperparameters . Stage 1 trains the student via ODE teacher forcing; Stage 2 trains the student and critic jointly via DMD with backward simulation, alternating 5 critic updates per student update. All trainings use LoRA adapters with the optimizer settings below.
Teacher
Stage 1
Stage 2 (student)
Stage 2 (critic)
Data
23 K real Hallo3 clips
30 K teacher trajectories
92 K conditions
Training steps
60 K
40 K
≈1,083
≈5,417
Global batch size
256
256
256
256
Learning rate
1e-5
1e-5
2e-7
2e-7
Gradient clipping
—
—
0.5
Initialization
Wan2.1 T2V 1.3 B
Teacher
Stage 1 merged
Teacher
Appendix
Table 7: AURA 1.3B training hyperparameters. Same optimizer settings as Tab. 6 (AdamW, LoRA on attention, rank and α128 , bf16); 5 critic updates per student update in Stage 2.
Method
Params
Res
Steps
DiT / VAE (fps) ↑
Total (fps) ↑
FID ↓
FVD @ (0/101/109 ) ↓
JEDi @( 0/101/109 ) ↓
CSIM ↑
Sync-C ↑
Sync-D ↓
WE ↓
Hallo3
14.5B
480p
50
0.24/3.2
0.22
42.8
185.4 / 184.7 / 180.1
0.25 / 0.15 / 0.14
0.74
5.06
9.9
0.0036
Wan-S2V
14B
720p
40
0.08/6.7
0.08
57.7
186.0 / 203.1 / 200.2
0.22 / 0.28 / 0.27
0.71
5.42
9.0
0.0032
TalkVerse
5B
720p
50
0.69/7.4
0.58
58.5
222.6 / 212.1 / 197.5
0.40 / 0.43 / 0.37
0.74
3.82
10.5
0.0019
Live Avatar
14B
480p
4
8.4/4.3
1.9
52.0
179.8 / 183.8 / 179.7
0.30 / 0.51 / 0.52
0.77
4.90
9.3
0.0039
AURA 5B (ours)
5B
720p
4
19.7/22.7
9.7
56.9
198.1 / 188.2 / 200.2
0.51 / 0.22 / 0.21
0.79
4.30
9.87
0.0032
AURA 1.3B (ours)
1.3B
480p
4
28.8/96.9
20.7
56.8
172.3 / 179.9 / 179.2
0.45 / 0.35 / 0.41
0.84
4.07
9.93
0.0036
Appendix
Table 8: Per-offset breakdown on Hallo3 test , complementing the averaged version in the main paper (Tab. 1 ). FVD and JEDi are reported as triplets at temporal offsets at three different (temporal) frames: @0/@101/@109 . WE : warping error. DiT / VAE : throughput of the diffusion transformer and VAE decoder alone, in frames per second on a single H 100 GPU. Total (fps): end-to-end throughput, including model offload and overhead. Bold : best; underlined : second best (applied per offset within each triplet). AURA 1.3B outputs 480 p at 16 Hz (like Wan-S2V and Live Avatar), the other AURA and TalkVerse rows 25 Hz.
Method
Params
Res
Steps
fps ↑
FID ↓
FVD ( @0/@101/@109 ) ↓
JEDi ( @0/@101/@109 ) ↓
CSIM ↑
Sync-C/D
WE ↓
Hallo3
14.5 B
480 p
50
0.22
39.1
192.2 / 208.4 / 206.9
0.26 / 0.25 / 0.22
0.71
6.23 / 9.4
0.0038
Wan-S2V
14 B
720 p
40
0.08
50.4
294.3 / 288.7 / 285.9
0.66 / 0.52 / 0.55
0.62
6.24 / 8.9
0.0061
TalkVerse
5 B
720 p
50
0.58
53.2
233.2 / 268.9 / 265.5
0.33 / 0.66 / 0.75
0.59
4.78 / 10.5
0.0059
Live Avatar
14 B
480 p
4
1.90
52.4
237.2 / 281.4 / 283.9
0.48 / 1.26 / 1.29
0.67
5.90 / 9.2
0.0057
AURA 5B (ours)
5 B
720 p
4
9.7
54.0
182.0 / 219.4 / 223.0
0.33 / 0.27 / 0.36
0.72
4.94 / 10.0
0.0043
AURA 1.3B (ours)
1.3 B
480 p
4
20.7
51.5
243.1 / 251.0 / 252.2
0.57 / 0.53 / 0.62
0.77
4.76 / 10.1
0.0046
Appendix
Table 9: Per-offset breakdown on CelebV-HQ test , complementing the averaged version in the main paper. FVD and JEDi are reported as triplets at temporal offsets @0/@101/@109 . WE : warping error. Sync-C/D : Sync-C ( ↑ ) / Sync-D ( ↓ ). fps : end-to-end throughput on a single H100 GPU for a 10 s clip; per-component DiT and VAE values appear in the main paper. Bold : best; underlined : second best (per offset within each triplet).
Figure 6: Quality–speed trade-off across audio-driven avatar generation models . The horizontal axis reports end-to-end generation time for 10 s of video on a single H 100 GPU (log scale, lower is better). The vertical axis reports FVD ( left ) and JEDi ( right ), averaged over the Hallo3 and CelebV-HQ test sets (lower is better). Bubble area is proportional to the number of model parameters. AURA ( green ) achieves quality comparable to multi-step baselines at a fraction of their generation time.
Metric
0–10s
10–20s
20–30s
30–40s
40–50s
50–60s
Mean
AURA 5B
FVD ↓
182.1
206.0
202.8
198.3
202.2
195.2
197.8
CSIM ↑
0.764
0.735
0.720
0.710
0.657
0.689
0.713
Sync-C ↑
5.00
5.22
5.00
4.85
4.74
4.79
4.93
Warp ↓
3.50
3.44
3.45
3.43
3.99
3.53
3.56
AURA 1.3B
Appendix
Table 10: Long-form evaluation. AURA 5B and AURA 1.3B ( 4 steps, fixed KV cache) on the 100 Hallo3 test references driven by 60 s of speech; metrics per 10 s segment (FVD: mean of three windows per segment). Warping error in 10−3 .
Figure 7: One minute of streaming. (Top) Frames of one Hallo3 test clip at t=0,10,…,60 s for AURA 5B and AURA 1.3B, next to the reference image. (Bottom) FVD, CSIM and Sync-C per 10 s segment over the 100 references (Tab. 10 ).
Figure 8: RoPE re-indexing of the reference frame. (Top) just after a re-indexation, the reference (orange) sits 40 latent positions ahead of the generation front. (Middle) after 30 generated frames, the reference is 10 positions ahead — the minimum offset reached during training. (Bottom) the reference is shifted by +30 , restoring the maximum +40 offset. The offset thus cycles in [+10,+40] , staying within the training-time regime.
Variant
FID ↓
FVD ↓
JEDi ↓
CSIM ↑
Sync-C ↑
Warp ↓
DiT (fps) ↑
n=1 (default)
56.9
195.5
0.31
0.79
4.30
0.0032
20
n=2
56.8
193.2
0.37
0.78
4.20
0.0035
18
Appendix
Table 11: Rolling cache window size. Effect of the number n of retained past chunks on quality and speed, on Hallo3 test ( 10 s clips). Enlarging the window from n=1 to n=2 slightly improves FVD but degrades JEDi, CSIM, Sync-C and WE, while increasing cache size and reducing throughput. FVD and JEDi are averaged over the three temporal offsets ( @0 , @101 , @109 ). Best per column in bold .
Figure 9: Qualitative comparison on the Hallo3 test set . Despite stronger FID/FVD scores (Table 1 ), Hallo3 generations exhibit recurrent hand artifacts ( bottom ) that distribution-based metrics fail to penalize. AURA ( top ) produces cleaner outputs at a fraction of the generation time.
Figure 10: Qualitative comparison between AURA 5B and its TalkVerse teacher . Given the same reference image ( left ) and a 10 s audio input, AURA 5B ( bottom ) generates video of comparable visual quality to the teacher ( top ) while running approximately 17× faster end-to-end on a single H100 GPU. Full videos are on the project page .