Exocentric-to-egocentric (Exo2Ego) video generation transforms third-person recordings into first-person viewpoints, unlocking immersive AR/VR experiences and scalable demonstration synthesis for embodied robotics. Extending it to a streaming setting is expected to further enable live applications. However, existing methods suffer from degraded manipulation regions, overfitting and confinement to fixed-length clips with prohibitive latency. To address these limitations, we present StreamEgo, a real-time framework for unbounded, generalizable Exo2Ego video generation. To resolve intricate manipulation dynamics while supporting the streaming pipeline, a streaming dual-branch prior provides pixel-aligned guidance for translating appearance and skeletal kinematics. We also introduce a region-aware loss curriculum to improve generalization and stabilize multi-objective diffusion training. Furthermore, we distill the model into a chunk-wise autoregressive model, augmented with a region-aware corruption mechanism and a first-chunk curriculum, enabling high-quality, generalizable real-time streaming Exo2Ego generation at 24.3 FPS on a single H100 GPU.
Figures & tables
Figure 1: Streaming exocentric-to-egocentric (Exo2Ego) video generation. Left: Tri-regional decomposition of the task of Exo2Ego. Right: Comparison of our method with state-of-the-art Exo2Ego method EgoX ( Kang et al., 2026 ) on Ego-Exo4D dataset ( Grauman et al., 2024 ) .
Figure 2: Overview of the bidirectional teacher. (a) Causal construction of the streaming dual-branch priors, comprising the dynamic ego prior and human pose prior. (b) Conditioning assembly and fine-tuning of the base Wan model ( Team Wan and others, 2025 ) under our region-aware curriculum.
Figure 3: Training pipeline for the streaming student model. The k -th egocentric chunk is denoised by the Wan DiT ( Team Wan and others, 2025 ) conditioned on its dual-branch priors and the memory buffer. We apply the proposed region-aware corruption and first-chunk curriculum during training.
Figure 4: Visual comparison from Grauman et al. (2024) ( Unseen ), Kwon et al. (2021) , and Sener et al. (2022) on synchronized key frames.
Table 1: Comparison on the New Action and Unseen from Ego-Exo4D ( Grauman et al., 2024 ) and H2O ( Kwon et al., 2021 ) . Bold : the best. Underline : second best. Ours (est.) runs Ours (long) with ego trajectories estimated from the exocentric video (Sec. 4.4 ).
Figure 5: Streaming long-video generation with the autoregressive student on Grauman et al. (2024) ( Unseen ) and Kwon et al. (2021) .
Figure 6: Ablation comparison on Grauman et al. (2024) . Left: exocentric input and its egocentric ground truth. Middle: ablations results for the streaming dual-branch prior. Right: ablation results and training log for the region-aware curriculum. Red circles and arrows mark failure regions.
Table 2: Ablation study on Ego-Exo4D ( Grauman et al., 2024 ) Unseen . Rows marked † evaluate region-aware loss weight scheduling on New Action . Teacher variants isolate conditioning and loss scheduling; student variants isolate autoregressive training strategies. Bold: The best .
Figure 7: Ablation comparisons for region-aware corruption and first-chunk curriculum.
Appendix figures & tables13 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 8: Additional qualitative comparison on synchronized key frames from Ego-Exo4D ( Unseen ) ( Grauman et al., 2024 ) , H2O ( Kwon et al., 2021 ) , and Assembly101 ( Sener et al., 2022 ) .
Figure 9: Four additional examples of the teacher ablations in Fig. 6 on Grauman et al. (2024) Unseen , covering the streaming dual-branch prior and region-aware loss weight scheduling ( w/o HSU). Each row shows one example. Gt accounts for egocentric ground truth.
Figure 10: Results for user study questions (1)-(4).
Table 7: EgoX ( Kang et al., 2026 ) on its original backbone and on ours, evaluated on Ego-Exo4D ( Grauman et al., 2024 ) and H2O ( Kwon et al., 2021 ) .
Figure 11: Streaming schedule of the student. Each incoming exocentric chunk triggers a fixed sequence of stages; the corresponding egocentric chunk is emitted when the next exocentric chunk arrives, giving a constant start-up latency of one chunk ( 1,375 ms).
Stage
Device
Depends on
Par.
Latency
(1) Streaming dual-branch priors
H100
—
A
0.58 s
(2) VAE encode, exo chunk + priors
H100
(1)
B
0.30 s
(3) VLM action prompt ( 6 keyframes)
H100
—
B
0.23 s
(4) Denoising, 3 steps (student)
H100
(2), (3)
C
1.36 s
(5) VAE decode, ego chunk
H100
(4)
D
0.91 s
Appendix
Table 8: Per-chunk stage costs of the streaming student ( 33 frames, 1,375 ms of video). Stages in the same parallel group are independent and can be executed concurrently; the dependency column states what each stage must wait for.
Configuration
Stage-1 A
Stage-1 B
Stage-1 C
Stage-2 init
Stage-2 post
Initialization
teacher
1A
1B
1C
2 init
Training steps
2,500
10,000
18,000
16,000
20,000
Learning rate
1×10−5
1×10−5
6×10−6
2×10−5
6×10−6
LR schedule
constant w/ 100 -step warmup
←
←
←
←
First-chunk probability
1.0
0.6
0.25
0.25
0.25
History corruption
off
off
on
on
on
Appendix
Table 9: Configuration of the causal adaptation (Stage 1) and of the pyramid token compression (Stage 2). Each sub-phase re-initializes the optimizer. † The Stage-1 sub-phases share a single LoRA configuration.
Configuration
Stage-3 ODE init
Stage-3 post
Initialization
Stage-2 post
Stage-3 ODE init
Training steps
3,759
2,250
Generator learning rate
2×10−6
2×10−6
Critic learning rate
—
4×10−7
LR schedule
constant
constant
Optimizer
AdamW, β1=0 , β2=0.999 , wd 10−3
←
Appendix
Table 10: Configuration of the few-step distillation (Stage 3).
Stage
Hardware
Time per chunk
Multi-HMR on 17 of 33 frames (head position)
A100
0.68 s
SemiUHPE on the head crops (head orientation)
A100
0.03 s
MoGe-2 on 3 frames (metric depth)
A100
0.19 s
Wearer tracking, One Euro filter
CPU
<0.01 s
Head tracking in total
–
0.91 s
Appendix
Table 11: Per-chunk latency of the causal head tracker ( 33 frames, 1.375 s of video).