Exocentric-to-egocentric (Exo2Ego) video generation transforms third-person recordings into first-person viewpoints, unlocking immersive AR/VR experiences and scalable demonstration synthesis for embodied robotics. Extending it to a streaming setting is expected to further enable live applications. However, existing methods suffer from degraded manipulation regions, overfitting and confinement to fixed-length clips with prohibitive latency. To address these limitations, we present StreamEgo, a real-time framework for unbounded, generalizable Exo2Ego video generation. To resolve intricate manipulation dynamics while supporting the streaming pipeline, a streaming dual-branch prior provides pixel-aligned guidance for translating appearance and skeletal kinematics. We also introduce a region-aware loss curriculum to improve generalization and stabilize multi-objective diffusion training. Furthermore, we distill the model into a chunk-wise autoregressive model, augmented with a region-aware corruption mechanism and a first-chunk curriculum, enabling high-quality, generalizable real-time streaming Exo2Ego generation at 24.3 FPS on a single H100 GPU.
Figures & tables
Figure 1: Streaming exocentric-to-egocentric (Exo2Ego) video generation. Left: Tri-regional decomposition of the task of Exo2Ego. Right: Comparison of our method with state-of-the-art Exo2Ego method EgoX ( Kang et al., 2026 ) on Ego-Exo4D dataset ( Grauman et al., 2024 ) .
Figure 2: Overview of the bidirectional teacher. (a) Causal construction of the streaming dual-branch priors, comprising the dynamic ego prior and human pose prior. (b) Conditioning assembly and fine-tuning of the base Wan model ( Team Wan and others, 2025 ) under our region-aware curriculum.
Figure 3: Training pipeline for the streaming student model. The k -th egocentric chunk is denoised by the Wan DiT ( Team Wan and others, 2025 ) conditioned on its dual-branch priors and the memory buffer. We apply the proposed region-aware corruption and first-chunk curriculum during training.
Figure 4: Visual comparison from Grauman et al. (2024) ( Unseen ), Kwon et al. (2021) , and Sener et al. (2022) on synchronized key frames.
Table 1: Comparison on the New Action and Unseen from Ego-Exo4D ( Grauman et al., 2024 ) and H2O ( Kwon et al., 2021 ) . Bold : the best. Underline : second best. Ours (est.) runs Ours (long) with ego trajectories estimated from the exocentric video (Sec. 4.4 ).
Figure 5: Streaming long-video generation with the autoregressive student on Grauman et al. (2024) ( Unseen ) and Kwon et al. (2021) .
Figure 6: Ablation comparison on Grauman et al. (2024) . Left: exocentric input and its egocentric ground truth. Middle: ablations results for the streaming dual-branch prior. Right: ablation results and training log for the region-aware curriculum. Red circles and arrows mark failure regions.
Table 2: Ablation study on Ego-Exo4D ( Grauman et al., 2024 ) Unseen . Rows marked † evaluate region-aware loss weight scheduling on New Action . Teacher variants isolate conditioning and loss scheduling; student variants isolate autoregressive training strategies. Bold: The best .
Figure 7: Ablation comparisons for region-aware corruption and first-chunk curriculum.
Appendix figures & tables13 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 8: Additional qualitative comparison on synchronized key frames from Ego-Exo4D ( Unseen ) ( Grauman et al., 2024 ) , H2O ( Kwon et al., 2021 ) , and Assembly101 ( Sener et al., 2022 ) .
Figure 9: Four additional examples of the teacher ablations in Fig. 6 on Grauman et al. (2024) Unseen , covering the streaming dual-branch prior and region-aware loss weight scheduling ( w/o HSU). Each row shows one example. Gt accounts for egocentric ground truth.
Figure 10: Results for user study questions (1)-(4).
Table 7: EgoX ( Kang et al., 2026 ) on its original backbone and on ours, evaluated on Ego-Exo4D ( Grauman et al., 2024 ) and H2O ( Kwon et al., 2021 ) .
Figure 11: Streaming schedule of the student. Each incoming exocentric chunk triggers a fixed sequence of stages; the corresponding egocentric chunk is emitted when the next exocentric chunk arrives, giving a constant start-up latency of one chunk ( 1,375 ms).
Stage
Device
Depends on
Par.
Latency
(1) Streaming dual-branch priors
H100
—
A
0.58 s
(2) VAE encode, exo chunk + priors
H100
(1)
B
0.30 s
(3) VLM action prompt ( 6 keyframes)
H100
—
B
0.23 s
(4) Denoising, 3 steps (student)
H100
(2), (3)
C
1.36 s
(5) VAE decode, ego chunk
H100
(4)
D
0.91 s
Appendix
Table 8: Per-chunk stage costs of the streaming student ( 33 frames, 1,375 ms of video). Stages in the same parallel group are independent and can be executed concurrently; the dependency column states what each stage must wait for.
Configuration
Stage-1 A
Stage-1 B
Stage-1 C
Stage-2 init
Stage-2 post
Initialization
teacher
1A
1B
1C
2 init
Training steps
2,500
10,000
18,000
16,000
20,000
Learning rate
1×10−5
1×10−5
6×10−6
2×10−5
6×10−6
LR schedule
constant w/ 100 -step warmup
←
←
←
←
First-chunk probability
1.0
0.6
0.25
0.25
0.25
History corruption
off
off
on
on
on
Appendix
Table 9: Configuration of the causal adaptation (Stage 1) and of the pyramid token compression (Stage 2). Each sub-phase re-initializes the optimizer. † The Stage-1 sub-phases share a single LoRA configuration.
Configuration
Stage-3 ODE init
Stage-3 post
Initialization
Stage-2 post
Stage-3 ODE init
Training steps
3,759
2,250
Generator learning rate
2×10−6
2×10−6
Critic learning rate
—
4×10−7
LR schedule
constant
constant
Optimizer
AdamW, β1=0 , β2=0.999 , wd 10−3
←
Appendix
Table 10: Configuration of the few-step distillation (Stage 3).
Stage
Hardware
Time per chunk
Multi-HMR on 17 of 33 frames (head position)
A100
0.68 s
SemiUHPE on the head crops (head orientation)
A100
0.03 s
MoGe-2 on 3 frames (metric depth)
A100
0.19 s
Wearer tracking, One Euro filter
CPU
<0.01 s
Head tracking in total
–
0.91 s
Appendix
Table 11: Per-chunk latency of the causal head tracker ( 33 frames, 1.375 s of video).
Generating egocentric video from a single exocentric video is an emerging and important topic for AR/VR and physical AI. Compared with conventional novel view synthesis, exo-to-ego generation is a significantly harder task because the standard geometric conditioning becomes highly unreliable under extreme view changes and large unobservable regions. We present Grounded-Exo2Ego, a principled framework that addresses these challenges at both the architectural and data levels. Architecturally, Grounded-Exo2Ego is a dual-branch video diffusion model that couples a geometric anchoring branch, which conditions the generation on the rendering of a 3D reconstruction, with a novel semantic grounding branch, which goes beyond the prevailing geometry-based approach and improves quality by synthesizing challenging regions based on object-level context. Additionally, we found that the overlooked issue of camera-reconstruction misalignment severely undermines exo-to-ego learning. We thus introduce a camera re-localization algorithm that resolves this issue and substantially improves quality across all metrics. We further develop a fully automated synthetic data engine that generates and renders rigged 3D characters in procedurally generated environments. Evaluation on the challenging EgoExo4D dataset shows that our method outperforms recent state-of-the-art approaches by large margins across all metrics. Detailed ablations validate improvements from each of our contributions at both the data and architectural level.
Exo-to-egocentric video generation aims to synthesize what a person sees from their own viewpoint given third-person footage and a target head trajectory. The task requires transferring appearance and semantics across large viewpoint changes while hallucinating content never observed by the exocentric camera. Existing approaches either impose additional input requirements, such as a ground-truth initial egocentric frame or multiple synchronized exocentric views, or remain limited to category-specific settings. EgoX is the first to address cross-activity and in-the-wild generalization, but requires a human-provided caption of the non-existent egocentric view at inference and introduces a computationally expensive geometry-guided attention bias that can propagate reconstruction errors and suppress textual and visual context. We therefore propose Ego-Forge, a caption-free and bias-free framework for exo-to-egocentric generation. It introduces Dynamic Captioning, which derives conditioning tokens directly from the model's hidden states and adapts them to the diffusion timestep and network depth, replacing external text conditioning. By scaling training by an order of magnitude and using all available exocentric viewpoints, Ego-Forge learns cross-view correspondence implicitly and eliminates the need for geometry-guided attention, requiring only a lightweight depth prior. Ego-Forge achieves state-of-the-art performance on Ego-Exo4D, runs faster end-to-end, requires no external annotation at inference, and generalizes to in-the-wild scenes, including cases where over-reliance on geometry blocks appearance inference. Our model and source code will be made publicly available.
Foundation video generation models such as WAN 2.2 exhibit strong text- and image-conditioned synthesis abilities but remain constrained to the same-view generation setting. In this work, we introduce Exo2EgoSyn, an adaptation of WAN 2.2 that unlocks Exocentric-to-Egocentric(Exo2Ego) cross-view video synthesis. Our framework consists of three key modules. Ego-Exo View Alignment(EgoExo-Align) enforces latent-space alignment between exocentric and egocentric first-frame representations, reorienting the generative space from the given exo view toward the ego view. Multi-view Exocentric Video Conditioning (MultiExoCon) aggregates multi-view exocentric videos into a unified conditioning signal, extending WAN2.2 beyond its vanilla single-image or text conditioning. Furthermore, Pose-Aware Latent Injection (PoseInj) injects relative exo-to-ego camera pose information into the latent state, guiding geometry-aware synthesis across viewpoints. Together, these modules enable high-fidelity ego view video generation from third-person observations without retraining from scratch. Experiments on ExoEgo4D validate that Exo2EgoSyn significantly improves Ego2Exo synthesis, paving the way for scalable cross-view video generation with foundation models. Source code and models will be released publicly.