Ego-Exo4D Human Meshes Dataset: 4D Human Motion Reconstruction for Ego-Exo Captures

Abhiram Maddukuri; Georgios Pavlakos

cs.CVSep 24, 2026

Ego-Exo4D Human Meshes Dataset: 4D Human Motion Reconstruction for Ego-Exo Captures

Authors: Abhiram Maddukuri, Georgios Pavlakos

Organizations: The University of Texas at Austin

Abstract

Ego-Exo4D is a large-scale dataset providing synchronized egocentric and multi-view exocentric video, a rich resource for skill learning and assessment, procedural activity understanding, and embodied AI. However, the dataset ships with only sparse 3D human pose annotations, and reconstructing dense human motion from its multi-view captures is nontrivial. To this end, we present Ego-Exo4D-HM, a large-scale dataset of 4D human motion reconstructions for Ego-Exo4D's captures, and release the accompanying reconstruction pipeline. The code, dataset, and documentation can be found at https://abhiram824.github.io/egoexo4d_human_meshes.

Figures & tables

Figure 1 : Recovered SMPL-H meshes overlaid on exocentric video across the reconstructed Ego-Exo4D sequences.

Figure 2 : Comparison of our pipeline with MAMMA Cuevas-Velasquez et al. (2026) .

Explore similar work

EgoHTR: Egocentric 4D Demonstrations of Human Terrain Traversal
Jul 15, 2026Alex Brandes, Haig Conti Georges Sajelian, Manthan Patel +11Training-Set Long-Tail Motions Synthesis
ETH Zurich · TU Munich · Stanford +1
Human4K: A Large-Scale 4K Multi-View Mocap Dataset for Whole-Body 3D Human Reconstruction
Jul 15, 2026Tianshun Han, Ziyu Shi, Lijian Liu +7Avatar Reconstruction Training-Set Long-Tail Motions
School of Computer Science and Engineering, the Faculty of Innovation Engineering, Macau University of Science and Technology, Macau 999078, China · State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS), Institute of Automation, Chinese Academy of Sciences (CASIA), Beijing 100190, China · School of Artificial Intelligence, University of Chinese Academy of Sciences (UCAS), Beijing 100049, China +7
EgoExoMoCap: Distributed Ego-Exo Human Motion Capture
Jul 17, 2026Jiaxi Jiang, Bharat Lal Bhatnagar, Nan Yang +6Motion Capture Egocentric Motion Generation
1Meta Reality Labs · *This work was done during an internship at Meta. · 2ETH Zürich

Jul 15, 2026, cs.ROJ/K move · Enter open · S save

EgoHTR: Egocentric 4D Demonstrations of Human Terrain Traversal

Alex Brandes, Haig Conti Georges Sajelian, Manthan Patel, Dominik Hollidt+10

ETH Zurich · TU Munich · Stanford+1

Training-Set Long-Tail Motions Synthesis Wearable Devices

Open paper AskPDF arXiv

Deploying humanoid robots in unstructured terrain remains an open problem. While classic reinforcement learning struggles with the sheer complexity of real-world interactions, more promising methods leveraging human priors remain limited to models lacking contextual awareness. The restricted motion synthesis is a direct consequence of existing dataset pipelines failing to capture human-scene sequences in challenging environments. To bridge this gap between humanoid learning and scene reconstruction, we introduce the Egocentric Human-Terrain Reconstruction (EgoHTR) dataset. We develop and open-source a reconstruction pipeline capturing 55 scene-aligned 4D human motion sequences in diverse, complex environments using a multi-sensor setup of egocentric wearables and a portable 3D scanner. The resulting dataset comprises over 150k frames, which we evaluate against motion-capture ground truth, demonstrating state-of-the-art accuracy and establishing a rigorous benchmark for human motion analysis and synthesis. Further, we leverage this data to train perceptive locomotion policies, demonstrating hardware deployment on a Unitree G1 for reconstructed reference motions. Our pipeline enables community-driven dataset extensions and factors the problem to help researchers build foundational, context-aware robots that reliably traverse uneven terrain.

Abstract

Explore similar work

EgoHTR: Egocentric 4D Demonstrations of Human Terrain Traversal

Human4K: A Large-Scale 4K Multi-View Mocap Dataset for Whole-Body 3D Human Reconstruction

EgoExoMoCap: Distributed Ego-Exo Human Motion Capture