AtomEgo: Exploring Ego-Robot Integration for Embodied Foundation Model Pretraining

Di Wu; Dongchen Zheng; Junhe Sheng; Zhongxing Wei; Songxin Zhang; Zejian Xie; Xiaoquan Sun; Junyang Zheng

cs.ROSep 18, 2026

AtomEgo: Exploring Ego-Robot Integration for Embodied Foundation Model Pretraining

Authors: Di Wu, Dongchen Zheng, Junhe Sheng, Zhongxing Wei, Songxin Zhang, Zejian Xie, Xiaoquan Sun, Junyang Zheng,

Organizations: INFIFORCE · Tongji University · HKU · Lionrock Artificial Intelligence Laboratory ·

Abstract

Embodied foundation models are constrained by the limited scale and diversity of robot demonstrations, motivating the use of large-scale egocentric human interaction data. However, how to effectively incorporate such data into embodied-model pre-training remains unclear because of substantial embodiment and action-space gaps between humans and robots. We present AtomEgo, a systematic study of ego--robot co-training supported by a curated corpus of approximately 2,659 hours and a scalable data processing pipeline. Across vision--language--action and world--action model architectures, we investigate three representative paradigms: joint co-training with domain-specific action heads, progressive ego-to-robot transfer through embodiment alignment, and joint video--action modeling. We evaluate these paradigms through multi-task real-robot experiments and language-conditioned cross-embodiment representation analysis. Our results reveal a simple principle: Data Scale * Alignment Quality --> Capability Gain; egocentric data can improve generalization, but their value depends on how effectively they are aligned and utilized. This principle can provide practical guidance for scalable ego--robot pre-training.

Figures & tables

Figure 1: Qualitative overview of the three data pools used in AtomEgo pre-training: open-source robot data, open-source egocentric data, and in-house task-matched human–robot alignment data.

Group	Source	Episodes	Hours	Proportion
Robot data	Piper (in-house)	5,829	25.5	1.745%
	AgiBot World Beta [ 2 ]	21,837	314.0	6.537%
	DROID [ 12 ]	64,124	343.0	19.196%
	RoboCOIN [ 29 ]	93,352	655.7	27.945%
	RoboMIND [ 28 ]	83,152	221.7	24.892%
Ego data	EgoVerse [ 21 ]	64,464	1,079.5	19.297%

Table 1: Composition of the expanded co-training recipe. Proportions are calculated by training episode count.

Range	Width	Physical meaning	Action semantics
$0{:}7$	7	Left arm joints	Relative
$7{:}13$	6	Left end-effector position and Euler rotation	Absolute
$13{:}16$	3	Reserved	Masked
$16$	1	Left gripper	Absolute
$17{:}29$	12	Left hand joints	Absolute
$29{:}36$	7	Right arm joints	Relative

Table 2: Semantic layout of the unified 80-dimensional state–action space. Indices are zero-based and end-exclusive.

Figure 2: Overview of joint ego–robot co-training with domain-specific action heads (Atom-DH). See Section 4.2 for details.

Figure 3: Overview of progressive ego-to-robot transfer through curriculum-based embodiment alignment (Atom-CL). See Section 3 for details.

Stage	Objective and training data	Initialization	Updates	Trainable components
1	Egocentric prior learning on Ego data	Base PaliGemma VLM	100,000	Full model
2	Embodiment alignment on task-aligned human and robot data	Stage 1 checkpoint	50,000	Vision encoder and action expert
3	Robot pre-training on robot datasets	Stage 2 checkpoint	97,728	Full model

Table 3: Three-stage pre-training curriculum for progressive ego-to-robot transfer.

Figure 4: Overview of joint world–action modeling for ego–robot co-training (Atom-WAM). See Section 4 for details.

Figure 5: Task-matched ID and OOD evaluation settings for the seven real-robot tasks. Each OOD setting preserves the task objective while changing one primary spatial, appearance, or physical factor.

Panel of Table 4 — Table 4: Per-task real-robot results under ID and OOD conditions.

Panel of Table 5 — Table 5: Language-conditioned cross-embodiment representation results on seven tasks. Higher raw matched-task similarity indicates stronger consistency, while lower normalized different-task similarity indicates better task separation.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 8: Hardware platform used for real-robot evaluation.

Task	ID setup	OOD type	OOD perturbation	OOD specification and controls	Success criterion
Cook Bread	Standard pot and bread poses	Spatial	Rotate pot	Pot yaw $=45^{\circ}$ CCW; pot center and other object poses unchanged	Bread is placed in the pot, and the pot is placed on the cloth
Hang Cup	Standard cup mass	Physical	Increase cup mass	Mass $m_{\rm ID}\!\rightarrow m_{\rm OOD}$ ; cup appearance and initial pose unchanged	Handle is inserted and the cup hangs stably for the predefined duration
Place Fruits	Standard target set	Appearance	Add visually similar non-target fruit as a distractor	Distractor identity and pose fixed; target identities and initial poses unchanged	All three target fruits are placed in the basket while the distractor remains outside
Place Plate	Standard rack pose	Spatial	Translate rack	Rack displacement $\Delta\mathbf{x}_{\rm rack}$ fixed; plate pose and rack orientation unchanged	Plate is aligned, inserted, released, and remains stable in the rack
Place Pot	Standard pot pose	Spatial	Rotate pot	Pot yaw $=60^{\circ}$ ; pot center and cloth pose unchanged	Pot is grasped by the handle and placed on the cloth
Press Button	Standard illumination	Appearance	Reduce illumination	Illuminance $L_{\rm ID}\!\rightarrow L_{\rm OOD}$ ; camera and button poses unchanged	Button activation is registered and the end effector reaches the predefined retreat region

Appendix

Table 6: Seven-task real-robot evaluation protocol. Each OOD condition changes one primary factor while holding other task-relevant variables fixed whenever applicable. Symbolic quantities denote protocol parameters fixed before execution, including illumination, mass, displacement, and stability duration.

Figure 9: Uniformly sampled task-completion progressions for the seven real-robot evaluation tasks. Each row pair shows front and head views at ten time points from a complete task execution.

Hyperparameter	Robot pre-training	Multi-task post-training
Training data	34 robot datasets; 150.1M source frames; EgoVerse excluded	Two in-house Piper multi-task datasets
Action representation	Unified80	Unified80
Hardware	$3\times 8$ NVIDIA B200 GPUs	$1\times 8$ NVIDIA B200 GPUs
FSDP devices	4	4
Global batch size (train / validation)	1,536 / 96	512 / 96
Training steps	97,728	20,000

Appendix

Table 7: Key hyperparameters for robot pre-training and multi-task post-training.

Variant	ID success	OOD success	Combined success
Atom-DH (+OT)	38.57% (27/70)	31.43% (22/70)	35.00% (49/140)
Atom-DH (w/o OT)	44.29% (31/70)	42.86% (30/70)	43.57% (61/140)

Appendix

Table 8: Ablation of the training-only OT regularizer on the seven-task real-robot evaluation. Each ID or OOD entry aggregates 70 trials; Combined pools all 140 trials.

Robot Learning

Robot learning develops methods for robots to acquire skills and adapt from demonstrations, interaction, data, and physical models.

See all 321 papers

HumanScale: Egocentric Human Video Can Outperform Real-Robot Data for Embodied Pretraining
Jun 18, 2026Juncheng Ma, Jianxin Bi, Yufan Deng +19Robot Foundation Models Cross-Embodiment Robot Learning
PKU · NUS · MIT +2
ACE-Ego-0: Unifying Egocentric Human and Robotic Data for VLA Pretraining
Jun 15, 2026Hao Li, Ganlong Zhao, Yufei Liu +8Cross-Embodiment Robot Learning Vision-Language-Action Models
ACE Robotics · CUHK MMLab · SJTU +2
Data Pyramid for Embodied Manipulation: A Survey
Jul 27, 2026Yifan Ye, Yankai Fu, Yaoxu Lv +26Robotic Data Collection Robot Foundation Models
PKU · NTU · HKUST +8

Jun 18, 2026J/K move · Enter open · S save

HumanScale: Egocentric Human Video Can Outperform Real-Robot Data for Embodied Pretraining

Juncheng Ma, Jianxin Bi, Yufan Deng, Xuanran Zhai+18

PKU · NUS · MIT+2

Robot Foundation Models Cross-Embodiment Robot Learning Egocentric Video Understanding Robot Learning

Open paper AskPDF arXiv

Embodied foundation models are expected to benefit from data scaling like large language models, but face a much tighter data bottleneck. Teleoperated real-robot trajectories remain the dominant pretraining source due to their precise action supervision and embodiment alignment, yet their scalability is limited by high collection cost, acquisition difficulty, and low behavioral and environmental diversity. These limitations have sparked interest in egocentric human video as a scalable, substantially lower-cost, and more diverse alternative for embodied model pretraining. However, its effectiveness compared to teleoperated real-robot data remains underexplored. To address this question, we conduct a systematic study comparing egocentric human video and teleoperated real-robot trajectories as pretraining data sources for embodied foundation models, under fixed post-training and validation protocols. Surprisingly, we find that egocentric data, when processed through a carefully designed filtering and labeling pipeline, is not merely a viable substitute for model pretraining but can lead to superior performance. With the same amount of pretraining data, models pretrained on egocentric data achieve a 24% lower validation loss on real-robot action prediction, as well as 52.5% and 90% higher success rates on in-distribution and out-of-distribution real-robot task execution, respectively. This finding verifies a scalable paradigm for embodied foundation models: pretrain on egocentric human video to learn diverse world representations, then adapt with a small amount of labeled real-robot data for action-space alignment. We hope this study encourages broader exploration of egocentric data and offers guidance for data quality assessment before costly robot data collection.

Abstract

Robot Learning

Explore similar work

HumanScale: Egocentric Human Video Can Outperform Real-Robot Data for Embodied Pretraining

ACE-Ego-0: Unifying Egocentric Human and Robotic Data for VLA Pretraining

Data Pyramid for Embodied Manipulation: A Survey