LeRobot: An Open-Source Library for End-to-End Robot Learning
Authors: Remi Cadene, Simon Aliberts, Francesco Capuano, Michel Aractingi, Adil Zouitine, Pepijn Kooijmans, Jade Choghari, Martino Russi, Caroline Pascal, Steven Palma, Mustafa Shukor, Jess Moss, Alexander Soare, Dana Aubakirova, Quentin Lhoest, Quentin Gallouédec, Thomas Wolf
Organizations: Hugging Face · University of Oxford
Robotics is undergoing a significant transformation powered by advances in high-level control techniques based on machine learning, giving rise to the field of robot learning. Recent progress in robot learning has been accelerated by the increasing availability of affordable teleoperation systems, large-scale openly available datasets, and scalable learning-based methods. However, development in the field of robot learning is often slowed by fragmented, closed-source tools designed to only address specific sub-components within the robotics stack. In this paper, we present lerobot, an open-source library that integrates across the entire robot learning stack, from low-level middleware communication for motor controls to large-scale dataset collection, storage and streaming. The library is designed with a strong focus on real-world robotics, supporting accessible hardware platforms while remaining extensible to new embodiments. It also supports efficient implementations for various state-of-the-art robot learning algorithms from multiple prominent paradigms, as well as a generalized asynchronous inference stack. Unlike traditional pipelines which heavily rely on hand-crafted techniques, lerobot emphasizes scalable learning approaches that improve directly with more data and compute. Designed for accessibility, scalability, and openness, lerobot lowers the barrier to entry for researchers and practitioners to robotics while providing a platform for reproducible, state-of-the-art robot learning.
Figures & tables
Figure 1: lerobot is an open-source library for end-to-end robot learning. It covers the entire stack, from middleware motor interfaces to large-scale data collection and dataset streaming, supporting an optimized inference stack, scalable implementations of SOTA robot learning algorithms, and providing support for training custom models as well as easily reusing pre-trained ones.
Figure 2: Some of the explicit and implicit models for autonomous motion.
Figure 3: Classical robotics uses modular, model-based pipelines with hand-crafted features, while robot learning employs monolithic, data-driven policies that learn directly from interaction data.
Figure 4: (A) Low-cost, open-source robots like SO-10X and ALOHA cost a fraction of proprietary industrial arms, using consumer-grade parts and 3D-printable designs. (B) Decentralized efforts to collect expert demonstrations in the form of trajectories surpassed centralized efforts for the collection of large amounts of real-world robotics data.
Table 5
Figure 5: Numbers and trends of downloads, datasets, and episodes by robot type over time. The number of episodes in each dataset has been explicitly tracked starting in October 2024 only. For completeness, we report the top-5 robots grouped in Other , for each of the metrics considered, in Table 4 .
Figure 6: The different robot learning algorithms currently supported by lerobot .
Peak Memory
# Params
CPU
MPS
RTX 4090
A100
ACT
52M
817.4MB
462MB
211.24 MB
211.24 MB
Diffusion Policy
263M
1.22GB
224MB
1.12 GB
1.12 GB
π0
3.5B
4.13GB
97MB
13.32 GB
13.32 GB
SmolVLA
450M
1.69GB
555MB
1.75 GB
1.75 GB
Table 2: Peak memory consumption of policy models currently supported by lerobot . All models are run in full precision (fp32). Diffusion and Flow Models are run with 10 denoising steps at inference. All models maintain their original outputs shapes.
Avg Inference Latency (ms)
# Params
CPU
MPS
RTX 4090
A100
ACT
52M
182.313 ± 40.82
42.667 ± 10.085
5.013 ± 0.061
13.77 ± 0.445
Diffusion Policy
263M
(100%)
3453.838 ± 39.271
369.788 ± 0.193
613.893 ± 10.173
π0
3.5B
(100%)
(100%)
209.381 ± 2.762
568.978 ± 2.937
SmolVLA
450M
2028.461 ± 302.59 (2%)
721.826 ± 57.748
99.244 ± 1.195
278.833 ± 1.886
Table 3: Average and standard deviation inference latency over 100 forward passes for policy models currently supported by lerobot. Diffusion and Flow Models are run with 10 denoising steps at inference time. (x%) indicates the percentage of samples that timed-out before the 5000ms hard stop (0% omitted).
Figure 7: Numbers and trends of uploads and downloads of robot learning models by policy type over time. TD-MPC ( Hansen et al., 2022 ) , HIL-SERL ( Luo et al., 2024 ) and VQ-BET ( Lee et al., 2024 ) are absent from all visualizations as they are not typically uploaded by users.
Figure 8: Overview of the generalized inference schema supported by lerobot , whereby a remote server can be used to host compute-expensive policies for inference, while the robot client receives a stream of the actions chunks to enact. The schema provides scalability and flexibility through the possibility to fully customize the function f used to aggregate overlapping chunks.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Table 4: Breakdown of the Other category by top-5 robot platforms across datasets, downloads, and episodes.
Figure 9: Openly-available datasets with the largest number of downloads using the LeRobotDataset format. The most downloaded datasets are academic benchmarks released by the research community ( Collaboration et al., 2025 ; Khazatsky et al., 2025 ) .
Figure 10: Timing performance of StreamingLeRobotDataset versus a regular LeRobotDataset .
Figure 15
Table 5: Comparison between regular (Sync) and optimized (Async) inference. We evaluate the SmolVLA implementation provided in lerobot on three real-world performed using the SO-100 arm, consisting of (1) pick and place cubes (2) stacking cubes on top of each other and (3) sorting cubes. lerobot ’s decoupled inference schema achieves similar success rates (left) but results in significantly reduced cycle times (middle) and thus higher throughput (right), over the 10 test episodes (60s each) for the task considered.
Robot learning increasingly depends on large and diverse data, yet robot data collection remains expensive and difficult to scale. Egocentric human data offer a promising alternative by capturing rich manipulation behavior across everyday environments. However, existing human datasets are often limited in scope, difficult to extend, and fragmented across institutions. We introduce EgoVerse, a collaborative platform for human data-driven robot learning that unifies data collection, processing, and access under a shared framework, enabling contributions from individual researchers, academic labs, and industry partners. The current release includes 1,362 hours (80k episodes) of human demonstrations spanning 1,965 tasks, 240 scenes, and 2,087 unique demonstrators, with standardized formats, manipulation-relevant annotations, and tooling for downstream learning. Beyond the dataset, we conduct a large-scale study of human-to-robot transfer with experiments replicated across multiple labs, tasks, and robot embodiments under shared protocols. We find that policy performance generally improves with increased human data, but that effective scaling depends on alignment between human data and robot learning objectives. Together, the dataset, platform, and study establish a foundation for reproducible progress in human data-driven robot learning. Videos and additional information can be found at https://egoverse.ai/
Ryan Punamiya, Simar Kareer, Zeyi Liu +37
Georgia Institute of Technology · Stanford University · University of California San Diego +5
A critical bottleneck in robot learning is the scarcity of task-labeled, segmented training data, despite the abundance of large-scale robotic datasets recorded as long, continuous interaction logs. Existing datasets contain vast amounts of diverse behaviors, yet remain structurally incompatible with modern learning frameworks that require cleanly segmented, task-specific trajectories. We address this data utilization crisis by formalizing robotic sequence retrieval: the task of extracting reusable, task-centric segments from unlabeled logs using only a few reference examples. We introduce ROSER, a lightweight few-shot retrieval framework that learns task-agnostic metric spaces over temporal windows, enabling accurate retrieval with as few as 3-5 demonstrations, without any task-specific training required. To validate our approach, we establish comprehensive evaluation protocols and benchmark ROSER against classical alignment methods, learned embeddings, and language model baselines across three large-scale datasets (e.g., LIBERO, DROID, and nuScenes). Our experiments demonstrate that ROSER consistently outperforms all prior methods in both accuracy and efficiency, achieving sub-millisecond per-match inference while maintaining superior distributional alignment. By reframing data curation as few-shot retrieval, ROSER provides a practical pathway to unlock underutilized robotic datasets, fundamentally improving data availability for robot learning.
Zillur Rahman, Eddison Pham, Alejandro Daniel Noel +1
University of Nevada, Las Vegas · University of Toronto · LatentWorlds AI, NL
General-purpose robot reward models are typically trained to predict absolute task progress from expert demonstrations, providing only local, frame-level supervision. While effective for expert demonstrations, this paradigm scales poorly to large-scale robotics datasets where failed and suboptimal trajectories are abundant and assigning dense progress labels is ambiguous. We introduce Robometer, a scalable reward modeling framework that combines intra-trajectory progress supervision with inter-trajectory preference supervision. Robometer is trained with a dual objective: a frame-level progress loss that anchors reward magnitude on expert data, and a trajectory-comparison preference loss that imposes global ordering constraints across trajectories of the same task, enabling effective learning from both real and augmented failed trajectories. To support this formulation at scale, we curate RBM-1M, a reward-learning dataset comprising over one million trajectories spanning diverse robot embodiments and tasks, including substantial suboptimal and failure data. Across benchmarks and real-world evaluations, Robometer learns more generalizable reward functions than prior methods and improves robot learning performance across a diverse set of downstream applications. Code, model weights, and videos at https://robometer.github.io/.