cs.ROOct 19, 2026

Learning from Fleet Experience: Collective Experience Learning Across 96 Robots

Authors: Chao Yu, Shu'ang Yu, Hao Lin, Hongzhi Zang, Yuxiang Yan, Chenxi Jiang, Zhen Guo, Chuankang Li, 

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Robot deployment generates experience that can support policy improvement, but continued reliance on expert corrective actions makes learning difficult to scale across growing fleets. We propose learning from fleet experience: using deployed robots' own executions as a source of supervision for further policy improvement. We call this paradigm Collective Experience Learning (CEL), in which robots pool execution experience to update a shared policy that returns to the fleet. Following expert-supervised initialization, we realize CEL through failure-guided experience retrieval fine-tuning (FERT). FERT uses failure contexts to retrieve relevant segments from successful trajectories and prioritizes their recorded actions for supervised updates, without collecting additional expert demonstrations or corrective actions. Operators provide binary episode-outcome labels and environment resets. A fault-tolerant asynchronous training system (FATS) coordinates robot execution, cloud inference and policy optimization despite local interruptions. Across five real-world manipulation tasks, the resulting policies achieve success rates of 93.3-99.2%, evaluated over 120 trials per task on robots excluded from training data collection. With collection time per robot held fixed, larger fleets provide more experience and generally improve performance across the evaluated sizes. On multi-stage phone packaging, learning with 96 robots raises success from 79.2% to 98.3%. These results support CEL as a scalable approach to improving robot policy reliability through experience generated during deployment.

Figures & tables

Figure 1: Overview of Collective Experience Learning (CEL). We propose a new training paradigm CEL, so that robots improve from fleet experience rather than continued human experience, enabling large-scale online training with up to 96 robots. We develop a training algorithm FERT under CEL paradigm, which uses failure contexts to retrieve relevant successful segments and prioritize their recorded actions for supervised updates. To support scalable online training, FATS coordinates fault-tolerant asynchronous execution, cloud inference and policy optimization. Across five real-world manipulation tasks, policies improve after FERT, and success rates generally scale with fleet size.
Figure 2: Sources of action supervision for robot policy learning: human demonstrations, expert corrections and fleet experience. CEL uses robot-generated actions for further policy improvement after expert-supervised initialization. Our implementation retains human-provided episode-outcome labels and environment resets.
Figure 3: Failure-guided experience retrieval fine-tuning. Robots asynchronously collect autonomous experience with binary task-outcome labels. Observations are encoded using visual and proprioceptive features. Failure contexts retrieve related segments from successful trajectories, and these segments receive increased sampling priority during policy fine-tuning. Updated policies return to the fleet to generate further experience.
 InteractionExperience
UnitObservation and action chunkTrajectory fields assembled into an episode
Size0.45 MB of images per model input1.58 GB per Phone Packaging episode
TimingEach inference requestFields stream during execution, episodes complete about every 4 min
RequirementReply before the robot’s next interactionComplete record and outcome before use in training
PathRobot observations to cloud inference, actions back to the robotCloud-side assembly reuses inference data and receives additional fields from the robot site
Table 1: Interaction and experience paths in FATS. Sizes describe model-input images and a complete episode record, respectively.
Figure 4: Fault-tolerant asynchronous training system (FATS). a , The SwarmLink data plane connects robot workers at the edge to shared cloud inference and assembles streamed trajectory fields for training. The HotSwarm control plane manages membership and worker identity, and recovers failed workers. b , Synchronous collection makes each policy update wait for every robot, which leaves faster robots idle. In asynchronous collection, robots continue independently. c , HotSwarm restarts the worker of a failed robot in place while the other robots continue, and the action computed for the failed worker (#1) is discarded. d , Inference data and execution records are joined in the cloud, and an episode is released to training only after its required fields and outcome arrive. e , Robot time available for collection after fault handling, measured for FATS in the 96-robot run and modeled for restarting the entire system after each fault event with restart times between 2 and 10 min ( Section D.4 ). Durations in b and c are illustrative.
ArtifactHazard if the identity is not checkedResponse
Membership recordA peer builds a connection to a dead process and waits indefinitelyRefuse and wait for a fresh record
Control message in flightA late failure report retires the replacementIgnore
Connection held by a peerThe replacement joins an outdated connection groupRebuild under a new epoch
Data in the pipelineTraining pairs actions with another worker’s observationsDrop and receive again
Queued transferThe transfer runs on a destroyed connectionReplay on the new connection
Table 2: Artifacts that can outlive a failed worker, the hazard each poses to its replacement, and the response of HotSwarm.
Figure 5: Overview of the five manipulation tasks performed with AgiBot G2 robots: Flower Arrangement, Clothes Folding, Phone Charging, Stool Assembly and Phone Packaging. A detailed step-by-step description is provided in Appendix E .
Figure 6: Main results across five tasks. Each subfigure compares the success rates of human teleoperation, SFT, the selected multi-robot HG-DAgger (labelled (M)HG-DAgger), RLT, RECAP, and FERT on one task. For FERT, we report the checkpoint trained with the largest robot fleet. Error bars indicate 95% Wilson confidence intervals.
Task name(M)HG-DAgger (%) ↓\downarrowFERT (%) ↓\downarrowRelative failure reduction (%) ↑\uparrow
Flower Arrangement1.670.8350.0
Clothes Folding14.171.6788.2
Phone Charging40.002.5093.8
Stool Assembly18.336.6763.6
Phone Packaging20.831.6792.0
Task avg19.002.6786.0
Table 3: Failure rate table. (M)HG-DAgger reports the performance of the checkpoints used to initialize FERT. FERT reports results obtained with the maximum number of training robots for each task.
Figure 7: Scaling with fleet size across five tasks. Each panel shows success rate as a function of the number of training robots. The blue and pink curves represent HG-DAgger and FERT, respectively. HG-DAgger starts from SFT, whereas FERT starts from the selected (M)HG-DAgger checkpoint; horizontal dashed lines in different colours indicate the success rates of these initial policies. Error bars indicate 95% Wilson confidence intervals.
Figure 8: Ablation studies of FERT on Phone Packaging with 16 training robots. Pink bars indicate the full FERT configuration, while gray bars indicate the corresponding ablated variants. In (a), FR denotes failure-guided retrieval. In (b), the interval is measured in optimizer steps. In (c), each value specifies both hpreh_{\mathrm{pre}} and hposth_{\mathrm{post}} , with L=90L=90 frames held fixed. In (d), ρ\rho is the priority retention factor; larger values retain previous priorities for longer. Error bars indicate 95% Wilson confidence intervals.
Figure 9: FATS in the 96-robot training run. a , Episodes completed per minute in 5-min bins. Marks above the curve show fault events, with height proportional to the number of robots affected, and the shaded interval is the lunch break. b , Fault incidents by initiating cause and duration. Circles mark incidents in which the robot was restarted, and squares mark incidents in which recovery was blocked. c , Episodes collected when each robot’s measured episodes and pauses are replayed under other designs, relative to the run. A full restart stops all robots for 2, 5 or 10 minutes after each fault event, and synchronous rounds make each policy update wait for one episode from every robot. The two synchronous variants replace the pauses that overlap a fault, or all pauses, with the robot’s median pause. d , Completed episodes per robot, sorted by count. Statistics are computed from the detailed traces in Section H.2 .
Figure 10: Measured communication-operation durations across observation, action and experience channels. Bars report the fraction and count of operations in each duration interval, and each panel shows the total number of events.
Appendix figures & tables21 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 11: Multi-robot HG-DAgger with human interventions. Human operators take over execution when correction is needed. Successful trajectories are stored with intervention indicators, and eligible intervention chunks are used for supervised policy updates.
Figure 12: Robot-site network and cloud resources for FATS. The site supports 100 robots, with 96 used in the largest training run. Six cloud servers provide 48 NVIDIA A800 GPUs for inference and policy optimization. The robot site and cloud communicate through a dedicated 30 Gbps link.
TaskLanguage prompt
Flower arrangement“Put the flowers into the vase.”
Clothes folding“Fold the cloth.”
Phone charging“Charge the phone.”
Stool assembly“Assemble the stool with a screwdriver.”
Phone packaging“Pack the phone.”
TaskHead-camera crop boxTwo wrist-cameras crop box
Appendix
Table 4: Task prompts and observation crop parameters. Wrist crops apply to both the left and right cameras.
ParameterSetting
Max learning rate2.5×10−52.5\times 10^{-5}
Global batch size1024
Learning-rate scheduleCosine
Learning-rate warmup80 optimizer steps
Learning-rate schedule horizon10,000 optimizer steps
Gradient norm clipping4.0
Appendix
Table 5: Key FERT settings. Batch sizes are global.
TaskData sourceTrajectoriesIntervention
  TotalSuccessfulFailedfraction
Flower arrangement(M)HG-DAgger160160074.6%
 Newly collected data4813352.4%
Clothes folding(M)HG-DAgger148148068.0%
 Newly collected data118397918.0%
Phone charging(M)HG-DAgger716716039.9%
Appendix
Table 6: RECAP training data sources and composition. Intervention fractions are measured over frames.
ParameterSetting
Value peak learning rate10−410^{-4}
Value global batch size2,048
Policy peak learning rate5×10−55\times 10^{-5}
Policy global batch size256
Number of value bins201
Value support[−1,0][-1,0]
Appendix
Table 7: Key RECAP settings. Batch sizes are global.
ParameterSetting
Actor learning rate3×10−43\times 10^{-4}
Critic learning rate3×10−43\times 10^{-4}
Global batch size256
Discount factor0.99
Update-to-data ratio5
Critic-to-actor update ratio2:12:1
Appendix
Table 8: Key RLT settings shared across tasks, except for the overrides in Table 9 . Deviation budgets are in normalized action coordinates.
TaskParameterSetting
Phone ChargingDeviation budget0.2 for the 7 right-arm dimensions; 0 for the left arm and grippers
Flower ArrangementDeviation budget0.5 for the 14 arm dimensions; 0 for the grippers
 Exploration standard deviation0.005
Clothes FoldingDeviation budget0.5 for the 14 arm dimensions; 0 for the grippers
 Exploration standard deviation0.005
Appendix
Table 9: Task-specific RLT settings. Unlisted settings follow Table 8 . Deviation budgets are in normalized action coordinates.
TaskIntervention fractionComplete episodes
Flower Arrangement30%30
Clothes Folding20%30
Phone Packaging40%40
Stool Assembly80%100
Phone Charging40%70
Appendix
Table 10: Online RLT intervention fractions and complete-episode counts by task. All values are approximate.
Table 11: Task success rates for SFT, (M)HG-DAgger, RLT, RECAP, and FERT. Results are reported for two held-out test robots, with 60 trials per robot for each evaluated checkpoint. Robot 1 and Robot 2 denote the held-out test robots; their rows report successful trials out of 60. Avg. (%) pools successes over 120 trials. Bold values indicate the highest average success rate for each task.
 FERT (16 robots)w/o FRNo extensionRetention ρ\rhoSync interval
    0.80.951001500
Robot 153/6033/6034/6044/6050/6046/6048/60
Robot 251/6044/6035/6041/6049/6044/6048/60
Avg. (%)86.6764.1757.5070.8382.5075.0080.00
Appendix
Table 12: Success rates for the FERT ablation study on Phone Packaging with 16 training robots. FERT denotes the full configuration; w/o FR disables failure-guided retrieval; No extension sets hpre=hpost=0h_{\mathrm{pre}}=h_{\mathrm{post}}=0 . Retention ρ\rho is the priority retention factor, and Sync interval is the policy synchronization interval in optimizer steps. Each configuration is evaluated over 60 trials on each of two held-out test robots. The robot rows report successful trials out of 60, and Avg. (%) pools successes over 120 trials. Bold indicates the highest average success rate.
IDFlower ArrangementClothes FoldingPhone ChargingStool AssemblyPhone Packaging
120/2020/2012/2020/2016/20
219/2016/2018/2020/2013/20
320/2019/2012/2017/2019/20
412/2015/2012/2015/2019/20
518/2014/2019/2014/2018/20
620/2015/2013/2014/205/20
Appendix
Table 13: Human teleoperation success rates across five tasks. Each of the six operators completes 20 trials per task, yielding 120 trials per task in total. ID denotes the operator; each entry reports successful trials out of 20. Avg. (%) pools successes over all 120 trials per task.
Figure 13: Human teleoperation success rates across tasks and operators. Each operator is evaluated over 20 trials per task.
Figure 14: Detailed per-robot trial timelines during a 96-robot training run. Successful trials, failed trials and system errors are marked separately; blank intervals include environment resets and operator breaks.
Figure 15: Active robot count during a 96-robot training run. A robot is active while executing actions; inactive intervals include environment resets, operator breaks and robot errors. Light curves show raw measurements, and dark curves show Gaussian-smoothed measurements ( σ=60\sigma=60 s).
Figure 16: Error Types and Occurrence Times During a 96-Robot Training Run. The x-axis indicates time, and the y-axis lists the IDs of robots that encountered at least one error. Colors distinguish different error types.
Figure 17: Visualize the retrieval effect in Flower Arrangement. The first row shows a clip of the robot failing to insert the second flower, while the next three rows show three matched successful clips. FERT effectively matched the clip of successfully inserting the second flower.
Figure 18: Visualize the retrieval effect in Clothes Folding. The top row shows a failure segment in which the robot attempts to fold a T-shirt in half but fails to grasp its right corner with the right gripper. The following rows below show the corresponding retrieved segments. With an appropriate feature representation, FERT retrieves the same critical step across clothes of different colors.
Figure 19: Visualize the retrieval effect in Phone Charging. It illustrates the most easily mistaken step in the phone charging: inserting the charger plug into the socket, which requires high precision.
Figure 20: Visualize the retrieval effect in Stool Assembly. The figure shows a failed attempt to place a screw into a screw hole alongside the corresponding retrieved successful segments.
Figure 21: Visualize the retrieval effect in Phone Packaging. The figure illustrates the lid-placement stage: placing the lid over the box at an angle. This step requires high precision despite limited visibility. The top two rows show a failed attempt in which the left arm repeatedly adjusts the lid’s angle, but the lid still topples over. The three examples below show successful attempts.
More in this topic

Robot Policy Learning

Robot policy learning develops controllers that map sensory observations, task goals, and interaction history to actions for embodied robots.

See all 369 papers

Explore similar work

CardsList

  1. CAPEX: Efficiently Distilling Foundation Model Behavior into Deployable Robot Policies through Experience-Adaptive Reasoning

    Sep 26, 2026Shivam Aarya, Zhang Xi-Jia, Chengyue Huang +6Robot Demonstration GenerationSynthetic Data Generation for Robotics

    School of Interactive Computing Georgia Institute of Technology Atlanta, GA 30332, USA

  2. Learning While Deploying: Fleet-Scale Reinforcement Learning for Generalist Robot Policies

    May 1, 2026Yi Wang, Xinchen Li, Pengwei Xie +13Robot Policy LearningRobotic Manipulation

    Shanghai Innovation Institute. · AGIBOT Finch. · Columbia University.

  3. FAR: Failure-Aware Retry for Test-Time Recovery and Continual Policy Improvement

    Jul 1, 2026Haoran Hao, Shahram Najam Syed, Jeffrey Ichnowski +1Robot Policy LearningRobot Policy Adaptation

    Carnegie Mellon University