Robotic assembly presents a long-standing challenge due to its requirement for precise, contact-rich manipulation. While simulation-based learning has enabled the development of robust assembly policies, their performance often degrades when deployed in real-world settings due to the sim-to-real gap. Conversely, real-world reinforcement learning (RL) methods avoid the sim-to-real gap, but rely heavily on human supervision and lack generalization ability to environmental changes. In this work, we propose a hybrid approach that combines a simulation-trained base policy with a real-world residual policy to efficiently adapt to real-world variations. The base policy, trained in simulation using low-level state observations and dense rewards, provides strong priors for initial behavior. The residual policy, learned in the real world using visual observations and sparse rewards, compensates for discrepancies in dynamics and sensor noise. Extensive real-world experiments demonstrate that our method, SPARR, achieves near-perfect success rates across diverse two-part assembly tasks. Compared to the state-of-the-art zero-shot sim-to-real methods, SPARR improves success rates by 38.4% while reducing cycle time by 29.7%. Moreover, SPARR requires no human expertise, in contrast to the state-of-the-art real-world RL approaches that depend heavily on human supervision.
Figures & tables
Fig. 2 : Overview of experimental tasks . (Top) 10 AutoMate [ 6 ] assembly tasks. AutoMate provides a dataset of 100 assembly tasks with diverse parts. We choose 10 out of 100 tasks with near-perfect specialist policy pre-trained in simulation. (Bottom) 3 tasks on NIST board. NIST (National Institute of Standards and Technology) provides Assembly Task Boards as performance benchmarks to evaluate robotic assembly technologies. We consider the peg and gear insertion tasks on task board #1.
Fig. 3 : Illustration of asymmetric policy combination . Combining state-based base policies from simulation and image-based residual policies learned in the real-world.
Fig. 4 : Performance on 10 AutoMate tasks . We evaluate the success rate ( ↑ higher is better) and cycle time ( ↓ lower is better) averaged over 20 episodes. SERL, AutoMate, and SPARR (Ours) transfer simulation-trained policies to the real world without human effort, where SPARR achieves substantially higher success rates and shorter cycle times. HIL-SERL (Oracle) serves as an upper bound, assuming access to near-optimal human demonstrations and continuous human supervision.
Fig. 5 : Policy deployment on different socket poses for AutoMate task 00731. Each box indicates the (x, y) coordinates of the socket pose and the corresponding success rate (0–1) during evaluation. The training socket pose is at (0.48, 0), and for evaluation, the socket is displaced by 2 cm : up (0.50, 0), down (0.46, 0), left (0.48, –0.02), and right (0.48, 0.02). (a) Base policy from simulation. (b) Base policy with a state-based residual policy. (c) SPARR (Ours): base policy with image-based residual policy. The color bar represents success rate from 0 (yellow) to 1 (green). SPARR achieves higher success rates (darker green) and demonstrates robustness to socket pose variations.
Fig. 6 : Adaptation of simulation policies from AutoMate tasks to NIST tasks. We show images from wrist-mounted cameras here. Fig. 2 (bottom) shows the NIST tasks from the front camera view.
Fig. 7 : Performance on NIST assembly tasks. SPARR outperforms the baseline in success rate ( ↑ higher is better) and cycle time ( ↓ lower is better).
Fig. 8 : Visualization of base, residual and combined actions. On a trajectory completing rectangular peg insertion task, the residual action in yaw axis disagrees with the base action and changes the plug rotation when the combined action is executed in the environment. All these actions are clamped to the range [-1,1] as incremental target pose.
Success rate ( ↑ )
Cycle time (s) ( ↓ )
Task
01041
01036
01041
01036
AutoMate
0.8
0.8
10.76
4.82
SPARR w/o demo update
0.95
0.8
8.1
4.31
SPARR (Ours)
1.0
1.0
6.81
5.45
TABLE I : Comparison of method variants. SPARR shows higher success rate than the variant without demo buffer update.
Success rate ( ↑ )
Cycle time (s) ( ↓ )
Task
00015
00768
00015
00768
AutoMate
0.75
0.65
7.66
10.19
SPARR w/o base action input
0.8
0.8
6.12
8.76
SPARR (Ours)
1.0
0.95
3.51
6.11
TABLE II : Comparison of method variants. SPARR outperforms the variant without base action as input to residual policy.
Scaling data volume and diversity is critical for generalizing embodied intelligence. While synthetic data generation offers a scalable alternative to expensive physical data acquisition, transferring robotic manipulation policies from simulation to the real world (sim-to-real) remains a formidable challenge due to the domain gap. This paper presents HyperSim, a holistic framework spanning from synthetic data generation to policy training and seamless real-world deployment. To systematically bridge the sim-to-real gap, HyperSim is realized through three core pillars: high-fidelity environment synthesis, adversarial trajectory generation, and sim-and-real co-training. Collectively, these modules address domain discrepancies by enhancing visual fidelity, expanding data coverage, and enforcing domain-invariant representations. We rigorously validate HyperSim through a large-scale empirical study involving 400 real-world task executions across two representative manipulation models. Assessed across three fine-grained metrics, our complete pipeline achieves remarkable sim-to-real success rates of 80% and 95% with ACT and π_{0}, respectively. Furthermore, policies trained on our adversarial trajectories exhibit significantly enhanced robustness against dynamic uncertainties, achieving a 35% higher completion rate under physical perturbations.
Junyi Dong, Haotian Luo, Ziwei Xu +11
CloudRobo Lab, Huawei Cloud Computing Technologies Co.,Ltd. · Shanghai Jiao Tong University · The University of Hong Kong
The NIST Assembly Task Board #1 remains a challenging benchmark for learning-based methods in robotic manipulation. Over short horizons, it demands precise, contact-rich control of small, tight-fitting parts. Over long horizons, it requires recovery and coordination across extended action sequences. Demonstrations, task decomposition, and dense progress rewards have enabled research advances by providing guidance on how to assemble. But how much of the strategy must be specified? The progression from AlphaGo to AlphaGo Zero showed that RL at scale can discover superhuman strategies without prescribing how to win. AlphaNIST asks whether a sparse binary task reward and task diversity admitted by this task-agnostic reward can similarly let a robot discover how to assemble. In simulation, we train a single policy from scratch across all 20 assembly tasks on the NIST Assembly Task Board #1, reaching 98.2% overall training success without demonstrations, progress shaping, scripted primitives, or per-task tuning. The framework also trains policies to complete ten assembly pairs in a single episode without a prescribed order, with task success rewarded only when all ten pairs are simultaneously complete. The learned policies discover dexterous and planning-like behaviors: choosing grasps for later assembly, using surrounding geometry to reorient parts, and preparing one part, assembling another, then returning to finish the first without an explicit planner. For videos, see our website: https://alpha-nist.github.io/
Contact-rich assembly remains challenging because it requires submillimeter spatial accuracy and reliable interpretation of forces during sustained contact. Although simulation-based reinforcement learning offers a scalable training paradigm, discrepancies in visual observations, contact dynamics, and force/torque (F/T) measurements often limit policy transfer. We observe that proprioception is comparatively consistent across domains because joint positions are expressed in a shared calibrated coordinate system and joint velocities are computed consistently in simulation and on hardware. Based on this observation, we present PACE (Proprioception-Anchored Cross-Modal Encoder), which supervises temporal visual and F/T representations by predicting proprioceptive state transitions. Static domain-specific factors, including lighting, texture, and sensor bias, contain little information about joint motion; optimizing the proposed objective therefore suppresses their influence on the learned representation while retaining task-relevant motion cues. Policies trained on frozen PACE features are directly deployed on hardware without real-world fine-tuning or object-pose tracking. Across four contact-rich assembly tasks, PACE attains an average real-world success rate of 93.3% and only a 2.7-percentage-point sim-to-real drop, while remaining robust to perturbations that substantially degrade pose-based and learned-fusion baselines.
Yuhan Wang, Yurou Chen, Hongye Jiang +2
Shanghai Jiao Tong University · University of Sussex