cs.ROOct 16, 2026
SaveBlind Precise Assembly: Proprioceptive Sim-to-Real Peg-in-Hole for Diverse Parts
Organizations: Massachusetts Institute of Technology
Abstract
Every robot already carries high-resolution contact sensors: its own joints. Legged robots and humanoids exploit them to walk, run, and track whole-body motions without vision, but there the contacts are large and forceful. Is joint sensing also enough for precise, contact-rich assembly, where the robot must find a millimeter-scale opening, mesh gear teeth, or catch a thread through light contact? We study this question by training blind assembly policies that observe only a short history of joint positions, velocities, and their own actions, without visual, force, torque, or tactile sensing. The poses of both the fixture and the part are unknown to the policy, and the grasp is randomized substantially. The policy must therefore assemble the parts through contact, sensed exclusively through its own proprioception. Trained with reinforcement learning entirely in simulation over randomized parts, grasps, and contacts, such policies learn peg insertion, gear meshing, and nut threading. The peg insertion policy transfers zero-shot to the real world and inserts pegs of different shapes and sizes in 73% of trials. Controlled comparisons with matched visual policies in simulation show that blind policies generalize far better to unseen part shapes and sizes, though they can be slower. Training such policies needs thousands of different assemblies, each simulated with its clearance intact, so we also release an open-source generator that builds them procedurally from exact convex pieces for MuJoCo; it is more accurate and faster than convex decomposition or MuJoCo's signed-distance collision. Videos and interactive simulations are at https://younghyopark.me/blind-proprioceptive-assembly/.
Figures & tables
Figure 1: Blind insertion. A policy trained only in simulation inserts pegs of different shapes and sizes in the real world using only its joint encoders and previous actions.
Figure 2: Peg insertion, gear meshing, and nut threading. (a–c) A start of each task, faded, and the same grasp at success: the peg seated in its hole, the gear seated on its shaft in mesh with both anchor gears, and the nut turned until its thread has caught. (d–f) Stadium pegs, gear sets, and nuts and bolts generated by mjfit and used for training, to scale.
Figure 3: Training fixture poses and grasps. Gear and bolt fixtures span the peg’s ranges.
Figure 4: Assets generated by mjfit one color per convex piece of the generated collision geometry. Labels give the number of convex pieces (pcs) and the CPU time per physics step with the parts in contact.
Figure 5: Same parts, three decompositions , one color per convex piece (Ø16 mm plate; M10 nut cut open). Ours errs only toward extra play; CoACD and V-HACD put material inside the hole and fill the nut’s 0.1 mm thread clearance.
Figure 6: Speed against accuracy for a 16 mm round peg with 100 µm clearance, (a) on CPU, one environment on one thread, and (b) on GPU, per environment step across 256 MuJoCo Warp worlds, with the error also measured on Warp. Each line is one method’s Pareto front over all its settings; lower left is better, and hollow markers fill the clearance (dotted line). (c) The peg, centered and aligned, dropped from 1 mm above each method’s default plate and shown 0.2 s later.
Figure 7: Policy Architecture . A temporal convolution compresses the last proprioceptive observations (joint positions and velocities , gripper pose and twist , and the previous action ) into a latent , and an MLP maps and the current observation to joint targets for a joint impedance controller. During training, we provide contact supervision from privileged simulator state. A contact decoder (amber) predicts the contact state ( ) from . Its cross-entropy with the simulator’s labels (left, near half of the fixture cut away) trains the encoder, but never reaches the policy.
Figure 8: Proprioception alone is enough to learn all three tasks. Blind policies, which sense only their own joints, reach the same near-perfect training success as visual policies with a wrist camera. Lines are the mean of two training seeds.
Figure 9: Proprioception guides efficient search without vision. Because the blind policy must search through contact, it follows a less direct path and seats later than the visual policy (a, b). Nonetheless, it travels far less (c) than an optimistic scripted spiral search baseline ( Chhatpar & Branicky, 2001 ) .
Figure 10: The blind policies’ joint histories encode the task’s mode, in simulation and in the real world. A linear probe fitted only in simulation reads the mode from each policy’s history latent. Each matrix gives its confusions (%); beside it, one start state is decoded at every step (T: true mode, D: decoded). On all three tasks, the probe cleanly separates modes that call for different actions, such as lifted, engaged, and stuck. (a) On the robot, it decodes the same sequence of peg insertion modes as in simulation.
Figure 11: Proprioceptive policies remain more robust to geometric variation. Success of the blind and visual policies (both with a 30-step history) on each task, in distribution (left of the dashed line) and on unseen parts, each widening one object property beyond its training range while the others stay within theirs.
Figure 12: Real-world results of the blind policy, with no fine-tuning, beside the same policy in simulation on the same peg (2,048 episodes). Error bars are 95% Wilson intervals. Details in Appendix C .
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
| Peg insertion | Gear meshing | Nut threading | |
| Part and fixture | |||
| Part | stadium prism | involute spur gear, held by an 18 mm boss; 1–2 anchor gears | ISO metric nut and bolt, coarse and fine pitch |
| Size | scale 1.35–2.45; opening 38–91 20–54 mm; length 50–60 mm scale | module 2.375–3.125 mm; 16–24 teeth | M36 and M42 |
| Shape | aspect ratio 1.25–2.5 (log), area-preserving | pressure angle 21.25–23.75 ∘ ; tooth width 0.45–0.47 pitch; face width 9.75–13.25 mm | across flats and height 0.95–1.05 ISO 4032 |
| Clearance | 1.0–1.5 mm per side | backlash 0.4–0.6 mm per anchor; bore 0.75–1.0 mm | flank 0.1–0.2 mm (log) |
| Fixture | hole 10 mm scale deep in an 80 mm scale square plate | shaft radius 4.5–5.5 mm, 4.75–8.25 mm above the gear | bolt stick-out 7.5–22.5 mm above its minimum |
Table 1: Study training ranges. The study policies of all three tasks (section 3 ) train on these ranges; Appendix A.2 gives the wider ranges of the deployed peg policy and the full-range gear and nut policies. Ranges marked (log) are sampled log-uniformly and the rest uniformly. Geometry is drawn once per world; poses and contact parameters at every reset. No observation noise is added. ‡ The peg’s hole center; the gear’s and bolt’s fixtures draw the same offsets from their own nominal positions.
| Parameter | Deployed peg policy |
|---|---|
| Part and fixture | |
| Cross-section | random convex † |
| Opening, long short side | 39–91 30–52 mm |
| Peg length | 25–35 mm, below the handle |
| Hole depth | 20 mm |
| Plate | convex, 5–70 mm margin |
Table 2: Deployed peg policy’s training ranges , where they differ from the study peg’s (Table 1 ); scale, aspect ratio, and clearance are the study’s. † Polygonal, smooth, and mixed outlines in equal shares, each inside a stadium of the given scale and aspect ratio and redrawn until its hole fits the printed fixture insert (at most 106 52 mm), under an 80 44 40 mm stadium handle.
| Probe | Setup | Exact answer |
|---|---|---|
| Wall offset | Rays from the hole axis at 720 angles and 11 heights; no solver | Wall on the analytic outline |
| S0 contact onset | Aligned peg placed at known gaps; collision only | Contact appears at gap 0 |
| S1 drop | centered free peg dropped from 1 mm; from mm to µm | Passes with zero wall contact iff |
| S2 lateral play | Planar slides; 0.1 N push in 24 directions | Support of the feasible translation set |
| S3 tilt play | Slides and one hinge; 1.5 mN m torque | Maximum tilt at which both faces fit |
| S4 yaw play | Slides and a vertical hinge (square hole) |
Table 3: Passive probes. Errors are in micrometers of wall position; negative values mean that phantom material stopped the peg early.
| Representation | CPU, 1 thread | GPU, 512 worlds | Error at µm |
|---|---|---|---|
| mjfit , square hole | 93 | 970 | 0.2 µm |
| mjfit , circle, 7–32 sides | 82–29 | 1120–1070 | 2.6 µm |
| mjfit , circle, 63–281 sides | 16–4.9 | 875–442 | 0.4 µm |
| Mesh SDF, octree depth 5–8 | 2.9–2.2 | 590–840 | fills, or 23–79 µm |
| CoACD, threshold 0.01 (839 hulls) | 1.8 | 254 | fills |
| Other CoACD and V-HACD settings | 3.3–70 | 680–1900 | fills |
Table 4: Real-time factor at a 1 ms time step with the peg pressed against the wall; ranges span each method’s settings, best of multi-point collision on and off. GPU figures sum over 512 MuJoCo Warp worlds on one RTX 5090. The last column gives the worst settled error at a 20 µm radial clearance.
Figure 13: Study peg cross-sections, to scale. The study policies train on stadiums; the unseen random convex pegs used only for evaluation have one vertex on each side of a matching stadium’s bounding box and the same clearance.
| Blind proprioceptive (ours) | Visual | |
| Success (%) | 99.4 | 97.2 |
| Time to seat (s) | 0.80 | 0.44 |
| Insertions per minute | 65 | 82 |
| Lateral travel before seating (mm) | 37 | 20 |
| Time partly engaged (s) | 0.38 | 0.22 |
| Tilt when partly engaged / seated ( ∘ ) | 6.7 / 1.1 | 5.2 / 1.1 |
Table 5: In-distribution behaviour (sections 3.1 and 3.2 ), from 8,192 episodes per seed, averaged over both seeds (success, time, rate), and from the 3,964 of 4,096 matched starts where both seed-1 policies succeed (travel, engagement, tilt). Medians unless stated; insertions per minute count every attempt, with failures at their full 5 s. Starts lie a median 10 mm from the hole center.
Figure 14: Contact supervision speeds up learning. Training success of the blind study policies of figure 8 (blue) and of the same policies trained without the contact loss (gray).
Figure 15: Latent-swap test for the policies of figure 10 in simulation. Each state keeps its own current observation but receives the history latent of a state in another mode; each cell is the mean change in action relative to a swap within the receiving state’s own mode.
| Peg insertion | Gear meshing | Nut threading | |
| Clearance | |||
| Training | 1.0–1.5 mm per side | backlash 0.4–0.6 mm per anchor | flank 0.1–0.2 mm |
| Tighter | 0.5–0.625 mm | 0.2–0.25 mm | 0.05–0.0625 mm |
| Aspect / depth | |||
| Training | aspect ratio 1.25–2.5 | tooth height 1.0 module above the pitch circle | ISO thread depth |
| Mild | Slender: 3.54–4.20 | Taller teeth: 1.1–1.2 modules | Deeper thread: 1.1–1.2 , same flank angle |
Table 6: Unseen parts of figure 11 . Each row widens one property beyond its study training range (first row of each group); the bar’s name in the figure is in italics where it differs by task. Size bins pool parts smaller and larger than training.
| Clearance | Aspect / depth | Size | Shape | ||||
| Senses | In dist. | Tighter | Mild | Strong | Mild | Extreme | Random |
| Peg insertion | |||||||
| Proprioception (blind) | 99.4 | 93.8 | 97.1 | 93.3 | 97.1 | 91.9 | 36.2 |
| RGB (visual) | 97.2 | 54.1 | 62.7 | 40.1 | 57.7 | 16.4 | 3.9 |
| Gear meshing | |||||||
| Proprioception (blind) | 99.9 | 99.9 | 99.9 | 99.9 | 99.9 | 99.9 | 99.9 |
Table 7: Success on unseen parts (%), every bar of figure 11 ; Table 6 names each task’s columns. Peg insertion is the mean over two training seeds per policy; gear meshing and nut threading are one seed. Both policies have a 30-step history. Best per task and column in bold.
More in this topic
Robotic Assembly
Robotic assembly studies how robots perceive, plan, and join parts reliably, from precision insertion to collaborative construction.
See all 85 papers