As real-robot data collection scales, collection-system design shapes both operator efficiency and the behaviors learned by robot policies. Workspace limitations, control-interface friction, and latency can introduce pauses, detours, and corrective motions into otherwise successful demonstrations. We investigate these effects across embodiment, teleoperation control, and data consistency, combining a kinematic study of 25 dual-arm robots, controlled teleoperation experiments, and real-robot evaluation of fine-tuned π0.5 policies. The embodiment study shows that positional reachability overstates orientation-aware task coverage and that modest shoulder motion can substantially expand the effective workspace. Teleoperation experiments show that parallelizing body-part control reduces task completion time by 43-53%, while compliant control improves success and reduces peak contact forces on constrained and contact-rich tasks. Policy-learning experiments show that demonstrations must capture the visual information underlying action choices: active-search demonstrations improve arm-selection accuracy from chance (50%) to 100%. Calibration experiments reveal that mixed-calibration training data remain usable in our tested configurations when an inference-aligned subset is retained, whereas inference-time miscalibration causes all 30 trials to fail. Policies also retain approximately 90% of the extra path length in detour-contaminated demonstrations, showing how inefficient operator motions persist in learned behavior. Together, these findings connect collection-system design to demonstration quality and downstream policy execution. They motivate evaluating effective workspace, control efficiency, observation completeness, calibration alignment, and motion quality before scaling collection, and addressing system-induced compensations at their source.
Figures & tables
Figure 2: Orientation coverage of three morphologies. Top row: right-arm reachability and coverage in a vertical slice through the right shoulder; each position is tested against 64 discretized tool-axis directions, color shows the fraction that is solvable, and gray dots are unreachable (a position counts as reachable if at least one direction is solvable). Middle and bottom rows: coverage of the downward subset (9 directions within 45 ∘ of gravity) and the forward subset (11 directions within 45 ∘ of +X ) on a horizontal slab just above the table, each morphology at its optimal table height; the dashed rectangle marks the task region (table footprint, 5–35 cm high).
Figure 3: Effect of shoulder translation on humanoid and non-humanoid workspaces. Representative link lengths and joint limits for each class are derived from 25 mainstream robot URDFs, and planar reachable cross-sections are compared at four fixed end-effector orientations φ . Within each orientation: (a) humanoid, where dark orange is the reachable region with a fixed shoulder and light orange the extension when the shoulder may translate horizontally and vertically; (b) non-humanoid; (c) overlap of the two classes. Axes in mm.
Dataset
Pitch / roll
Yaw / roll
AMASS ( Mahmood et al., 2019 )
1.61 [1.51, 1.70]
1.88 [1.72, 2.05]
PICO motion capture
1.52 [1.22, 1.87]
1.21 [1.04, 1.43]
Table 1: Waist rotation relative to roll in human motion data. Geometric means of the pitch/roll and yaw/roll ratios of spine3-relative-to-pelvis rotation in a forward–left–up frame, with grouped-bootstrap 95% confidence intervals.
Figure 4: Effect of end-effector length on coverage and joint-limit cost. Left and middle: TCP distances of 0.14308 m and 0.24308 m over the same 10,075 sampled positions. Targets constrain TCP position and tool-axis direction, with free rotation about the tool axis. Color shows the 75th percentile (P75) of joint-limit cost over successful directions (shared color scale; blue is low, red is high). Right: P75(extended) − P75(original) over directions solvable in both; positive values indicate higher cost after extension.
Figure 5: Input-channel mapping. Left (1-to-N): a single input channel drives all body parts, which take turns via mode switching. Right (M-to-N): three channels drive five parts—headset for base and head, IK for the waist, and controllers for arms and grippers—so all parts can move at once.
Figure 6: Action sequences under the three configurations for the floor-pick task (configurations as in Table 2 ). Colors distinguish base, waist, head, arms, and end effectors; hatching marks actions solved automatically by IK. Total durations are measured medians over 30 trials; action order comes from task logs, while individual segment lengths are evenly allocated for illustration and do not estimate per-action durations.
Task
A
B
C
Floor pick → table behind
35.7
26.7 ( − 25%)
16.7 ( − 53%)
Wipe two spots on a table
27.0
19.3 ( − 28.5%)
15.3 ( − 43%)
Table 2: Completion time after decoupling control of body parts , median of 30 trials per configuration (s); parentheses give the reduction relative to A. A: every body part on the hand controllers; B: A plus IK control of the waist; C: B plus headset control of base and head.
Figure 7: Replays of representative demonstrations under absolute and relative mapping , shown as frames sampled at equal intervals from the operator’s tracked body motion.
Success (%)
Peak force (N)
Task
Contact type
Force
Position
Force
Position
Block stacking
Pick and place
100
100
12.1
11.2
Door opening
Mechanism-constrained
100
83.3
34.6
120.5
Whiteboard wiping
Contact-rich
100
80.0
29.2
94.4
Box carrying (L / R hand)
Bimanual closed chain
100
86.7
32.7 / 25.7
72.6 / 85.3
Table 3: Success rate (%) by task and control mode , 30 trials per configuration, and peak contact force (N) on representative trajectories.
Category
Hours
Share
Clearly requires compliant control
38,353
18.5%
Benefits from compliant control
80,161
38.7%
Table 4: Share of collection hours needing compliant control , out of 207,030 hours across 20,253 task records.
Figure 8: Representative end-effector contact-force profiles under position control (gray) and force control (orange). Vertical axes: contact force magnitude (N).
Figure 9: Delay accumulation in teleoperation. The command path runs from operator input through the network link ( Tctrl ) and control pipeline ( Tproc : pose filter, retargeting, IK, joint filter) to the body parts, which respond in parallel ( Ti ); visual feedback returns through the robot camera and video link ( Tvideo ). Timing is measured from command issue to display of the matching feedback.
Figure 10: Latency sensitivity by target size. Operators control the robot end effector in simulation to reach randomly generated target positions and orientations as quickly as possible, with target sizes of 2, 4, and 8 cm and varying injected latency; 10 trials per condition. The horizontal axis is the injected latency; the system runs at 30 Hz, giving a fixed baseline latency of about 33 ms.
Display
Mean
SD
Sessions used
2D screen
62.11
13.82
9 / 9
3D screen
63.14
9.84
7 / 9
VR headset
18.71
5.35
7 / 9
Table 5: Time to discomfort by display device : session duration before reported discomfort (min), nine sessions per device. Designated outliers (3D screen: 17 and 25 min; VR headset: 3 and 8 min) are excluded from the statistics.
Figure 11: Evaluation tasks used in the data-quality experiments: bimanual block transfer (§ 4.3 ), out-of-view block pick-and-place (§ 4.1 ), and test-tube handover and insertion (§ 4.2 ).
Figure 12: Demonstration setups. LOS-known: the operator sees the block, the robot camera does not, and the arm is chosen before seeing. BLOS-known: neither operator nor robot camera sees the block initially, and the arm is still chosen before seeing. BLOS-search: the robot camera sees the block before the arm is chosen.
Protocol
Correct arm (%)
Pick-and-place success (%)
LOS-known
50
33
BLOS-known
53
40
BLOS-search
100
60
Table 6: Arm choice and pick-and-place success for the three demonstration protocols, 30 trials each. A random arm choice is correct 50% of the time.
Figure 13: Calibration items affecting robot data : joint zero, kinematics (MDH), torque linearity, torque-sensor zero, dynamics, and camera, followed by a whole-robot consistency check. Joint zero (orange outline) is the item varied in our experiment.
Figure 14: Training mixtures for the zero-offset experiment. Each mix has 100 episodes in four groups of 25. A: all clean. A′ : 25 clean plus groups Q, R, S with biased offsets. B: ±P , ±Q′ , no clean data, zero-mean offsets. B′ : P, Q, R, S, no clean data, biased offsets. The table lists per-joint offsets (degrees) on J1–J7; bars show real-robot pick-and-place success.
Figure 15: How each mixture shifts the joint-state distribution , shown for two representative joints across the four training mixes.
Figure 16: Speed profiles of clean and detour demonstrations (top) and of the corresponding policies at inference (bottom).
Detour share in training
Metric
Source
0%
20%
50%
80%
Joint acceleration RMS (rad/s 2 )
Demo
0.26
0.31
0.40
0.48
Policy
0.25
0.24
0.30
0.32
Joint jerk RMS (rad/s 3 )
Demo
3.29
3.82
4.73
5.46
Policy
3.23
3.28
3.98
4.21
Wrist path length (m)
Demo
0.83
0.88
0.97
1.04
Table 7: Motion metrics versus detour share in training. Demo: the collected demonstrations of each mixture; Policy: real-robot rollouts of the policy trained on it. Values are means across runs. Handover mismatch is ∣vnew−vold∣/∣vold∣ at the boundary between consecutive action chunks.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 17: What a zero-position offset does to the data. The same recorded episode, shown clean and with a fixed offset on every joint. The offset shifts the recorded state, cancels in the delta action, and the resulting end-effector error depends on the pose. Gray: clean recording; orange: the same recording with the offset.
Figure 18: The same command lands somewhere else on a miscalibrated robot. A policy outputs joint angles; the robot that executes them has its own zero offsets, so the end effector misses the target.
Human demonstrations offer a scalable way to collect manipulation data, but their contacts may be unstable or infeasible when transferred to a robot hand. Collecting demonstrations directly on the target robot avoids this mismatch but substantially increases the cost of data collection. To address this trade-off, we present Touch2Robot, a framework that lets humans collect demonstrations while seeing how the target robot hand would contact the object. We capture human hand motion, tactile-glove measurements, and object motion during human manipulation. These recordings guide object-specific RL policies to reproduce the demonstrated object motion while favoring contacts consistent with the recorded human touch. We distill the learned behaviors into a unified real-time retargeter that maps incoming human observations and object geometry to robot hand configurations. During collection, the predicted robot configuration is synchronized with the tracked object pose in simulation to reconstruct robot-object contacts, which are visualized to help the demonstrator adapt subsequent interactions to the target hand. Across four real-world tasks, Touch2Robot improves average real-robot replay completion from 37.9% to 72.1% over visual-only feedback, while reducing the collection time per replay-successful demonstration from 58.6s to 18.2s. Reconstructed target-hand contacts achieve 44.2% F1 against real-robot tactile measurements, and policies trained on Touch2Robot demonstrations improve downstream Diffusion Policy performance by 29.1 percentage points over visual-only feedback. These results show that bringing robot touch into the human demonstration loop improves both the quality and efficiency of scalable dexterous data collection. Project webpage: https://Touch2Robot.github.io/.
Shengcheng Luo, Xiaoyang Cheng, Hong Ying +6
ShanghaiTech University · Beijing Institute for General Artificial Intelligence (BIGAI) · Shanghai Jiao Tong University +1
Adapting visuomotor policies to new manipulation tasks often requires substantial manual engineering or teleoperated data collection. Simulation can provide task-specific data at scale, but constructing the scene, designing expert behavior, and configuring data generation still require significant per-task effort. We present ARSTAG, an agentic Real2Sim2Real system that turns a single RGB image and a natural-language instruction directly into robot policy-learning data. A hierarchy of language agents constructs a task-scoped simulation scene, generates robot-feasible demonstrations, and expands the training distribution through task-consistent randomization, while a coordinator agent manages cross-stage feedback and recovery. Across seven manipulation tasks spanning grasping, placement, and stacking, the ARSTAG-generated demonstrations enable sim-to-real transfer of three visuomotor policy architectures to a dual-arm robot, with pi0.5 achieving an average real-world success rate of 74.6%. Ablations show that task-consistent randomization substantially improves robustness, and policy performance increases with generated dataset size. Project webpage: https://boweili666.github.io/ARSTAG/.
Bowei Li, Yuner Zhang, Changliu Liu
Carnegie Mellon University, Pittsburgh, PA 15213, USA
How can robot policies learn more effectively from a fixed demonstration budget? The first Real-world Embodied AI Learning (REAL-I) Challenge at ICRA 2026 examined this question through simulation, real-robot evaluation, and an on-site final on a shared dual-arm humanoid platform. We describe the challenge tasks, data and deployment interfaces, and competition results, then compare the approaches contributed by NUS-CLEAR, RCL-Lab, and DeepTouch AI. Their systems combined pretrained vision-language-action models and task-specific imitation policies with different strategies for data curation, staged adaptation, checkpoint selection, and action-space design. The team reports highlight the importance of adapting to the deployment environment while retaining prior capabilities, treating demonstration quality at an appropriate temporal scale, and suppressing errors in inactive robot components. They also expose the limitations of offline action-prediction metrics for forecasting closed-loop success. These observations motivate a view of fixed-data robot learning that integrates data, adaptation, evaluation, and deployment.