Organizations: Institute of Industrial Science, The University of Tokyo, Japan · Interfaculty Initiative in Information Studies, Graduate School of Interdisciplinary Information Studies, The University of Tokyo, Japan
Training humanoid robots to learn human behaviors requires expensive data collection, and human-robot interaction data can be even more difficult and costly to collect. Training a specialized reaction model for each humanoid embodiment can also be time-consuming and resource-intensive. Human-to-humanoid motion retargeting provides a more scalable alternative, allowing human-human reaction data to serve as a common motion representation for different humanoid embodiments. However, due to morphological and kinematic differences between humans and humanoid robots, a reaction preferred in human motion space may not be the most suitable one for execution on a particular robot embodiment. In this work, we propose an inference-time reaction search framework that leverages stochastic sampling from a generative reaction model and jointly considers reaction quality and humanoid embodiment feasibility. Given the observed interaction, our framework generates multiple plausible reaction candidates. Each candidate is retargeted to the target humanoid and evaluated using complementary interaction and embodiment-specific feasibility criteria, allowing candidate selection under the physical constraints of the target embodiment. Since the reaction generator remains frozen, the framework can adapt to different humanoid embodiments at inference time without retraining. Experimental results demonstrate that the proposed framework maintains overall human reaction quality while improving the feasibility of retargeted humanoid reactions across different embodiments. The proposed pipeline improves humanoid retargeting feasibility by 1.61%. Moreover, human-space and G1 best-candidate selections disagree in 78.98% of samples, demonstrating substantial embodiment dependence. Code will be released.
Figures & tables
Fig. 1: Overview of the proposed inference-time search framework. An interaction semantic condition is generated using the observed interaction history for the frozen ReMoGen proposal generator. Each of the K reactions is retargeted, filtered using human validity, robot validity, and semantic floor criteria, and scored in both human and robot space to select the final humanoid reaction candidate. The example shows a high-five interaction, where the actor is on the right and the reactor is on the left. Candidate 4 is selected over candidate 0 as it preserves the intended interaction while providing more suitable motion references for humanoid retargeting. The visualized G1 motions further illustrate how the final retargeted humanoid motion can be performed in the human–robot interaction scene.
Split / [-1pt]Semantic Input
Method
K
FID ↓
R@3 ↑
MM Dist. ↓
Diversity →
Peak Jerk ↓
Collision Proxy ↓
Contact F1 ↑
Δpy (pp) ↑
(a) Inter-X test: predicted class from observed history
Test / Predicted
ReMoGen
1
1.661
0.130
12.855
6.614
0.078
10.240
0.770
0.00
Test / Predicted
Ours
16
1.737
0.132
12.841
6.666
0.072
10.188
0.770
+0.21 [+0.10, +0.32]
(b) Validation-only semantic-input diagnostic
Val. / Predicted
ReMoGen
1
2.468
0.130
12.732
6.916
0.079
10.111
0.784
0.00
Val. / Predicted
Ours
16
2.607
0.133
12.625
6.917
0.074
10.069
0.784
+0.45
TABLE I: Human reaction quality on Inter-X. (a) Comparison using a semantic condition inferred from the observed interaction history. (b) Validation-only semantic-input diagnostic.
Fig. 2: Candidate-sampling scaling, with K=1 denoting the single-sample no-search baseline. (a) G1 feasibility gain and interaction improvement for the humanoid reaction search space. (b) Semantic gain and median search latency for the human reaction search space.
Split
Method
K
Feas. Gain -1pt ↑
Δ [-1pt]Interaction ↓
Δpy -1pt ↑
Foot [-1pt]Slide ↓
Joint [-1pt]Jerk ↓
Robot Valid -1pt ↑
(a) Official Inter-X test
Test
ReMoGen single / candidate 0
1
0.00
0.000
0.00
0.143
44.15
96.15
Test
Ours
16
+1.61 [+1.51,+1.72]
-0.120
+0.21
0.127
42.01
96.73
(b) Validation-only selector ablation
Val.
ReMoGen single / candidate 0
1
+0.00
+0.000
+0.00
0.142
46.81
94.48
Val.
Random human-valid
16
+0.94
+0.013
-0.14
0.130
43.71
94.44
TABLE II: Fixed-window humanoid retargeting on Inter-X. (a) Comparison on the official test split. (b) Selector component ablation on the validation split.
Fig. 3: Human- and robot-space candidate rankings can be different. The same observed history produces multiple human reaction candidates (top), which are retargeted to the target humanoid (bottom). The highlighted example shows a candidate ranking change due to the retargeting needs.
Space
Method
K
FID ↓ / [-1pt]Feas. ↓
MM Dist. ↓ / [-1pt]Gain ↑
Interact. ↓
Slide ↓
Jerk ↓
Jump ↓
Robot Valid -1pt ↑
(a) Human reaction quality
Human
Predicted-class ReMoGen single
1
0.665
6.099
–
–
–
–
–
Human
Predicted-class Ours
16
0.587
6.052
–
–
–
–
–
(b) Whole-branch G1 transfer
Robot
ReMoGen single full branch
1
0.848
0.00%
3.425
0.166
59.66
0.182
80.08
Robot
Ours full branch
16
0.815
+3.54 [+3.39,+3.69]%
3.408
0.132
53.23
0.166
84.74
TABLE III: Full autoregressive rollout results on the official Inter-X test split.
Fig. 4: Candidate selection differences between G1 and H1-2. The G1 best and H1-2 best candidates show the optimal retargeted motion for each morphology, and the “transfer” block shows the retargeted reaction candidate from the other morphology’s optimal candidate.
Setting
K
Feas. Gain(%) ↑
Δ Interaction ↓
Robot Valid ↑
(a) Retargeter generalization
GMR
16
+1.61
-0.120
96.15 → 96.73%
NMR
16
+1.24
-0.099
84.65 → 85.80%
(b) Dataset generalization
InterHuman fixed
16
+2.50
-0.121
80.58 → 83.74%
InterHuman full
16
+3.85
-0.034
75.84 → 79.89%
TABLE IV: Generalization across retargeters, datasets, and robot morphologies. InterHuman dataset results use the official test split. The results are improvements over the ReMoGen single-sample baseline.
Method
K /Budget
Human Semantic [-1pt]Gain (pp) ↑
G1 Semantic Δ (pp) ↑
Feas. Gain (%) ↑
Δ Interaction ↓
Robot Valid (%) ↑
(a) Flat candidate-sampling scaling
Flat sampling
1
+0.00
+0.00
+0.00
+0.000
94.48
Flat sampling
16
+0.43
+0.17
+1.55
-0.134
95.11
Flat sampling
32
+0.54
+0.12
+2.03
-0.164
95.25
Flat sampling
64
+0.74
+0.16
+2.52
-0.213
95.35
Flat sampling
128
+0.80
+0.09
+3.01
-0.248
95.67
TABLE V: Candidate-sampling scaling and matched-budget iterative refinement results.
Metric
Baseline
Ours
Δ [-1pt][95% CI]
Tracking success (%)
92.6886
92.6886
+0.0000 [-1pt][-0.1168,+0.1168]
Joint RMSE (rad)
0.1782
0.1780
-0.0002 [-1pt][-0.0005,+0.0000]
Root position RMSE (m)
0.0397
0.0393
-0.0004 [-1pt][-0.0006,-0.0003]
Root orientation RMSE (rad)
0.1692
0.1678
-0.0014 [-1pt][-0.0021,-0.0009]
Wrist RMSE (m)
0.0526
0.0521
-0.0005 [-1pt][-0.0007,-0.0004]
Torque RMS (Nm)
13.1829
13.1609
-0.0220 [-1pt][-0.0298,-0.0142]
TABLE VI: TWIST tracking performance on the Inter-X test split.
Component
Median
p90
Human reaction search, K=16
41.26
42.62
GMR, one 10-frame candidate (2 hist. + 8 pred.)
30.85
40.13
Parallel GMR + robot metrics, K=16
64.81
82.72
Complete loaded K=16 path
108.68
128.99
TABLE VII: Runtime of the search and retargeting pipeline (ms).
Institute of Humanoid Robots, Department of Precision Machinery and Precision Instrumentation, University of Science and Technology of China, Hefei, Anhui 230026, China