Precise SE(3) End-Effector Tracking in Whole-Body Humanoid Control

Joohwan Seo; Xiaofeng Guo; Jinkun Cao; Roberto Horowitz; Rocky Duan; Guanya Shi; Koushil Sreenath

cs.ROOct 7, 2026

Precise SE(3) End-Effector Tracking in Whole-Body Humanoid Control

Authors: Joohwan Seo, Xiaofeng Guo, Jinkun Cao, Roberto Horowitz, Rocky Duan, Guanya Shi, Koushil Sreenath

Organizations: University of California, Berkeley · Amazon FAR · Carnegie Mellon University

Abstract

Precise end-effector tracking during humanoid whole-body motion is challenging due to floating-base oscillations, gravity, dynamic coupling, and locomotion-induced disturbances. We propose ResGAC, a whole-body humanoid controller for precise end-effector pose tracking that combines geometric admittance control (GAC) with residual reinforcement learning. GAC provides structured $\SE$ task-space feedback and generates nominal arm joint-position targets, while residual RL compensates for unmodeled dynamics and coordinates locomotion and balance in the shared joint-position action space. The left-invariant geometric formulation allows the same GAC law to be used across manipulation reference frames. This enables the use of a ground-attached heading frame that preserves planar locomotion while removing pelvis roll, pitch, and heave from the manipulation reference, thereby reducing reference-induced end-effector motion during locomotion. ResGAC is validated on a real Unitree G1 humanoid. Across four standing end-effector tracking benchmarks, ResGAC consistently outperforms representative baselines, including SONIC, achieving lower translational and rotational errors. Real-world experiments further demonstrate reduced propagation of pelvis motion to the desired end-effector pose using the proposed ground-attached heading frame. ResGAC achieves $90\%$ success in a standing peg-in-hole task compared with $50\%$ for SONIC, and accurate world-frame $\SE$ end-effector pose tracking during lower-body motion. Experimental videos are included in the supplementary material and are also available on the project website: https://resgac.github.io/ResGAC-website/.

Figures & tables

Group	Term	Form	$\sigma$ / Param.	$w_{i}$
Manip.	EEF position	$\exp{(L_{2}^{2}+L_{1})}$	$0.001/0.01$	$4.0$
	EEF orientation	$\exp{(SO(3))}$	$0.02$	$3.0$
Loco.	Linear velocity	$\exp{(L_{2}^{2})}$	$0.25$	$2.0$
	Angular velocity	$\exp{(L_{2}^{2})}$	$0.25$	$1.5$
	Swing-foot trajectory	$\exp{(L_{2}^{2})}$	$0.008$	$5.0$
	Pelvis orientation	$\exp{(L_{2}^{2})}$	$0.04$	$1.0$

TABLE I : Reward terms used for ResGAC training in the final curriculum phase. The total reward is

r_{t}=\sum_{i}w_{i}r_{i,t}

\exp(\cdot)

denotes an exponential kernel. Values correspond to the final training stage; curriculum-dependent changes are summarized separately.

	A	B	C	D	E	F
Task	Stand Fixed	Stand Traj.	Walk Fixed	Walk Traj.	Walk Traj.	Walk Traj.
Iterations	$\leq 10$ k	$\leq 10$ k	$\leq 20$ k	$50$ k	$60$ k	$60$ k
Cmd. Range	$0$	$0$	$.3/.3/.5$	$.7/.7/.7$	$.7/.7/.7$	$.7/.7/.7$
$\sigma_{p}$	$.02$	$.01$	$.02$	$.005\!\rightarrow$ $.001$	$.001/.01$	$.001/.01$
$\sigma_{R}$	$.20$	$.10$	$.10$	$.10\!\rightarrow$ $.02$	$.02$	$.02$
$\sigma_{v}$	$.25$	$.25$	$.25$	$.25\!\rightarrow$ $.10$	$.10$	$.10$

TABLE II : Progressive training curriculum of ResGAC. Command ranges denote maximum absolute

(v_{x},v_{y},\omega_{z})

commands. For stages E and F,

\sigma_{p}

denotes

(\sigma_{2},\sigma_{1})

of the combined

L_{2}^{2}+L_{1}

kernel.

Controller	Smooth		Setpoint		Infinity		Pick-and-Place
Controller	Pos. [mm]	Rot. [ ∘ ]	Pos. [mm]	Rot. [ ∘ ]	Pos. [mm]	Rot. [ ∘ ]	Pos. [mm]	Rot. [ ∘ ]
E2E-RL	43.6 $\pm$ 31.1	9.36 $\pm$ 7.47	33.9 $\pm$ 20.9	4.87 $\pm$ 3.24	33.6 $\pm$ 14.9	8.36 $\pm$ 4.42	35.5 $\pm$ 31.6	6.02 $\pm$ 1.56
GAC+Decoupled-RL	98.5 $\pm$ 32.5	14.80 $\pm$ 6.16	94.5 $\pm$ 24.9	13.31 $\pm$ 3.78	50.6 $\pm$ 12.6	9.47 $\pm$ 1.87	74.1 $\pm$ 37.5	12.25 $\pm$ 3.68
SONIC-IK-WBT	41.7 $\pm$ 27.6	18.86 $\pm$ 13.37	25.1 $\pm$ 17.6	15.42 $\pm$ 11.01	32.3 $\pm$ 32.4	14.06 $\pm$ 13.01	21.9 $\pm$ 5.9	15.31 $\pm$ 5.94
SONIC-Teleop	39.6 $\pm$ 40.3	23.23 $\pm$ 20.52	25.3 $\pm$ 19.6	18.32 $\pm$ 11.56	18.4 $\pm$ 9.0	11.00 $\pm$ 5.40	17.3 $\pm$ 9.9	24.63 $\pm$ 9.09
SONIC-Teleop-v1.1	34.3 $\pm$ 29.0	17.60 $\pm$ 26.86	19.7 $\pm$ 9.6	8.65 $\pm$ 8.63	15.3 $\pm$ 6.4	6.44 $\pm$ 2.77	14.5 $\pm$ 3.9	6.23 $\pm$ 3.17

TABLE III : End-effector tracking performance during standing in real-world hardware experiment. Errors are reported as mean

\pm

standard deviation of the per-sample error magnitudes, aggregated over temporal samples and both arms. For the setpoints tracking, only steady-state samples after each command transition are included. Lower values indicate better tracking performance, with the best and second-best results shown in bold and underlined, respectively. The command reference frame is the pelvis frame

\{P\}

for all controllers except SONIC-Teleop-v1.1, which uses the heading frame

\{H\}

Fig. 2 : End-effector error convergence for the setpoint arm-sweeping trajectory. The

25

setpoint intervals are aligned at each command transition (

t=0

) and aggregated. Solid curves indicate the mean position and orientation error magnitudes, while shaded regions indicate one standard deviation.

	Ground-Heading ( $H_{0}$ )			Pelvis ( $P$ )
Metric	Pelvis Std.	EEF MAE	Ratio	Pelvis Std.	EEF MAE	Ratio
Roll [ ∘ ]	1.218	0.807	0.790	1.322	1.113	0.958
Pitch [ ∘ ]	3.794	2.600	0.954	4.107	2.757	0.964
$z$ [mm]	6.280	12.56	0.884	6.219	22.50	1.068

TABLE IV : Comparison of pelvis-frame

P

and ground-attached heading-frame

H_{0}

representations during walking arm holding using ResGAC. “EEF MAE” denotes the mean absolute error (MAE) from the fixed world-frame hold target, while “Ratio” denotes the motion-transmission ratio

\rho=\mathrm{EEF~Std.}/\mathrm{Pelvis~Std.}

. Lower values indicate better world-frame holding and weaker transmission, respectively.

Fig. 3 : World-frame

YZ

tracking in the Mime-man demo, comparing ResGAC and E2E-RL. ResGAC achieves position and orientation MAEs of

\mathbf{8.8\;\mathrm{mm}}

\mathbf{2.69^{\circ}}

, respectively, compared with

\mathbf{30.0\;\mathrm{mm}}

and

\mathbf{13.75^{\circ}}

for E2E-RL.

Explore similar work

Learning Holistic Whole-Body Loco-Manipulation with a Bipedal Mobile Manipulator
Sep 16, 2026Zhongyu Chen, Yuxuan Nai, Qian Chen +8Whole-Body Control Humanoid Loco-Manipulation
ZJU-UIUC Institute. · LimX Dynamics.
PAKE: Learning Whole-Body Loco-Manipulation with Partial Kinematic Embeddings
Jul 13, 2026Zhengmao He, Moonkyu Jung, Hyeongjun Kim +4Humanoid Loco-Manipulation Legged Robots
The Hong Kong University of Science and Technology (Guangzhou). · Korea Advanced Institute of Science and Technology. · ETH Zurich. +1
M3imic: Learning a Versatile Whole-Body Controller for Multimodal Motion Mimicking
Jun 3, 2026Zuxing Lu, Ziang Zheng, Yao Lyu +7Whole-Body Control Mimic
School of Automation, Southeast University, Nanjing 210096, China · School of Vehicle and Mobility, Tsinghua University, Beijing 100084, China · Department of Robotics, Mohamed Bin Zayed University of Artificial Intelligence, Abu Dhabi 7909, UAE

Sep 16, 2026, cs.ROJ/K move · Enter open · S save

Learning Holistic Whole-Body Loco-Manipulation with a Bipedal Mobile Manipulator

Zhongyu Chen, Yuxuan Nai, Qian Chen, Yidong Zhu+7

ZJU-UIUC Institute. · LimX Dynamics.

Whole-Body Control Humanoid Loco-Manipulation

Open paper AskPDF arXiv

Bipedal loco-manipulation enables robots to interact with objects beyond the nominal workspace of their arms by coordinating locomotion and manipulation. Realizing this capability requires a low-level whole-body controller that translates task-level manipulation goals into coordinated arm and leg motions while maintaining balance. We present a unified whole-body controller trained with reinforcement learning that directly maps 6-DoF end-effector targets to coordinated actions for the bipedal base and robotic arm. Given only an end-effector target, the learned controller autonomously coordinates reaching, postural adaptation, and stepping without explicit base-velocity or footstep commands. A reward-gating strategy regulates the trade-offs among end-effector tracking, locomotion, and balance during training, while a temporal context estimator combines windowed Transformer encoding, recurrent GRU memory, and auxiliary dynamics prediction to extract dynamics-relevant information from observation history. Real-robot experiments demonstrate that the same controller supports reaching, postural adaptation, and stepping under commands from VR teleoperation, a learned diffusion policy, and scripted trajectories, providing a common end-effector interface for diverse manipulation tasks.

Abstract

Explore similar work

Learning Holistic Whole-Body Loco-Manipulation with a Bipedal Mobile Manipulator

PAKE: Learning Whole-Body Loco-Manipulation with Partial Kinematic Embeddings

M3imic: Learning a Versatile Whole-Body Controller for Multimodal Motion Mimicking