Force-controlled loco-manipulation requires a whole-body policy to coordinate locomotion and arm motion while regulating end-effector interaction forces. This is challenging under floating-base dynamics and changing support contacts, particularly when end-effector force/torque sensing is unavailable for control. This paper presents a force-aware reinforcement learning approach with hybrid sensorless force estimation for wheeled-legged loco-manipulation. The proposed method provides a structured estimate of the end-effector force as an explicit policy observation, enabling force-guided contact behavior without using an end-effector force/torque sensor for control. The force estimate is obtained by combining generalized momentum observation, contact-constrained wrench projection, and temporal residual learning: the model-based components extract the physically structured part of the whole-body disturbance, while the residual network compensates the remaining motion-dependent bias. The estimated force is integrated into a mode-conditioned whole-body policy with an axis-wise force/position selector, allowing free-space motion, pure force regulation, and hybrid force/position control within one controller. Simulation results demonstrate improved sensorless force estimation and force-control performance. Hardware experiments further validate the proposed controller through quantitative valve-rotation and hybrid wiping evaluations, together with force-guided door opening and zero-force human-guided motion on a real wheeled-legged platform.
Figures & tables
Fig. 1: Hardware demonstrations of the proposed force-aware controller: (A) valve rotation with whole-body motion, (B) hybrid force/position board wiping, (C) opening a spring-loaded door by force control, and (D) compliant zero-force/human-guidance interaction. These tasks require the robot to coordinate locomotion, arm motion, and physical interaction with the environment.
Fig. 2: Conceptual comparison and simplified pipeline for force-controlled loco-manipulation. Existing methods represent interaction through learned force estimates [ 12 ] or history features with or without auxiliary force prediction [ 13 , 14 ] . The proposed method combines a physics-prior estimate f^eobs with a learned residual Δf^e to obtain a structured force estimate f^e . The force estimation supports more task-effective position, force, and hybrid force/position behaviors.
Fig. 3: Overview of the proposed physics-structured interaction representation for force-aware loco-manipulation. The hybrid sensorless estimator combines physics-structured estimation and temporal residual learning to produce the corrected end-effector force estimate f^e . This estimate is provided to the RL policy together with proprioceptive observations, locomotion commands, and end-effector commands. During training, virtual spring–damper interactions and a multi-critic formulation are used to learn both locomotion and manipulation objectives. The learned policy uses the estimated force feedback for position control, force regulation, and hybrid force/position loco-manipulation without an end-effector force/torque sensor.
Fig. 4: Representative fz force-estimation results under inertial-parameter mismatch. The inertial parameters used by the generalized momentum observer and wrench projection are kept fixed, while the simulated robot inertial parameters are scaled by αm=0.8 , 1.0 , and 1.2 . The compared methods are GMO-PI, GMO-CQP, TCN, and the proposed estimator (Ours).
αm
Method
ex
ey
ez
evec
0.8
GMO-PI
5.89
5.76
14.76
16.97±0.91
0.8
GMO-CQP
1.33
1.25
5.69
5.99±0.13
0.8
TCN
2.90
3.24
3.10
5.41±0.33
0.8
Ours
1.17
1.11
1.17
2.02±0.31
1.0
GMO-PI
5.37
5.67
3.76
8.72±1.11
1.0
GMO-CQP
1.19
1.19
1.14
2.06±0.39
TABLE I: Force-estimation RMSE under inertial-parameter mismatch.
Fig. 5: Pure force-control comparison in the lateral box-dragging task. The snapshots on the right side show the resulting box motion, where the proposed method moves the box closer to the expected lateral displacement range. The left plots show the ground-truth tangential force fy and normal force fz generated by each controller. Compared with the reproduced baselines [ 12 , 13 ] , the proposed method tracks the commanded fy more accurately while maintaining fz near the desired 20N level.
Method
ey [N]
ez [N]
ybox [m]
Portela [ 12 ]
6.21±0.19
12.69±0.80
[−0.199,0.007]
Zhi [ 13 ]
5.42±0.03
5.44±0.06
[−0.049,0.039]
Ours
0.98±0.04
3.98±0.04
[−0.205,0.220]
TABLE II: Box-dragging performance over ten trials.
Fig. 6: Representative temporal sequences of the hardware experiments. (A) Valve rotation while coordinating arm interaction and whole-body motion. (B) Hybrid force/position wiping with tangential end-effector motion and sustained normal contact. (C) Force-guided opening of a spring-loaded door through coordinated manipulation and base motion. (D) Zero-force human-guided motion, in which external interaction is used to guide the whole-body robot motion. Images in each row are ordered from left to right in time.
Fig. 7: Representative valve-rotation trial during in-place manipulation and manipulation during locomotion. The top panel shows the valve rotation angle, while the middle and bottom panels compare the commanded, estimated, and measured interaction forces along the y and z axes. The dashed vertical line marks the transition from in-place manipulation to manipulation during locomotion.
Fig. 8: Representative hybrid force/position wiping trial. The desired and kinematically estimated end-effector positions along the board-tangential x and y directions are shown in the top and middle panels, respectively. The bottom panel shows the commanded normal force, the proposed sensorless force estimate, and the external F/T sensor measurement.