Force-Aware Reinforcement Learning with Hybrid Sensorless Force Estimation for Wheeled-Legged Loco-Manipulation

Xuanqi Zeng; Jiaming Wang; Tianlin Zhang; Lingwei Zhang; Botian Xu; Weipeng Xia; Zhongyu Li; Yun-Hui Liu

cs.ROSep 12, 2026

Force-Aware Reinforcement Learning with Hybrid Sensorless Force Estimation for Wheeled-Legged Loco-Manipulation

Authors: Xuanqi Zeng, Jiaming Wang, Tianlin Zhang, Lingwei Zhang, Botian Xu, Weipeng Xia, Zhongyu Li, Yun-Hui Liu

Organizations: Department of Mechanical and Automation Engineering at the Chinese University of Hong Kong, Shatin, Hong Kong

Abstract

Force-controlled loco-manipulation requires a whole-body policy to coordinate locomotion and arm motion while regulating end-effector interaction forces. This is challenging under floating-base dynamics and changing support contacts, particularly when end-effector force/torque sensing is unavailable for control. This paper presents a force-aware reinforcement learning approach with hybrid sensorless force estimation for wheeled-legged loco-manipulation. The proposed method provides a structured estimate of the end-effector force as an explicit policy observation, enabling force-guided contact behavior without using an end-effector force/torque sensor for control. The force estimate is obtained by combining generalized momentum observation, contact-constrained wrench projection, and temporal residual learning: the model-based components extract the physically structured part of the whole-body disturbance, while the residual network compensates the remaining motion-dependent bias. The estimated force is integrated into a mode-conditioned whole-body policy with an axis-wise force/position selector, allowing free-space motion, pure force regulation, and hybrid force/position control within one controller. Simulation results demonstrate improved sensorless force estimation and force-control performance. Hardware experiments further validate the proposed controller through quantitative valve-rotation and hybrid wiping evaluations, together with force-guided door opening and zero-force human-guided motion on a real wheeled-legged platform.

Figures & tables

Fig. 1: Hardware demonstrations of the proposed force-aware controller: (A) valve rotation with whole-body motion, (B) hybrid force/position board wiping, (C) opening a spring-loaded door by force control, and (D) compliant zero-force/human-guidance interaction. These tasks require the robot to coordinate locomotion, arm motion, and physical interaction with the environment.

Fig. 2: Conceptual comparison and simplified pipeline for force-controlled loco-manipulation. Existing methods represent interaction through learned force estimates [ 12 ] or history features with or without auxiliary force prediction [ 13 , 14 ] . The proposed method combines a physics-prior estimate $\hat{\bm{f}}_{\mathrm{e}}^{\mathrm{obs}}$ with a learned residual $\Delta\hat{\bm{f}}_{\mathrm{e}}$ to obtain a structured force estimate $\hat{\bm{f}}_{\mathrm{e}}$ . The force estimation supports more task-effective position, force, and hybrid force/position behaviors.

Fig. 3: Overview of the proposed physics-structured interaction representation for force-aware loco-manipulation. The hybrid sensorless estimator combines physics-structured estimation and temporal residual learning to produce the corrected end-effector force estimate $\hat{\bm{f}}_{\mathrm{e}}$ . This estimate is provided to the RL policy together with proprioceptive observations, locomotion commands, and end-effector commands. During training, virtual spring–damper interactions and a multi-critic formulation are used to learn both locomotion and manipulation objectives. The learned policy uses the estimated force feedback for position control, force regulation, and hybrid force/position loco-manipulation without an end-effector force/torque sensor.

Fig. 4: Representative

f_{z}

force-estimation results under inertial-parameter mismatch. The inertial parameters used by the generalized momentum observer and wrench projection are kept fixed, while the simulated robot inertial parameters are scaled by

\alpha_{m}=0.8

1.0

, and

1.2

. The compared methods are GMO-PI, GMO-CQP, TCN, and the proposed estimator (Ours).

$\alpha_{m}$	Method	$e_{x}$	$e_{y}$	$e_{z}$	$e_{\mathrm{vec}}$
0.8	GMO-PI	5.89	5.76	14.76	$16.97\pm 0.91$
0.8	GMO-CQP	1.33	1.25	5.69	$5.99\pm 0.13$
0.8	TCN	2.90	3.24	3.10	$5.41\pm 0.33$
0.8	Ours	1.17	1.11	1.17	$\bm{2.02\pm 0.31}$
1.0	GMO-PI	5.37	5.67	3.76	$8.72\pm 1.11$
1.0	GMO-CQP	1.19	1.19	1.14	$2.06\pm 0.39$

TABLE I: Force-estimation RMSE under inertial-parameter mismatch.

Fig. 5: Pure force-control comparison in the lateral box-dragging task. The snapshots on the right side show the resulting box motion, where the proposed method moves the box closer to the expected lateral displacement range. The left plots show the ground-truth tangential force $f_{y}$ and normal force $f_{z}$ generated by each controller. Compared with the reproduced baselines [ 12 , 13 ] , the proposed method tracks the commanded $f_{y}$ more accurately while maintaining $f_{z}$ near the desired $20~\mathrm{N}$ level.

Method	$e_{y}$ [N]	$e_{z}$ [N]	$y_{\mathrm{box}}$ [m]
Portela [ 12 ]	$6.21\pm 0.19$	$12.69\pm 0.80$	$[-0.199,\,0.007]$
Zhi [ 13 ]	$5.42\pm 0.03$	$5.44\pm 0.06$	$[-0.049,\,0.039]$
Ours	$\bm{0.98\pm 0.04}$	$\bm{3.98\pm 0.04}$	$\bm{[-0.205,\,0.220]}$

TABLE II: Box-dragging performance over ten trials.

Fig. 6: Representative temporal sequences of the hardware experiments. (A) Valve rotation while coordinating arm interaction and whole-body motion. (B) Hybrid force/position wiping with tangential end-effector motion and sustained normal contact. (C) Force-guided opening of a spring-loaded door through coordinated manipulation and base motion. (D) Zero-force human-guided motion, in which external interaction is used to guide the whole-body robot motion. Images in each row are ordered from left to right in time.

Fig. 7: Representative valve-rotation trial during in-place manipulation and manipulation during locomotion. The top panel shows the valve rotation angle, while the middle and bottom panels compare the commanded, estimated, and measured interaction forces along the

y

and

z

axes. The dashed vertical line marks the transition from in-place manipulation to manipulation during locomotion.

Fig. 8: Representative hybrid force/position wiping trial. The desired and kinematically estimated end-effector positions along the board-tangential

x

and

y

directions are shown in the top and middle panels, respectively. The bottom panel shows the commanded normal force, the proposed sensorless force estimate, and the external F/T sensor measurement.

Robotic Loco-Manipulation

See all 30 papers

Oct 26, 2026

Learning Force-Reactive Humanoid Loco-Manipulation

Many everyday tasks require controlling contact force as well as motion. On a humanoid, this force must often be coordinated across both hands and the whole body, and cannot be specified by kinematic trajectories alone. Therefore, we present FLORE, a framework for learning force-reactive humanoid loco-manipulation. We collect robot-free demonstrations using two force-sensing UMI-style handheld grippers for hand motion and contact wrenches, and wearable trackers for body motion. A force-reactive diffusion policy takes vision, proprioception, and measured wrenches as input, then predicts actions and per-hand force commands. A hybrid compliance whole-body controller with force tracking executes these commands while maintaining balance: an integral of the force error measured by real force/torque (F/T) sensors lets each hand track force accurately along the commanded direction and remain compliant along the others. In simulation, the controller tracks per-hand force commands of varied magnitude and direction, with little loss in whole-body motion accuracy. Across three contact-rich tasks on the robot, the force-reactive policy succeeds in 80% of rollouts, compared with 26.7% for a position-only policy trained on the same demonstrations and executed by the same controller. For more details, please visit our website: https://louis-leee.github.io/FLoRe/.

Sihang Li, Xiaofeng Guo, Jinkun Cao +5

Amazon FAR (Frontier AI & Robotics) · New York University · Carnegie Mellon University +1

Humanoid Loco-Manipulation Whole-Body Robotic Manipulation Force-Controlled Robotic Manipulation Compliant Robot Control

Ask PDF

Sep 3, 2026

FWBC-VLA: Force-Aware Whole-Body Compensation for Contact-Rich Loco-Manipulation

Contact-rich loco-manipulation requires a bridge between semantic action generation and physical interaction control. Existing Vision-language-action (VLA) models generate task-level actions from visual and linguistic observations, but cannot interpret the physical interactions induced by those actions. While the whole-body control (WBC) policy can stabilize the robot, it cannot distinguish task-relevant interaction forces from forces induced by external disturbances during manipulation. Although force/torque sensors provide direct measurements of physical interactions, retrofitting them entails additional hardware costs and substantial integration effort, particularly for platforms not designed with sensor integration in mind. To address this problem, we propose FWBC-VLA, a force-aware framework that bridges task-level VLA action generation and low-level whole-body compensation control for wheeled-legged robots. First, we introduce HSR-Force, a sensorless residual-torque estimator for inferring contact strength and its temporal variation. These contact estimates are then encoded as tokens and injected into the VLA action expert during action decoding, enabling the policy to perceive contact onset, sustained loading, and release. For loco-manipulation tasks, all parameters of the pretrained VLA backbone are fine-tuned on our WL&Arm Dataset, which comprises more than 5,000 episodes. Moreover, the robot's proprioceptive state, the Jacobian-derived body-frame force estimate, and the estimated contact state are jointly fed into a compensation generator to produce corrective actions. The manipulation-centric actions are subsequently combined with the corrective actions and passed to the WBC policy for execution. Real-world experiments on whiteboard wiping and door opening with a door closer demonstrate the effectiveness of our FWBC-VLA in contact-rich loco-manipulation.

Yutian Zhang, Siyuan Ma, Liwen Yang +6

1Zhejiang University · 3Tsinghua University · 2Shanghai Artificial Intelligence Laboratory +2

Contact-Rich Robotic Manipulation Whole-Body Control VLMs for Robotics Loco-Manipulation

Ask PDF

Apr 29, 2026

Learning Tactile-Aware Quadrupedal Loco-Manipulation Policies

Quadrupedal loco-manipulation is commonly built on visual perception and proprioception. Yet reliable contact-rich manipulation remains difficult: vision and proprioception alone cannot resolve uncertain, evolving interactions with the environment. Tactile sensing offers direct contact observability, but scalable tactile-aware learning framework for quadrupedal loco-manipulation is still underexplored. In this paper, we present a tactile-aware loco-manipulation policy learning pipeline with a hierarchical structure. Our approach has two key components. First, we leverage real-world human demonstrations to train a tactile-conditioned visuotactile high-level policy. This policy predicts not only end-effector trajectories for manipulation, but also the evolving tactile interaction cues that characterize how contact should develop over time. Second, we perform large-scale reinforcement learning in simulation to learn a tactile-aware whole-body control policy that tracks diverse commanded trajectories and tactile interaction cues, and transfers zero-shot to the real world. Together, these components enable coordinated locomotion and manipulation under contact-rich scenarios. We evaluate the system on real-world contact-rich tasks, including in-hand reorientation with insertion, valve tightening, and delicate object manipulation. Compared to vision-only and visuotactile baselines, our method improves performance by 28.54% on average across these tasks.

Pokuang Zhou, Yuhao Zhou, Quan Khanh Luu +7

Purdue University, West Lafayette, IN, USA. · Istituto Italiano di Tecnologia, Genoa, Italy. · Columbia University, New York, NY, USA.

Legged Robotics RL for Robotics Contact-Rich Robotic Manipulation Tactile Sensing for Robotic Manipulation Robotic Loco-Manipulation

Ask PDF

Abstract

Robotic Loco-Manipulation

Explore similar work

Learning Force-Reactive Humanoid Loco-Manipulation

FWBC-VLA: Force-Aware Whole-Body Compensation for Contact-Rich Loco-Manipulation

Learning Tactile-Aware Quadrupedal Loco-Manipulation Policies