Contact-Guided Exploration for Non-Prehensile Locomanipulation with Multi-Critic RL
Authors: Simone Tolomei, Mayank Mittal, Franco Angelini, Manolo Garabini, Paolo Salaris, Marco Hutter
Organizations: Centro di Ricerca E. Piaggio, Dipartimento di Ingegneria dell’Informazione, Universit`a di Pisa, Italy · ETH Zürich, Switzerland · NVIDIA
Non-prehensile manipulation offers versatile skills for moving and rearranging heavy or bulky objects, particularly when combined with a mobile manipulation platform. However, both model-based and model-free approaches struggle with the complex hybrid dynamics and the sparsity of the contact in these tasks. To address these challenges, we propose a contact-guided exploration strategy implemented within a Multi-Critic Reinforcement Learning (RL) framework. A dedicated exploration critic is trained with a dense contact-seeking reward that guides the end-effector toward meaningful contact points; its influence is progressively decayed to recover a task-optimal policy. We obtain candidate interaction points from a general-purpose grasping algorithm, enabling the exploration mechanism to generalise across various object geometries. We evaluate the approach on multiple tasks, including box pushing, chair transportation, and a dishwasher opening task. Finally, we validate the chair transportation policy through extensive experiments on a quadrupedal mobile manipulator, demonstrating deployable non-prehensile manipulation in the real world.
Figures & tables
Fig. 1: Learning Non-Prehensile Manipulation via Contact-guided exploration. We guide reinforcement learning exploration using object-centric contact priors. The colored dashed lines illustrate the learned policy’s ability to execute contact-rich behaviours (e.g., pushing a chair to a target) by focusing on high-probability interaction regions.
Fig. 2: Overview of the System Architecture. (Top Left) Contact candidates are generated from object meshes using surface sampling or grasping algorithm to guide exploration. (Center) The Multi-Critic module employs a shared backbone with three specialized heads for task ( rtask ), exploration ( rexp ), and regularization ( rreg ) rewards. These are combined using a schedule-weighted advantage ∑Aiwi , where the influence of exploration ( wexp ) and regularization ( wreg ) is decayed according to a predefined schedule. (Bottom Left) The Actor utilizes an LSTM-based policy to output target arm joint positions qa∗ , base height, and base velocity. The base commands are realized by a frozen, pre-trained locomotion policy that generates low-level leg joint targets ql∗ . (Right) The versatility of the architecture is demonstrated across three non-prehensile tasks: chair transportation, box pushing, and dishwasher opening.
Fig. 3: Contact Candidates on Complex Geometries. Red markers indicate candidate contact points computed via a grasping algorithm. During training, a single candidate point is sampled to serve as a target for the end-effector position. The dataset includes both procedurally randomized chair models and a curated subset of CAD models from the IKEA furniture dataset.
Reward Term
Weight
Task Group (Goal Achievement: rtask )
Object Vel. to Goal
10.0
Object Pos. Tracking
15.0
Group Weight wtask
0.75
Exploration Group (Contact Guidance: rexp )
EE Tracking Contact Point
3.0
TABLE I: Reward Structure and Simulation Parameters
Fig. 4: Manipulation Strategy for Target-Directed Transportation. The learned policy exhibits an orientation-aware strategy to maximize control authority over the chair. The agent is initialized relative to the object with various target locations (colored discs). The overlapping silhouettes illustrate the continuous transition from initial approach to stabilized pushing, demonstrating the policy’s ability to balance base locomotion with precise arm coordination.
Fig. 5: Quantitative performance and failure mode analysis. Comparison of success rates and termination causes for (a) Box Pushing and (b) Chair Transportation. All methods use the contact-guided exploration reward during training. While vanilla PPO and PPO with Weight Schedule (WS) suffer from significant rates of Timeouts and Missed Contacts, the addition of the Multi-Critic architecture substantially improves efficiency. The full proposed method (Multi-Critic PPO + WS) achieves the highest success rates (over 90%), effectively mitigating Tipover events.
Fig. 6: Experimental validation and zero-shot real-world generalization. We deploy the policy on a quadrupedal manipulator and evaluate it on diverse physical geometries. Panels (a)–(c) show successful transportation of multiple chair models, while panel (d) shows robustness on a LOVBACKEN three-legged table. The policy generalizes to the table’s backrest-free topology, reorienting the base to maintain an optimal contact normal and transferring contact-guided exploration from simulation to the real world without additional fine-tuning.
Object
Successes
Total Runs
Success Rate
Primary Benchmark
ADDE
27
37
72.90%
SANDSBERG
8
14
57.14%
Case Studies
VIHALS
3
3
100.00%
LOVBACKEN
2
4
50.00%
TABLE II: Hardware Experiments on IKEA Objects
Fig. 7: Reactive recovery behavior under contact failure. Following an initial slip or failed contact attempt, the agent does not enter an unstable state but instead executes a recovery sequence: first re-positioning the base, then increasing the arm’s swing amplitude and re-establishing a stable contact point to resume the transportation task.
Fig. 8: Robustness to physical perturbations and payload changes. (a) The agent transports a chair with an additional 5 kg payload, showing robustness to large, unseen variations in inertia and friction. (b) The system rejects real-time disturbances when a human abruptly shifts the object, with the policy immediately re-planning the contact trajectory.
Fig. 9: Emergent contact sequencing for articulated object manipulation. In the dishwasher opening task, the contact-guided exploration provides candidate points (red markers) on both the handle and the door panel. The learned policy demonstrates a multi-stage strategy: (Top) the agent first uses the handle to initiate the opening motion. (Bottom) As the door kinematics change, the agent switch contact to the panel, utilizing a pushing motion to lower the door.