Contact-based robotic manipulation is constrained by end-effector reachability and the mechanical properties of objects. Airflow can exert forces without direct robot-object contact, enabling manipulation of objects beyond the reach or when direct contact is undesirable. However, the lack of suitable simulation has hindered the application of reinforcement learning to airflow-based manipulation, limiting the broader adoption of this mechanism compared with contact-based methods. To address this gap, we developed a robot- and task-agnostic airflow extension for Isaac Lab that supports massively parallel training. The extension uses experimentally identified airflow models to compute the forces and torques exerted on objects. Based on this extension, we train a hybrid push-blow (HPB) policy that combines contact-based pushing with airflow manipulation and a rectangular reorientation (RR) policy that controls a cuboid's heading using airflow alone. To enable physical execution, we design a lightweight, high-speed blower that coexists with a gripper. In zero-shot real-world deployment, HPB achieves an 80% success rate over 20 trials, while RR achieves a mean absolute heading error of 2.6 degrees across four target headings. The project materials are available at https://github.com/iit-DLSLab/BeyondContact.
Figures & tables
Fig. 2: Simulated airflow velocity field of a free turbulent round jet. Axial and radial positions are normalized by the nozzle outlet diameter D .
Fig. 3: Example of the airflow extension across diverse target objects and robotic platforms. Surface primitives are sampled from basic geometric shapes or mesh geometry in USD assets, while blowers can be mounted on various platforms, including quadrupeds and robotic arms.
Fig. 4: Force computation pipeline of the extension. The object’s surface is decomposed into primitives, and local airflow-induced forces are computed based on the airflow configuration and each primitive’s projected area normal to the blower axis. These forces are then aggregated into the net force and torque on the object and applied in Isaac Sim.
Fig. 5: Exploded view of the custom blower and system architecture showing control board interfaces and power distribution.
Parameter
Value
Parameter
Value
Learning rate
3×10−4
Clipping
0.2
Discount factor, γ
0.995
GAE parameter, λ
0.95
Epochs
6
Mini-batches
8
Value-loss coefficient
1.0
Entropy coefficient
0.005
Gradient norm
1.0
Initial STD
1.0
TABLE I: Key PPO hyperparameters shared by the two tasks.
Reward term
Formula
Weight
HPB Task-level Rewards
Dense goal reward
1−tanh(dt/σd)
2.0
Goal progress
(dt−1−dt)/σd
3.0
Goal inside
Iin
8.0
Goal stable
Iinexp(−∥vb,xy∥22/σv2)
2.0
Nozzle–object range
exp(−(d−d∗)2/σrange2)
0.05
TABLE II: Reward function terms and weights.
Parameter
Range
Object mass
[0.8,1.2]×mdesign
Object static friction
[0.8,1.2]
Object dynamic friction
[0.7,1.1]
Joint stiffness
[0.8,1.2]
Joint damping
[0.8,1.2]
Outlet airflow velocity, U0
[24.5,30.0] m/s
TABLE III: Domain randomization parameters.
Curriculum
Initial
Final
Update
HPB Task Curriculum
Square target side length (m)
0.24
0.20
Success rate
Target centroid x -range (m)
[0.55,0.75]
[0.40,1.40]
Success rate
Target centroid y -range (m)
[−0.08,0.08]
[−0.15,0.15]
Success rate
Air penalty, sair
0
1
5–15k steps
RR Task Curriculum
TABLE IV: Curriculum settings for HPB and RR. Step counts are expressed in thousands (k).
Dataset
Model
RMSE ( m/s )
R2
MARE (%)
k±SE ( m−1 )
B±SE ( − )
x0±SE ( mm )
CFD ( D=40mm )
Exponential
1.74
0.927
30.2
2.293 ± 0.044
—
—
Inverse
0.35
0.997
3.4
—
3.324 ± 0.037
-36.1 ± 2.2
CFD ( D=31.7mm )
Exponential
1.94
0.927
30.2
2.893 ± 0.055
—
—
Inverse
0.39
0.997
3.4
—
5.324 ± 0.037
-28.6 ± 1.7
EDF without Nozzle
Exponential
0.61
0.994
6.6
2.648 ± 0.065
—
—
Inverse
1.07
0.981
10.9
—
4.573 ± 0.279
15.9 ± 14.1
TABLE V: Centerline model-identification results for CFD simulations and experimental EDF measurements, including fitted parameters and their standard errors (SE).
Fig. 6: Centerline velocity comparison for the no-nozzle and nozzle configurations. Axial distance is normalized by D .
Fig. 7: Measured and fitted radial velocity profiles for no-nozzle and nozzle configurations. Radius is normalized by D , and color identifies the axial station x/D .
Fig. 8: HPB task policy deployment. The arm first pushes the cube into a position suitable for blowing (①–②). It then aims the blower at the cube, activates the airflow, and adjusts the blowing direction to guide the cube toward the target region (③). Once the cube enters the target region, the airflow command decreases to zero, leaving the cube in the target region (④–⑤). The plots below show the measured angles of the six joints and the airflow reference; the red dashed lines mark the times corresponding to the images above.
Training manipulation policies in contact-rich settings is a core challenge in robotics. While Reinforcement Learning (RL) has demonstrated its strength in such settings, it may struggle to sufficiently explore and discover complex manipulation strategies. To address this, we combine two basic ideas: First, designing appropriate reset strategies (the start state distribution of episodes) has shown promise in improving RL exploration and effectiveness. Second, while model-based approaches to finding trajectories through manipulation are hard, recent work showed that model-based approaches to sampling states on constrained manifolds can be highly efficient. Based on these observations, we propose a novel state sampler that enables efficient goal-conditioned RL in complex contact-rich manipulation tasks. Our sampler explicitly takes into account the structure of contact in order to provide a rich covering of diverse contact modes. By combining constrained sampling resets with projected interpolation and curriculum learning, our novel approach outperforms RL without constrained sampling and alternative reset methods, and effectively trains universal, prehensile and non-prehensile manipulation policies in contact-rich whole-body and in-hand settings. See the project page https://www.user.tu-berlin.de/mtoussai/26-CSRL/ for videos and full source code access.
Marc Toussaint, Cornelius V. Braun, Armand Jordana +5
Technische Universität Berlin · Robotics Institute Germany · LAAS-CNRS +1
Reinforcement learning explores effectively in domains such as Atari games, navigation, and locomotion, where novelty over states or dynamics is a sufficient signal. In contrast, dexterous manipulation requires rich physical hand--object interactions, but existing methods often suffer from unstable contact-based novelty signals, inefficient distance novelty signals, or reliance on task-specific priors. We propose ContactExplorer, a general exploration method for dexterous manipulation tasks. ContactExplorer represents contact as the intersection between object surface points and hand keypoints, encouraging dexterous hands to discover diverse and novel contact patterns, namely which fingers contact which object regions. It maintains a contact counter conditioned on discretized object states obtained via learned hash codes. This counter is leveraged in two complementary ways: (1) a count-based contact coverage reward that promotes exploration of novel contact patterns, and (2) an energy-based reaching reward that guides the agent toward under-explored contact regions. We evaluate ContactExplorer on seven contact-rich manipulation tasks and five dexterous hand embodiments. Experimental results show that ContactExplorer substantially improves sample efficiency and success rates over existing exploration methods, that it reduces the need for task-specific priors, and that it remains effective across hand embodiments and transfers to the real world. Project page is https://contact-explorer.github.io.
Zixuan Liu, Ruoyi Qiao, Chenrui Tie +5
School of Computing, National University of Singapore · RoboScience
Contact-rich manipulation underlies many everyday human activities, yet evaluating robots on such tasks remains difficult: the tasks involve diverse contact behaviors and are hard to reproduce across laboratories. Drawing on observations of everyday human manipulation, we identify eight essential skills that provide a practical framework for studying contact-rich manipulation. We introduce ContactBench 1.0, an extensible benchmark that instantiates these skills through 36 reference tasks and more than 40 object assets. The assets capture task-relevant geometry and contact behavior, can be fabricated reproducibly on consumer-grade 3D printers, and are each paired with a matching simulation model. Controlled task variations in both the real world and simulation enable evaluation of how well robot systems generalize. We evaluate recent representative vision-only and visual-tactile policy learning methods on six benchmark tasks. The results reveal that while vision-only policies struggle to perceive or react to contact events, visual-tactile policies do not meaningfully improve their performance, highlighting that contact-rich manipulation remains a significant challenge for current policy learning methods. By providing common task definitions alongside physical and simulated assets, ContactBench 1.0 offers a shared testbed for developing, evaluating, and comparing contact-rich manipulation systems. The benchmark will be open-sourced at https://jingyi-xiang.github.io/contactbench.
Jingyi Xiang, Yixuan Wang, Jeannette Bohg +1
University of Illinois Urbana-Champaign, Urbana, IL, USA · Stanford University, Palo Alto, CA, USA