Learning Beyond Rigid Manipulation with Hybrid Soft-Rigid Robots
Authors: Mariano Ramírez Montero, Ebrahim Shahabi, Giovanni Franzese, Jens Kober, Barbara Mazzolai, Cosimo Della Santina
Organizations: Cognitive Robotics, Delft University of Technology, Delft, The Netherlands · Technology Innovation Institute (TII), Abu Dhabi, United Arab Emirates · Bioinspired Soft Robotics Lab, Istituto Italiano di Tecnologia, Genova, Italy · Institute of Robotics and Mechatronics, German Aerospace Center (DLR), Oberpfaffenhofen, Germany
Rigid-soft robot hybrids promise to unite the precision and repeatability of rigid manipulators with the compliance and adaptability of soft structures. Yet, full-fledged rigid-soft manipulators remain largely unexplored, in part because the intelligence required to exploit this combination is still missing. Here, we introduce a hybrid robotic system obtained by the series interconnection of a rigid arm and a soft manipulator, and systematically examine its ability to perform both established and previously inaccessible manipulation tasks. We show that a tailored imitation learning algorithm -- generally applicable to this class of hybrid systems -- leverages the complementary roles of rigid precision and soft embodiment to acquire manipulation skills and generalize them in a single shot across task variations. As a result, the system matches or exceeds the performance of rigid robots in classical tasks such as stacking, while enabling qualitatively new behaviors, including grasping objects through narrow openings.
Figures & tables
Figure 1: Overview of the proposed hybrid system combining computational and physical intelligence. a A human provides a single kinesthetic demonstration by moving the rigid part of the proposed rigid-soft hybrid manipulator. The learned policy is then transported to a new task instance using keypoint-based deformation, enabling generalization without retraining. The errors introduced by the transportation process, together with possible variations in the physical surroundings (e.g., different screwdriver), are absorbed by the soft arms’ physical intelligence. b The soft arm can adapt passively to interactions with objects and the ground, compensating for uncertainties and variations in the environment and in the relative positioning of the soft arm base without requiring changes to the control input. Panels c and d show a side-by-side comparison of demonstration and generalization for a pick-and-place task. The first column reports a summary of the two keypoints (source s~i and target t~i ) used in this task, and of the two trajectories (demonstrated in c and transported in d at the rigid robot end effector.
Figure 2: Proposed soft–rigid hybrid platform. a The integrated system, composed of a 7-DoF rigid co-bot (Franka Panda), an RGB-D camera (RealSense D405) for keypoint localization, and a cable-driven soft arm mounted at the wrist. The soft arm is actuated via two tendons by motors enclosed in the actuator case. Four buttons on the robot are used to control and log tendon references during demonstrations. b Ventral bending of the soft arm when pulling one of the tendons. c Twisting of the soft arm when pulling the other tendon. Panels d and e show the platform executing compound motions where both rigid and soft components are actuated simultaneously. f A qualitative example of a task we can do with our platform, drilling with a toy assembly.
Figure 3: A summary of our tested grasping tasks. a One of the 10 tested configurations for the non-prehensile manipulation task. b One of the 20 tested configurations for the stacking task, and we also note the tolerance of ±1 cm in this task due to the difference in radius of the two objects. c One of the 10 positions tested for dynamic grasping, and we also note the average speed of 0.8 m/s during the grasp. The success rate is also shown for all tasks in their respective panels.
Figure 4: Executions of the non-prehensile manipulation task. a Plot of the transported trajectories, where they have been spatially aligned by translating the picking locations to the origin for clarity. b Initial and goal locations for the object from a top-view, with the arrows indicating paired locations. c A sequence from an execution closely resembling the original demonstration, where the object is approached and dragged along a similar trajectory. d A more demanding configuration, in which the robot must extend further to reach past the object before initiating the pull. In both instances, the manipulation is achieved without grasping, relying instead on compliant, non-prehensile contact.
Figure 5: Executions of the stacking task. Panel (A) depicts the given demonstration, and transported trajectories in different colors, where these have all been spatially aligned to the picking locations for clarity. Panel (B) shows the initial object locations and the goal stacking locations from a top-view, with the colored arrows indicating the paired locations. Panels (C 1 -C 3 ) visualize different initial configurations and their robot executions. Note how these different configurations showcase generalization, since the task is completed in configurations that are dissimlar to that of the demonstration.
Figure 6: Executions of the dynamic grasping task. a Demonstration and transported trajectories for each new configuration. b Initial object locations from a top-view. c, d, e Several executions of this task, where the stills show initial contact, soft arm wrapping, and succesful picking. This task highlights the coordination between the fast motions that can be achieved via the rigid robot part and the physical adaptability of the soft arm.
Figure 7: A summary of our soft arm-specific tasks. a The hybrid platform manipulating an object through a narrow gap. b The hybrid platform combining the movement of the rigid and soft parts to search for an object through a narrow gap using a small camera mounted on the soft arm; the view of this camera is shown in the bottom right of each frame.
Figure 8: Comparison between our hybrid platform, a rigid gripper, a soft hand, and a soft gripper when grasping with noisy observations . a An example of the rigid gripper tests, with the higher transparency cylinder showing where the robot believes the object is. b Plotted noise vectors that were tested for each of these cases, showing which were successes or failures. A convex hull is also drawn using the successes for each case to approximate the success regions. c The estimated areas for each of these cases shown in a bar plot; note how the hybrid has the largest area.
Figure 9: Success rates for pick-and-place across object size and weight. Heatmap showing success rates over 10 trials for combinations of object diameter and mass. Lighter and smaller objects are handled reliably, while performance degrades as mass and size increase.
Figure 10: Pictorial representation of the proposed algorithm. On the left we show the demonstration trajectory X , the source task-level keypoints S~=(s~1,s~2) , and how they are re-targeted to robot-centric keypoints S=(s1,s2) . A new task instance is defined by T~=(t~1,t~2) , which yields target robot-level keypoints T=(t1,t2) , shown on the right target space. A learned deformation map ΦT transports the original trajectory X to the generalized one in target space ΦT(X) while preserving its structure.
Hybrid rigid-soft robots combine the precision of rigid manipulators with the compliance and adaptability of soft arms, offering a promising approach for versatile grasping in unstructured environments. However, coordinating hybrid robots remains challenging, due to difficulties in modeling, perception, and cross-domain kinematics. In this work, we present a novel augmented reality (AR)-based physical human-robot interaction framework that enables direct teleoperation of a hybrid rigid-soft robot for simple reaching and grasping tasks. Using an AR headset, users can interact with a simulated model of the robotic system integrated into a general-purpose physics engine, which is superimposed on the real system, allowing simulated execution prior to real-world deployment. To ensure consistent behavior between the virtual and physical robots, we introduce a real-to-simulation parameter identification pipeline that leverages the inherent geometric properties of the soft robot, enabling accurate modeling of its static and dynamic behavior as well as the control system's response.
Huishi Huang, Jack Klusmann, Haozhe Wang +9
Advanced Robotics Centre (ARC), National University of Singapore (NUS), 117608, Singapore. · Institute of High Performance Computing (IHPC), Agency for Science, Technology and Research (A*STAR), 138632, Singapore. · Chair of Cognitive Systems, Technical University of Munich (TUM), 80333, Munich. +2
In animals such as elephants and octopuses, acquiring non-visual information about an object and physically engaging with it are inseparable processes mediated by rich, large-area interactions between compliant appendages and the environment. Soft robots provide a natural platform for translating this principle into engineered systems. Yet current robotic intelligence makes limited use of physical interaction, treating it primarily as a disturbance to be rejected or, at best, as a means of compensating for object misalignment. Here, we introduce a physical intelligence framework in which distributed compliant interactions jointly reveal task-relevant information and organize manipulation behavior. This results in an intrinsically partially observable problem: key task-relevant information is never measured directly, but must instead be inferred from the history of physical interactions. We propose a reinforcement-learning architecture that addresses this challenge by learning a memory-based control policy end-to-end. The key innovations making this possible are (i) a pretrained exploration policy that provides a reference for broad workspace exploration, (ii) joint optimization that integrates exploration and grasping objectives within a single recurrent policy, and (iii) a two-stage sim-to-real adaptation including observation mapping and policy fine-tuning. We demonstrate this principle through blind whole-arm grasping with a hybrid rigid-soft robotic arm that we equip with IMUs embedded directly within its compliant structure, providing its only source of proprioceptive sensing. The learned policy successfully identifies and grasps various objects by autonomously coordinating workspace exploration, object encounter and localization, inference of grasp-relevant properties, and stable whole-arm wrapping.
Cognitive Robotics Department, Faculty of Mechanical Engineering, Delft University of Technology, Delft 2628 CD, The Netherlands. · School of Engineering, Newcastle University, UK.
Manipulation in confined environments, such as threading a manipulator through narrow apertures, remains a fundamental challenge, especially for conventional rigid robots. Hybrid rigid-soft manipulators offer promise but face two compounding planning challenges: backbone shapes feasible in free space become infeasible under environmental contact, and planning rigid and soft segments independently ignores their kinematic coupling. We present THREAD, the first diffusion-based trajectory planner for hybrid manipulation, learning a generative prior over physically realizable backbone trajectories conditioned on local environment geometry, with physics-inspired losses encoding curvature, smoothness, and collision constraints jointly across both segments. Trained in simulation, THREAD achieves 92.4% task success with 5x fewer collisions than the strongest baseline. We show cross-embodiment real-world transfer with minimal online updates, successfully threading through apertures as small as 1.3x the soft segment diameter.