Module Number Adaptive Visual Shape Control for Serial Modular Soft Robots

Kyohei Akamine; Takato Horii; Yusuke Sakaue; Hiroki Ishizuka

cs.ROAug 30, 2026

Module Number Adaptive Visual Shape Control for Serial Modular Soft Robots

Authors: Kyohei Akamine, Takato Horii, Yusuke Sakaue, Hiroki Ishizuka

Organizations: Graduate School of Engineering Science, The University of Osaka, Osaka 560-0043, Japan · Faculty of Science and Technology, Sophia University, 7-1 Kioi-cho, Chiyoda-ku, Tokyo 102-8554, Japan

Abstract

Image based shape control provides a simple means of controlling the whole body configuration of soft robots. However, existing data driven approaches are typically developed for fixed robot structures and require new control data when the number of modules changes. This paper presents a module number adaptive visual shape control method for serial modular soft pneumatic robots. A controller trained only on single module actuation shape data is reused for robots with one to five modules by decomposing whole body camera images into local module patches. A single common module segmenter localizes individual modules across all tested configurations, while the same local controller is applied to every extracted patch. Geometric data augmentation improves transferability to downstream modules, and a lightweight mask reconstruction network reconstructs a synthetically removed actuator mask channel. Experiments on physical robots demonstrate shape control across varying numbers of modules and under environmental changes and payload loading. The results show that single module control learning enables scalable whole body control without configuration specific control data collection.

Figures & tables

Method	Sensor	Scale	Feedback	VC
Almanzor et al. [ 14 ]	RGB	Fixed	Visual	$\times$
Monteiro et al. [ 15 ]	RGB	Fixed	Open-loop	$\times$
Chen et al. [ 19 ]	MoCap+markers	$3{\to}2,4$	Configuration	$\times$
Ours	RGB	$1{\to}5$	Visual	$\checkmark$

TABLE I: Comparison with prior work.

Fig. 1: A soft robot is constructed by connecting modules, each consisting of three soft pneumatic actuators. (a) Five-module soft robot. Its total length is 875 mm, and it can bend up to 75 ∘ in each direction. (b), (c) A module and its constituent pneumatic artificial muscle actuators. (d) Pressure–contraction curve. The actuator contracts by up to approximately 20%.

Fig. 2: Architecture of the proposed control model, which deploys a module controller trained on a single module to multi-module configurations without additional multi-module control-data collection or controller retraining. (a) Module extractor using the module segmenter (MS) to extract $N$ module patches. (b) Module controller using the actuator segmenter (AS) and voltage prediction model $\mathcal{I}_{\theta}$ . (c) Mask reconstruction network $\mathcal{R}_{\phi}$ that reconstructs a missing actuator-mask channel.

Fig. 3: Experimental setup. (a) Control system. The shape of the soft robot is captured, and the pneumatic pressure applied to each actuator is determined based on the observed shape. (b) Experimental setup. The distance between the soft robot and the camera was 3.3 m.

Fig. 4: The results confirm that a soft robot with multiple modules can be controlled using a module controller trained only on a single module. (a) The target posture is shown in red and the actual posture in blue. The actual posture converged toward the target posture. (b) Temporal change in the image error between the actual and target postures. For all tested module configurations, the image error was substantially reduced from its initial value and generally converged within approximately 20 s. The shaded regions indicate the standard deviation across the five target postures. (c) Final-step image error with and without data augmentation. The lower image error obtained with data augmentation confirms its importance in training the module controller.

Fig. 5: Results of environmental-variation experiments simulating indoor conditions. The target posture is shown in red and the actual posture in blue. The proposed method enabled the actual posture to converge toward the target posture despite changes in illumination, the placement of nearby objects, and synthetic image-plane translation and rotation.

Fig. 6: The robot converged to a posture close to the target even with an attached payload. The target posture is shown in red and the actual posture in blue. (a) Comparison between the target and actual postures. (b) Temporal change in the image error between the target and actual postures. The image error converged within approximately 20 s. The shaded regions indicate the standard deviation across the five target postures.

Fig. 7: The results confirm that the missing mask could be reconstructed and that the soft robot could still be controlled. (a) The target posture is shown in red and the actual posture in blue. When the mask was missing, the robot hardly moved and did not reach the target posture. In contrast, when the mask was reconstructed, the actual posture converged toward the target posture. (b) Temporal change in the image error between the actual and target postures. When mask reconstruction was applied, the image error decreased over time. The shaded regions indicate the standard deviation across the five target postures.

Soft Robotics

Soft robotics studies robots with compliant bodies and actuators that deform to move, sense, and interact with uncertain environments.

See all 230 papers

A Continual Learning Framework for Adaptive Control of Modular Soft Robots
Jul 7, 2026Nilay Kushawaha, Muhammad Sunny Nazeer, Baljinder Singh Bal +2Continual Robot Learning Continuum Robot Control
1The BioRobotics Institute, Scuola Superiore Sant’Anna, 56024, Pontedera, Italy · Department of Excellence in Robotics and AI, Scuola Superiore Sant’Anna, 56127, Pisa, Italy · Department of Mechanical Engineering, National University of Singapore, 119077, Singapore +1
3D Shape Control of Extensible Multi-Section Soft Continuum Robots via Visual Servoing
Feb 22, 2026Abhinav Gandhi, Shou-Shan Chiang, Cagdas D. Onal +1Continuum Robot Control Vision-Based Robot Control
Robotics Engineering Department, Worcester Polytechnic Institute 100 Institute Rd, Worcester, MA 01609, USA
Real-Time Shape Control of Multi-Segment Soft Robotic Arms Using Koopman Operators with Global and Local Observables
Sep 2, 2026Jiahe Wang, Eron Ristich, Sultan Haidar Ali +5Model Predictive Control Feedback Control

Jul 7, 2026J/K move · Enter open · S save

A Continual Learning Framework for Adaptive Control of Modular Soft Robots

Nilay Kushawaha, Muhammad Sunny Nazeer, Baljinder Singh Bal, Cecilia Laschi+1

1The BioRobotics Institute, Scuola Superiore Sant’Anna, 56024, Pontedera, Italy · Department of Excellence in Robotics and AI, Scuola Superiore Sant’Anna, 56127, Pisa, Italy · Department of Mechanical Engineering, National University of Singapore, 119077, Singapore+1

Continual Robot Learning Continuum Robot Control Soft Robotics Adaptive Control Robot Learning

Open paper AskPDF arXiv

Soft robots have attracted significant attention in applications such as medical intervention, rehabilitation, and robotic manipulation due to their inherent compliance, flexibility, and high degrees of freedom. Modular soft robots (MSRs), composed of multiple interconnected segments, represent an emerging class of robotic systems with highly deformable and reconfigurable structures capable of performing complex tasks. However, designing controllers for MSRs remains challenging due to their nonlinear dynamics, modeling complexity, and hyper-redundant nature. Existing approaches typically require controllers to be retrained from scratch whenever the robot morphology changes. In this work, we address these challenges through a continual learning inspired control framework capable of incrementally adapting to changes in robot morphology while preserving previously acquired knowledge. Specifically, the proposed framework enables the controller to sequentially learn new MSR configurations without forgetting previously learned ones. In addition, for MSRs with fixed configurations, the same framework can be employed in a distributed manner to learn module-specific dynamics, enabling localized control and improved precision. The proposed approach is validated through closed-loop trajectory tracking experiments in simulation using a tendon-driven soft robot, as well as on a real-world three-module pneumatic soft robotic arm. Furthermore, we demonstrate the adaptive capabilities of the framework through a reaching experiment in which the controller selectively activates only the necessary modules to reach a virtual target position, thereby reducing computational overhead.

Abstract

Soft Robotics

Explore similar work

A Continual Learning Framework for Adaptive Control of Modular Soft Robots

3D Shape Control of Extensible Multi-Section Soft Continuum Robots via Visual Servoing

Real-Time Shape Control of Multi-Segment Soft Robotic Arms Using Koopman Operators with Global and Local Observables