Ferrofluids are liquids containing magnetic nanoparticles, enabling shape control through external magnetic fields. This deformability supports their use as soft robotic bodies that can navigate confined spaces and manipulate objects. Their nonlinear continuum dynamics, however, make reliable control challenging. Existing work shows how spatial magnetic fields from electromagnet arrays can establish and maintain desired ferrofluid configurations. A complementary objective is to exploit the fluid's dynamics under limited actuation to reach prescribed shapes that may not be sustainable at equilibrium. Here, we use deep model-based reinforcement learning to control these transient shapes. Learning directly through physical interaction, our policies select current commands for a single fixed electromagnet based on visual observations, guiding the moving ferrofluid toward prescribed geometries. The learned policies repeatedly attain a variety of shapes spanning polygons, asterisks and pentominoes. These results demonstrate that learning directly from physical interaction enables reliable fluid shaping, opening new possibilities for controlling ferrofluid-based soft robots.
Figures & tables
Fig. 2: Experimental platform for ferrofluid shaping. A downward-facing camera observes the fluid in a glass jar. A 3D-printed case supports the jar, and an item-profile frame holds the camera. The shaping electromagnet sits beneath the jar. Section II-A describes the internal arrangement.
Fig. 3: Schematic cutaway of the apparatus (not to scale). The jar rests above the electromagnet; the cutaway shows its central pole and annular winding. The heat sink sits below the magnet.
Fig. 4: Hall-probe readings in a plane through the magnet axis at 0.5 A. Points mark sampled positions. The signed logarithmic color scale gives readings in gauss and retains their polarity. The schematic magnet half-section below shares the map’s radial scale.
Fig. 5: Application of DreamerV3 [ 27 ] to ferrofluid shaping. Recorded images, actions and rewards train the world model. The policy is trained through imagined interaction with this model. The control loop runs at 55 Hz. The target mask enters the IoU record improvement reward path. Learning and interaction run concurrently. The controller receives updated policy weights before each episode (dashed arrow), then holds them fixed throughout it. The thumbnails show a recorded trajectory sampled every ten control steps at 64×64 resolution. The overlap image shows the intersection (dark), target-only (orange) and observation-only (blue) regions.
Fig. 6: Schematic ferrofluid silhouettes during stretching (a) and recoil (b). Solid lines show the same current outline, while dashed lines show earlier outlines and arrows indicate motion, illustrating how identical silhouettes can evolve differently.
Target
Untrained policy
Final policy
Polygons
Triangle
0.757±0.001
0.868±0.020
Square
0.857±(<0.001)
0.925±0.009
Pentagon
0.904±(<0.001)
0.916±0.003
Hexagon
0.931±(<0.001)
0.941±(<0.001)
Asterisks
TABLE I: Untrained- and final-policy episode returns
Target
Crossings
Time (s)
Target
Crossings
Time (s)
Triangle
122/150
2.35±0.86
L
143/150
4.03±1.09
Square
127/150
3.05±0.25
N
134/150
4.64±0.93
Pentagon
0/150
—
P
86/150
7.13±2.73
Hexagon
0/150
—
T
128/150
3.66±1.61
3 rays
98/150
3.02±0.80
U
52/150
8.80±2.50
4 rays
28/150
4.24±0.19
V
116/150
5.68±1.31
TABLE II: Final-policy threshold crossings and timing
Fig. 7: Area overlap and the interpretation of IoU. Left: a filled circle positioned and sized to maximize overlap with the fixed target mask, colored by intersection and unmatched regions. Right: selected ferrofluid photographs with their logged single-frame IoU, computed using shape registration in Section III-C . A circle overlaps substantially with the hexagon despite lacking corners, whereas its overlap with the three-ray asterisk is lower.
Fig. 8: Polygon targets, example silhouettes, episode returns and capped first-attainment times. Each column shows the target, two smaller examples at lower and intermediate IoUs, the best recorded frame, episode return, and capped first-attainment time at the IoU threshold shown in each lower panel ( Section IV-A ). Image labels report single-frame IoU. Thin curves show three seeds and the bold curve their mean, with trailing 10-episode averages. Plus signs mark unsuccessful episodes at their recorded end times. Each plot row shares a vertical scale; horizontal axes count training steps in thousands, excluding resets. Each policy trains for 100 000 steps (400 episodes of 250 steps).
Fig. 9: Three- through six-ray asterisks, using the image, episode-return and capped first-attainment-time layout of Fig. 8 . Each plot row shares a vertical scale. Curves use trailing 10-episode averages; plus signs mark unsuccessful episodes. Each policy trains for 100 000 steps (400 episodes of 250 steps).
Fig. 10: Pentomino targets F, I, L, N, P and T, using the image, episode-return and capped first-attainment-time layout of Fig. 8 . Curves use trailing 10-episode averages; plus signs mark unsuccessful episodes. All 12 pentominoes share a vertical scale for each plot row. Each policy trains for 300 000 steps (500 episodes of 600 steps).
Fig. 12: Formation and decay of the three-ray asterisk during training (seed 2, episode 214), shown through camera frames, current commands and registered IoU. Numbered points on the IoU curve identify the photographs, with peak IoU at frame 8 and shading marking the best seven-observation window.
Target
Span (s)
Target
Span (s)
Triangle
0.13±0.01
L
0.08±0.01
Square
0.16±(<0.01)
N
0.11±0.02
Pentagon
0.34±0.06
P
0.13±0.01
Hexagon
0.31±0.03
T
0.12±0.03
3 rays
0.09±0.02
U
0.08±0.03
4 rays
0.08±0.01
V
0.08±0.01
TABLE III: Persistence within 0.05 IoU of each episode’s peak
Precise shape control of Deformable Linear Objects (DLOs) is crucial in robotic applications such as industrial and medical fields. However, existing methods face challenges in handling complex large deformation tasks, especially those involving opposite curvatures, and lack efficiency and precision. To address this, we propose a two-stage framework combining Reinforcement Learning (RL) and online visual servoing. In the large-deformation stage, a model-based reinforcement learning approach using an ensemble of dynamics models is introduced to significantly improve sample efficiency. Additionally, we design a self-curriculum goal generation mechanism that dynamically selects intermediate-difficulty goals with high diversity through imagined evaluations, thereby optimizing the policy learning process. In the small-deformation stage, a Jacobian-based visual servo controller is deployed to ensure high-precision convergence. Simulation results show that the proposed method enables efficient policy learning and significantly outperforms mainstream baselines in shape control success rate and precision. Furthermore, the framework effectively transfers the policy trained in simulation to real-world tasks with zero-shot adaptation. It successfully completes all 30 cases with diverse initial and target shapes across DLOs of different sizes and materials. The project website is available at: https://anonymous.4open.science/w/sc-mbrl-dlo-EB48/
Zhaowei Liang, Song Wang, Zhao Jin +2
All authors are with the Department of Mechanical Engineering, Tsinghua University, Beijing, China.
The capability to encode spatially distinct magnetization patterns within soft materials enables remote control over complex deformations. This characteristic is especially important for microfluidic platforms, where limited dynamic control of channel boundaries and laminar flow conditions usually restrict fluid transport and interactions. The present study introduces a shape-programmable magnetic soft membrane actuator as an active microchannel component that can dynamically modulate its shape under magnetic fields and therefore the microfluidic environment. The membrane is magnetically programmed using a template-based approach, which allows it to be controllably deformed in the form of a sinusoid under the influence of an external magnetic field. The membrane's integration into a microchannel converts a passive channel wall into a dynamically changeable interface, allowing active fluid manipulation and enhancing micromixing in laminar flow conditions. The proposed approach establishes a versatile platform for wirelessly controlled deformable interfaces in next-generation microfluidic and lab-on-chip systems.
Direnc Akyildiz, Fatih Kocabas, Xiangyi Tan +1
Bio-integrated Robotics Lab at Mechanical Engineering Department, University of Wisconsin -Madison, Madison, WI 53706 USA · Department of Electronic and Electrical Engineering, University College London, London, WC1E 7JE UK · Biomedical Engineering Department, University of Wisconsin - Madison, Madison, WI 53706 USA +1
Autonomous contact-based micromanipulation is challenging because surface and interfacial interactions at the microscale are difficult to model accurately, limiting the use of conventional model-based control and sim-to-real learning. We present a closed-loop sim-to-real reinforcement learning (RL) approach for microfiber shape control on a surface. The central idea is to train geometric shape regulation in a simplified frictionless simulator and rely on real-time visual feedback during deployment to iteratively correct the observed effects of unmodeled surface interactions. An RL policy trained entirely in simulation is transferred directly to a physical dual-gripper micromanipulation system operating at 40 Hz, without retraining or domain adaptation. Using silk microfibers as a testbed, the policy achieves a mean point-wise shape error of 270 ± 80 μm across twenty-four diverse initial configurations. Across nine specimens covering all combinations of three fiber diameters (50, 80, and 120 μm) and three manipulated lengths (10 mm, 15mm, and 20 mm), the same policy achieves sub-millimeter final shape error without any retraining or retuning. These results show that a policy learned in a simplified simulator can achieve repeatable real-world microfiber shape regulation under surface contact, provided that the task-relevant effects of the sim-to-real mismatch remain observable and correctable within the closed feedback loop.