PneuTac: Tactile Manipulation with Soft Pneumatic Robots via Unified MPM-Gaussian Splatting Simulation

Shaohong Zhong; Marco Pontin; Joe Watson; Perla Maiolino; Ingmar Posner

cs.ROSep 29, 2026

PneuTac: Tactile Manipulation with Soft Pneumatic Robots via Unified MPM-Gaussian Splatting Simulation

Authors: Shaohong Zhong, Marco Pontin, Joe Watson, Perla Maiolino, Ingmar Posner

Organizations: Oxford Robotics Institute, Department of Engineering Science, University of Oxford, OX1 3PJ Oxford, U.K.

Abstract

Soft robots and tactile sensors have demonstrated great potential in delicate manipulation tasks. Soft pneumatic robots enable safe contact through compliance, and vision-based tactile sensors offer high-resolution touch perception. However, learning tactile manipulation with compliant robots has been challenging, bottlenecked by the lack of efficient simulation. Existing simulators typically model them in isolation, and exhibit large calibration gaps that are difficult to overcome efficiently. We present PneuTac, a unified framework for tactile-feedback manipulation with soft pneumatic robots. We leverage the material point method (MPM) for modelling the dynamics of the soft robot and the deformable tactile membrane, and 3D Gaussian splatting (3DGS) for rendering. Real-to-sim modelling is done with a simple vision-based method, to then train action and perception networks for efficient simulation with surrogate models. We use the framework to drive a tactile-guided pipeline to collect demonstrations in simulation. Through experiments on a custom-designed pneumatic soft finger with a tactile sensing tip, together with additional cross-device evaluations, we show that PneuTac is capable of accurately modelling soft robots with tactile sensors, and that policies trained with simulation-augmented demonstrations outperform baselines trained on the same real data on three real-world contact-rich compliant manipulation tasks, making it a practical framework for tactile manipulation on compliant hardware.

Figures & tables

Fig. 1: Overview of PneuTac. The framework leverages MPM-3DGS coupling for efficient real-to-sim modelling from CAD models. This supports accurate modelling of soft robot deformation and tactile contact rendering, which facilitates policy learning for real-world contact-rich compliant manipulation.

Fig. 2: PneuTac pipeline. Left: image-based real-to-sim calibration. Middle: residual-torque prediction ( $f_{r}$ ) and probe-to-MPM indentation mapping ( $f_{m}$ ) for fast surrogate simulation. Right: generation of simulated demonstrations guided by real trajectories and tactile observations.

Fig. 3: Real-to-sim comparison of the soft finger and three-chamber arm in novel configurations. The finger’s back view is rendered from the side opposite the calibration camera.

View	PneuTac	Tex-Mesh	Par-Splat	Def-NeRF
Front $\downarrow$	$\mathbf{50.07\pm 0.83}$	$51.13\pm 1.12$	$54.58\pm 1.59$	$55.06\pm 11.72$
Back $\downarrow$	$77.61\pm 11.72$	$79.09\pm 9.08$	$78.62\pm 8.43$	$\mathbf{68.96\pm 12.77}$

TABLE I: RMSE comparison of soft finger rendering methods for front and back views. Values are reported as mean

\pm

SEM.

Metric	PneuTac	Tacchi	DOT-Sim	Taxim
RMSE $\downarrow$	$\mathbf{10.18\pm 0.58}$	$10.86\pm 0.29$	$14.14\pm 0.23$	$14.13\pm 0.61$
PSNR (dB) $\uparrow$	$\mathbf{28.11\pm 0.54}$	$27.44\pm 0.23$	$25.13\pm 0.14$	$25.20\pm 0.36$
NCC $\uparrow$	$\mathbf{0.982\pm 0.001}$	$0.952\pm 0.004$	$0.913\pm 0.004$	$0.915\pm 0.011$

TABLE II: Comparison of tactile rendering methods. Values are reported as mean

\pm

SEM.

Fig. 4: Tactile image comparison. Columns 1-3: the Digit sensor attached to the soft finger; columns 4-5: a second, unmodified Digit sensor.

Task	Sim+Real	Real	N-Aug	Real-20	No-Tac	PPO
Switch	$83.3\pm 3.3$	$60.0\pm 5.8$	$73.3\pm 3.3$	$\mathbf{90.0\pm 5.8}$	$30.0\pm 5.8$	$63.3\pm 3.3$
Egg	$\mathbf{90.0\pm 5.8}$	$26.7\pm 8.8$	$46.7\pm 6.7$	$40.0\pm 10.0$	$56.7\pm 3.3$	$80.0\pm 5.8$
Card	$\mathbf{66.7\pm 3.3}$	$33.3\pm 8.8$	$40.0\pm 5.8$	$50.0\pm 5.8$	$63.3\pm 3.3$	$10.0\pm 5.8$

TABLE III: Real-world task success rates (%). Values are reported as mean

\pm

SEM across 3 seeds, with 10 trials per seed and task.

Panel of Figure 8 — Fig. 5: Contact-rich manipulation with the tactile soft finger. Left to right: switch flipping, egg carton opening, and card pulling.

Explore similar work

ETac: A Lightweight and Efficient Tactile Simulation Framework for Learning Dexterous Manipulation
Apr 22, 2026Zhe Xu, Feiyu Zhao, Xiyan Huang +1Tactile Contact-Rich Dexterous Manipulation
School of Information Science and Technology at ShanghaiTech University, Shanghai 201210, China
Reduced-order Neural Modeling with Differentiable Simulation for High-Detail Tactile Perception
May 6, 2026Yuhu Guo, Zhikai Shen, Jiasheng Qu +4Tactile Deformable Linear Objects
The University of Manchester, UK · The Chinese University of Hong Kong, Hong Kong SAR, China · University of Leeds, UK +1
Imagining the Sense of Touch: Touch-Informed Manipulation via Imagined Tactile Representations
Jul 2, 2026Zhiyuan Zhang, Adeesh Desai, Jyun-Chi Hu +7Tactile Contact-Rich Dexterous Manipulation
Department of Industrial Engineering, Purdue University, West Lafayette, IN 47907, USA · Department of Mechanical Engineering, Texas A&M University, College Station, TX 77843, USA

Apr 22, 2026, cs.ROJ/K move · Enter open · S save

ETac: A Lightweight and Efficient Tactile Simulation Framework for Learning Dexterous Manipulation

Zhe Xu, Feiyu Zhao, Xiyan Huang, Chenxi Xiao

School of Information Science and Technology at ShanghaiTech University, Shanghai 201210, China

Tactile Contact-Rich Dexterous Manipulation Dexterous Manipulation Robotic Manipulator

Open paper AskPDF arXiv

Tactile sensors are increasingly integrated into dexterous robotic manipulators to enhance contact perception. However, learning manipulation policies that rely on tactile sensing remains challenging, primarily due to the trade-off between fidelity and computational cost of soft-body simulations. To address this, we present ETac, a tactile simulation framework that models elastomeric soft-body interactions with both high fidelity and efficiency. ETac employs a lightweight data-driven deformation propagation model to capture soft-body contact dynamics, achieving high simulation quality and boosting efficiency that enables large-scale policy training. When serving as the simulation backend, ETac produces surface deformation estimates comparable to FEM and demonstrates applicability for modeling real tactile sensors. Then, we showcase its capability in training a blind grasping policy that leverages large-area tactile feedback to manipulate diverse objects. Running on a single RTX 4090 GPU, ETac supports reinforcement learning across 4,096 parallel environments, achieving a total throughput of 869 FPS. The resulting policy reaches an average success rate of 84.45% across four object types, underscoring ETac's potential to make tactile-based skill learning both efficient and scalable.

Abstract

Explore similar work

ETac: A Lightweight and Efficient Tactile Simulation Framework for Learning Dexterous Manipulation

Reduced-order Neural Modeling with Differentiable Simulation for High-Detail Tactile Perception

Imagining the Sense of Touch: Touch-Informed Manipulation via Imagined Tactile Representations