Contact-Rich Robotic Manipulation

Follow this topic Ask about this topic

Momentum

59 papers in the last four weeks, up 436% on the four weeks before. 0.6% of all new papers.

Jul 13Week of Sep 28

Latest papers 263

Aug 3, 2026cs.RO

Hybrid Impedance-Admittance Control with Multi-Link Aerial Robot for Contact-Rich Surface Sliding Task

Multi-link aerial robots can actively deform their articulated structures during flight, giving them strong potential for aerial manipulation. However, they still face substantial challenges in contact-rich aerial manipulation tasks such as surface sliding, which requires both disturbance robustness and compliance to uncertain surface geometry. Force-control strategies such as impedance and admittance control are commonly employed to address these requirements. Although impedance control can provide disturbance-resistant interaction and admittance control can offer compliant adaptation, their opposite force--motion causalities prevent their simultaneous implementation when applied through the same actuation source, such as the rotor thrusts used by conventional aerial robots. To overcome this limitation, we propose a hybrid impedance--admittance control strategy for a multi-link aerial robot. The articulated morphology enables a functional separation of force and motion regulation across joint and rotor actuation sources. In this framework, admittance behavior is generated through joint angle regulation to enhance adaptive interaction, while impedance behavior is achieved by modulating rotor thrust to regulate the sliding motion. This structural coordination allows the robot to leverage the complementary strengths of both control paradigms. As a result, the multi-link aerial robot achieves resilient and adaptive surface sliding. Experimental results demonstrate robust and compliant sliding performance on unknown surfaces.

Zicheng Luo, Maolin Lei, Jinjie Li +3

Department of Mechanical-Engineering, The University of Tokyo, Bunkyo-ku, Tokyo 113-8656, Japan

Contact-Rich Robotic Manipulation Admittance Control Impedance Control Aerial Manipulation

Aug 3, 2026cs.RO

A Tilt-Rotor UAV with a Gripper for Stable Contact-Based Tasks via Environmental Anchoring

Maintaining a stable pose during physical interaction is a significant challenge for aerial robots, often limiting their use in contact-based tasks. This paper presents a novel uncrewed aerial vehicle (UAV) platform designed to transition from unconstrained flight to a stable, constrained work platform via environmental anchoring. Our system comprises: 1) a multirotor with a tilt-rotor mechanism that decouples pitch from forward motion, enabling stable hover at non-zero pitch angles, and 2) a novel underactuated, cable-driven, prismatic gripper featuring compliance to adapt to irregular geometries, designed to stabilize the UAV by anchoring it to its environment. We present the design and prototyping of the complete system and validate its performance through a series of real-robot flight tests. Results demonstrate that anchoring significantly improves stability for interaction tasks, reducing positional drift RMSE by over 95% compared to a free-flight baseline, even under windy conditions. The anchored system can withstand longitudinal reaction forces up to 75N while maintaining a stable pose. Furthermore, across a range of target geometries and orientations, the system demonstrated consistent stability with a positional drift RMSE that never exceeded 3mm. These results establish the viability of our approach for complex physical interaction tasks, such as sampling tree health by drilling or sensor installation in hard-to-reach locations. Watch our UAV at: https://youtu.be/HDQ8S4ZW3Ls

Joshua Taylor, Nursultan Imanberdiyev, Wei-Yun Yau +2

National University of Singapore (NUS), Singapore. · Institute for Infocomm Research (I2R), A*STAR, Singapore.

Aerial Robotics Contact-Rich Robotic Manipulation Aerial Manipulation

Aug 3, 2026cs.RO

TWINS: A Tactile Wearable Isomorphic Arm Networked System for Contact-Rich Manipulation Learning

Recent advances in robot learning for manipulation have increased the importance of collecting real-world demonstration data. However, existing robotic systems primarily focus on end-effector manipulation, making it difficult to teach and execute manipulation tasks involving body-surface contact with the arms and chest. This paper presents TWINS (Tactile Wearable Isomorphic Arm Networked System), a robotic system for manipulation involving body-surface contact. TWINS consists of a Wearable Dual-Arm Device, which is worn by the operator, and an Isomorphic Robot with the same joint configuration and external dimensions. Distributed tactile sensors embedded in the chest and arms enable the measurement of body-surface contact synchronized with joint motion. Using the Wearable Dual-Arm Device, we collected demonstrations for four manipulation tasks involving body-surface contact. We then trained imitation learning policies using the collected demonstrations and deployed them on the Isomorphic Robot, enabling manipulation guided by body-surface tactile observations. Experimental results demonstrate that TWINS provides a unified robotic system for demonstration, learning, and execution of manipulation involving body-surface contact. https://mmurooka.github.io/twins-project-page/

Takahide Kitamura, Masaki Murooka, Natsuki Yamanobe +1

National Institute of Advanced Industrial Science and Technology (AIST), 2-3-26 Aomi, Koto-ku, 135-0064, Tokyo, Japan.

Bimanual Robotic Manipulation Contact-Rich Robotic Manipulation Robot Imitation Learning Robotic Tactile Sensing

Aug 2, 2026cs.RO

Demystifying When and Why VLAs Fail in Contact-Rich Tasks and How to Fix Them

We address the problem of understanding when and why Vision-Language-Action models struggle with contact-rich manipulation tasks that require precise physical interaction. Prior work has primarily focused on addressing contact failures through force-augmented architectures and training-time regularizers, yet the root causes of these failures remain underexplored. We identify two distinct failure modes underlying this gap. Precision failures are rooted in a flow-matching policy training mismatch, and force failures arise from the distinctive structure of force signals. We address each failure mode with a targeted mechanism and combine them into FACT, which achieves 66% average success rate across five contact-rich tasks against 41% for the best prior baseline, in an evaluation spanning almost 2,500 real-world rollouts.

Carlota Parés-Morlans, Nils Kuhn, Isabel Liu +2

Stanford University

Robotic Control Contact-Rich Robotic Manipulation Vision-Language-Action Models

Aug 2, 2026cs.RO

Learning Physical Interaction: A Survey of Tactile- and Force-aware Robot Learning

Physically grounded robot intelligence requires robots to perceive, reason about, and regulate their interactions with the physical world. This capability is particularly critical in contact-sensitive manipulation, where successful task execution depends not only on visual perception and motion generation, but also on force regulation and adaptive control. In this context, recent robot learning methods have made substantial progress by integrating force, tactile, vision, language, and proprioceptive sensing into learned manipulation policies. In parallel, many systems adopt multi-phase architectures that combine high-level policies, action-refinement modules, and low-level controllers to bridge semantic task understanding with reactive physical execution. Despite these advances, existing surveys have not explicitly reviewed force- and tactile-aware robot learning from a unified perspective that jointly captures multimodal sensing and multi-phase system design. This survey addresses this gap by proposing TF-ART, a Tactile/Force-Aware Robot learning Taxonomy for multimodal and multi-phase frameworks, which maps individual methods into a unified hierarchical structure. The framework characterizes how recent works organize observation modalities, encode and fuse heterogeneous sensory inputs, generate and refine actions across multiple phases, and connect learned policies to reactive robot-end control. Building on this methodological view, we further examine the task settings and infrastructure requirements of physical interaction, thereby integrating both algorithmic and practical perspectives on force- and tactile-aware robot learning.

Shilin Shan, Chuhao Zhou, Ruize Wang +30

Nanyang Technological University · Stanford University · University of California, Berkeley +9

Contact-Rich Robotic Manipulation Robotic Tactile Sensing Robot Learning

Aug 2, 2026cs.RO

CAAT: Contact-Aware Attention Scaling and Tactile Masking for Data-Efficient Contact-Rich Manipulation

In contact-rich manipulation, visual observations primarily guide motion in free space, whereas tactile observations become particularly informative during contact. However, standard Transformer-based visuo-tactile policies typically rely on either token concatenation or learnable gating. These approaches lack explicit contact-aware priors, making it difficult to efficiently learn effective cross-modal representations from demonstrations. To address this limitation, we propose CAAT, a lightweight contact-aware framework that explicitly incorporates contact priors through attention scaling and dynamic tactile masking. Specifically, CAAT emphasizes visual information before contact and tactile information during contact. It also suppresses static background tokens by comparing the current tactile observation with a non-contact reference. CAAT can be integrated into commonly used Transformer-based policies without modifying their action decoders. In simulation, integrating CAAT with ACT improves the average success rate by 18.0 percentage points over direct visuo-tactile fusion and by 10.0 percentage points over gated fusion. In real-world experiments using a visuo-tactile UMI platform, CAAT achieves an average success rate of 60.0% across ACT, Diffusion Policy, and

π_0

, outperforming the strongest baseline by an average of 21.1 percentage points. These results demonstrate that explicit contact priors and dynamic tactile masking are effective in improving visuo-tactile policy learning and task performance of diverse policy architectures. https://mrjiangjm.github.io/caat/

Jiaming Jiang, Yuzhe Huang, Hao Liang +7

ShanghaiTech University · Beijing Institute for General Artificial Intelligence · Beihang University +2

Contact-Rich Robotic Manipulation Visuo-Tactile Representation Learning Robot Imitation Learning Multimodal Sensor Fusion

Aug 1, 2026cs.RO

Grasp Execution Without a Planner: Configuration-Space Grasp Distance Fields with Certified Safety & Guaranteed Quality

Mainstream plan-then-track approaches to multifingered grasp execution entail selecting a grasp, planning a collision-free trajectory, and tracking the resulting trajectory via a feedback controller. Pose-estimation error during execution or scene motion can invalidate this open-loop commitment and trigger replanning. We thus present Grasp Distance Fields (GDFs), smooth softmin distances to finite sets of arm-hand grasp configurations. Using their negative gradients as feedback, we jointly select and execute grasps without planning a trajectory. A CBF-CLF quadratic program (QP) enforces self-collision, workspace, object, and obstacle-clearance constraints, while its CLF slack quantifies obstruction of task progress. We bound the softmin approximation error by

\log N/ρ

and prove forward invariance of the filtered safe set. To handle changes in contact topology, we combine a hysteretic contact-mode transition with a wrench-quality CBF that limits degradation of the realized force-closure margin relative to hold onset. Using our method, a fixed-base manipulator and a Unitree G1 equipped with the same underactuated hand grasp and lift 46 of 50 test objects amid clutter and moving obstacles. The realized grasps also retain a median 94% of their synthesized quality margin, and each QP solve requires 0.09 ms within a 20 ms control interval. Project page: www.clintonenwerem.com/gdf.

Clinton Enwerem, John S. Baras, Calin Belta

Department of Electrical and Computer Engineering and the Institute for Systems Research (ISR), University of Maryland, College Park, MD, USA

Robotic Control Robotic Grasping Signed Distance Functions Contact-Rich Robotic Manipulation

Aug 1, 2026cs.RO

Disentangling Visuo-Tactile Foresight: Oracle-Guided Interface Discovery for World Action Models

Contact-rich manipulation remains challenging because successful control depends on physical interaction cues that are often weakly observable from vision alone. Recent tactile world action models jointly model future visual observations and tactile signals to guide action generation, but how such futures should be structured for effective use by the action expert remains underexplored. Directly studying this question with learned world action models is difficult because end-to-end behavior entangles physically invalid visual futures, unreliable predictions, inaccurate or cross-modally inconsistent tactile forecasts, and an unreadable future-to-action interface. To make this interface independently studyable, we introduce Oracle Visuo-Tactile Foresight (OVTF), a controlled framework that supplies paired RGB and tactile futures from successful trajectories verified in simulation. By fixing the future provider, OVTF isolates the interface and asks a cleaner question: if the future is successful and physically executable, what representation allows the action expert to absorb its benefit? Within OVTF, we propose Asymmetric Phase-Local Future Memory (AFM), in which visual memory reads future vision, each tactile memory jointly attends to its own tactile stream and phase-aligned future vision, and cross-tactile access is blocked. We compare AFM with Modality-Isolated Future Memory (IFM), which removes visual-to-tactile access and processes each future modality independently. Across seven tasks on the UniVTAC simulation benchmark, AFM achieves 32.0% average success, compared with 23.7% for IFM and 14.9% for UniVTAC-ACT. This controlled comparison shows that selective phase-aligned visual-tactile routing provides a more actionable future-to-action bridge than complete modality isolation.

Zihang Yao, Chaoyue Ding, Yingying Yu

Brigham Young University · Brigham Young University, Provo, Utah, USA · Beijing Academy of Science and Technology +1

World Models for Robotics Contact-Rich Robotic Manipulation Tactile Sensing for Robotic Manipulation Tactile World Models

Aug 1, 2026cs.RO

Learning to Predict Contact Force Distributions from Vision Leveraging Object Geometry Priors

Based on vision and prior experience, humans can make rough physical predictions and adjust their manipulation strategies. This paper aims to endow robots with a similar ability. To collect paired data of vision and forces, we use a rigid-body simulator commonly adopted in robotics. However, unlike simulators that output noisy point forces, humans are able to make consistent predictions even in unfamiliar situations. Based on this observation, we hypothesize that predicting smooth force distributions rather than raw point forces can improve both force prediction itself and downstream task performance. To validate this hypothesis, we construct a model that predicts three-dimensional force distributions from a single RGB image of piled daily objects. The target distribution is generated by applying statistical smoothing to point forces obtained from the simulator. Moreover, by incorporating object geometry into the smoothing process, we aim to account for variations in contact states and achieve more consistent vision-based predictions. We conduct extensive evaluations in both simulation and real environments. Results show that our approach improves prediction accuracy, enhances downstream task performance through smoothing, and further benefits from geometry-guided smoothing. Remarkably, the trained model generalizes effectively to real-world scenes despite being trained solely in simulation.

Ryo Hanai, Yukiyasu Domaea, Ixchel G. Ramirez-Alpizar +3

Artificial Intelligence Research Center, National Institute of Advanced Industrial Science and Technology (AIST), Tokyo, Japan · Intelligent Systems Research Institute, National Institute of Advanced Industrial Science and Technology (AIST), Tokyo, Japan · Institute of AI and Robotics, Graduate School of Fundamental Science and Engineering, Waseda University, Tokyo, Japan

Contact-Rich Robotic Manipulation Robotic Manipulation

Jul 31, 2026cs.RO

Compliant Sphere Lattice Contact: Distributed Contact Modeling for Sphere-Based Robot Representations

Contact planning in robotics requires models that are both computationally efficient and physically accurate. Sphere-based robot representations satisfy the first requirement by enabling fast collision checking and differentiable geometry, but sacrifice physical accuracy by relying on point contact which cannot capture contact patch area, pressure distributions, rotational stiffness, or frictional moments. We introduce Compliant Sphere Lattice Contact (CSLC), a distributed contact model that operates natively on sphere representations by modeling the robot interface as a compliant lattice of surface spheres connected through anchor and lateral springs. When pressed against an object, the lattice deforms to produce a spatially distributed contact patch that improves the physical accuracy of sphere-based contact. We validate CSLC across two independent solvers and show preliminary results demonstrating contact patch formation and improved grasp stability.

Nataliya Nechyporenko, Ava Abderezaei, Alessandro Roncone

Robotic Grasping Contact-Rich Robotic Manipulation

Jul 31, 2026cs.RO

TRACT: Temporally Routed Action Chunks with Chronological Phase Authority for Contact-Rich Manipulation

Action chunking shortens the effective decision horizon of robot imitation learning by predicting multiple future actions, while conventional phase conditioning describes the current control instant. When a predicted horizon crosses a procedural boundary, assigning the current phase to the entire chunk creates a structural temporal mismatch. We present TRACT, which factorizes phase-structured action chunking into an accepted current phase and a single CURRENT-to-NEXT boundary inside the future horizon. A task-local graph constrains chronological phase authority, and a cumulative boundary distribution monotonically routes future queries through phase-specific query and action paths. For contact execution, a causal response-deficit integrator compares policy intent with ACK-eligible subsequent motion, accumulates arm compensation when directional response is suppressed, and decays after confirmed recovery. Across six real-robot variants with ten trials each, full TRACT achieves 10/10 full-sequence success, 99.00 [88.75, 100.00]% median [min, max] wipe completion, zero observed phase ambiguity, and zero stalls. Under the current complete method package and evaluation setting, the routed representation obtains better observed task results than the flat package (6/10 vs. 3/10 success; 77.08% vs. 8.03% median wipe completion). Chronological authority reduces observed phase ambiguity from 8/10 to 0/10, and response integration reduces stalls from 4/10 to 0/10. The package comparison does not isolate routing from other generator-package differences.

Jiahao Liu, Kento Kawaharazuka, Tasuku Makabe +1

Department of Mechano-Informatics Graduate School of Information Science and Technology, The University of Tokyo Tokyo, Japan

Robotic Control Robot Skill Learning Contact-Rich Robotic Manipulation Robot Imitation Learning Action Chunking

Jul 31, 2026cs.RO

MDIR: A Task-Manifold Impedance Retargeting Method for Contact-Rich Teleoperation

Fixed Cartesian impedance makes contact-rich teleoperation demonstrations practical, but gains that secure progress and contact support also determine impact and force variability. We study single-demonstration controller-to-controller impedance retargeting. Given one fixed Cartesian impedance command sequence {K0, D0, xcmd}, Manifold-Decomposed Impedance Retargeting (MDIR) deterministically reparameterizes the recorded controller into an executable task-channel variable-impedance command. MDIR targets this local retargeting problem by preserving projected task-channel responses near the demonstrated trajectory. It represents the source response in operational work, exertion, and support channels with a passive residual complement under a control-chain metric, computes an executable Cartesian-to-Manifold Retargeting (C2M) baseline, and applies Manifold-Constrained Parameter Optimization (MPO) to select a feasible representative with lower wrist-force peaks, impulse, force variability, and nominal controller power. Across planar wiping, pick-and-place, and pushing on a Franka Panda, the full MDIR controller passes Task Check in all 15 closed-loop executions and reduces all four aggressiveness metrics relative to the fixed-impedance demonstrations.

Liu Jiahao, Kento Kawaharazuka, Tasuku Makabe +1

Department of Mechano-Informatics, Graduate School of Information Science and Technology, The University of Tokyo, Tokyo, Japan

Contact-Rich Robotic Manipulation Robot Teleoperation Impedance Control

Jul 31, 2026cs.RO

TacPrint: A Wearable Fingertip Tactile Sensor for Human-to-Robot Contact Reproduction

Human-centric data collection is emerging as a significant paradigm for robot skill acquisition, but seamlessly integrating low-cost, scalable tactile sensing systems that capture fine-grained fingertip interactions without compromising natural operation remains a key challenge. This reduces the reliability of human-to-robot transfer in contact-rich tasks. In this work, we present TacPrint, a wearable fingertip tactile sensor, where protrusions on the inner surface of the silicone skin are aligned one-to-one with 24 capacitive taxels to enable localized capacitive responses. A real-to-sim-to-real pipeline estimates a 35

\times

26 contact-depth map from 24-channel capacitive signals. Against simulation-generated labels, the model achieved a contact-region RMSE of 0.223

\pm

0.161 mm, a weighted-centroid error of 1.213

\pm

2.379 pixels, and an IoU of 0.829

\pm

0.169. With measured capacitive inputs, the network-predicted depth evaluated at the guide-calibrated contact center showed a mean absolute error of 0.085

\pm

0.057 mm across all 40 controlled trials, while the mean contact-position error was 0.250

\pm

0.208 mm across the 37 trials whose reference contact regions were not truncated by the sensing boundary. In human-to-robot replay, tactile-guided compensation increased grasping and wiping success rates from 0% to 91.67% and 90%, respectively. In closed-loop grasping, dense-depth feedback achieved success rates of 87.5% over all tested positions and 85% under edge-contact conditions, compared with 67.5% and 45% for raw-taxel feedback.

Yongxi Liu, Chaofan Zhang, Xingyu Zhang +4

Institute of Automation, Chinese Academy of Sciences · Imprintx Robotics · Beijing Academy of Artificial Intelligence

Robotic Grasping Tactile Sensing Contact-Rich Robotic Manipulation Tactile Sensing for Robotic Manipulation

Jul 30, 2026cs.RO

FA-RDP: A Frequency-Adaptive Reactive Diffusion Policy for Contact-Rich Manipulation

In contact-rich manipulation, action multimodality and reactivity dominate different stages of a single episode. Before contact, multiple trajectories might be equally valid, making it important to preserve diverse action modes. After contact, geometric constraints and force limits narrow the solution space, while successful execution demands rapid responses to force feedback. However, standard diffusion policies use a fixed inference frequency and sampling steps throughout the episode, forcing a fundamental compromise: low-frequency, multi-step sampling better preserves pre-contact multimodality but responds slowly to force feedback, whereas high-frequency sampling improves reactivity but tends to collapse distinct pre-contact modes. To resolve this tradeoff, we present FA-RDP, a frequency-adaptive reactive diffusion policy. A shared multi-frequency visual-force Transformer predicts action chunks at both low and high frequencies, while a learned multimodality indicator dynamically selects multi-step low-frequency sampling before contact and one-step high-frequency sampling as action ambiguity decreases. We further introduce Manifold Consistency Distillation (MCD), which reparameterizes the diffusion network to predict actions on the robot action manifold while retaining DDPM-based residual supervision. Experiments on three contact-rich manipulation tasks show that FA-RDP achieves the highest success rate while preserving diverse pre-contact trajectory modes. Code and videos are available at https://fa-rdp.github.io.

Lifeng Zhuo, Wendi Chen, Han Xue +4

1Shanghai Jiao Tong University · 2Shanghai Innovation Institute · 3Noematrix Ltd.

Contact-Rich Robotic Manipulation Visuomotor Policy Learning Diffusion Distillation Diffusion Policy Diffusion Model Inference Acceleration

Jul 30, 2026cs.RO

TacWAM: Anchor-Guided World Action Model with Mechanics-Aware Tactile Prediction

World Action Models (WAMs) combine future-state prediction with robot action generation, but existing approaches largely rely on visual futures. Visual prediction captures scene structure and object motion, yet provides limited supervision for force, deformation, shear, and slip during contact-rich manipulation. This creates two design requirements: tactile futures should carry meaningful physical information, and they should not become privileged cues for action generation. We present TacWAM, a mechanics-aware tactile WAM that addresses this challenge in three steps. First, a Spatially Aligned Fusion (SAF) Tactile Encoder maps tactile appearance, dense force fields, and deformation flow into a shared latent prediction space, with bilateral force and torque reconstruction preserving global contact information. Second, a tactile history encoder provides temporal context so future tactile prediction reflects how force and deformation change beyond the current tactile observation. Third, Anchor-Guided Tri-Modal (AGT) Attention separates current visual and tactile anchors, future prediction tokens, and action tokens, allowing future tactile states to supervise training without being directly read by the action branch. We evaluate TacWAM on four real-world contact-rich manipulation tasks covering fragile grasping, sustained surface contact, and dynamic in-hand manipulation. TacWAM achieves an average success rate of 75.0%, exceeding the strongest evaluated baseline by 37.5 percentage points. Staged ablations show consistent degradation when tactile history is removed and access to future prediction targets is relaxed. These results indicate that future tactile supervision can improve contact-aware action learning when combined with informative tactile representations and deployment-consistent information constraints.

Lei Jin, Yiding Ma, Xin Zhang +3

1Tsinghua University · 2Manifold AI

World Models for Robotics Contact-Rich Robotic Manipulation Visuo-Tactile Representation Learning World Action Models Tactile Sensing for Robotic Manipulation Tactile World Models

Jul 29, 2026cs.RO

ContactFlow: A video action conditioning that transfers across embodiments

World models offer a promising route toward robot planning by enabling agents to imagine and verify the consequences of actions before execution. However, current video-based world models often struggle to capture the physical constraints that govern manipulation, particularly contact. Further, their action conditioning is often constrained to specific embodiments such as parallel grippers. We propose \emph{Contact Flow}, an embodiment-agnostic action representation that encodes manipulation through the trajectory of 3D contact points between an actor and a target object. By discarding actor-specific appearance and kinematics, Contact Flow provides a shared conditioning signal for both human demonstrations and robotic execution. Therefore, we can train a large-scale video generative model on both human and robotic object interaction videos conditioned on Contact Flow, yielding a world model that predicts physically plausible manipulation outcomes. We integrate this model into a propose-imagine-verify-act pipeline, where generated rollouts are assessed by a vision-language model before execution. Experiments on the DROID dataset and real-world tabletop manipulation tasks demonstrate that Contact Flow enables transfer between human demonstrations and different robotic embodiments.

Sami Azirar, Enrico Pallotta, Jan Nogga +3

University of Bonn, Lamarr Institute

Action-Conditioned Video Generation Contact-Rich Robotic Manipulation Action-Conditioned World Models Cross-Embodiment Robot Learning

Jul 27, 2026cs.RO

FeelWorld: Visuo-Tactile World Model for Hierarchical Contact Prediction and Planning

Humans plan physical interactions by imagining the possible outcomes of candidate actions. However, existing visual world models primarily capture appearance dynamics while overlooking the tactile states that govern contact-rich interactions, potentially producing imagined futures that appear visually plausible but violate physical dynamics. We introduce FeelWorld, a hierarchical visuo-tactile world model that jointly predicts future visual latents and three tactile states. FeelWorld organizes these states hierarchically as contact state, a 3D tactile latent that encodes force-related information, and slip state. These states are jointly predicted by a shared latent dynamics model with explicit supervision. To prevent irrelevant tactile signals during free-space motion from degrading visual prediction, we introduce a contact-gated asymmetric attention mechanism that maintains a visual-only prediction pathway before contact and enables joint visuo-tactile dynamics prediction during contact. The model is further trained with autoregressive rollouts and context noise injection to improve robustness to compounding errors. The predicted contact and slip states also support contact-aware CEM planning. Experiments on chip grasping, fruit grasping, and USB insertion show that FeelWorld reduces 10-step LPIPS from 0.084 to 0.058 and maintains an LPIPS that is 61% lower than that of the visual baseline after an 80-step autoregressive rollout. FeelWorld also achieves an average zero-shot planning success rate of 81.7%, providing an effective approach for incorporating tactile sensing into world models.

Wenxuan Ma, Chaofan Zhang, Chao Xue +4

Institute of Automation, Chinese Academy of Sciences · Imprintx Robotics · Beijing Academy of Artificial Intelligence

Contact-Rich Manipulation World Models for Robotics Contact-Rich Robotic Manipulation World Models World Model-Based Planning Model-Based Planning+1

Jul 26, 2026cs.RO

$N_0$ -TWAM: Scaling Tactile-Native World-Action Model for Contact-Rich Manipulation

We present

N_0

-TWAM, a tactile-native world-action model for contact-rich manipulation that predicts both future vision and future contact. To our knowledge, it is the first tactile world-action model trained at large scale, and it shows strong capability on contact-rich tasks. We pre-train

N_0

-TWAM at large scale with visuo-tactile joint training over tactile-rich demonstrations spanning six embodiments and 450 tasks. We use NeoForce, a unified force-based tactile representation, to form a physically grounded contact signal that conditions action generation. To improve long-horizon and multi-stage manipulation, we introduce tactile contact events for task staging and advance through them during execution. For real-time efficiency, we adopt an asymmetric Mixture-of-Transformers architecture that pairs a full-width expert for video prediction with slim experts for downstream action and tactile prediction. Evaluations on both real and simulated benchmarks justify the capabilities of

N_0

-TWAM across a range of contact-rich tasks, and demonstrate the benefit of data scaling for precise tactile and action prediction. In summary,

N_0

-TWAM endows a world-action model with predictive capabilities to foresee vision, touch and action, building a solid foundation for fine-grained manipulation on open contact-rich tasks. The codebase and model checkpoints will be made publicly available to foster further research and development in tactile-enabled robotic manipulation.

NeoteAI Team, Fudan TEAI Team

NeoteAI Team · Fudan TEAI Team

Long-Horizon Robotic Manipulation World Models for Robotics Contact-Rich Robotic Manipulation Visuo-Tactile Representation Learning Tactile World Models

Jul 26, 2026cs.RO

$N_0$ -VTLA: Scaling Vision-Tactile-Language-Action Model with Latent Tactile Tokens

We present

N_0

-VTLA, a vision-tactile-language-action (VTLA) foundation model capable of (1) fine-grained contact-rich manipulation with tactile perception and tactile-feedback control, and (2) offline policy improvement from stored deployment data. Building on current vision-based backbones, we propose a training recipe for tactile integration consisting of visuo-tactile pre-training, staged tactile-pathway integration, and advantage-conditioned offline policy improvement. During pre-training, the policy learns broad contact priors from NeoData, our large-scale visuo-tactile robot dataset; to our knowledge,

N_0

-VTLA is the first VTLA model pretrained on tactile data at scale. During post-training, we augment the policy with a predictive tactile pathway that distills the contact patterns learned at scale into the fine motion adjustments required by downstream tactile-centric manipulation. For offline policy improvement, we introduce ALTER, an advantage-conditioned offline reinforcement learning method that converts relative progress and trajectory-event comparisons into binary advantage labels for policy training on a fixed deployment corpus, further improving task-specific learning on contact-rich skills such as deformable object manipulation. Across contact-rich benchmarks,

N_0

-VTLA outperforms strong baselines by wide margins: it wins all nine real-robot NeoReal tasks and reaches 63.8% mean success on a twenty-task simulation suite, against 44.0% for the strongest baseline.

N_0

-VTLA policies trained with ALTER reach 75-95% success on three long-horizon real-robot tasks. These results lay a foundation for versatile tactile-driven manipulation policies.

NeoteAI Team, Fudan TEAI Team

NeoteAI Team · Fudan TEAI Team

Long-Horizon Robotic Manipulation Contact-Rich Robotic Manipulation Visuo-Tactile Representation Learning Offline RL Tactile Sensing for Robotic Manipulation

Jul 24, 2026cs.RO

ViTacWorld: Scaling Visuo-Tactile World Models for Contact-Rich Robot Manipulation

Contact-rich robot manipulation requires physical interaction cues that are often invisible to cameras, making tactile sensing essential for robust control. However, scaling visuo-tactile robot learning remains difficult because real tactile interaction data are expensive to collect, hardware-dependent, and limited in task and scene diversity. We present ViTacWorld, an action-conditioned visuo-tactile world model for scalable contact-rich robot manipulation. ViTacWorld leverages public real tactile datasets and a constructed simulation environment to scale visuo-tactile-action data, exploiting the fact that tactile signals are directly grounded in physical contact and can exhibit a smaller simulation-to-real gap than purely visual observations. The model is first pretrained with large-scale real and simulated visuo-tactile trajectories, and then finetuned with real-world policy rollouts to better match downstream manipulation behaviors. Given robot actions, ViTacWorld predicts temporally aligned visual observations and tactile feedback, enabling visuo-tactile-action rollout generation. To the best of our knowledge, ViTacWorld is the first framework that uses a world model for robot visuo-tactile-action trajectory generation and policy evaluation. It serves two roles: synthesizing rollouts to improve downstream tactile policies, and evaluating policies by predicting action-conditioned visuo-tactile outcomes under controlled action sequences. Experiments on contact-rich manipulation tasks show that ViTacWorld generates physically meaningful rollouts, improves policy performance through scalable data augmentation, and enables action-conditioned policy evaluation. Project page: https://vitacworld.github.io/

Yunao Huang, Shiyu Sang, Suting Ni +6

ShanghaiTech University · InstAdapt

World Models for Robotics Contact-Rich Robotic Manipulation World Models Synthetic Data Generation for Robotics Tactile Sensing for Robotic Manipulation Tactile World Models

Jul 24, 2026cs.RO

Impedance Control of Ship-Borne Manipulators via Optimization-based Task-Space Inverse Dynamics

Ship-borne manipulators operating in maritime environments are subject to stochastic wave-induced base motions that introduce kinematic disturbances and dynamic coupling, degrading trajectory tracking accuracy and complicating safe, contact-rich manipulation. This paper proposes a torque-level optimization-based control framework that integrates high-precision trajectory tracking with task-space impedance for ship-borne manipulators. The controller is formulated using task-space inverse dynamics (TSID) and solved via quadratic programming to explicitly compensate for the dynamic coupling introduced by base motion. To enable accurate feedforward compensation, an error-state Kalman filter (ESKF) is developed to estimate the base state by fusing inertial measurements with end-effector pose feedback. The framework is validated in simulation and real-world experiments using a 7-DOF manipulator mounted on a 6-DOF Stewart platform. The proposed method reduces real-world end-effector position tracking error by over 25.7% compared with the best baseline. Furthermore, the controller enables dynamic peg-in-hole insertion with 1~mm clearance under base motion, increasing the success rate while reducing average contact forces by 45%, demonstrating precise and compliant manipulation in contact-rich environments.

Lingxiao Meng, Bi-Ke Zhu, Xuheng Gao +5

Robot Trajectory Tracking Contact-Rich Robotic Manipulation Impedance Control Robotic Manipulation

Jul 23, 2026cs.RO

FORGE-plus: Force-Budgeted Recovery for Contact-Rich Assembly with a Frozen LLM Supervisor

Force-conditioned reinforcement learning (RL) enables tight-clearance assembly under a commanded force ceiling, but practical deployment requires determining an appropriate force limit for each object and recovering from insertion failures without exceeding it. We present a two-layer framework in which a frozen, text-only large language model (LLM) assigns a per-object force ceiling before execution and selects recovery maneuvers from a fixed action menu using compact textual force signatures. The LLM never controls force directly: a low-level controller enforces the force ceiling, the recovery policy cannot increase it, and the hidden breaking-force threshold is known only to the evaluator. We evaluate the framework on fragile bottle placement and 0.4 mm diametral-clearance gear insertion using two grippers (Robotiq 2F-140 and Franka Panda hand). A single policy passes 256/256 evaluation episodes on both fragile and robust objects without breakage, correctly predicts release timing, and completes a full table-pick-and-insert pipeline with a mean peak force of 5.4 N. Under injected in-grip slip, the force-signature recovery strategy resolves 40% and 64% of failures on the two grippers, whereas a press-harder baseline is either ineffective or causes frequent breakage. We also report negative results, including the failure of PPO to solve the task under strict force constraints and unsuccessful learned release strategies. All experiments are conducted in rigid-body simulation with hidden force-threshold breakage; no sim-to-real claim is made.

Kyupaeck Jeff Rah, Midum Oh

Independent Researcher

Reinforcement Learning Language Model-Based Control Contact-Rich Robotic Manipulation Robot Failure Recovery Constrained RL Robotic Manipulation+1

Jul 23, 2026cs.RO

URF: A Unified Robot Control-Policy Framework for Stable Contact Aware Manipulation

Learning-based manipulation policies usually predict robot actions from sensory observations and leave their execution to a separate low-level controller. In rigid contact, this separation can be problematic: the same motion to a virtual target or compliant motion command can lead to unstable contact, tracking error, excessive loading, or tool damage, depending on the low-level controller. In this paper, we propose a \textit{Unified Robot Control-Policy Framework} (URF), which connects compliant action prediction with unified impedance-admittance control. Given multimodal observations, URF predicts a virtual target, a stiffness matrix, and an impedance-admittance switch ratio. The switch ratio determines when the controller should behave more like admittance control for accurate motion tracking and when it should move toward impedance control for safer rigid contact. Because demonstration data do not provide ground-truth environment stiffness, we construct switch-ratio labels from measured contact forces and use them to supervise controller-mode prediction. Across box-flipping and line-pressing tasks, URF achieves higher task success rates while reducing failure modes observed with admittance-only execution, including rapid force buildup, large force oscillations, tool breakage, and robot safety stops. These results suggest that contact-aware policies benefit from predicting not only compliant actions but also the controller behavior used to execute them. Project page: https://jiyou384.github.io/urf_project_page/

Jiyou Shin, Youngjin Seo, Jaeseog Won +5

The authors are with the Faculty of Mechanical Engineering, Sungkyunkwan University, Suwon 2066, South Korea · The authors are with the Faculty of Intelligent Robotics, Sungkyunkwan University, Suwon 2066, South Korea

Robotic Control Contact-Rich Robotic Manipulation Impedance Control Robotic Manipulation Force-Controlled Robotic Manipulation

Jul 22, 2026cs.RO

FELT: Generating Tactile Signals from Vision for Visuo-Tactile Manipulation

The sense of touch is central to manipulation, especially when vision is occluded or ambiguous. Although combining vision and touch improves manipulation, learning robust visuo-tactile policies requires substantial tactile data. Such data remains scarcer than visual data, because tactile sensors are fragile, specialized, and hard to standardize. To address this, we present Feature-Extracted Latent Tactile (FELT), a learning-based framework that synthesizes per-finger pressure tactile images from RGB observations, reducing the need for tactile-equipped data collection. FELT uses a large frozen visual encoder and a lightweight query decoder to predict tactile signals in a single feed-forward pass. To respect the physical topology of dual-finger tactile sensors, FELT decodes the left and right tactile sensor panels through separate branches, capturing the asymmetric contact patterns during interactions such as wiping, insertion, and in-hand rotation. At inference time, FELT only requires RGB data, allowing us to augment existing vision-only data with tactile observations, either as generated tactile images or as latent tactile features. Experiments on four contact-rich manipulation tasks demonstrate that both generated tactile images and latent tactile features improve policy success over vision-only baselines, with latent feature requiring no real tactile sensor during policy training or deployment. Supplementary material is available on our anonymous website: https://felt-tactile.github.io/.

Zinan Li, Yiyang Ling, Yuming Gu +8

Viterbi School of Engineering, University of Southern California · Columbia University · Starpilot

Contact-Rich Robotic Manipulation Visuo-Tactile Representation Learning Tactile Sensing for Robotic Manipulation

Jul 20, 2026cs.RO

FM-VLA: Force-based Memory for Vision-Language-Action Models in Contact-Rich Manipulation

Vision-language-action (VLA) models have achieved impressive generalization in robotic manipulation, and recent memory-augmented VLAs have relaxed the Markovian assumption by conditioning on past images or language summaries. Vision-based memory approaches address this by conditioning on sampled past image frames, but they are computationally expensive and fundamentally limited when temporal events are visually ambiguous, e.g., pushing a button multiple times with small movements. We propose FM-VLA, a VLA model with force-based memory, enabling temporal context reasoning for non-Markovian, contact-rich manipulation. We encode force histories into compact force memory tokens with a variational autoencoder (VAE) pretrained with force time series reconstruction. By projecting force latent representations and short state history as additional conditioning tokens to the action expert module, we enable VLAs to leverage accumulated contact event history to guide manipulation. We evaluate FM-VLA on three memory-dependent tasks, including finding a hidden block, pressing a button, and wiping a dish for a specific number of times. Our lightweight force memory achieves over 80% success rate with minimal inference overhead, significantly outperforming baseline approaches. Project page: https://qft-333.github.io/FM-VLA-Page/

Ruicheng Li, Qixiu Li, Ruichun Ma +8

Tsinghua University · Microsoft Research · Fudan University +1

Memory-Augmented VLMs Language-Conditioned Robot Manipulation Contact-Rich Robotic Manipulation Robotic Manipulation Vision-Language-Action Models

Jul 20, 2026cs.RO

HCPG-Flow:Hierarchical Contact-Progress Guidance for Flow-Policy Robot Manipulation

Flow policies can represent multimodal action distributions for robot manipulation, yet a robot must execute one action at each control step. When several proposals are sampled, critic-based ranking makes data collection depend on value estimates over candidate actions that may be weakly represented in replay. We introduce HCPG-Flow, an analytic rollout-time selector that augments SAC-Flow with hierarchical, object-centric contact-progress guidance while preserving its actor and critic objectives. HCPG switches from end-effector approach to task progress after contact, scores each proposal by the first-order reduction of a task-relevant distance, standardizes scores within the candidate set, and executes a temperature-controlled action embedding. Across ten simulated tasks, HCPG improves mean success over SAC-Flow on both benchmarks, including a 9.5 percentage-point gain on Maniskill. Four physical tasks further show high success with a 17.4% reduction in successful completion steps.Project page: https://hitxraz.github.io/HCPG-Flow/

Guanghu Xie, Mingxu Li, Shuo Zhang +5

State Key Laboratory of Robotics and Systems, Harbin Institute of Technology, Harbin 150001, Heilongjiang, China

Contact-Rich Robotic Manipulation Robotic RL Robotic Manipulation

Jul 17, 2026cs.RO

BayesContact: Uncertain Pose Estimation via Visuo-Tactile Proposals and Simulation-based Inference

Contact-rich manipulation requires pose estimates that are often more accurate than what depth-only sensing provides. Existing methods, relying on vision and contact, employ costly offline training procedures that need to be retrained for new environments and geometries. We propose BayesContact, a Simulation-Based Inference framework for visuo-tactile pose estimation in peg-in-hole insertion. BayesContact maintains a particle belief over object pose and fuses depth observations with force/torque-derived contact evidence. We employ simulation based forward models to approximate these observation likelihoods. For each pose hypothesis, a renderer predicts depth measurements and a physics simulator predicts contact outcomes under guarded probing actions; both are scored against real observations to update the belief. The resulting multimodal belief also enables information-gain-based probing for active disambiguation. Across simulated geometries and real-robot experiments, BayesContact improves pose observability and insertion success over vision-only inference by 30%

Aditya Kamireddypalli, Matias Mattamala, Joao Moura +3

School of Informatics, University of Edinburgh, Scotland. · Department of Mechanical and Mechatronics Engineering, University of Waterloo, Canada.

Simulation-Based Inference Contact-Rich Robotic Manipulation Object Pose Estimation Robotic Insertion

Jul 17, 2026cs.RO

Data and Learning Where it Matters for Contact-Rich Manipulation

Learned policies trained end-to-end on large datasets often remain brittle in high-precision tasks and struggle with generalization. We find that these limitations largely stem from a lack of structure and focus in data collection. Our key insight is to leverage dense data collection only for the critical segment of contact-rich tasks and to rely on traditional planning during simple free-space motion. We propose an automated data-collection scheme in combination with offline deep reinforcement learning for the critical segment of the task, eliminating reliance on a teleoperator's skill and on online policy updates. Across four challenging real-world tasks, using only 2 to 2.5 hours of autonomous data collection, we achieve an average success rate of 96%, compared to the strongest baseline at 55%. Notably, performance remains high in out-of-distribution scenarios where end-to-end approaches struggle. Our results pave the way for targeted data collection for contact-rich tasks and for high success rates in precision applications.

Oliver Hausdörfer, Linus Schwarz, Gabor Marko +7

1TU Munich · 2Siemens AG

Robot Skill Learning Contact-Rich Robotic Manipulation Robotic Data Collection Robot Policy Generalization

Jul 17, 2026cs.RO

IMBench: A Benchmark for Intuitive Robotic Manipulation

Humans combine reasoning and motor control to solve complex manipulation tasks under diverse constraints. They build an understanding of the physical world that helps them convert reasoning into actions and quickly adapt to new scenes, tasks, and rules. We refer to this capability as intuitive manipulation. Existing benchmarks fail to capture this integration: they evaluate physical reasoning in isolation from execution, or measure policy performance without requiring explicit reasoning. We introduce IMBENCH, a benchmark designed to evaluate intuitive manipulation as an integrated capability spanning perception, physical reasoning, action generation, and iterative execution. Our tasks require models to infer task-relevant physical structure and generate feasible action sequences under explicit constraints, including contact-rich manipulation, tool use, and multi-stage dependencies. We introduce a benchmark of 35 tasks, 14K filtered trajectories, and scalable tools for generating diverse scenarios. Experiments reveal a consistent gap: vision language models show partial physical reasoning ability but fail to produce executable plans, while state-of-the-art vision-language-action models struggle to satisfy task constraints and generalize across scenarios. These results identify intuitive manipulation as a missing axis in current foundation models and generalist robot policies, and position IMBENCH as a step toward evaluating and enabling more integrated, adaptive physical intelligence.

Anurag Maurya, Sukhvansh Jain, Prajwal Avhad +9

Manav Robotics, Bengaluru, India

Contact-Rich Robotic Manipulation Robotic Manipulation Robot Manipulation Benchmarks VLMs for Robotics

Jul 16, 2026cs.RO

Representation-Aligned Tactile Grounding for Contact-Rich Robotic Manipulation

Tactile-enhanced vision-language-action (VLA) policies have been introduced for contact-rich manipulation, where critical interaction states are often hidden from vision. Future tactile prediction is a promising way to use touch because it turns tactile outcomes into supervision for action-induced contact dynamics. Yet VLA policies contain representations with different roles, from perceptual encoding to motor prediction, making it unclear where this supervision should be applied. We study this as a representation-alignment problem. Through a linear probe analysis, we find that future tactile states are most predictable from intermediate action-expert features, rather than from vision-language features or final action states. Motivated by this observation, we introduce a lightweight Latent Tactile Predictor (LTP), which predicts compact future tactile embeddings from the identified intermediate representation. By avoiding direct prediction of noisy raw tactile signals, LTP provides an action-outcome grounding signal that aligns intermediate action representations with future contact consequences. Experiments on real-world contact-rich manipulation tasks show that representation-aligned tactile grounding outperforms less aligned or multi-interface tactile prediction, highlighting the importance of where tactile supervision is applied.

Ruilin Chen, Jingkai Jia, Tong Yang +8

Shanghai Key Lab of Intelligent Information Processing, College of Computer Science and Artificial Intelligence, Fudan University · College of Intelligent Robotics and Advanced Manufacturing, Fudan University · TeleAl, China Telecom +2

Contact-Rich Robotic Manipulation Visuo-Tactile Representation Learning Tactile Sensing for Robotic Manipulation Representation Alignment