In visually ambiguous manipulation such as detecting button click tactile feedback is often the sole source of ground truth. However, fusing tactile data poses a significant challenge due to a spatiotemporal mismatch: tactile perception requires high-frequency processing with long-horizon memory (System 1), whereas visual policies operate at low control frequencies (System 2). Existing architectures struggle to bridge this gap: Transformers are computationally prohibitive for high-frequency loops (>100Hz), while LSTMs suffer from forgetting over extended interaction histories. In this paper, we introduce TacMamba, a hierarchical architecture that aligns high-bandwidth tactile reflexes with low-frequency visual planning. Our approach comprises three core contributions: (1) a custom high-frequency tactile interface designed for flexible integration; (2) a Mamba-based Tactile History Compressor that encodes continuous force history into a compact state with O(1) inference latency (0.45 ms), enabling plug-and-play fusion with VLA models without joint pre-training and (3) a Tactile-Guided Dual-Stage Training strategy that leverages temporal discrimination for self-supervised representation learning and phase-uniform sampling to mitigate data sparsity. Experiments on discrete counting and implicit state switching demonstrate that TacMamba achieves 100% success rates, significantly outperforming the visual-only pi_0.5 baseline, while strictly satisfying hard real-time constraints.
Figures & tables
Fig. 1: The TacMamba System Architecture. The framework bridges the spatiotemporal discrepancy between high-speed reflexes and low-frequency reasoning. Left (System 1): The tactile encoder processes 1D force streams at 100Hz using Mamba Models, recursively updating the hidden state ht in real-time. Right (System 2): This compressed hidden state ht is projected and asynchronously injected as a soft prompt into the low-frequency ( ∼ 1Hz) Vision-Language-Action (VLA) planner.
Fig. 2: System Overview. (a) Modular morphology-based tactile fingertip design, where an integrated compliant contact body supports both fingertip and fingerpad interactions and mechanically projects distributed contacts onto a single-axis force sensor; lateral side panels protect the internal structure, and a reconfigurable dual-clamp interface enables attachment to generic parallel grippers. (b) FEM-based characterization of force projection, showing peak contact pressure versus applied load for fingertip and fingerpad contacts, together with (c) representative visualizations of total deformation, equivalent (von Mises) stress, and contact pressure.
Fig. 3: The TacMamba Network Architecture. Top: The core TacMamba backbone processes continuous tactile streams via a hierarchical Selective SSM, utilizing RevIN and Channel Independence for feature extraction. Bottom-Left: An expanded view of the Mamba block mechanism, illustrating how input-dependent parameters ( Δt,Bt,Ct ) model hybrid dynamics. Bottom-Right: The auxiliary discriminator network, employed exclusively during the training phase to facilitate robust feature learning and enforce temporal causality.
Fig. 4: Efficiency Analysis. Inference latency and memory growth.
Figure 5
Fig. 7: Qualitative folding results comparison. (a) Successful cloth folding workflow executed by the full TacMamba model. The zoomed-in views highlight precise and stable grasps on the fabric corners enabled by high-frequency tactile feedback. (b) and (c) Typical failure modes observed when the phase-prioritized sampling strategy is disabled (ablation): ”Failed to grasp” (missing the corner due to visual ambiguity) and ”Shape crumpled” (improper tension control causing deformation).
Method
Success Rate
External Visual Baselines ∗ :
OpenVLA [ 29 ]
6%
CogACT [ 30 ]
15%
MemoryVLA [ 11 ]
58%
Our Implementations:
π0.5 [ 2 ] (Early stage, 5k steps)
37.5%
TABLE II: Performance on Sequential Button Pressing Task
Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China · Zhongguancun Academy, Beijing 100094, China · Dexmal, Beijing 100096, China
SKL-MAIS, Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · TARS Robotics +2