cs.ROSep 23, 2026

GLoTouch: Global-to-Local Haptic Perception Using a Parallel Gripper for Object Search, Recognition, and Grasping Without External Vision

Authors: Zonglin Li, Wanruo Zhang, Yiming Wang, Kun Song, Xinyi Zhou, Daolin Ma

Organizations: Independent Researcher · Shanghai Jiao Tong University · The University of Hong Kong

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Perceiving objects in the environment is a fundamental capability of autonomous robots. In dark or low-light environments, external cameras often fail to reliably perceive object positions and geometry; when visual sensing is unavailable, completing target search, recognition, and grasping through touch alone becomes a key robot manipulation capability. This task must simultaneously address container-scale spatial exploration and object-scale fine-grained geometric perception, which is particularly challenging for low-degree-of-freedom parallel grippers. However, a unified framework remains lacking for connecting container-scale spatial exploration with object-scale fine-grained geometric perception and grasping. To address this challenge, we present GLoTouch, a global-to-local haptic perception and manipulation framework built on a parallel gripper. In the global stage, the gripper holds a passive long-reach probe, combining force measurements with known tool geometry to localize contacts and actively estimate candidate-object positions, coarse contours, and heights. In the local stage, the robot sets down the probe and uses the bilateral visuotactile sensors on the same gripper to directly acquire local haptic observations, which are matched against a given target 3-D model without object-specific training. We evaluate the framework in both simulation and real-robot experiments. Source code will be open-sourced.

Figures & tables

Fig. 1: GLoTouch pipeline. Given the target STL model and workspace, A Global Haptic Exploration performs coverage-driven probe search, tool-geometry-aware contact localization, and surrounding touch positioning to estimate a candidate object CiC_{i} . After the probe is set down, B Local Haptic Recognition acquires stable tactile contacts, predicts finite-window support geometry from the target STL, and ranks candidates for retrieval. The ranking bars in B are illustrative similarity scores; the implemented decision ranks candidates by matching loss, where lower is better (Sec. V ).
Fig. 2: Task lighting setting: deployment is in darkness or low light; photographs are captured under temporary illumination for documentation only. The controller uses no external visual input.
Fig. 3: Global exploration and localization. (a) Wrist force magnitude; red arrows mark contact-triggered stops. (b) Exploration trajectory, contact points, estimated object centers, and fitted contours.
Fig. 4: Contact-mode validation on a calibration square. (a) Ground truth, contacts, square fits, and centers. (b) Errors from the true contour (mm).
Fig. 5: UniVTAC simulation sequence. Blue : probe-mediated global exploration. Black : tactile acquisition. Green : grasping.
MethodMean discovered (/5)Complete at 5 m (%)
Random descent1.23/3.17/4.0230.0
Global search1.35/3.30/4.4455.0
TABLE I: Global-search ablation on 100 randomized five-object layouts. Discovered values are ordered as 1/3/5 m budgets; complete discovery is at 5 m.
TargetWukongNailongAppleTrainTruckTotal
Successes16/2019/2020/2014/2015/2084/100
TABLE II: Simulation retrieval by target in five-object scenes. Each target is evaluated in 20 trials.
STL \ ObjectNailongWukongTrainAppleTruck
Nailong0.270.482.400.651.69
Wukong1.020.542.271.471.85
Train1.871.330.781.370.93
Apple1.590.892.960.082.24
Truck1.912.022.051.250.90
TABLE III: Mean matching-loss matrix under the full matcher. Rows: target STLs; columns: physical objects. Diagonal entries: correct pairings.
MethodTop-1 (%)Margin
Width/action only68.30.267
+ Local depth78.30.343
+ Collection-mode prior (Full)90.00.402
TABLE IV: Progressive ablation on the five-object development set. Margin is the mean first–second loss difference under each method’s own loss scale and is not directly comparable across rows.
TargetWukongNailongAppleTrainTruckTotal
Successes7/108/1010/105/108/1038/50
TABLE V: Real-robot end-to-end retrieval by target in five-object scenes. Each target is evaluated in 10 trials.
Fig. 6: Real-robot five-object sequence. Red : global probe search. Blue : probe return and local recognition. Green : target retrieval. Dark frames indicate a temporary lights-off period; the tactile-driven pipeline continues to execute.
More in this topic

Tactile Sensing for Robotic Manipulation

Tactile sensing for robotic manipulation uses contact signals to help robots perceive forces, prevent slips, and handle objects reliably.

See all 180 papers

Explore similar work

CardsList
Sep 18, 2026

ZeroTouch: Tactile-Supervised Visual Contact Estimation for Contact-Rich Manipulation

Reliable robotic grasping benefits from estimating the evolving physical interaction and selecting a grasp-dependent compression target. Tactile sensors provide direct interaction measurements but require dedicated hardware at deployment. We introduce ZeroTouch, a tactile-supervised framework that predicts dense contact deformation, the instantaneous six-axis wrench, and a grasp-dependent desired compression target from wrist RGB observations, gripper state, and local gravity direction. Tactile measurements are used only as privileged supervision during training and are not required at deployment. On the full validation set, the complete architecture reduces normal-force MAE from 2.017 N for a state-only baseline to 0.531 N. In physical evaluation with 20 trials per condition, ZeroTouch achieves 95% success on an unseen object, 80% in a seen-object/unseen-grasp condition, and 90% under a content/load shift. Under the same evaluation protocol, OpenVLA achieves 25%, 40%, and 55%, while SmolVLA achieves 10%, 25%, and 35%, respectively.

Dmitriy Kosenkov, Daniia Zinniatullina, Miguel Altamirano Cabrera +3

Intelligent Space Robotics Laboratory, Skolkovo Institute of Science and Technology, Moscow, Russian Federation.

Robotic GraspingContact-Rich Robotic ManipulationContact Force EstimationVisual-to-Tactile PredictionRobotic PerceptionForce-Controlled Robotic Manipulation
AskPDF
Jun 23, 2026

TACTFUL: Tactile-Driven Exploration For Object Localization and Identification in Confined Environments

Humans effortlessly locate and identify objects by touch alone, even without vision. In contrast, robotic systems rely heavily on vision and struggle with autonomous tactile exploration and object identification. We present TACTFUL, a vision-free tactile exploration framework that enables a multi-fingered robot to autonomously explore confined workspaces, discover objects through contact, and identify them via tactile reconstruction. Trained entirely on real hardware without simulation, our system learns a single policy that balances global workspace exploration with local surface refinement through a dynamic reward schedule. Our results demonstrate that tactile sensing, when paired with structured learning, can serve as an effective primary modality for object-level reasoning, achieving 77% success with 0.015 m average reconstruction error and outperforming baseline approaches on real-world objects.

Shivani Kamtikar, Chung Hee Kim, Camilla Tabasso +3

Amazon Fulfillment Technologies & Robotics, Westborough, MA, USA. · Siebel School of Computing and Data Science, University of Illinois at Urbana-Champaign, Champaign, IL, USA. · Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, USA.

RL for RoboticsRobotic Tactile SensingRobotic Object Search
AskPDF
Mar 11, 2026

TacLoc: Global Tactile Localization on Objects from a Registration Perspective

Pose estimation is essential for robotic manipulation, particularly when visual perception is occluded during gripper-object interactions. Existing tactile-based methods generally rely on tactile simulation or pre-trained models, which limits their generalizability and efficiency. In this study, we propose TacLoc, a novel tactile localization framework that formulates the problem as a one-shot point cloud registration task. TacLoc introduces a graph-theoretic partial-to-full registration method, leveraging dense point clouds and surface normals from tactile sensing for efficient and accurate pose estimation. Without requiring rendered data or pre-trained models, TacLoc achieves improved performance through normal-guided graph pruning and a hypothesis-and-verification pipeline. TacLoc is evaluated extensively on the YCB dataset. We further demonstrate TacLoc on real-world objects across two different visual-tactile sensors.

Zirui Zhang, Boyang Zhang, Fumin Zhang +1

Object Pose EstimationPoint Cloud RegistrationTactile Sensing
AskPDF