Physics-Informed Sliding-Window Particle Filtering for Tactile-Only In-Hand 6-DoF Object Pose Refinement

Lingjun Shao; Ying Zhang; Xiangfei Li; Xiangyang Li; Huan Zhao; Zhenyu Wang; Han Ding

cs.ROAug 18, 2026

Physics-Informed Sliding-Window Particle Filtering for Tactile-Only In-Hand 6-DoF Object Pose Refinement

Authors: Lingjun Shao, Ying Zhang, Xiangfei Li, Xiangyang Li, Huan Zhao, Zhenyu Wang, Han Ding

Organizations: State Key Laboratory of Intelligent Manufacturing Equipment and Technology, School of Mechanical Science and Engineering, Huazhong University of Science and Technology, Wuhan 430074, China. · Electric Power Research Institute, Guangdong Power Grid Co Ltd., Guangzhou 510080, China. · HUST-Wuxi Research Institute, Wuxi 214174, China.

Abstract

This paper studies tactile-only 6-DoF pose refinement and belief maintenance for grasped objects in static and short quasi-static in-hand configurations where vision is unavailable or heavily occluded. The key difficulty is tactile partial observability: whole-hand taxel contacts are sparse, intermittent, and ambiguous under limited excitation and object symmetries. We propose a physics-informed particle filter on $\mathrm{SE}(3)$ that updates pose beliefs from dense whole-hand tactile measurements. The likelihood combines active-contact signed-distance consistency, force-normal alignment, friction-cone feasibility, zero-force negative evidence, and optional feasibility guards. A sliding-window log-likelihood fuses recent tactile frames to reduce single-frame ambiguity, while a potential-field-guided proposal steers particles away from hand--object penetration. Symmetry-aware resampling preserves multiple plausible modes. Experiments on an Allegro Hand V5 with five objects show lower normalized ADD-S than tactile-only geometric, particle-filter, and learning baselines, and ablations confirm the benefits of temporal fusion, potential guidance, and mode preservation.

Figures & tables

Param.	Value	Param.	Value	Param.	Value
$\tau_{c}$	$0.5$	$F_{\rm ref},\omega_{\max}$	$1.0$ , $14.0$	$\sigma_{\rm sd},\lambda_{\rm sd}$	$0.002$ , $6.0$
$c_{d},\delta_{\min},\delta_{\max}$	$0.0018$ , $0$ , $0.007$	$\sigma_{\rm ang},\lambda_{\rm ang}$	$25^{\circ}$ , $1.0$	$\mu,\sigma_{\rm fric},\lambda_{\rm fric}$	$0.6$ , $1.0$ , $1.0$
$m_{0},\sigma_{0},\lambda_{0}$	$0.001$ , $0.002$ , $1.0$	$m_{f},\sigma_{\rm front},\lambda_{\rm front}$	$0.001$ , $0.002$ , $10.0$	$\epsilon_{\rm out},\lambda_{\rm out}$	$0.006$ , $1.0$
$\sigma_{g},\lambda_{F},\lambda_{\tau}$	$0.002$ , $0.5$ , $0.5$	$N$	$500$	$\sigma_{\parallel}^{\mathrm{i/f}}$	$0.03/0.002\,\mathrm{m}$
$\sigma_{\perp}^{\mathrm{i/f}}$	$0.01/0.001\,\mathrm{m}$	$\sigma_{\rm yaw}^{\mathrm{i/f}}$	$10.0/0.8^{\circ}$	$\sigma_{\rm tilt}^{\mathrm{i/f}}$	$3.0/0.04^{\circ}$
$N_{a}$	$200$	$\phi(\cdot)$	piecewise, $k=6.0$ , hold $0.2$	$\gamma_{\rm explore}$	$0.10$

TABLE I: Key full-model parameters. Superscript i/f denotes initial/final values.

Method	Screwdriver	Connector	Cuboid	Flat bottle	Banana	Average	Time / update (s)
ICP	1.103/0.235	0.517/0.171	0.621/0.210	0.745/0.260	0.698/0.241	0.737/0.306	0.018
ICP-force	1.038/0.290	0.516/0.229	0.602/0.226	0.702/0.285	0.671/0.263	0.706/0.299	0.024
Covariance T2G [ 6 ]	0.530/0.319	0.857/0.821	0.646/0.402	0.912/0.613	0.748/0.521	0.739/0.635	0.092
Contact-SDF PF	0.487/0.238	0.512/0.171	0.452/0.162	0.562/0.218	0.496/0.189	0.502/0.209	0.118
Single-frame Physics PF ( $W=1$ )	0.590/0.238	0.545/0.210	0.522/0.196	0.624/0.250	0.557/0.223	0.568/0.214	0.156
Dikhale-TactileOnly ∗ [ 11 ]	0.895/0.420	0.730/0.350	0.585/0.260	0.812/0.410	0.768/0.360	0.758/0.381	0.047

TABLE II: Normalized ADD-S (median/IQR) and runtime. Contact-SDF PF and Single-frame Physics PF are internal ablations; the other comparators are external baselines.

Fig. 1: Representative configurations of the screwdriver, connector, cuboid, flat bottle, and banana on the tactilely instrumented Allegro Hand V5.

Object	$s_{o}$ / Grasp type	Trials / frames per trial
Screwdriver	33.7, only grasp	200 / 240
Connector	12.8, grasp & pinch	200 / 240
Cuboid	18.0, grasp & pinch	200 / 240
Flat bottle	26.8, only grasp	200 / 240
Banana	20.4, only pinch	200 / 240

TABLE III: Summary of the evaluated object set and tactile sequences.

Setting	ADD-S Med. / IQR	Conv. steps	Fail. rate
Full model / $1.0\times$ nominal	0.375 / 0.176	12.4	7.6%
Component ablations
Contact-SDF PF	0.461 / 0.235	21.3	14.2%
Single-frame Physics PF ( $W=1$ )	0.568 / 0.224	19.8	15.6%
Without potential guidance	0.421 / 0.206	45.1	16.8%
Without mode preservation	0.426 / 0.218	–	29.0%

TABLE IV: Ablation and proposal-scale sensitivity on screwdriver and connector sequences.

Fig. 2: Effect of sliding-window length. Mean normalized ADD-S for

W=1,2,5,10

over 400 trajectories (200 per object) from the screwdriver and connector sets.

Setting	Variant	med. ADD-S / IQR	Fail. rate	Trig. rate
Nominal	w/o reloc.	0.386 / 0.153	6.1%	–
Nominal	Ours	0.361 / 0.124	2.6%	2.4%
Pert. init.	w/o reloc.	0.684 / 0.336	28.7%	–
Pert. init.	Ours	0.418 / 0.179	8.2%	12.1%

TABLE V: Relocalization ablation; perturbed initialization adds random 30-mm translation and

30^{\circ}

rotation offsets.

Object Pose Estimation

Object pose estimation recovers an object's 3D position and orientation from images, depth, point clouds, or contact sensing.

See all 187 papers

Bayesian Active Object Recognition and 6D Pose Estimation from Multimodal Contact Sensing
Mar 22, 2026Haodong Zheng, Gabriele M. Caddeo, Andrei C. Jalba +3Object Pose Estimation Robotic Tactile Sensing
You Only Touch Once: 6-DoF Object Pose Estimation from Single Tactile Contact
Jun 27, 2026Pengfei Ye, Yuxiang Ma, Haonan Chen +5Object Pose Estimation Robotic Manipulation
MIT CSAIL · Harvard University · University of Cambridge
Geometrically Plausible Object Pose Refinement using Differentiable Simulation
Mar 22, 2026Anil Zeybek, Rhys Newbury, Snehal Dikhale +3Object Pose Estimation Robotic Manipulation
Technical University of Munich, Germany · Physical AI, Australia · Monash University, Australia +1

Mar 22, 2026J/K move · Enter open · S save

Bayesian Active Object Recognition and 6D Pose Estimation from Multimodal Contact Sensing

Haodong Zheng, Gabriele M. Caddeo, Andrei C. Jalba, Wijnand A. IJsselsteijn+2

Object Pose Estimation Robotic Tactile Sensing

Open paper AskPDF arXiv

We present an active tactile exploration framework for joint object recognition and 6D pose estimation. The proposed method integrates wrist force/torque sensing, GelSight tactile sensing, and free-space constraints within a Bayesian inference framework that maintains a belief over object class and pose during active tactile exploration. By combining contact and non-contact evidence, the framework reduces ambiguity and improves robustness in the joint class-pose estimation problem. To enable efficient inference in the large hypothesis space, we employ a customized particle filter that progressively samples particles based on new observations. The inferred belief is further used to guide active exploration by selecting informative next touches under reachability constraints. For effective data collection, a motion planning and control framework is developed to plan and execute feasible paths for tactile exploration, handle unexpected contacts and GelSight-surface alignment with tactile servoing. We evaluate the framework in simulation and on a Franka Panda robot using 11 YCB objects. Results show that incorporating tactile and free-space information substantially improves recognition and pose estimation accuracy and stability, while reducing the number of action cycles compared with force/torque-only baselines. Code, dataset, and supplementary material will be made available online.

Abstract

Object Pose Estimation

Explore similar work

Bayesian Active Object Recognition and 6D Pose Estimation from Multimodal Contact Sensing

You Only Touch Once: 6-DoF Object Pose Estimation from Single Tactile Contact

Geometrically Plausible Object Pose Refinement using Differentiable Simulation