Mixture of Flow Matching and Particle Filter for Proprioception-Based Online Contact Estimation

Seo Wook Han; Min Jun Kim

cs.ROOct 21, 2026

Mixture of Flow Matching and Particle Filter for Proprioception-Based Online Contact Estimation

Authors: Seo Wook Han, Min Jun Kim

Organizations: Intelligent Robotic Systems Laboratory, School of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST), Daejeon 34141, South Korea

Abstract

In this article, we propose ContactFlowFilter, a novel approach for contact estimation that determines the number of active contacts, their locations, and the associated force vectors. We consider a robot manipulator equipped with joint torque sensors and a base force/torque sensor; such a proprioceptive-only setup presents a challenging inverse problem characterized by (i) multi-modality from non-unique solutions, (ii) temporal dependencies in state evolution, and (iii) discrete contact events involving contact addition/removal. Existing approaches, such as particle filters (PFs) and learning-based methods, often struggle to address these challenges simultaneously. To overcome this, ContactFlowFilter integrates a generative model into the PF framework. The recursive structure of the PF captures temporal dependencies, while the flow matching model serves as an auxiliary proposal distribution to track multi-modal beliefs and detect contact events. Extensive validation in both simulation and real-world experiments confirms the effectiveness of the proposed method. In real-world settings, ContactFlowFilter achieves the lowest dissimilarity from the target distribution, with errors of only 0.36 cm and 0.88 cm for single- and dual-contact scenarios, respectively, while operating at 0.69 ms and 2.69 ms.

Figures & tables

Panel of Figure 1 — Fig. 1: (a) This work focuses on the problem of contact estimation for robotic arms: identifying the number of active contacts, their locations ( $\bm{r}_{i}\in\mathbb{R}^{3}$ ), and the associated forces ( $\bm{F}_{i}\in\mathbb{R}^{3}$ ) using proprioceptive sensing (e.g., joint torque sensors (JTSs) and a base force/torque (F/T) sensor). (b) The proposed method addresses up to two active contacts, with the estimation results for the first and second contacts represented by magenta and blue arrows, respectively.

Panel of Figure 2 — Fig. 2: Examples of key challenges in the contact estimation problem: (a) Multi-modality : Two distinct contact forces ( $\bm{F}_{1a}$ and $\bm{F}_{1b}$ ) result in identical observations. (b) Temporal dependency : Current contact states depend on history, involving discrete changes in the number of contacts (e.g., when a new contact $\bm{F}_{2}$ occurs, $t\text{-}2\!\rightarrow\!t\text{-}1$ ) and continuous updates in locations or forces (e.g., when an existing contact $\bm{F}_{1}$ slides, $t\text{-}1\!\rightarrow\!t$ ). (c) Contact event detection : Determining the number of active contacts relies on identifying the exact timing of a transition and whether a contact is added or removed.

Fig. 3: Overview of the proposed ContactFlowFilter framework. The proposed method samples particles from two complementary proposal distributions: Top : The PF’s temporal prediction involves sampling from the previous time step, random position updates, and importance weighting (indicated by pink circles proportional to their weights). The same procedure applies to the particles around the second contact $\bm{F}_{2}$ ; only $\bm{F}_{1}$ is visualized for clarity. Bottom : The flow matching samples (indicated by pink and blue points) are transported along a probability path $(p^{[0]}\rightarrow p^{[1]})$ to represent the contact points. Right : These proposals are merged to integrate temporal context and multi-modality, forming the combined distribution for the subsequent recursive prediction.

	Model-based Approaches			Learning-based Approaches				Ours
	Line of force	Particle filter	QP-based algorithm	Probabilistic mapping	Binary classification	Regression	Diffusion model	Ours
Contact Number Support	$1$	$\geq 2$	$1$	$\geq 2$	$1$	$1$	$2$	$2$
Multi-Modal Distribution	$\times$	$\times$	$\bigcirc$	$\bigcirc$	$\times$	$\times$	$\bigcirc$	$\bigcirc$
Temporal Reasoning	$\times$	$\bigcirc$	$\times$	$\times$	$\times$	$\times$	$\triangle$	$\bigcirc$
Contact Force Estimation	$\bigcirc$	$\bigcirc$	$\bigcirc$	$\times$	$\times$	$\times$	$\times$	$\bigcirc$
References	[ 5 , 11 , 21 , 45 ]	[ 38 , 36 , 14 , 61 , 59 , 3 , 47 , 24 ]	[ 42 , 44 ]	[ 32 ]	[ 60 , 46 ]	[ 46 , 9 ]	[ 13 ]	–

TABLE I: Comparison of different approaches based on key capabilities.

Fig. 4: The Link 1 meshes of the Franka Emika Panda robot are used for (a) visualization and (b) motion model computation. Non-contact faces are removed (red circle), and the mesh is refined into uniform triangular faces (blue circle).

Fig. 5: The proposed motion model consists of two steps: (a) For the given particle $\bm{x}_{t-1,i}^{(m)}$ , the geodesic distances to all other faces $\{d_{j}\}_{j=1}^{n_{f}}$ are retrieved from a pre-computed lookup table. (b) The weights $\{w_{j}\}_{j=1}^{n_{f}}$ are set inversely proportional to the geodesic distances; the particle for current time step $\bm{x}_{t,i}^{(m)}$ is then sampled from the resulting distribution.

Fig. 6: (a) For each contact point $\bm{r}_{t,i}$ , the friction cone $\mathcal{F}_{c}$ is defined by the surface normal $\bm{n}$ . (b) The friction cone is approximated as a polyhedral cone spanned by support vectors $\{\bm{s}_{i,j}\}_{j=1}^{4}$ .

Fig. 7: (a) The results of the QP defined in ( 23 ) are visualized over the entire surface for a single contact ( $\bm{F}_{1}$ ) on link 2, where high-belief (low QP error) regions are observed on both links 1 and 2. (b) The conventional PF algorithm populates the particle set $\mathcal{X}_{t,1}$ on only a single link in each execution (see Trial 1 and Trial 2).

Fig. 8: (a) The first particle set $\mathcal{X}_{t,1}$ converges to the first contact point. (b) After the second contact ( $\bm{F}_{2}$ ) occurs, the delayed initialization of $\mathcal{X}_{t,2}$ causes the existing set $\mathcal{X}_{t,1}$ to drift away from the first contact point.

Panel of Figure 10 — Fig. 9: The training sample $\bm{X}^{[1]}\in\mathbb{R}^{N_{s}\times 6}$ for flow matching is constructed based on the number of contact points $n_{c}$ : (a) $n_{c}=1$ : All $N_{s}$ rows are filled with $\bm{r}_{1}$ , extending across the full width of the matrix. (b) $n_{c}=2$ : The matrix is divided into two blocks horizontally; the first three columns consist of $\bm{r}_{1}$ , and the remaining three consist of $\bm{r}_{2}$ .

Fig. 10: (a) The observation for flow matching is defined as a sequence where each link’s pose $\bm{T}_{l}$ is augmented with the virtual wrench $\mathcal{F}_{v,l}$ . (b) $\mathcal{F}_{v,l}$ is obtained by projecting the spatially distributed external base wrench $\hat{\bm{\mathcal{F}}}_{ext}$ and the first $l$ elements of the external joint torque, $(\hat{\tau}_{ext,1},\hat{\tau}_{ext,2},\cdots,\hat{\tau}_{ext,l})$ , onto the corresponding frame $\{L_{l}\}$ via the adjoint transformation $\text{Ad}^{-T}$ .

Fig. 11: (a) Global feature encoder : Encodes the correlated features of robot kinematics and external contacts. (b) Pointwise feature encoder : Captures the spatial context relative to the robot surface for each query point in $\bm{X}^{[k]}$ . (c) Decoder : Outputs the vector field by conditioning the pointwise features on the global feature.

Fig. 12: Asynchronous execution of the flow matching inference (Python) and the main algorithm loop (C++).

Fig. 13: Experiment A: Comparison of generative processes implemented with our architecture, using (a) Flow Matching [ 8 ] and (b) DDIM [ 51 ] . ( $\bullet$ -1) : Sample trajectories visualized over ODE time steps $k$ (128-step integration). ( $\bullet$ -2) : Accelerated generation paths with 2-step integration. ( $\bullet$ -3) : Resulting samples via the 2-step integration.

Generation method	Architecture	ODE-step	Single-contact ( $n_{c}=1$ )		Dual-contact ( $n_{c}=2$ )
			M-RMSE ( $\mathrm{cm}$ ) $\downarrow$	C-RMSE ( $\mathrm{cm}$ ) $\downarrow$	M-RMSE ( $\mathrm{cm}$ ) $\downarrow$	C-RMSE ( $\mathrm{cm}$ ) $\downarrow$
Flow Matching	✓ Proposed	2	0.17 ± 0.27	0.45 ± 0.63	0.72 ± 0.92	2.20 ± 2.75
		4	0.16 ± 0.27	0.44 ± 0.62	0.67 ± 0.89	2.18 ± 2.86
		8	0.15 ± 0.26	0.43 ± 0.62	0.62 ± 0.86	2.17 ± 2.91
		16	0.14 ± 0.26	0.42 ± 0.62	0.56 ± 0.85	2.16 ± 2.94
Flow Matching	✗ Baseline	2	0.40 ± 0.51	1.36 ± 1.88	1.17 ± 1.15	3.88 ± 3.59

TABLE II: Experiment A: Quantitative results of samples generated via dual-sampling (inference time: approx. 1.28

\mathrm{ms}

/step)

Fig. 14: Experiment B: Evaluation of particle set management for contact event detection under sequential addition (Phase 1) and removal (Phase 2) of contact forces.

	Suc. Rate $\uparrow$		Adj. Delay $\downarrow$		Fau. Rate $\downarrow$
Method	Phase 1 (%)	Phase 2 (%)	Phase 1 (ms)	Phase 2 (ms)	Overall (%)
✓ Proposed	99.60	99.65	3.21	26.74	0.75
✗ Baseline	97.30	91.55	2.89	51.46	16.50

TABLE III: Experiment B: Quantitative results of particle set management

Fig. 15: Experiment B: Sequential snapshots comparing the baseline and proposed methods. (a) The baseline method suffers from erroneous set initialization due to local minima caused by particle starvation. (b) The proposed method effectively performs contact event detection without such false positives.

Fig. 16: Experiment B: (a) Due to particle drift, the baseline method often fails to detect contact removal. (b) Conversely, the proposed method is robust against particle drift by utilizing the dual-particle distribution.

Fig. 17: Experiment B: A failure trial was observed in Phase 1 for both the baseline and proposed methods, as two adjacent contact forces are often indistinguishable from a single resultant force.

Scenario	Metric	Method
Scenario	Metric	ContactFlowFilter (Ours)	CPF	MCP-EP	Dual-Particle only
Single-Contact ( $n_{c}=1$ )	M-RMSE ( $\mathrm{cm}$ ) $\downarrow$	0.04 0.08	$1.84$ 5.27	$0.57$ 1.64	$0.18$ 0.13
	W-RMSE ( $\mathrm{cm}$ ) $\downarrow$	0.09 0.54	$1.94$ 5.55	$0.71$ 1.85	N/A
	M-RMSE ( $\mathrm{N}$ ) $\downarrow$	0.12 0.07	$0.14$ 0.16	$0.13$ 0.07	N/A
	W-RMSE ( $\mathrm{N}$ ) $\downarrow$	0.17 0.07	$0.20$ 0.16	$0.17$ 0.08	N/A
	PLS $\downarrow$	5.67 8.52	$6.83$ 10.38	$7.95$ 13.34	$6.56$ 16.84

TABLE IV: Experiment C: Quantitative results of the simulation evaluation

Fig. 18: Experiment C: The likelihood function, defined in ( 35 ), is visualized over the entire robot surface for two scenarios: (a) single-contact ( $n_{c}=1$ ) and (b) dual-contact ( $n_{c}=2$ ). In (b), the likelihood is computed for the second contact, resulting in a negligible value at the first contact point (indicated by pink circle).

Fig. 19: Experiment C: Comparison of three methods in multi-modal single-contact scenarios: (a) ContactFlowFilter, (b) CPF, and (c) Dual-Particle-Only.

Fig. 20: Experiment C: The first row (a- $\bullet$ ) presents the results of ContactFlowFilter, and the second row (b- $\bullet$ ) shows those of CPF. Each row illustrates the sequential contact scenario, evolving from a single-contact ( $n_{c}=1$ ) to a dual-contact ( $n_{c}=2$ ).

Scenario	Metric	Method
Scenario	Metric	ContactFlowFilter (Ours)	CPF	MCP-EP	Dual-Particle only
Single-Contact ( $n_{c}=1$ )	M-RMSE ( $\mathrm{cm}$ ) $\downarrow$	0.36 0.52	$1.36$ 3.99	$0.77$ 1.40	$0.51$ 0.63
	W-RMSE ( $\mathrm{cm}$ ) $\downarrow$	1.62 1.08	$2.88$ 5.48	$2.99$ 2.84	N/A
	M-RMSE ( $\mathrm{N}$ ) $\downarrow$	1.98 1.11	$2.36$ 2.66	$2.02$ 1.70	N/A
	W-RMSE ( $\mathrm{N}$ ) $\downarrow$	2.62 2.43	$3.86$ 4.29	$6.17$ 5.91	N/A
	PLS $\downarrow$	8.85 14.03	$12.36$ 34.12	$30.10$ 34.59	$16.76$ 30.76

TABLE V: Experiment D: Quantitative results of the real-world evaluation

Fig. 21: Experiment D: The real-world evaluation dataset was collected (a) using a tool for acquiring ground-truth contact states ( $\bm{r}_{i}$ and $\bm{F}_{i}$ ), (b) under a sequential dual-contact scenario.

Contact Force Estimation

See all 25 papers

Sep 23, 2026

CoPRE: Improving Sensitivity in Proprioceptive Contact Detection for Low-Cost Robot Arms

Contact detection during robotic manipulation allows robots to recognize unexpected contact and adapt their motion accordingly. However, in low-cost robot arms without dedicated force or tactile sensors, detecting weak contacts from proprioception is challenging because the resulting changes in joint-level proprioceptive signals can be small compared to normal variation and noise caused by robot motion itself. We introduce Contact-free Proprioceptive Response Estimation (CoPRE), improving proprioceptive contact detection sensitivity using only contact-free motion, without additional force sensors, contact labels, or analytical dynamics models. CoPRE estimate the expected joint torques under contact-free motion from proprioceptive state history and commanded motion, while removing recent observations that may already reflect contact. It then computes the residual between the expected and observed joint torque estimates, and maps this residual to a contact score using a noise-weighted Jacobian. Real-robot experiments on ARX Arm and Unitree G1 show that CoPRE achieves 74.1% and 82.2% recall on the tested contact trials, compared with 0%/0% on ARX and 16.3%/42.2% on G1 for the learned torque-prediction and inverse-dynamics baselines. CoPRE also reaches 90% detection rate for pushing force at 3.5 N on ARX and 5.5 N on G1. To demonstrate the downstream utility of our method, we implement belief-space manipulation planning for obstacle-aware object placement and book insertion where detected contacts update the spatial belief and enable the robot to retreat from blocked motions, adjust its pose, and retry. Project website at https://copre-arm.github.io

Yuxiao Zhu, Jinzhou Li, Yifei Dong +6

Duke University · KTH Royal Institute of Technology · Carnegie Mellon University

Robotic Perception Robotic Manipulation Contact-Rich Robotic Manipulation

Ask PDF

May 2, 2026

SixthSense: Task-Agnostic Proprioception-Only Whole-Body Wrench Estimation for Humanoids

Humanoid robots are entering our physical world at scale, yet as oversized toys--good at singing and dancing, but short on force-interaction capabilities for practical tasks. Bridging this gap necessitates prioritizing reliable contact perception as a fundamental requirement. Estimating external wrenches in humanoids is complicated by floating-base dynamics and indeterminate contact locations. Existing analytical frameworks require idealistic assumptions and hard-to-obtain measurements, which are often unavailable in practice. To bridge this gap, we propose SixthSense, a task-agnostic approach that infers whole-body contact timing, location, and wrenches from proprioception and IMU data alone. To capture the multi-modal dynamics between unstructured contact inputs and the uncertain motion outputs, we employ conditional flow matching to tokenize proprioceptive histories and estimate a spatiotemporally sparse contact-event flow. SixthSense serves as a plug-and-play perception module for applications including collision detection, physical human-robot interaction, and force-feedback teleoperation. Experiments across standing, walking, and whole-body motion-tracking policies showcased unprecedented performance in diverse behaviors.

Xingzhou Chen, Xiayan Xu, Yan Ning +8

The Hong Kong University of Science and Technology, Hong Kong SAR, China · Zhejiang University, Hangzhou, China · Tencent Robotics X, Shenzhen, China

Robotic Perception Humanoid Robotics Contact Force Estimation

Ask PDF

Oct 7, 2026

Tactile Reconstruction of Contact Task Frames and Forces for Hybrid Force/Motion Control

Hybrid force/motion control requires knowledge of the interaction force and of a task frame defining the force- and motion-controlled directions. These quantities are usually obtained from force/torque sensing or model-based residuals, often assuming also a nominal environment model. This work addresses the online estimation of the contact force and a possibly time-varying task frame using only soft optical tactile sensing, under the assumption of locally planar contact with a negligible contact moment. The proposed method maps a single image of the deformed elastomer of a soft optical tactile sensor to observable contact variables: indentation depth, two surface-to-sensor tilt angles, and 3D contact force, each with a per-sample uncertainty estimate. The mapping is learned through a self-labeling acquisition procedure, in which a manipulator imposes controlled contacts while an auxiliary Force/Torque sensor is used offline to provide ground-truth labels. The tactile measurement is then fused with robot proprioceptive data in an Extended Kalman Filter, producing a continuously updated estimate of the contact task frame and of the interaction force. Control experiments with a DigiTac sensor mounted on a UR10 manipulator demonstrate closed-loop contact force regulation against a flat rigid board in linear and angular motion by a human operator, with touch as the only exteroceptive feedback.

Antonio Rapuano, Simone Orelli, Barbara Bazzana +2

Department of Computer, Control and Management Engineering, Sapienza University of Rome, 00185 Rome, Italy. · Robotics and Mechatronics Department, Faculty of Electrical Engineering, Mathematics and Computer Science, University of Twente, 7500 AE Enschede, The Netherlands.

Contact Force Estimation Force-Controlled Robotic Manipulation Tactile Sensing Vision-Based Tactile Sensing

Ask PDF

Abstract

Contact Force Estimation

Explore similar work

CoPRE: Improving Sensitivity in Proprioceptive Contact Detection for Low-Cost Robot Arms

SixthSense: Task-Agnostic Proprioception-Only Whole-Body Wrench Estimation for Humanoids

Tactile Reconstruction of Contact Task Frames and Forces for Hybrid Force/Motion Control