MotionDLO: Hybrid Event- and Frame-Based Tracking of Deformable Linear Objects

Annalena Hartmann; Priyamvada Ajithkumar; Patrick Bründl; Jörg Franke

cs.ROAug 23, 2026

MotionDLO: Hybrid Event- and Frame-Based Tracking of Deformable Linear Objects

Authors: Annalena Hartmann, Priyamvada Ajithkumar, Patrick Bründl, Jörg Franke

Organizations: Institute for Factory Automation and Production Systems, Friedrich-Alexander-Universität Erlangen-Nürnberg, Fürther Straße 246b, 90429 Nürnberg, Germany

Abstract

Reliably tracking moving deformable linear objects (DLOs) while simultaneously ensuring robustness, accuracy, and temporally consistent state estimation remains a fundamental challenge in robot perception. We introduce MotionDLO, a real-time tracking framework specifically designed to overcome these limitations in temporal continuity and latency. The method exploits the high temporal resolution and sparsity of event-based cameras and combines segmentation with the Coherent Point Drift (CPD) algorithm under the principles of Motion Coherence Theory. This integration enables temporally consistent shape estimation while maintaining a low computational overhead. Existing event-based tracking methods are typically computationally efficient but exhibit reduced accuracy compared to frame-based approaches, or alternatively compromise event sparsity to achieve competitive performance. To resolve this trade-off, we propose a hybrid event- and frame-based tracking architecture that preserves the complementary strengths of both sensing modalities. The event stream ensures high-frequency motion updates, while frame-based information stabilizes spatial accuracy and object identity. We demonstrate that the proposed framework reliably associates DLO instances across video sequences, enabling robust perception for robotic manipulation tasks. Experimental results validate real-time performance at 12 ms update rates and accurate shape tracking with an point-to-curve error as measurement of accuracy of up to 0.43 mm, supporting dynamic path adaptation during manipulation. The source code and demonstration datasets are publicly available.

Figures & tables

Task	Category	Methods	Advantages	Limitations
Single-frame segmentation	Generic instance segmentation	CNN-based: YOLACT [ 4 ] , YOLACT++ [ 5 ] , SOLOv2 [ 52 ] , CondInst [ 49 ] ; Two-stage: Mask R-CNN [ 29 ] , Mask R-CNN + PointRend [ 19 ] ; Transformer: Mask2Former [ 13 ] , QueryInst [ 20 ]	General-purpose masks via decoupled heads, boundary refinement, or attention	Poor fidelity on pixel-wide DLOs Transformer variants are often too computationally demanding for closed-loop robotic systems with strict latency requirements.
	DLO-specific graph / skeleton pipelines	Ariadne [ 17 ] , Ariadne+ [ 11 ] , FASTDLO [ 9 ] , RT-DLO [ 8 ] , mBEST [ 15 ]	DLO-tailored segmentation, crossing handling, centerline extraction	Per-frame only, no temporal propagation or cross-frame identity
	Foundation-model / vision-language segmentation	SAM family: SAM [ 31 ] , SAM 2 [ 46 ] , SAM 3 [ 12 ] ; Vision-language: CLIPSeg [ 36 ] , ISCUTE [ 32 ] , DLO Perceiver [ 7 ]	Zero-shot or text-guided DLO selection; useful for initialization and drift correction	Masks only, no centerline state or temporal identity High GPU cost for real-time inference
Cross-frame tracking	Registration-based	TrackDLO [ 54 ] , MultiDLO [ 55 ]	Temporally coherent state via point-set registration and Motion Coherence Theory	Degrade under fast motion RGB-D bound and capped at camera frame rate
	Learning-based	UniStateDLO [ 38 ]	Cross-frame state estimation via learned generative diffusion	Complex multi-stage RGB-D / point-cloud pipeline

TABLE I: Comparison of existing single-frame DLO segmentation and cross-frame tracking methods.

Task	Category	Methods	Advantages	Limitations
Motion & feature estimation	Event-camera sensing	Gallego et al. [ 23 ] , Gehrig et al. [ 24 ]	Microsecond latency, HDR, blur-robust edge output	Raw events lack DLO masks or ordered state
	Event-based feature / object trackers	Async corners: Alzugaray et al. [ 3 ] ; Moving-object detection: Mitrokhin et al. [ 40 ] ; Hybrid frame-event: Messikommer et al. [ 39 ] ; Spiking fusion: Yang et al. [ 56 ]	Fast feature- or object-level tracking; hybrid and spiking variants add blur robustness and neuromorphic efficiency	Sparse features or generic moving objects, not DLO geometry or identity
	Event optical flow	EV-FlowNet [ 59 ] , E-RAFT [ 25 ] , time-surface matching [ 42 ] , triplet matching [ 48 ]	High-rate motion fields for warping under fast motion	No DLO instance identity or ordered centerline
Object / structure segmentation	Event-based thin-structure tracking	Powerlines: Dietsche et al. [ 18 ] ; Cable-suspended loads: Panetsos et al. [ 44 ]	Event-based tracking of thin, cable-like structures	Constrained setups, not cluttered multi-DLO manipulation
	Open-vocabulary event perception	SEAL [ 34 ]	Multimodal open-vocabulary event segmentation	Outdoor driving benchmarks, not DLO manipulation

TABLE II: Overview of event-based segmentation and tracking methods relevant to MotionDLO.

Fig. 1: Bandwidth–latency trade-off for DLO observation. (Left) Along the Pareto front (green), a 250 fps high-speed camera attains low latency at the cost of Mb/s, while a 20 fps frame camera remains bandwidth-efficient but incurs 50 ms latency and pronounced motion blur. The event-based camera combines sub-millisecond latency with an order-of-magnitude lower bandwidth than the high-speed camera while yielding a sparse, blur-free signal. (Right) Within a single 50 ms inter-frame interval of the low-speed camera, the DLO traverses 30 mm across the field of view, a displacement that defeats CPD-based frame trackers. The event stream resolves this motion as a continuous sequence of intermediate shapes.

Fig. 2: The tracking result $Y_{t}$ is computed at 12 ms intervals by the CPD tracker operating on a geodesic MCT representation. At each update, the tracker receives polyline input from one of two branches: the frame-based branch (A) or the event-based branch (B). Branch selection depends on the DLO motion state, which is determined by the threshold parameter $\rho_{\mathrm{thr}}$ defined as the count of DLO-attributed events within a fixed temporal window. A motion state of $v_{\mathrm{DLO}}=0$ triggers branch (A), while $v_{\mathrm{DLO}}\neq 0$ triggers branch (B). Temporal alignment between the two branches is ensured via hardware trigger timestamps; spatial alignment is established through the homography $H$ .

Fig. 3: The CPD tracker producing $Y_{t}$ builds upon established components from prior work, (a) arc-length parametrization and (b) MCT geodesic kernel. To account for DLO velocity $v_{\mathrm{DLO}}$ and improve robustness at higher speeds, (c) chain Laplacian regularization and (d) velocity warm-start are introduced. The assumed constant DLO diameter $d$ is taken into account via component (e) to condition the tracker on event-based input and reduce computational cost. The assumed DLO diameter $d$ is taken into account via component (f) to condition the tracker on event-based input and reduce computational cost. Robustness against measurement errors is addressed through the component (g).Robustness against measurement errors is addressed through components (h) and (i).

Fig. 4: Sequence diagram of the respective timing between the different instances

Fig. 5: A single DLO is grasped in an entangled scenario and manipulated by the robot gripper (left). An event-based camera, a frame-based camera, and a depth sensor are mounted in a fixed configuration to jointly observe the DLO motion trajectory (right).

Fig. 6: DLO 1 is a standard pneumatic plastic tube with a reflective surface and a hollow core. DLO 2 is a PROFINET cable, while DLO 3 is a switchgear cabinet wire with a copper core of 1.5 mm diameter.

Robot Speed (%)	Measure	px/s	mm/s
50	average	1398	893.7
50	max	2339	1495.0
75	average	1923	1229.0
75	max	2363	1510.3
100	average	2312	1477.7
100	max	3529	2255.1

TABLE III: TCP speed measurements at different robot speed settings.

Fig. 7: Qualitative evaluation of the observed ID switch. The input image shows the RGB image at three consecutive timestamps, while RT-DLO, SAM 3 and MotionDLO show the respective results.

	RT-DLO		SAM 3		MotionDLO
Time	ID switch	Instances	ID switch	Instances	Switch	Instances
1	0	2	0	1	0	1
2	1	2	0	1	0	1
3	2	2	0	1	0	1
4	2	2	0	1	0	1
5	2	2	0	1	0	1

TABLE IV: Count of ID switches between consecutive timestamps for the DLO 3 at robot speed 75

\%

sequence. Ideal result for the single DLO scenario is no ID switch and one detected DLO.

Fig. 8: Segmentation results for the multiple wire scenario, where the rightmost DLO is moved and needs to be tracked.

	RT-DLO		SAM 3		MotionDLO
Time	ID switch	Instances	ID switch	Instances	Switch	Instances
1	0	2	0	3	0	1
2	2	4	0	3	0	1
3	2	2	0	3	0	1
4	3	3	0	3	0	1
5	2	3	0	3	0	1

TABLE V: Count of ID switches between consecutive timestamps. Ideal result for the multiple DLO scenario is no ID switch and detection of the moving DLO, depending on the algorithm.

Inference time in ms	Recentness map	ROI masking	Mask extraction	Skeletonization	CPD	Complete
MotionDLO	0.32	2.07	2.01	3.68	3.16	12.34
RT-DLO	-	3.20	14.32	-	-	-
TrackDLO	-	-	-	(669)	15.22	-

TABLE VI: Inference time in ms of the MotionDLO pipeline decomposed by processing stage.

Fig. 9: Ground truth generated for DLO 2 and its tracking result. Left: Frame based ground truth generation based on images at the beginning of the movement without motion blur and its respective tracking result. Right: Event-based ground truth generation based on the spare representation and its tracking result. For visualization purposes only 10 of the used 20 nodes are displayed.

Fig. 10: TrackDLO and MotionDLO compared at robot speeds 5

\%

and 50

\%

. At robot speed 50

\%

TrackDLO starts to fail.

Fig. 11: Tracking error at defined timesteps of MotionDLO for all three DLOs under the moving condition at robot speed 100

\%

Fig. 12: The mean arc-length paired and point-to-curve error of MotionDLO under a static condition for all three DLOs.

DLO	Metric	Mean $\bar{e}$	Median	Std $\sigma$	Unit
DLO 1	Arc-length paired	$5.83$	$4.64$	$2.69$	px
	Arc-length paired	$1.06$	$0.84$	$0.49$	mm
	Point-to-curve	$3.05$	$2.94$	$1.20$	px
	Point-to-curve	$0.55$	$0.54$	$0.22$	mm
DLO 2	Arc-length paired	$5.35$	$3.78$	$3.76$	px
	Arc-length paired	$1.02$	$0.72$	$0.72$	mm

TABLE VII: MotionDLO tracking accuracy on DLO 1, DLO 2, and DLO 3 (

n=15

frames, robot speed 100

\%

). Scales:

s=5.50\,\mathrm{px/mm}

(DLO 1),

s=5.23\,\mathrm{px/mm}

(DLO 2),

s=\text{18.12}\,\mathrm{px/mm}

(DLO 3).

Visual Object Tracking

Visual object tracking follows objects across video frames, maintaining their locations and identities despite motion, occlusion, and changing appearance.

See all 175 papers

Event-based Motion & Appearance Fusion for 6D Object Pose Tracking
Mar 9, 2026Zhichao Li, Chiara Bartolozzi, Lorenzo Natale +1Object Pose Estimation Event-Based Vision
Event-driven Perception for Robotics, Istituto Italiano di Tecnologia, Italy · Humanoid Sensing and Perception, Istituto Italiano di Tecnologia, Italy · University of Genoa, Genoa, Italy
Event6D: Event-based Novel Object 6D Pose Tracking
Mar 30, 2026Jae-Young Kang, Hoonhee Cho, Taeyeop Lee +4Object Pose Estimation Event-Based Vision
KAIST · NVIDIA
RoPT: Robust 6DoF Object Pose Tracking in Video
Oct 20, 2026Yongqing Liang, Yuzhong Chen, Sihang Li +7Visual Object Tracking Object Pose Estimation
New York University · General Motors

Mar 9, 2026J/K move · Enter open · S save

Event-based Motion & Appearance Fusion for 6D Object Pose Tracking

Zhichao Li, Chiara Bartolozzi, Lorenzo Natale, Arren Glover

Event-driven Perception for Robotics, Istituto Italiano di Tecnologia, Italy · Humanoid Sensing and Perception, Istituto Italiano di Tecnologia, Italy · University of Genoa, Genoa, Italy

Object Pose Estimation Event-Based Vision

Open paper AskPDF arXiv

Object pose tracking is a fundamental and essential task for robotics to perform tasks in the home and industrial settings. The most commonly used sensors to do so are RGB-D cameras, which can hit limitations in highly dynamic environments due to motion blur and frame-rate constraints. Event cameras have remarkable features such as high temporal resolution and low latency, which make them a potentially ideal vision sensors for object pose tracking at high speed. Even so, there are still only few works on 6D pose tracking with event cameras. In this work, we take advantage of the high temporal resolution and propose a method that uses both a propagation step fused with a pose correction strategy. Specifically, we use 6D object velocity obtained from event-based optical flow for pose propagation, after which, a template-based local pose correction module is utilized for pose correction. Our learning-free method has comparable performance to the state-of-the-art algorithms, and in some cases out performs them for fast-moving objects. The results indicate the potential for using event cameras in highly-dynamic scenarios where the use of deep network approaches are limited by low update rates.

Abstract

Visual Object Tracking

Explore similar work

Event-based Motion & Appearance Fusion for 6D Object Pose Tracking

Event6D: Event-based Novel Object 6D Pose Tracking

RoPT: Robust 6DoF Object Pose Tracking in Video