MotionDLO: Hybrid Event- and Frame-Based Tracking of Deformable Linear Objects
Authors: Annalena Hartmann, Priyamvada Ajithkumar, Patrick Bründl, Jörg Franke
Organizations: Institute for Factory Automation and Production Systems, Friedrich-Alexander-Universität Erlangen-Nürnberg, Fürther Straße 246b, 90429 Nürnberg, Germany
Reliably tracking moving deformable linear objects (DLOs) while simultaneously ensuring robustness, accuracy, and temporally consistent state estimation remains a fundamental challenge in robot perception. We introduce MotionDLO, a real-time tracking framework specifically designed to overcome these limitations in temporal continuity and latency. The method exploits the high temporal resolution and sparsity of event-based cameras and combines segmentation with the Coherent Point Drift (CPD) algorithm under the principles of Motion Coherence Theory. This integration enables temporally consistent shape estimation while maintaining a low computational overhead. Existing event-based tracking methods are typically computationally efficient but exhibit reduced accuracy compared to frame-based approaches, or alternatively compromise event sparsity to achieve competitive performance. To resolve this trade-off, we propose a hybrid event- and frame-based tracking architecture that preserves the complementary strengths of both sensing modalities. The event stream ensures high-frequency motion updates, while frame-based information stabilizes spatial accuracy and object identity. We demonstrate that the proposed framework reliably associates DLO instances across video sequences, enabling robust perception for robotic manipulation tasks. Experimental results validate real-time performance at 12 ms update rates and accurate shape tracking with an point-to-curve error as measurement of accuracy of up to 0.43 mm, supporting dynamic path adaptation during manipulation. The source code and demonstration datasets are publicly available.
General-purpose masks via decoupled heads, boundary refinement, or attention
Poor fidelity on pixel-wide DLOs Transformer variants are often too computationally demanding for closed-loop robotic systems with strict latency requirements.
High-rate motion fields for warping under fast motion
No DLO instance identity or ordered centerline
Object / structure segmentation
Event-based thin-structure tracking
Powerlines: Dietsche et al. [ 18 ] ; Cable-suspended loads: Panetsos et al. [ 44 ]
Event-based tracking of thin, cable-like structures
Constrained setups, not cluttered multi-DLO manipulation
Open-vocabulary event perception
SEAL [ 34 ]
Multimodal open-vocabulary event segmentation
Outdoor driving benchmarks, not DLO manipulation
TABLE II: Overview of event-based segmentation and tracking methods relevant to MotionDLO.
Fig. 1: Bandwidth–latency trade-off for DLO observation. (Left) Along the Pareto front (green), a 250 fps high-speed camera attains low latency at the cost of Mb/s, while a 20 fps frame camera remains bandwidth-efficient but incurs 50 ms latency and pronounced motion blur. The event-based camera combines sub-millisecond latency with an order-of-magnitude lower bandwidth than the high-speed camera while yielding a sparse, blur-free signal. (Right) Within a single 50 ms inter-frame interval of the low-speed camera, the DLO traverses 30 mm across the field of view, a displacement that defeats CPD-based frame trackers. The event stream resolves this motion as a continuous sequence of intermediate shapes.
Fig. 2: The tracking result Yt is computed at 12 ms intervals by the CPD tracker operating on a geodesic MCT representation. At each update, the tracker receives polyline input from one of two branches: the frame-based branch (A) or the event-based branch (B). Branch selection depends on the DLO motion state, which is determined by the threshold parameter ρthr defined as the count of DLO-attributed events within a fixed temporal window. A motion state of vDLO=0 triggers branch (A), while vDLO=0 triggers branch (B). Temporal alignment between the two branches is ensured via hardware trigger timestamps; spatial alignment is established through the homography H .
Fig. 3: The CPD tracker producing Yt builds upon established components from prior work, (a) arc-length parametrization and (b) MCT geodesic kernel. To account for DLO velocity vDLO and improve robustness at higher speeds, (c) chain Laplacian regularization and (d) velocity warm-start are introduced. The assumed constant DLO diameter d is taken into account via component (e) to condition the tracker on event-based input and reduce computational cost. The assumed DLO diameter d is taken into account via component (f) to condition the tracker on event-based input and reduce computational cost. Robustness against measurement errors is addressed through the component (g).Robustness against measurement errors is addressed through components (h) and (i).
Fig. 4: Sequence diagram of the respective timing between the different instances
Fig. 5: A single DLO is grasped in an entangled scenario and manipulated by the robot gripper (left). An event-based camera, a frame-based camera, and a depth sensor are mounted in a fixed configuration to jointly observe the DLO motion trajectory (right).
Fig. 6: DLO 1 is a standard pneumatic plastic tube with a reflective surface and a hollow core. DLO 2 is a PROFINET cable, while DLO 3 is a switchgear cabinet wire with a copper core of 1.5 mm diameter.
Robot Speed (%)
Measure
px/s
mm/s
50
average
1398
893.7
max
2339
1495.0
75
average
1923
1229.0
max
2363
1510.3
100
average
2312
1477.7
max
3529
2255.1
TABLE III: TCP speed measurements at different robot speed settings.
Fig. 7: Qualitative evaluation of the observed ID switch. The input image shows the RGB image at three consecutive timestamps, while RT-DLO, SAM 3 and MotionDLO show the respective results.
RT-DLO
SAM 3
MotionDLO
Time
ID switch
Instances
ID switch
Instances
Switch
Instances
1
0
2
0
1
0
1
2
1
2
0
1
0
1
3
2
2
0
1
0
1
4
2
2
0
1
0
1
5
2
2
0
1
0
1
TABLE IV: Count of ID switches between consecutive timestamps for the DLO 3 at robot speed 75 % sequence. Ideal result for the single DLO scenario is no ID switch and one detected DLO.
Fig. 8: Segmentation results for the multiple wire scenario, where the rightmost DLO is moved and needs to be tracked.
RT-DLO
SAM 3
MotionDLO
Time
ID switch
Instances
ID switch
Instances
Switch
Instances
1
0
2
0
3
0
1
2
2
4
0
3
0
1
3
2
2
0
3
0
1
4
3
3
0
3
0
1
5
2
3
0
3
0
1
TABLE V: Count of ID switches between consecutive timestamps. Ideal result for the multiple DLO scenario is no ID switch and detection of the moving DLO, depending on the algorithm.
Inference time in ms
Recentness map
ROI masking
Mask extraction
Skeletonization
CPD
Complete
MotionDLO
0.32
2.07
2.01
3.68
3.16
12.34
RT-DLO
-
3.20
14.32
-
-
-
TrackDLO
-
-
-
(669)
15.22
-
TABLE VI: Inference time in ms of the MotionDLO pipeline decomposed by processing stage.
Fig. 9: Ground truth generated for DLO 2 and its tracking result. Left: Frame based ground truth generation based on images at the beginning of the movement without motion blur and its respective tracking result. Right: Event-based ground truth generation based on the spare representation and its tracking result. For visualization purposes only 10 of the used 20 nodes are displayed.
Fig. 10: TrackDLO and MotionDLO compared at robot speeds 5 % and 50 % . At robot speed 50 % TrackDLO starts to fail.
Fig. 11: Tracking error at defined timesteps of MotionDLO for all three DLOs under the moving condition at robot speed 100 % .
Fig. 12: The mean arc-length paired and point-to-curve error of MotionDLO under a static condition for all three DLOs.
Visual object tracking follows objects across video frames, maintaining their locations and identities despite motion, occlusion, and changing appearance.
Object pose tracking is a fundamental and essential task for robotics to perform tasks in the home and industrial settings. The most commonly used sensors to do so are RGB-D cameras, which can hit limitations in highly dynamic environments due to motion blur and frame-rate constraints. Event cameras have remarkable features such as high temporal resolution and low latency, which make them a potentially ideal vision sensors for object pose tracking at high speed. Even so, there are still only few works on 6D pose tracking with event cameras. In this work, we take advantage of the high temporal resolution and propose a method that uses both a propagation step fused with a pose correction strategy. Specifically, we use 6D object velocity obtained from event-based optical flow for pose propagation, after which, a template-based local pose correction module is utilized for pose correction. Our learning-free method has comparable performance to the state-of-the-art algorithms, and in some cases out performs them for fast-moving objects. The results indicate the potential for using event cameras in highly-dynamic scenarios where the use of deep network approaches are limited by low update rates.
Zhichao Li, Chiara Bartolozzi, Lorenzo Natale +1
Event-driven Perception for Robotics, Istituto Italiano di Tecnologia, Italy · Humanoid Sensing and Perception, Istituto Italiano di Tecnologia, Italy · University of Genoa, Genoa, Italy
Event cameras provide microsecond latency, making them suitable for 6D object pose tracking in fast, dynamic scenes where conventional RGB and depth pipelines suffer from motion blur and large pixel displacements. We introduce EventTrack6D, an event-depth tracking framework that generalizes to novel objects without object-specific training by reconstructing both intensity and depth at arbitrary timestamps between depth frames. Conditioned on the most recent depth measurement, our dual reconstruction recovers dense photometric and geometric cues from sparse event streams. Our EventTrack6D operates at over 120 FPS and maintains temporal consistency under rapid motion. To support training and evaluation, we introduce a comprehensive benchmark suite: a large-scale synthetic dataset for training and two complementary evaluation sets, including real and simulated event datasets. Trained exclusively on synthetic data, EventTrack6D generalizes effectively to real-world scenarios without fine-tuning, maintaining accurate tracking across diverse objects and motion patterns. Our method and datasets validate the effectiveness of event cameras for event-based 6D pose tracking of novel objects. Code and datasets are publicly available at https://chohoonhee.github.io/Event6D.
Accurate 6-DoF object pose tracking is essential for robotic interaction, yet conventional monocular pipelines frequently suffer from catastrophic divergence under rapid object motion. We present RoPT, a robust tracking framework engineered to overcome this limitation by unifying visual perception, temporal kinematics, and global re-localization into a tightly coupled closed loop. Unlike existing open-loop trackers, RoPT delivers three critical capabilities that fundamentally alter system resilience. First, the visual perception component tracks object masks to robustly handle partial and severe occlusions. Second, a kinematic factor graph regularizes frame-to-frame updates, forecasts forward pose predictions, and drives a parallel multi-hypothesis verification scheme to bypass local minima. Finally, an active re-localization module transforms the pipeline into a closed-loop system, making it failure-aware and capable of deterministic self-recovery. We demonstrate the effectiveness of our framework across challenging benchmarks alongside real-world robotic demonstrations, achieving significantly higher tracking survivability under rapid motion.