cs.ROOct 7, 2026

Video-to-Model: Automatic Modeling of Deformable Linear Objects

Authors: Akshun Sharma, Kimia Forghani, Yancy Diaz-Mercado

Organizations: Department of Mechanical Engineering, University of Maryland College Park, Maryland, USA

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

This paper presents a video-to-model framework for automatically modeling the motion of a deformable suture thread from an input video. We utilize a recently developed CBF--CLF--QP numerical model that simplifies the characterization of deformable string motion through the selection of a small number of parameters. A perception module first localizes and tracks the thread in video, producing an ordered sequence of thread nodes. The observed thread motion is then processed by a spatio-temporal CNN network that estimates the effective parameters of a structured CBF--CLF--QP model. These parameters are used to simulate the thread under a user-defined needle velocity input. Experiments using unseen thread configurations and motion demonstrate that the framework can reliably reconstruct the thread behavior from video, automatically configure the structured model, and reproduce the expected thread motion with low tracking error. The proposed approach reduces the need for manual parameter tuning and provides a step toward automatic video-based modeling of deformable linear objects.

Figures & tables

Figure 1: Overview of the proposed video-to-model framework for automatic thread-model configuration. An input video is fed into a perception module to detect the centerline of the DLO. This object is then fed into a parameter estimator module which enables a CBF–CLF–QP simulation of the object.
Figure 2: Thread-localization and tracking pipeline. An input RGB frame is processed by the U-Net to obtain a thread-probability map, which is thresholded and cleaned using morphological closing and small-component removal. The resulting mask is skeletonized, pruned, and traced to obtain an ordered thread centerline. The green and red markers indicate the two centerline endpoints.
Figure 3: Different needle trajectories (thread tip) used to generate the training dataset.
MaterialEstimateγ\boldsymbol{\gamma}α\boldsymbol{\alpha}κ\boldsymbol{\kappa}QP error (mm)
1Ground Truth11.17413.5610.642–
 No Segmentation10.46015.6330.5070.060
 U-Net2.4892.2670.5260.639
2Ground Truth9.8961.0580.623–
 No Segmentation15.7571.8150.4330.043
 U-Net1.1050.8320.6810.095
Table 1: Parameter identification and QP replay for the asymmetric flower-petal trajectory. Errors are mean node-trajectory errors.
Figure 4: Asymmetric-flower-petal needle trajectory used for evaluation, comprising a forward flower-petal sweep, a variable-speed reverse, and a return to the initial position.
Figure 5: Representative thread configurations during the asymmetric flower-petal motion experiment. Black denotes the ground-truth trajectory, cyan denotes the U-Net observations, and magenta denotes the trajectory predicted by our model.

Explore similar work

CardsList

  1. DeformMaster: An Interactive Physics-Neural World Model for Deformable Objects from Videos

    May 10, 2026Can Li, Zhoujian Li, Ren Li +4World Model LearningWorld Models

    Nankai University · Rightly Robotics, A4X · Zhejiang University +2

  2. BendTwin: Robust Dense-to-Sparse Physical Reconstruction with Bending-Aware Differentiable Spring-Mass Models

    Aug 6, 2026Yixiong Jing, Qi Wang, Lin Chen +6Structural MechanicsSparse-View 3D Reconstruction

    University of Cambridge · Institute of Automation, Chinese Academy of Sciences · Northwestern Polytechnical University +1

  3. BLARM: Animating 3D Objects from Video via Blending Latent Rigid Motion Primitives

    Aug 31, 2026Pradyumn Goyal, Yizhak Ben-Shabat, Hsueh-Ti Derek Liu +6Motion Representation Learning

    Roblox · UMass Amherst · TU Crete