Deep Learning-Based Tracking and Lineage Reconstruction of Ligament Breakup

Vrushank Ahire; Vivek Kurumanghat; Mudasir Ganaie; Lipika Kabiraj

cs.CVApr 9, 2026

Deep Learning-Based Tracking and Lineage Reconstruction of Ligament Breakup

Authors: Vrushank Ahire, Vivek Kurumanghat, Mudasir Ganaie, Lipika Kabiraj

Organizations: Department of Computer Science and Engineering, IIT Ropar, Rupnagar, 140001, Punjab, India · Department of Mechanical Engineering, IIT Ropar, Rupnagar, 140001, Punjab, India

Abstract

The disintegration of liquid sheets into ligaments and droplets involves highly transient, multi-scale dynamics that are difficult to quantify from high-speed shadowgraphy images. Identifying droplets, ligaments, and blobs formed during breakup, along with tracking across frames, is essential for spray analysis. However, conventional multi-object tracking frameworks impose strict one-to-one temporal associations and cannot represent one-to-many fragmentation events. In this study, we present a two-stage deep learning framework for object detection and temporal relationship modeling across frames. The framework captures ligament deformation, fragmentation, and parent-child lineage during liquid sheet disintegration. In the first stage, a Faster R-CNN with a ResNet-50 backbone and Feature Pyramid Network detects and classifies ligaments and droplets in high-speed shadowgraphy recordings of an impinging Carbopol gel jet. A morphology-preserving synthetic data generation strategy augments the training set without introducing physically implausible configurations, achieving a held-out F1 score of up to 0.872 across fourteen original-to-synthetic configurations. In the second stage, a Transformer-augmented multilayer perceptron classifies inter-frame associations into continuation, fragmentation (one-to-many), and non-association using physics-informed geometric features. Despite severe class imbalance, the model achieves 86.1% accuracy, 93.2% precision, and perfect recall (1.00) for fragmentation events. Together, the framework enables automated reconstruction of fragmentation trees, preservation of parent-child lineage, and extraction of breakup statistics such as fragment multiplicity and droplet size distributions. By explicitly identifying children droplets formed from ligament fragmentation, the framework provides automated analysis of the primary atomization mode.

Figures & tables

Study	Key Method	Detect & Classify	Synthetic Data	Temporal Tracking	1 $\to$ N Breakup	Lineage Reconstruction	Key Limitation
Sibirtsev et al. (2023) [ 23 ]	Mask R-CNN	✓	✗	✗	✗	✗	Frame-by-frame; no temporal modeling
Ade et al. (2024) [ 1 ]	U-Net (holography)	✓	✓	✗	✗	✗	Detection only; no lineage modeling
Zhang et al. (2025) [ 27 ]	YOLOv8 (spray ligaments)	✓	✓	✓	✗	✗	Tracking limited to 1-to-1
Jose et al. (2024, 2025) [ 10 , 11 ]	Mask R-CNN / instance segmentation	✓	✓	✗	✗	✗	Frame-wise statistics; no temporal learning
Lim et al. (2024) [ 16 ]	YOLO-based segmentation	✓	✓	✗	✗	✗	Segmentation-focused; no tracking
Tretola et al. (2024) [ 24 ]	Hybrid DL + post-processing	✓	✗	✓	✗	✗	Breakup handled via heuristics; not learned

Table 1: Comparison of learning-based methods for various recently reported spray and breakup analysis. ✓ = supported; ✗ = not addressed.

Figure 1: Experimental setup for impinging jets atomization and high-speed shadowgraphy. A pressurized piston-cylinder system drives the liquid through two impinging nozzles, forming a liquid sheet that breaks into ligaments and droplets. The imaging pipeline extracts localized $256\times 256$ patches with object-level bounding box annotations (ligament L, droplet D). The objects are shown with indices and alternate red and blue boxes for easy visualization (best viewed in color).

Figure 2: Representative examples of morphology-preserving data generation showing (A) original image, (B) clean image, and (C) generated synthetic image.

Figure 3: Proposed two-stage deep learning framework. Study I performs frame-wise detection using Faster R-CNN (ResNet-50+FPN) trained on original and morphology-preserving synthetic images, outputting bounding boxes and class labels. Study II encodes object pairs from consecutive frames using physics-informed geometric features and passes them to a Transformer-augmented MLP that predicts MOVE , BREAKUP , or NONE relationships, enabling fragmentation lineage reconstruction.

Experiment	Setting	Original	Synthetic	Total
Exp1	Nog_050_Nsyn_00200	50	200	250
	Nog_100_Nsyn_00150	100	150	250
	Nog_150_Nsyn_00100	150	100	250
	Nog_200_Nsyn_00050	200	50	250
	Nog_250_Nsyn_00000	250	0	250
Exp2	Nog_250_Nsyn_00000	250	0	250

Table 2: Training data configurations for evaluating the impact of original-to-synthetic data composition on object detection performance.

Model	Architecture Details
BasicMLP	$5\rightarrow 64\rightarrow 32\rightarrow 3$
	Linear + BN + ReLU + Dropout (0.3)
ResidualMLP	$5\rightarrow 64\rightarrow$ ResBlock $\times 2\rightarrow 32\rightarrow 3$
	Linear(64) + BN + ReLU + Dropout (0.2) + Linear(64) + BN
AttentionMLP	$5\rightarrow 64\rightarrow$ Attention $\rightarrow 32\rightarrow 3$
	$\mathbf{a}=\sigma(W_{2}\tanh(W_{1}\mathbf{x}))$

Table 3: Neural architectures evaluated for temporal relationship classification.

Figure 4: Droplet diameter histograms for Experiment 1 comparing CV-derived and model-predicted distributions across five training configurations. Mixed training settings better preserve the overall shape of the droplet-size distribution.

Figure 5: Scatter plots for Experiment 1 comparing predicted and ground-truth geometric features across five training configurations. Mixed original–synthetic settings exhibit closer agreement with the reference annotations than the all-original baseline. A and P denote area and perimeter, respectively, where ba and pd denote base and predicted , respectively.

Setting	Precision	Recall	$F_{1}$ -score
Nog_050_Nsyn_00200	0.836	0.859	0.847
Nog_100_Nsyn_00150	0.832	0.856	0.844
Nog_150_Nsyn_00100	0.813	0.836	0.825
Nog_200_Nsyn_00050	0.827	0.879	0.852
Nog_250_Nsyn_00000	0.760	0.778	0.769

Table 4: Object detection performance for Experiment 1 (fixed total = 250 images, varying original/synthetic ratio). Evaluated on 37 held-out original images. Bold indicates the best value per column.

Setting	Precision	Recall	$F_{1}$ -score
Nog_250_Nsyn_00000	0.764	0.776	0.770
Nog_250_Nsyn_00250	0.837	0.880	0.858
Nog_250_Nsyn_00500	0.847	0.864	0.856
Nog_250_Nsyn_01000	0.847	0.850	0.849

Table 5: Object detection performance for Experiment 2 (fixed 250 original images, increasing synthetic data). Evaluated on 37 held-out original images. Bold indicates the best value per column.

Figure 6: Droplet diameter histograms for Experiment 2. Increasing synthetic data improves agreement between CV and model-predicted droplet-size distributions up to a moderate augmentation level.

Figure 7: Scatter plots for Experiment 2 illustrating the relationship between predicted and ground-truth geometric features as the synthetic data volume increases. Agreement improves with moderate augmentation, while the gains become marginal at higher synthetic ratios. A and P denote area and perimeter, respectively, where ba and pd denote base and predicted , respectively.

Setting	Precision	Recall	$F_{1}$ -score
Nog_050_Nsyn_00250	0.862	0.864	0.863
Nog_100_Nsyn_00250	0.848	0.861	0.855
Nog_150_Nsyn_00250	0.835	0.853	0.844
Nog_200_Nsyn_00250	0.849	0.877	0.863
Nog_250_Nsyn_00250	0.861	0.882	0.872

Table 6: Object detection performance for Experiment 3 (fixed 250 synthetic images, varying original data volume). Evaluated on 37 held-out original images. Bold indicates the best value per column.

Figure 8: Droplet diameter histograms for Experiment 3 showing strong agreement between CV-derived and model-predicted droplet-size distributions across all settings, with the best agreement observed at higher original data volumes.

Figure 9: Scatter plots for Experiment 3 illustrating the effect of increasing original data volume while keeping the synthetic set fixed. Agreement with ground-truth geometry improves as more original images are included. A and P denote area and perimeter, respectively, where ba and pd denote base and predicted , respectively.

Exp	Setting	mAP50	mAP50–95	Precision	Recall	$F_{1}$ -score	AP50_D	AP50_L	AP50–95_D	AP50–95_L
Exp1	Nog_050_Nsyn_00200	0.763 $\pm$ 0.007	0.469 $\pm$ 0.006	0.670 $\pm$ 0.006	0.763 $\pm$ 0.010	0.713 $\pm$ 0.008	0.697 $\pm$ 0.012	0.830 $\pm$ 0.008	0.272 $\pm$ 0.010	0.666 $\pm$ 0.010
	Nog_100_Nsyn_00150	0.728 $\pm$ 0.017	0.450 $\pm$ 0.014	0.749 $\pm$ 0.006	0.790 $\pm$ 0.008	0.769 $\pm$ 0.006	0.743 $\pm$ 0.011	0.713 $\pm$ 0.027	0.358 $\pm$ 0.008	0.542 $\pm$ 0.022
	Nog_150_Nsyn_00100	0.763 $\pm$ 0.008	0.511 $\pm$ 0.003	0.801 $\pm$ 0.003	0.795 $\pm$ 0.008	0.798 $\pm$ 0.005	0.743 $\pm$ 0.010	0.783 $\pm$ 0.010	0.406 $\pm$ 0.004	0.616 $\pm$ 0.008
	Nog_200_Nsyn_00050	0.787 $\pm$ 0.008	0.546 $\pm$ 0.006	0.845 $\pm$ 0.001	0.831 $\pm$ 0.014	0.838 $\pm$ 0.007	0.786 $\pm$ 0.018	0.789 $\pm$ 0.008	0.478 $\pm$ 0.009	0.614 $\pm$ 0.006
	Nog_250_Nsyn_00000	0.776 $\pm$ 0.010	0.580 $\pm$ 0.007	0.936 $\pm$ 0.002	0.812 $\pm$ 0.015	0.869 $\pm$ 0.009	0.801 $\pm$ 0.016	0.752 $\pm$ 0.008	0.546 $\pm$ 0.011	0.614 $\pm$ 0.005
Exp2	Nog_250_Nsyn_00000	0.807 $\pm$ 0.007	0.619 $\pm$ 0.007	0.937 $\pm$ 0.003	0.811 $\pm$ 0.009	0.869 $\pm$ 0.006	0.798 $\pm$ 0.011	0.817 $\pm$ 0.010	0.543 $\pm$ 0.008	0.694 $\pm$ 0.011

Table 7: 5-fold cross-validation results for all experimental configurations in Study I. Values are reported as mean

\pm

standard deviation. D: droplet, L: ligament. Bold indicates the best mean value within each experimental block for the primary metrics mAP50, mAP50–95, precision, recall, and

F_{1}

-score.

Figure 10: Comparison of object detection and classification results for models trained with different data compositions. Ligaments and droplets are denoted by red and green bounding boxes, respectively. (A) Detections produced by a model trained only on original images (Nog_250_Syn_000). (B) Detections produced by a model trained on a mixed original–synthetic dataset (Nog_100_Syn_250). Purple-highlighted regions indicate representative failure and recovery cases discussed in the text.

Model	Avg Acc.	Avg Prec.	Avg Rec.	Avg $F_{1}$ $\pm$ Std	Overall $F_{1}$
TransformerMLP	0.876	0.936	0.876	0.898 $\pm$ 0.006	0.898
FeatureInteractionMLP	0.841	0.921	0.841	0.867 $\pm$ 0.009	0.867
AttentionMLP	0.838	0.921	0.838	0.864 $\pm$ 0.011	0.864
ResidualMLP	0.834	0.920	0.834	0.861 $\pm$ 0.012	0.861
BasicMLP	0.806	0.909	0.806	0.836 $\pm$ 0.012	0.836

Table 8: Model comparison for temporal relationship classification (Study II). Bold indicates the best value.

Figure 11: Visualization of ligament and droplet tracking at a downstream location following initial sheet breakup. (A) Ground-truth correlations between two time instances separated by $\Delta t=0.20$ ms, obtained via manual labelling. Lighter-shaded objects represent structures at $t=0$ , while darker-shaded objects represent the corresponding structures at $t=0.20$ ms. (B) Model-predicted correlations for the same two time instances. Lighter-shaded objects correspond to $t=0$ , while darker-shaded objects with prominent borders denote structures identified and correlated by the trained model at $t=0.20$ ms.

Overall Performance
Metric / Configuration	Value
Accuracy	0.861
Precision	0.932
Recall	0.861
$F_{1}$ -score	0.887
Class-wise Recall

Table 9: Final test results for the TransformerMLP model on the held-out test split in Study II.

Figure 12: Correlation matrix for the five geometric input features together with the class label. Off-diagonal feature correlations are moderate, with the largest absolute correlation observed between centroid distance and normalized centroid distance, confirming that the selected features are complementary rather than redundant. Near-zero correlations involving area_ratio indicate that size evolution provides information largely independent of spatial displacement and overlap cues.

Multi-Object Tracking

Multi-object tracking (MOT) links detections across video frames to maintain object identities through motion, occlusion, and changing viewpoints.

See all 193 papers

SplashSplat: Reconstructing Splashing Liquids from Real-World Multi-View Videos
Sep 17, 2026Peiyu Liu, Dingxi Zhang, Federico Tombari +33D Gaussian Splatting 3D Reconstruction
EPFL · ETH · Google +1
SpectralSplats: Robust Differentiable Tracking via Spectral Moment Supervision
Mar 25, 2026Avigail Cohen Rimon, Amir Mann, Mirela Ben Chen +13D Gaussian Splatting Differentiable Rendering
Technion - Israel Institute of Technology · NVIDIA
GLACIER: Rethinking Mass Spectrum Prediction as an Object Detection Problem
Jun 28, 2026Rui-Xi Wang, Runzhong Wang, Connor W. ColeyMass Spectrometry
Massachusetts Institute of Technology Cambridge, MA 02139

Sep 17, 2026J/K move · Enter open · S save

SplashSplat: Reconstructing Splashing Liquids from Real-World Multi-View Videos

Peiyu Liu, Dingxi Zhang, Federico Tombari, Marc Pollefeys+2

EPFL · ETH · Google+1

3D Gaussian Splatting 3D Reconstruction Multi-View 3D Reconstruction Dynamic Scene Reconstruction

Open paper AskPDF arXiv

A splash lives for a fraction of a second: sheets tear into ligaments and droplets, appearance is view-dependent and nearly textureless, and little persists long enough to track. Reconstruction research has consequently focused on smoke, synthetic liquids, or gently deforming surfaces. To our knowledge, no synchronized multi-view dataset of splashing liquids exists. We therefore introduce a benchmark of 20 real scenes, from coherent streams to violent splashes, captured by seven synchronized, calibrated 4K cameras at 60 fps, with manually refined per-view liquid and container masks and fixed evaluation splits. We further present SplashSplat, built on a single principle: impose physical structure only where the observations can constrain it. Per-frame liquid SDFs fused from the masks provide the geometry, level-set transport between consecutive SDFs yields a coarse velocity field, and Lagrangian carriers advected along this flow, corrected against each new observation and reseeded where coverage is lost, decode local Gaussians for differentiable rendering. SplashSplat outperforms state-of-the-art dynamic Gaussian splatting methods on our real captures and on a synthetic benchmark, with physically more plausible motion and a lower training cost. The same representation supports temporal interpolation and style transfer without re-optimization.

Abstract

Multi-Object Tracking

Explore similar work

SplashSplat: Reconstructing Splashing Liquids from Real-World Multi-View Videos

SpectralSplats: Robust Differentiable Tracking via Spectral Moment Supervision

GLACIER: Rethinking Mass Spectrum Prediction as an Object Detection Problem