Organizations: Department of Computer Science and Engineering, IIT Ropar, Rupnagar, 140001, Punjab, India · Department of Mechanical Engineering, IIT Ropar, Rupnagar, 140001, Punjab, India
The disintegration of liquid sheets into ligaments and droplets involves highly transient, multi-scale dynamics that are difficult to quantify from high-speed shadowgraphy images. Identifying droplets, ligaments, and blobs formed during breakup, along with tracking across frames, is essential for spray analysis. However, conventional multi-object tracking frameworks impose strict one-to-one temporal associations and cannot represent one-to-many fragmentation events. In this study, we present a two-stage deep learning framework for object detection and temporal relationship modeling across frames. The framework captures ligament deformation, fragmentation, and parent-child lineage during liquid sheet disintegration. In the first stage, a Faster R-CNN with a ResNet-50 backbone and Feature Pyramid Network detects and classifies ligaments and droplets in high-speed shadowgraphy recordings of an impinging Carbopol gel jet. A morphology-preserving synthetic data generation strategy augments the training set without introducing physically implausible configurations, achieving a held-out F1 score of up to 0.872 across fourteen original-to-synthetic configurations. In the second stage, a Transformer-augmented multilayer perceptron classifies inter-frame associations into continuation, fragmentation (one-to-many), and non-association using physics-informed geometric features. Despite severe class imbalance, the model achieves 86.1% accuracy, 93.2% precision, and perfect recall (1.00) for fragmentation events. Together, the framework enables automated reconstruction of fragmentation trees, preservation of parent-child lineage, and extraction of breakup statistics such as fragment multiplicity and droplet size distributions. By explicitly identifying children droplets formed from ligament fragmentation, the framework provides automated analysis of the primary atomization mode.
Figures & tables
Study
Key Method
Detect & Classify
Synthetic Data
Temporal Tracking
1 → N Breakup
Lineage Reconstruction
Key Limitation
Sibirtsev et al. (2023) [ 23 ]
Mask R-CNN
✓
✗
✗
✗
✗
Frame-by-frame; no temporal modeling
Ade et al. (2024) [ 1 ]
U-Net (holography)
✓
✓
✗
✗
✗
Detection only; no lineage modeling
Zhang et al. (2025) [ 27 ]
YOLOv8 (spray ligaments)
✓
✓
✓
✗
✗
Tracking limited to 1-to-1
Jose et al. (2024, 2025) [ 10 , 11 ]
Mask R-CNN / instance segmentation
✓
✓
✗
✗
✗
Frame-wise statistics; no temporal learning
Lim et al. (2024) [ 16 ]
YOLO-based segmentation
✓
✓
✗
✗
✗
Segmentation-focused; no tracking
Tretola et al. (2024) [ 24 ]
Hybrid DL + post-processing
✓
✗
✓
✗
✗
Breakup handled via heuristics; not learned
Table 1: Comparison of learning-based methods for various recently reported spray and breakup analysis. ✓ = supported; ✗ = not addressed.
Figure 1: Experimental setup for impinging jets atomization and high-speed shadowgraphy. A pressurized piston-cylinder system drives the liquid through two impinging nozzles, forming a liquid sheet that breaks into ligaments and droplets. The imaging pipeline extracts localized 256×256 patches with object-level bounding box annotations (ligament L, droplet D). The objects are shown with indices and alternate red and blue boxes for easy visualization (best viewed in color).
Figure 2: Representative examples of morphology-preserving data generation showing (A) original image, (B) clean image, and (C) generated synthetic image.
Figure 3: Proposed two-stage deep learning framework. Study I performs frame-wise detection using Faster R-CNN (ResNet-50+FPN) trained on original and morphology-preserving synthetic images, outputting bounding boxes and class labels. Study II encodes object pairs from consecutive frames using physics-informed geometric features and passes them to a Transformer-augmented MLP that predicts MOVE , BREAKUP , or NONE relationships, enabling fragmentation lineage reconstruction.
Experiment
Setting
Original
Synthetic
Total
Exp1
Nog_050_Nsyn_00200
50
200
250
Nog_100_Nsyn_00150
100
150
250
Nog_150_Nsyn_00100
150
100
250
Nog_200_Nsyn_00050
200
50
250
Nog_250_Nsyn_00000
250
0
250
Exp2
Nog_250_Nsyn_00000
250
0
250
Table 2: Training data configurations for evaluating the impact of original-to-synthetic data composition on object detection performance.
Table 3: Neural architectures evaluated for temporal relationship classification.
Figure 4: Droplet diameter histograms for Experiment 1 comparing CV-derived and model-predicted distributions across five training configurations. Mixed training settings better preserve the overall shape of the droplet-size distribution.
Figure 5: Scatter plots for Experiment 1 comparing predicted and ground-truth geometric features across five training configurations. Mixed original–synthetic settings exhibit closer agreement with the reference annotations than the all-original baseline. A and P denote area and perimeter, respectively, where ba and pd denote base and predicted , respectively.
Setting
Precision
Recall
F1 -score
Nog_050_Nsyn_00200
0.836
0.859
0.847
Nog_100_Nsyn_00150
0.832
0.856
0.844
Nog_150_Nsyn_00100
0.813
0.836
0.825
Nog_200_Nsyn_00050
0.827
0.879
0.852
Nog_250_Nsyn_00000
0.760
0.778
0.769
Table 4: Object detection performance for Experiment 1 (fixed total = 250 images, varying original/synthetic ratio). Evaluated on 37 held-out original images. Bold indicates the best value per column.
Setting
Precision
Recall
F1 -score
Nog_250_Nsyn_00000
0.764
0.776
0.770
Nog_250_Nsyn_00250
0.837
0.880
0.858
Nog_250_Nsyn_00500
0.847
0.864
0.856
Nog_250_Nsyn_01000
0.847
0.850
0.849
Table 5: Object detection performance for Experiment 2 (fixed 250 original images, increasing synthetic data). Evaluated on 37 held-out original images. Bold indicates the best value per column.
Figure 6: Droplet diameter histograms for Experiment 2. Increasing synthetic data improves agreement between CV and model-predicted droplet-size distributions up to a moderate augmentation level.
Figure 7: Scatter plots for Experiment 2 illustrating the relationship between predicted and ground-truth geometric features as the synthetic data volume increases. Agreement improves with moderate augmentation, while the gains become marginal at higher synthetic ratios. A and P denote area and perimeter, respectively, where ba and pd denote base and predicted , respectively.
Setting
Precision
Recall
F1 -score
Nog_050_Nsyn_00250
0.862
0.864
0.863
Nog_100_Nsyn_00250
0.848
0.861
0.855
Nog_150_Nsyn_00250
0.835
0.853
0.844
Nog_200_Nsyn_00250
0.849
0.877
0.863
Nog_250_Nsyn_00250
0.861
0.882
0.872
Table 6: Object detection performance for Experiment 3 (fixed 250 synthetic images, varying original data volume). Evaluated on 37 held-out original images. Bold indicates the best value per column.
Figure 8: Droplet diameter histograms for Experiment 3 showing strong agreement between CV-derived and model-predicted droplet-size distributions across all settings, with the best agreement observed at higher original data volumes.
Figure 9: Scatter plots for Experiment 3 illustrating the effect of increasing original data volume while keeping the synthetic set fixed. Agreement with ground-truth geometry improves as more original images are included. A and P denote area and perimeter, respectively, where ba and pd denote base and predicted , respectively.
Exp
Setting
mAP50
mAP50–95
Precision
Recall
F1 -score
AP50_D
AP50_L
AP50–95_D
AP50–95_L
Exp1
Nog_050_Nsyn_00200
0.763 ± 0.007
0.469 ± 0.006
0.670 ± 0.006
0.763 ± 0.010
0.713 ± 0.008
0.697 ± 0.012
0.830 ± 0.008
0.272 ± 0.010
0.666 ± 0.010
Nog_100_Nsyn_00150
0.728 ± 0.017
0.450 ± 0.014
0.749 ± 0.006
0.790 ± 0.008
0.769 ± 0.006
0.743 ± 0.011
0.713 ± 0.027
0.358 ± 0.008
0.542 ± 0.022
Nog_150_Nsyn_00100
0.763 ± 0.008
0.511 ± 0.003
0.801 ± 0.003
0.795 ± 0.008
0.798 ± 0.005
0.743 ± 0.010
0.783 ± 0.010
0.406 ± 0.004
0.616 ± 0.008
Nog_200_Nsyn_00050
0.787 ± 0.008
0.546 ± 0.006
0.845 ± 0.001
0.831 ± 0.014
0.838 ± 0.007
0.786 ± 0.018
0.789 ± 0.008
0.478 ± 0.009
0.614 ± 0.006
Nog_250_Nsyn_00000
0.776 ± 0.010
0.580 ± 0.007
0.936 ± 0.002
0.812 ± 0.015
0.869 ± 0.009
0.801 ± 0.016
0.752 ± 0.008
0.546 ± 0.011
0.614 ± 0.005
Exp2
Nog_250_Nsyn_00000
0.807 ± 0.007
0.619 ± 0.007
0.937 ± 0.003
0.811 ± 0.009
0.869 ± 0.006
0.798 ± 0.011
0.817 ± 0.010
0.543 ± 0.008
0.694 ± 0.011
Table 7: 5-fold cross-validation results for all experimental configurations in Study I. Values are reported as mean ± standard deviation. D: droplet, L: ligament. Bold indicates the best mean value within each experimental block for the primary metrics mAP50, mAP50–95, precision, recall, and F1 -score.
Figure 10: Comparison of object detection and classification results for models trained with different data compositions. Ligaments and droplets are denoted by red and green bounding boxes, respectively. (A) Detections produced by a model trained only on original images (Nog_250_Syn_000). (B) Detections produced by a model trained on a mixed original–synthetic dataset (Nog_100_Syn_250). Purple-highlighted regions indicate representative failure and recovery cases discussed in the text.
Model
Avg Acc.
Avg Prec.
Avg Rec.
Avg F1± Std
Overall F1
TransformerMLP
0.876
0.936
0.876
0.898 ± 0.006
0.898
FeatureInteractionMLP
0.841
0.921
0.841
0.867 ± 0.009
0.867
AttentionMLP
0.838
0.921
0.838
0.864 ± 0.011
0.864
ResidualMLP
0.834
0.920
0.834
0.861 ± 0.012
0.861
BasicMLP
0.806
0.909
0.806
0.836 ± 0.012
0.836
Table 8: Model comparison for temporal relationship classification (Study II). Bold indicates the best value.
Figure 11: Visualization of ligament and droplet tracking at a downstream location following initial sheet breakup. (A) Ground-truth correlations between two time instances separated by Δt=0.20 ms, obtained via manual labelling. Lighter-shaded objects represent structures at t=0 , while darker-shaded objects represent the corresponding structures at t=0.20 ms. (B) Model-predicted correlations for the same two time instances. Lighter-shaded objects correspond to t=0 , while darker-shaded objects with prominent borders denote structures identified and correlated by the trained model at t=0.20 ms.
Metric / Configuration
Value
Overall Performance
Accuracy
0.861
Precision
0.932
Recall
0.861
F1 -score
0.887
Class-wise Recall
Table 9: Final test results for the TransformerMLP model on the held-out test split in Study II.
Figure 12: Correlation matrix for the five geometric input features together with the class label. Off-diagonal feature correlations are moderate, with the largest absolute correlation observed between centroid distance and normalized centroid distance, confirming that the selected features are complementary rather than redundant. Near-zero correlations involving area_ratio indicate that size evolution provides information largely independent of spatial displacement and overlap cues.
A splash lives for a fraction of a second: sheets tear into ligaments and droplets, appearance is view-dependent and nearly textureless, and little persists long enough to track. Reconstruction research has consequently focused on smoke, synthetic liquids, or gently deforming surfaces. To our knowledge, no synchronized multi-view dataset of splashing liquids exists. We therefore introduce a benchmark of 20 real scenes, from coherent streams to violent splashes, captured by seven synchronized, calibrated 4K cameras at 60 fps, with manually refined per-view liquid and container masks and fixed evaluation splits. We further present SplashSplat, built on a single principle: impose physical structure only where the observations can constrain it. Per-frame liquid SDFs fused from the masks provide the geometry, level-set transport between consecutive SDFs yields a coarse velocity field, and Lagrangian carriers advected along this flow, corrected against each new observation and reseeded where coverage is lost, decode local Gaussians for differentiable rendering. SplashSplat outperforms state-of-the-art dynamic Gaussian splatting methods on our real captures and on a synthetic benchmark, with physically more plausible motion and a lower training cost. The same representation supports temporal interpolation and style transfer without re-optimization.
3D Gaussian Splatting (3DGS) enables real-time, photorealistic novel view synthesis, making it a highly attractive representation for model-based video tracking. However, leveraging the differentiability of the 3DGS renderer "in the wild" remains notoriously fragile. A fundamental bottleneck lies in the compact, local support of the Gaussian primitives. Standard photometric objectives implicitly rely on spatial overlap; if severe camera misalignment places the rendered object outside the target's local footprint, gradients strictly vanish, leaving the optimizer stranded. We introduce SpectralSplats, a robust tracking framework that resolves this "vanishing gradient" problem by shifting the optimization objective from the spatial to the frequency domain. By supervising the rendered image via a set of global complex sinusoidal features (Spectral Moments), we construct a global basin of attraction, ensuring that a valid, directional gradient toward the target exists across the entire image domain, even when pixel overlap is completely nonexistent. To harness this global basin without introducing periodic local minima associated with high frequencies, we derive a principled Frequency Annealing schedule from first principles, gracefully transitioning the optimizer from global convexity to precise spatial alignment. We demonstrate that SpectralSplats acts as a seamless, drop-in replacement for spatial losses across diverse deformation parameterizations (from MLPs to sparse control points), successfully recovering complex deformations even from severely misaligned initializations where standard appearance-based tracking catastrophically fails.
Avigail Cohen Rimon, Amir Mann, Mirela Ben Chen +1
Technion - Israel Institute of Technology · NVIDIA
Predicting tandem mass spectra (MS/MS) from molecular structures represents a central task in analytical chemistry with direct relevance to clinical metabolomics, systems biology, and adjacent disciplines. In this work, we revisit the problem through the lens of object detection on molecular graphs. Molecular fragmentation, a central step in MS/MS prediction, can be approximated as detecting a set of subgraphs (i.e., fragments) and their associated spectral contributions. Existing fragment-based models follow a two-stage paradigm -- first generating candidate fragments and then scoring them -- analogous to two-stage R-CNNs in computer vision. Towards higher accuracy and faster inference, we introduce GLACIER, a single-stage transformer-based fragment detection neural network for molecular graphs. This unified formulation eliminates the need for candidate enumeration, enabling scalable and globally consistent modeling of molecular fragmentation. GLACIER is faster and more accurate than existing state-of-the-art by a significant margin, achieving 70.0% and 69.7% Top-1 retrieval accuracy with and without contrastive finetuning on the MassSpecGym dataset (from the previous SOTA of 64.0%) and 52.5% and 38.5% respectively on the NIST'20 dataset (from 33.2%). Furthermore, GLACIER provides nearly 8-fold inference speedup over our prior two-stage model. Code is available at https://github.com/coleygroup/ms-pred
Rui-Xi Wang, Runzhong Wang, Connor W. Coley
Massachusetts Institute of Technology Cambridge, MA 02139