Organizations: The National School of Artificial Intelligence (ENSIA), Algiers 16000, Algeria · Information Systems and Development Department, Algiers Airport, Algiers 16000, Algeria
Efficient luggage trolley management is critical for reducing congestion and ensuring asset availability in modern airports. Automated detection systems face two main challenges. First, strict security and privacy regulations limit large-scale data collection. Second, existing public datasets lack the diversity, scale, and annotation quality needed to handle dense, overlapping trolley arrangements typical of real-world operations. To address these limitations, we introduce a synthetic data generation pipeline based on a high-fidelity Digital Twin of Algiers International Airport using NVIDIA Omniverse. The pipeline produces richly annotated data with oriented bounding boxes, capturing complex trolley formations, including tightly nested chains. We evaluate YOLO-OBB using five training strategies: real-only, synthetic-only, linear probing, full fine-tuning, and mixed training. This allows us to assess how synthetic data can complement limited real-world annotations. Our results show that mixed training with synthetic data and only 40 percent of real annotations matches or exceeds the full real-data baseline, achieving 0.94 mAP@50 and 0.77 mAP@50-95, while reducing annotation effort by 25 to 35 percent. Multi-seed experiments confirm strong reproducibility with a standard deviation below 0.01 on mAP@50, demonstrating the practical effectiveness of synthetic data for automated trolley detection.
Figures & tables
Figure 1: Real-world dataset sample 1.
Figure 2: Real-world dataset sample 2.
Figure 3: Synthetic dataset sample from the Algiers Digital Twin.
Figure 4: Synthetic dataset sample 2 from the Algiers Digital Twin.
Figure 5: The semi-automated “Human-in-the-Loop” annotation workflow.
Subset
Frames
Boxes
Density
Viewpoints
Real World
1,504
14,080
10.17
Phone view
Synthetic
817
8616
10.55
Phone view
Total
2321
22696
10.36
-
Table I : Dataset Statistics
Real Data Subset (%)
Real Frames ( N )
Synthetic Frames
Combined Total (Strategy C)
Effective Real % in Mixed Batch
5%
65
817
882
∼ 7.4%
10%
131
817
948
∼ 13.8%
20%
261
817
1078
∼ 24.2%
30%
392
817
1209
∼ 32.4%
40%
522
817
1339
∼ 39.0%
50%
653
817
1470
∼ 44.4%
Table II : Dataset Composition Across Experimental Splits
Figure 6: Flowchart of the experimental methodology, illustrating the five distinct training protocols.
Split
Source
Frames
Notes
Training (80%)
Real-World
1,044
Operational footage
Synthetic
654
Phone View
Validation (20%)
Real-World
260
Sampling from it at matching % (e.g., 5-50%) for early stopping
Synthetic
163
100% utilized across all mixed experiments
Testing
Real-World
200
Strictly Held-Out (Manual annotation)
Table III : Dataset Partitioning and Source Breakdown
Parameter
Linear Probing (Strategy A)
Full Fine-Tuning (Strategy B)
Scratch / Mixed (Strategy C)
Pre-trained Weights
Synthetic
Synthetic
None (Random Init)
Backbone Freezing
First 10 Layers
0 (Unfrozen)
None
Initial LR ( lr0 )
0.0005
0.0005
0.001
Warmup Epochs
1.0
1.0
3.0
Epochs
100 (Patience: 25)
Image Size
640 × 640
Table IV : Training Hyperparameters: Comparison of Strategies
Model Variant
mAP50
mAP50-95
Precision
Recall
Real-Only (100% Data)
0.9424
0.8010
0.9375
0.9057
Synthetic-Only (Zero-Shot)
0.4162
0.2184
0.4501
0.4835
Table V : Baseline Performance Bounds (Test Set)
Real Data %
Strategy A: Linear Probing (Frozen)
Strategy B: Full Fine-Tuning (Unfrozen)
Strategy C: Mixed Training
mAP50
mAP50-95
Prec.
Recall
mAP50
mAP50-95
Prec.
Recall
mAP50
mAP50-95
Prec.
Recall
5%
0.8977
0.5455
0.8747
0.8048
0.9140
0.5953
0.8725
0.8291
0.9281
0.7158
0.9017
0.8817
10%
0.8515
0.4364
0.8586
0.7560
0.9113
0.6261
0.8725
0.8478
0.9326
0.6740
0.9088
0.8782
20%
0.8910
0.5895
0.9034
0.7874
0.9250
0.6027
0.8812
0.8507
0.9071
0.6593
0.8982
0.8280
30%
0.8691
0.5352
0.8376
0.8082
0.9210
0.6274
0.8712
0.8726
0.9320
0.7341
0.9133
0.8896
40%
0.9006
0.5934
0.8671
0.8229
0.9215
0.6784
0.8909
0.8712
0.9402
0.7301
0.9141
0.8916
Table VI : Comparative Results: Linear Probing vs. Full Fine-Tuning vs. Mixed Training
Figure 7: Data Efficiency Analysis. Mixed Training (Green) consistently outperforms both fine-tuning strategies. Notably, unfreezing the backbone (Strategy B, Orange) provides a substantial gain over linear probing (Strategy A, Blue), particularly as more real data becomes available.
Configuration
mAP50
mAP50-95
Precision
Recall
At 40% Real Data
Real-Only Baseline
0.9279
0.6902
0.8836
0.8457
Strategy C (Mixed)
0.9402
0.7301
0.9141
0.8916
At 50% Real Data
Real-Only Baseline
0.9360
0.6536
0.8817
0.8825
Strategy B (Unfrozen)
0.9430
0.7268
0.8890
0.9110
Table VII : Ablation Study: Impact of Synthetic Data at 40% and 50% Regimes
Figure 8: Operational Metrics Analysis. (a) Precision Comparison: All strategies maintain high precision, with Mixed Training (Green) consistently leading. (b) Recall Comparison: Strategy B (Orange) shows a strong recovery in Recall at 50% data, surpassing even the Mixed Strategy, suggesting that full fine-tuning becomes highly effective once sufficient real data is available to adapt the weights.
Strategy
Data %
mAP50
mAP50-95
Precision
Recall
Real-Only Baseline
Real-Only
10%
0.7663 ± 0.0000
0.4681 ± 0.0000
0.7797 ± 0.0000
0.6489 ± 0.0000
Real-Only
20%
0.8942 ± 0.0156
0.6287 ± 0.0578
0.8649 ± 0.0003
0.8164 ± 0.0302
Strategy A: Linear Probing (Frozen)
Fine-Tuning Frozen Backbone
5%
0.8653 ± 0.0413
0.5808 ± 0.0490
0.8147 ± 0.0562
0.7695 ± 0.0476
Fine-Tuning Frozen Backbone
10%
0.8811 ± 0.0248
0.5386 ± 0.0724
0.8395 ± 0.0295
0.8033 ± 0.0354
Table VIII : Multi-Seed Validation Results (Mean ± Standard Deviation across 3 seeds)
Figure 9: Qualitative comparison on a challenging test frame.
Synthetic data evaluation measures whether generated examples preserve the properties needed for a stated use, including fidelity, privacy, and utility.