TOGBench: Benchmarking and Improving Tiny-Object Generation Fidelity in Aerial Imagery
Organizations: GenGenAI, South Korea
Abstract
Advances in deep generative models now enable the synthesis of realistic imagery. However, aerial imagery is dominated by tiny objects, such as vehicles, ships, and aircraft, which occupy only a few pixels owing to the overhead, wide-area nature of aerial imaging. Existing generative models struggle substantially more with tiny objects than with larger ones. This limitation has been overlooked, as prior work on generative evaluation emphasizes prompt following or human preference rather than whether small objects are rendered faithfully. To address this gap, we introduce TOGBench, a quantitative, object-level benchmark for evaluating tiny-object generation fidelity in aerial imagery. We further propose two simple train-/inference-time techniques that improve the ability of layout-to-image generators to render small objects, raising their performance on TOGBench.
Figures & tables
| Method | Detectability | Appearance | Scene-level text–image metrics | Efficiency | |||||
|---|---|---|---|---|---|---|---|---|---|
| Pres. | Loc. | Class | Size | Dist. | CLIPScore (ViT-L/14) | PickScore | Rel. time | ||
| FLUX.1-dev (text-only) | 0.969 0.001 | 0.775 0.033 | 0.672 0.026 | 0.730 0.007 | 7.620 0.420 | 1.005 0.003 | 0.2604 0.0008 | 23.151 0.039 | |
| FLUX.1-dev layout | |||||||||
| guidance scale | 0.723 0.017 | 0.198 0.015 | 0.225 0.020 | 0.186 0.022 | 59.75 0.647 | 0.931 0.002 | 0.2437 0.0008 | 21.263 0.026 | |
| guidance scale (default) | 0.356 0.008 | 0.172 0.008 | 0.334 0.018 | 0.203 0.006 | 72.19 0.269 | 0.886 0.001 | 0.2592 0.0004 | 22.174 0.002 | |
| guidance scale | 0.372 0.013 | 0.180 0.007 | 0.354 0.011 | 0.218 0.009 | 70.66 0.871 | 0.894 0.001 | 0.2577 0.0004 | 22.074 0.007 | |
| Baseline checkpoint | Reweighted checkpoint ( ) | |||||||||
| 1.0 | 1.2 | 1.5 | 1.8 | 2.0 | 1.0 | 1.2 | 1.5 | 1.8 | 2.0 | |
| 1 | 72.52 † | 73.34 | 73.69 | 74.12 | 74.68 | 81.88 † | 82.47 | 82.77 | 83.30 | 83.55 |
| 1.2 | 76.43 | 76.74 | 76.35 | 77.77 | 77.47 | 84.62 | 84.98 | 85.02 | 85.48 | 85.43 |
| 1.5 | 80.89 | 80.96 | 80.47 | 81.08 | 80.81 | 86.04 | 86.11 | 86.06 | 86.04 | 85.80 |
| 1.8 | 81.73 | 82.03 | 81.71 | 81.53 | 80.98 | 85.47 | 84.89 | 84.62 | 83.94 | 83.28 |
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
| Class | Objects |
|---|---|
| airplane | 525 |
| bridge | 557 |
| storage-tank | 581 |
| ship | 635 |
| swimming-pool | 249 |
| vehicle | 1,266 |
| Family | Severity | Appearance | |
|---|---|---|---|
| Clean reference | n/a | 100.00 | 0.8175 |
| JPEG compression | Mild | 95.51 | 0.8513 |
| Moderate | 91.60 | 0.8646 | |
| Severe | 83.40 | 0.8872 | |
| Gaussian blur | Mild | 93.79 | 0.8892 |
| Moderate | 72.82 | 0.9501 |
| Perturbation | Presence | Localization | Class | Size |
|---|---|---|---|---|
| Baseline (generation) | 0.06 | 0.12 | 0.04 | 0.09 |
| Seed control | 0.07 | 0.13 | 0.03 | 0.08 |
| Drop 40% of boxes | 0.36 | 0.14 | 0.04 | 0.10 |
| Relabel all classes | 0.11 | 0.12 | 0.06 | 0.08 |
| Rescale boxes ( ) | 0.21 | 0.45 | 0.03 | 0.62 |
| Pres. | Loc. | Class | Size | App. | ||
|---|---|---|---|---|---|---|
| 1 (baseline) | 0.302 0.087 | 0.146 0.017 | 0.288 0.134 | 0.187 0.025 | 70.63 2.87 | 0.8947 0.0057 |
| 1.5 | 0.243 0.020 | 0.142 0.014 | 0.164 0.026 | 0.170 0.019 | 74.85 1.82 | 0.8883 0.0036 |
| 3 | 0.187 0.005 | 0.123 0.013 | 0.090 0.023 | 0.137 0.009 | 80.93 0.93 | 0.8848 0.0035 |
| 10 | 0.195 0.013 | 0.123 0.015 | 0.115 0.011 | 0.148 0.017 | 79.51 1.18 | 0.8877 0.0006 |
| Text | Image | Cond. | Measured odds | Predicted | |
|---|---|---|---|---|---|
| 1 | 0.2122 | 0.7223 | 0.0655 | 1.000 | 1.0 |
| 1.2 | 0.2101 | 0.7132 | 0.0767 | 1.185 | 1.2 |
| 1.5 | 0.2078 | 0.6996 | 0.0926 | 1.455 | 1.5 |
| 2 | 0.2046 | 0.6782 | 0.1171 | 1.892 | 2.0 |
| Method | Ensemble | YOLO26x | Dome-DETR | DQ-DETR |
|---|---|---|---|---|
| Guidance scale ‡ | 60.03 | 31.81 | 66.51 | 42.24 |
| Guidance scale (default) | 72.19 0.27 | 62.02 0.91 | 72.85 0.53 | 68.91 4.37 |
| Guidance scale ‡ | 68.87 | 61.75 | 69.84 | 68.79 |
| inference resolution ‡ | 75.89 | 64.58 | 77.94 | 75.79 |
| Ours (CLAG) | 80.14 1.00 | 74.80 6.49 | 82.18 0.95 | 78.87 1.38 |
| Ours (Layout-Weighted Loss) | 81.14 0.63 | 78.07 1.05 | 82.99 0.38 | 80.90 0.83 |
| airplane | bridge | storage-tank | ship | swimming-pool | vehicle | person | |
| Detectability | |||||||
| Baseline (guidance 3.5) | 81.35 2.26 | 36.49 1.55 | 82.06 1.85 | 87.54 3.46 | 86.65 1.84 | 90.82 0.76 | 83.60 0.87 |
| CLAG | 85.57 2.52 | 55.98 1.51 | 87.75 1.02 | 87.89 1.80 | 89.29 2.52 | 94.96 0.55 | 87.00 0.90 |
| Layout-Weighted Loss | 86.14 2.19 | 60.89 1.42 | 92.08 1.51 | 87.63 3.53 | 85.22 0.84 | 92.59 0.50 | 83.78 0.90 |
| CLAG + Layout-Weighted Loss | 87.91 1.49 | 75.12 1.54 | 97.87 0.21 | 84.13 4.73 | 85.55 1.04 | 95.11 0.86 | 84.88 0.87 |
| Appearance | |||||||
| Method | Over-gen. |
|---|---|
| Real images (anchor) | 4.71 |
| FLUX.1-dev (text-only) ‡ | 25.04 |
| Guidance scale ‡ | 1.88 |
| Guidance scale (default) | 10.11 0.84 |
| Guidance scale ‡ | 10.96 |
| inference resolution ‡ | 15.71 |
Image Generation
Image generation studies models that create or edit images from text, examples, layouts, and other structured controls.
See all 507 papers