Architectural Insights for Post-Tornado Damage Recognition
Authors: Robinson Umeike, Thang Dao, Shane Crawford, John van de Lindt, Blythe Johnston, Wanting, Wang, Trung Do, +3 more
Organizations: Lisa · The University of Alabama · Colorado State University · Argonne National Laboratory · Old Dominion University · University of South Alabama
Rapid and accurate building damage assessment in the immediate aftermath of tornadoes is critical for coordinating life-saving search and rescue operations, optimizing emergency resource allocation, and accelerating community recovery. However, current automated methods struggle with the unique visual complexity of tornado-induced wreckage, primarily due to severe domain shift from standard pre-training datasets and extreme class imbalance in real-world disaster data. To address these challenges, we introduce a systematic experimental framework evaluating 79 open-source deep learning models, encompassing both Convolutional Neural Networks (CNNs) and Vision Transformers, across over 2,300 controlled experiments on our newly curated Quad-State Tornado Damage (QSTD) benchmark dataset. Our findings reveal that achieving operational-grade performance hinges on a complex interaction between architecture and optimization, rather than architectural selection alone. Most strikingly, we demonstrate that optimizer choice can be more consequential than architecture: switching from Adam to SGD provided dramatic F1 gains of +25 to +38 points for Vision Transformer and Swin Transformer families, fundamentally reversing their ranking from bottom-tier to competitive with top-performing CNNs. Furthermore, a low learning rate of 1x10^(-4) proved universally critical, boosting average F1 performance by +10.2 points across all architectures. Our champion model, ConvNeXt-Base trained with these optimized settings, demonstrated strong cross-event generalization on the held-out Tuscaloosa-Moore Tornado Damage (TMTD) dataset, achieving 46.4% Macro F1 (+34.6 points over baseline) and retaining 85.5% Ordinal Top-1 Accuracy despite temporal and sensor domain shifts.
Figures & tables
Element
DS1 - Slight
DS2 - Moderate
DS3 - Extensive
DS4 - Complete
Roof Covering
2–15% of Roof Covering Damaged
15–50% of Roof Covering Damage
More than 50% of Roof Covering Damaged
More than 50% of Roof Covering Damaged (typically)
AND/OR
Window/Door
1 window or door failure
2 or 3 windows/doors failed
More than 3 windows/doors failed
More than 3 windows/doors failed (typically)
AND/OR
Roof Sheathing
No Roof Sheathing Failure
1–3 sections of roof sheathing failed
More than 3 sections AND less than 35% of roof sheathing failed
More than 35% of roof sheathing failed
AND/OR
Table 1: IN-CORE Damage State Classification Criteria for Wood-Frame Residential Structures.
Figure 1: Example of QSTD images for the 6-class IN-CORE taxonomy
Figure 2: (a) Class distribution of the QSTD dataset across training, validation, and test splits, highlighting the pronounced long-tail imbalance. (b) Overall class distribution of the TMTD dataset used for cross-event evaluation
Figure 3: Example of TMTD images for the 6-class IN-CORE taxonomy
Figure 4: The four-stage experimental design, from zero-shot baseline experiment to cross-event generalisation evaluation.
Architectural
Pre-training
Representative Models
Stage 0
Stage 1
Stage 2
Stage 3
Family
Sources
(Stage 0)
(Zero-Shot)
(Base Case)
(Optimization) §
(TTD)
AlexNet
†, ★
AlexNet [ 32 ]
2
2
58
✗
ConvNeXt
†
ConvNext-Small [ 44 ]
1
4
116
✓
DenseNet
†, ★
DenseNet161 [ 37 ]
2
5
145
✗
EfficientNet
†
EfficientNet-B3 [ 38 ]
1
12
348
✗
GoogLeNet
†
GoogLeNet [ 42 ]
1
1
29
✗
Table 2: Evaluated Architectural Families and Model Variants Across Experimental Stages.
Hyperparameter
Base Case
Search Space
Variations ‡
Domain-Specific Goal
(ψbase)
(Ψ)
Activation function
Model Default*
{ReLU, LeakyReLU, GELU, SiLU, ELU}
5
Investigate impact of non-linearity on learning complex damage textures and preventing dying neurons
Batch Size
64
{64, 32, 16, 8, 4}
4
Balance gradient stability with noise-induced regularization; assess impact on rare class representation
Optimizer
Adam
{Adam, SGD (momentum=0.9)}
1
Compare adaptive vs. momentum-based optimization for disaster imagery
Regularization
None
{None, Dropout with p∈{0.2,0.3,0.5} }
3
Combat overfitting to dataset-specific debris patterns and lighting artifacts
Enable multi-scale feature recognition from fine-grained (windows) to gross (structural) damage
Table 3: Comprehensive Hyperparameter Search Space and Optimization Strategy.
Rank ‡
Model
Pre-training
Macro-F1
Ordinal Top-1
Standard ¶ Top-1
Param ‖
Inference
(%)
Acc. (%)
Acc. (%)
(M)
Time (ms)
1
EfficientNet-B3
†
14.3
50.5
26.1
10.7
13.3
2
DenseNet161
★
12.6
75.9
61.0
26.5
21.5
3
AlexNet
★
11.3
33.5
23.2
57.0
0.6
4
GoogLeNet
†
8.4
14.8
9.4
5.6
7.1
5
AlexNet
†
6.9
39.5
8.3
57.0
0.6
Table 4: Zero-Shot Transfer Performance of Representative Models on QSTD Test Set.
Figure 5: (a) Zero-Shot Transfer Performance by Pre-training Source. (b) Scatter Plot of Macro F1 vs. Parameters for all 21 zero-shot models showing no correlation between parameter count and F1.
Figure 6: Ordinal confusion matrices for the top three zero-shot models on the QSTD test set. Color indicates ordinal distance (Dark Green = correct, Green = ±1 , Light Green = ±2 , Yellow = ±3 , Orange = ±4 , Red = ±5 ). (a) EfficientNet-B3 (ImageNet) shows scattered predictions. (b) AlexNet (Places365) overpredicts DS0 and DS5. (c) DenseNet161 (Places365) collapses to DS0-Undamaged
Figure 7: Ordinal confusion matrices of top-performing Stage 1 models for top three architecture families on the QSTD test set
Figure 8: Scatter plot of Macro F1 vs. Inference Time for all 79 fine-tuned models showing the Pareto frontier
Figure 9: Performance and Efficiency of Top Variants per Family Across Key Experimental Stages. Each subplot shows the best-performing variant from each architectural family for a specific configuration: (a) Stage 1 Base Case ( ψbase ), (b) Stage 2 Optimal Loss Function, (c) Stage 2 Optimal Input Resolution, and (d) Stage 2 Optimal Data Augmentation. (Top Row) Performance comparison using Macro F1 (blue), Ordinal Top-1 Acc (green), and Standard Top-1 Acc (red). (Bottom Row) Corresponding efficiency metrics: Parameter count (purple) and Inference Time (orange)
Figure 10: Hyperparameter Sensitivity Analysis Across Architectures. Box plots show the distribution of Macro F1 scores for all 79 fine-tuned models under varying configurations for nine key hyperparameters (*Base Case configuration): (Top row, L-R) Loss Function, Input Resolution, Data Augmentation; (Middle row, L-R) Optimizer, Learning Rate, LR Scheduler; (Bottom row, L-R) Dropout Regularization, Batch Size, Activation Function.
Figure 11: Effect of Input Resolution on Model Performance and Cost. (a) Average Macro F1 score across 73 convolutional model families. (b) Average computational cost (GFLOPs) and inference time (ms), showing a sharp increase at higher resolutions. (c) Performance scaling for representative top-5 performing models, highlighting architecture-specific peaks
Figure 12: Performance and Efficiency of Top Variants per Family Across Optimizer and Learning Rate. Each subplot shows the best-performing variant from each architectural family for a specific configuration
Figure 13: Performance and Efficiency of Top Variants per Family Across Scheduler, Batch Size, Dropout, and Activation Function. Each subplot shows the best-performing variant from each architectural family for a specific configuration
Figure 14: Pre-training Source Comparison: Average Macro F1 Across Stage 2 Optimizations (Places365 vs. ImageNet-1K)
Model Configuration
Macro F1
Ordinal
Std.
DS0
DS1
DS2
DS3
DS4
DS5
(%)
Top-1 Acc (%)
Top-1 Acc (%)
(F1)
(F1)
(F1)
(F1)
(F1)
(F1)
Naive Baseline
11.8
43.9
14.0
9.3
0.7
19.4
17.8
7.9
15.9
(ImageNet → TMTD)
QSTD-Optimized
46.4
85.5
58.3
63.5
20.6
22.6
36.2
75.5
59.1
(ImageNet → QSTD → TMTD)
Table 5: Zero-Shot Generalization Performance on the TMTD Dataset (ConvNeXt-Base).
Figure 15: Ordinal Confusion Matrices for Zero-Shot TMTD Test. (a) Naïve baseline model showing a poor classification performance. (b) QSTD-optimized model showing generalizable ordinal understanding of damage severity on an unseen dataset
Component
Recommendation
Impact
Notes
Optimizer
SGD with momentum = 0.9 for Vision Transformers; Adam acceptable for CNNs
+25 to +38 F1 points for ViT/Swin families
Most impactful optimization; elevates Transformers from bottom-tier to competitive performance
Learning Rate
1×10−4 (universal across all families)
+10.2 F1 points on average
Rates ≥1×10−3 consistently resulted in poorer performance
Input Resolution
256×256 to 448×448
+14.2% F1 for DS1–DS3
Higher resolutions show diminishing returns with increased cost
Mixed effects (48% improved; 49% degraded with Focal Loss)
Cross-entropy remains the most robust baseline on the QSTD benchmark
Pre-training Source
ImageNet (default); Places365 for AlexNet, DenseNet161, ResNet18
Up to +142% F1 (zero-shot)
Scene-centric pre-training benefits certain architectures
Table 6: Practical Optimization Guidelines based on QSTD Benchmark (Applicable Across All 79 Architectures).
Deployment
Recommended
Optimizer
Learning
Input
Aug-
Performance
Key
Scenario
Model
Rate
Resolution
mentation
Metrics
Advantages
Resource-Constrained (Edge devices, mobile platforms)
MobileNetV3-Large or ShuffleNetV2-x1.5
SGD (momentum=0.9)
1×10−4
256×256
Standard
62.0% F1, 4.2M params, ∼ 5ms inference
Optimal accuracy-efficiency trade-off for limited compute
Maximum Accuracy (GPU-enabled backend)
ConvNeXt-Base
SGD (momentum=0.9)
1×10−4
384×384
Standard
68.0% F1 (QSTD), 46.4% F1 (TMTD), 85.5% Ordinal Acc (TMTD)
Highest accuracy with strong cross-event generalization
Table 7: QSTD-Based Deployment Configurations for Automated Tornado Damage Assessment.
Appendix figures & tables19 assets
Supplementary material from the paper’s appendix.
Appendix
Class
Initial
Final
Net Change
Primary Adjustment
Annotation
Dataset
DS0 - Undamaged
3,744
3,375
−369 ( −9.9% )
Reclassified to DS1 upon detection of subtle roof/window damage
DS1 - Slight
480
606
+126 ( +26.3% )
Received reclassifications from DS0 and DS2
DS2 - Moderate
512
432
−80 ( −15.6% )
Many reclassified to DS3 based on roof sheathing criteria
DS3 - Extensive
127
368
+241 ( +189.8% )
Significant increase due to consistent application of sheathing failure thresholds
DS4 - Complete
440
455
+15 ( +3.4% )
Minor adjustments; clearest damage state with least ambiguity
Appendix
Table A1: Class Distribution Changes During Annotation and Cross-Validation.
Figure A1: QSTD Dataset Evolution and Refinement through Inter-Annotator Cross-Validation
Damage Class
Average Width (px)
Average Height (px)
DS0 - Undamaged
347.02
225.70
DS1 - Slight
371.34
238.19
DS2 - Moderate
367.34
229.52
DS3 - Extensive
366.38
235.88
DS4 - Complete
373.15
248.80
DS5 - Debris
1048.47
632.23
Appendix
Table A2: QSTD Image Dimensions by Damage Class. The table lists the mean width and height in pixels for images within each of the six damage state categories after initial cropping.
Figure A2: Figure showing the distribution of QSTD image resolutions (Height x Width) for each of the 6 damage classes, highlighting the different characteristics of the DS5 class
Strategy
Transformations
Key Parameters
Context
None
–
–
Provides a non-augmented baseline; used for validation and test sets.
Basic
Horizontal flip
p=0.3
Basic invariance to viewpoint.
(Base Case)
Resize
Target input resolution
Standard
Horizontal flip
p=0.5
Simulates moderate changes in camera angle and framing
Random rotation
θ∼U(−15°,15°)
Random crop
From padded input
Appendix
Table B1: Comprehensive List of Data Augmentation Strategies Used in Stage 1 and 2.
Table B2: Comprehensive List of All 79 Model Variants.
Rank
Model
Pre-training
Macro F1
Ordinal Top-1
Standard Top-1
Params
Inference
(Val) (%)
Acc (Val) (%)
Acc (Val) (%)
(M)
Time (ms)
1
EfficientNet-B6
†
66.0
91.5
77.7
40.75
22.89
2
EfficientNet-B3
†
64.4
88.7
77.0
10.71
13.29
3
EfficientNet-B4
†
63.9
89.8
77.4
17.56
16.65
4
EfficientNet-B1v2
†
63.4
89.1
75.5
6.52
11.70
5
ConvNeXt-Large
†
62.8
90.4
75.2
196.21
9.34
Appendix
Table C1: Complete Performance Results for All 79 Model Variants Trained Using the ψbase Configuration.
Metrics (Average)
CCE*
Focal Loss
Δ
Macro F1
52.9
52.4
−0.5
Standard Top-1
71.7
71.8
0.0
Ordinal Top-1
86.2
86.1
−0.1
Params (M)
58.77
61.54
+2.77
GFLOPs
8.98
9.85
+0.86
Inference Time (ms)
11.16
11.42
+0.26
Appendix
Table C2: Average Performance for Loss Function: CCE vs. Focal Loss ( γ=5 , α=0.5 ).
Architectural Family
Variants
Variants
Avg. F1 Change
Top F1
Top F1
Tested
Improved (%)
(Points)
Gain
Loss
AlexNet
2
0 (0)
−6.70
—
−13.4 (AlexNet-Places365)
ConvNeXt
4
0 (0)
−10.20
—
−17.7 (ConvNeXt-Base)
DenseNet
5
2 (40)
−1.70
+1.6 (DenseNet161)
−6.0 (DenseNet121)
EfficientNet
12
6 (50)
+0.50
+5.6 (EfficientNet-B2)
−3.4 (EfficientNet-B6)
GoogLeNet
1
0 (0)
−0.30
—
−0.3 (GoogLeNet)
Appendix
Table C3: Per-Family Impact of Focal Loss ( γ=5 , α=0.5 ) vs. CCE Baseline.
Metrics (Average)
224*
160
192
256
384
448
512
Macro F1
52.9%
53.0%
52.7%
52.8%
53.5%
53.9%
52.7%
Standard Top-1
71.7%
72.1%
71.8%
72.4%
72.4%
72.3%
72.1%
Ordinal Top-1
86.2%
86.4%
86.4%
86.5%
86.5%
86.9%
86.1%
Params (M)
58.77
52.57
52.57
52.57
52.57
52.57
52.57
GFLOPs
8.98
4.17
6.00
10.66
23.96
32.61
42.59
Inference Time (ms)
11.16
11.11
11.19
11.27
11.46
11.66
11.88
Appendix
Table C4: Average Macro F1 and Efficiency Metrics Across All Models for Input Resolution.
Figure C1: Figure showing the performance progression of QSTD image resolutions (Height x Width) for each of the 6 damage classes.
Figure C2: Heatmap of Per-Class F1 Score Percentage Change by Input Resolution, Relative to the 2242 Baseline. The blue-bounded region ( 2562 – 4482 ) highlights the optimal resolution range, where the most significant performance gains (+14.2 for DS1) are concentrated in the challenging intermediate damage states (DS1-DS3).
Augmentation Strategy
None
Basic*
Standard
Advanced
Heavy
Avg. Macro F1
51.6%
52.9%
54.0%
50.2%
46.8%
Avg. Standard Top-1
71.3%
71.7%
72.5%
71.9%
70.7%
Avg. Ordinal Top-1
85.7%
86.2%
86.4%
85.8%
84.6%
Avg. Params (M)
58.78
58.77
58.77
58.78
61.54
Avg. GFLOPs
8.98
8.98
8.98
8.98
9.85
Avg. Inference Time (ms)
11.23
11.16
11.23
11.21
11.43
Appendix
Table C5: Average Macro F1 and Efficiency Metrics Across All Models for Data Augmentation Strategies.
Metrics (Average)
Adam*
SGD
Δ
Macro F1
52.9%
59.8%
+6.9%
Standard Top-1
71.7%
74.6%
+2.9%
Ordinal Top-1
86.2%
89.3%
+3.2%
Params (M)
58.77
61.28
+2.51
GFLOPs
8.98
9.85
+0.86
Inference Time (ms)
11.16
11.42
+0.26
Appendix
Table C6: Average Performance for Optimizer (Adam vs. SGD).
Metrics (Average)
1×10−3 *
1×10−4
3×10−3
5×10−3
Macro F1
52.9%
63.1%
43.6%
40.9%
Standard Top-1
71.7%
76.2%
69.1%
68.3%
Ordinal Top-1
86.2%
90.3%
83.8%
82.5%
Params (M)
58.77
61.28
61.28
61.28
GFLOPs
8.98
9.85
9.85
9.85
Inference Time (ms)
11.16
11.43
11.40
11.41
Appendix
Table C7: Average Performance for Learning Rate Configurations.
Metrics (Average)
None*
Cosine
Step
Macro F1
52.9%
52.8%
53.8%
Standard Top-1
71.7%
71.8%
73.2%
Ordinal Top-1
86.2%
86.0%
86.9%
Params (M)
58.77
61.53
61.53
GFLOPs
8.98
9.85
9.85
Inference Time (ms)
11.16
11.46
11.39
Appendix
Table C8: Average Performance for LR Scheduler Strategies.
Metrics (Average)
64*
32
16
8
4
Macro F1
52.9%
51.0%
48.5%
45.7%
43.3%
Standard Top-1
71.7%
71.4%
70.8%
70.1%
69.2%
Ordinal Top-1
86.2%
85.4%
84.8%
84.1%
82.8%
Params (M)
58.77
61.28
61.28
61.28
61.28
GFLOPs
8.98
9.85
9.85
9.85
9.85
Inference Time (ms)
11.16
11.40
11.37
11.41
11.37
Appendix
Table C9: Average Performance for Batch Size Configurations.
Metrics (Average)
0.0*
0.2
0.3
0.5
Macro F1
52.9%
52.5%
52.7%
52.1%
Standard Top-1
71.7%
71.8%
71.8%
71.4%
Ordinal Top-1
86.2%
86.2%
86.2%
85.8%
Params (M)
58.77
61.28
61.28
61.28
GFLOPs
8.98
9.85
9.85
9.85
Inference Time (ms)
11.16
11.71
11.69
11.63
Appendix
Table C10: Average Performance for Dropout Rates.
Metrics (Average)
Default*
ReLU
Leaky ReLU
ELU
SiLU
GELU
Macro F1
52.9%
50.5%
49.6%
44.5%
46.5%
48.7%
Standard Top-1
71.7%
70.5%
70.3%
69.2%
69.8%
70.5%
Ordinal Top-1
86.2%
85.2%
84.5%
83.2%
83.8%
84.4%
Params (M)
58.77
61.28
61.28
61.28
61.28
61.28
GFLOPs
8.98
9.85
9.85
9.85
9.85
9.85
Inference Time (ms)
11.16
11.60
11.60
11.61
11.52
11.57
Appendix
Table C11: Average Performance for Activation Function Configurations.
We present TornadoNet, a comprehensive benchmark for automated street-level building damage assessment evaluating how modern real-time object detection architectures and ordinal-aware supervision strategies perform under realistic post-disaster conditions. TornadoNet provides the first controlled benchmark demonstrating how architectural design and loss formulation jointly influence multi-level damage detection from street-view imagery, delivering methodological insights and deployable tools for disaster response. Using 3,333 high-resolution geotagged images and 8,890 annotated building instances from the 2021 Midwest tornado outbreak, we systematically compare CNN-based detectors from the YOLO family against transformer-based models (RT-DETR) for multi-level damage detection. Models are trained under standardized protocols using a five-level damage classification framework based on IN-CORE damage states, validated through expert cross-annotation. Baseline experiments reveal complementary architectural strengths. CNN-based YOLO models achieve highest detection accuracy and throughput, with larger variants reaching 46.05% [email protected] at 66-276 FPS on A100 GPUs. Transformer-based RT-DETR models exhibit stronger ordinal consistency, achieving 88.13% Ordinal Top-1 Accuracy and MAOE of 0.65, indicating more reliable severity grading despite lower baseline mAP. To align supervision with the ordered nature of damage severity, we introduce soft ordinal classification targets and evaluate explicit ordinal-distance penalties. RT-DETR trained with calibrated ordinal supervision achieves 44.70% [email protected], a 4.8 percentage-point improvement, with gains in ordinal metrics (91.15% Ordinal Top-1 Accuracy, MAOE = 0.56). These findings establish that ordinal-aware supervision improves damage severity estimation when aligned with detector architecture. Model & Data: https://github.com/crumeike/TornadoNet
Robinson Umeike, Cuong Pham, Ryan Hausen +8
The University of Alabama · University of South Alabama · Johns Hopkins University +2
Decision-relevant building damage assessment is critical for prioritizing resources and recovery after a disaster, yet most automated methods either flatten damage into a single severity scale (no damage, minor, major, destroyed) or require paired pre- and post-event imagery that is often unavailable for emerging hazards. This paper presents Damage-TriageFormer, a single-image, post-event, footprint-conditioned model that produces a damage typology rather than a severity scale. We contribute: (1) DamageTriage-Bench, a new benchmark built from NOAA Emergency Response Imagery across Hurricane Michael (2018), Hurricane Helene (2024), and the 2025 Los Angeles wildfire complex, with five typology classes that distinguish roof damage from structural damage and, within each, partial from total extent; and (2) Damage-TriageFormer, which extends a DINOv3 ViT-L backbone with a Simple Feature Pyramid for higher-resolution instance pooling, a two-stage gated damage head, and an auxiliary severity-regression objective. Our model achieves macro F1 of 0.624 on validation and 0.619 on a held-out stratified test set, performing strongest where operational triage needs it most, with per-class F1 of 0.91 and 0.84 on undamaged buildings and total structural collapse, respectively. While the rare Total Roof Damage class remains difficult due to its limited examples and an inherently ambiguous label boundary, our results show that single-image post-event imagery can support actionable building damage typing, enabling targeted emergency response and resource allocation without a pre-event reference.
Yiming Xiao, Yu-Hsuan Ho, Sanjay Thasma +2
Texas A&M University · Resilitix Intelligence LLC · Institute for a Disaster Resilient Texas
Rapid and accurate post-disaster building damage assessment is essential, yet remains a challenging task. Unmanned Aerial Vehicle (UAV) imagery offers a timely and high-resolution view of affected areas, but existing Computer Vision (CV) models often demand large annotated datasets, generalize poorly across geographic regions and their assessment policies, and are confined to the specific tasks they were trained for. Large Vision-Language Models (LVLMs) offer a promising alternative through their strong reasoning and generalization capabilities, but fall short on precise, low-level perception tasks such as object detection and accurate bounding box generation. Furthermore, they often require a substantial amount of data for effective fine-tuning on domain-specific tasks. In this paper, we propose a hybrid framework that decouples detection from damage assessment, combining the precision of CV models with the reasoning power of LVLMs. A CV model first detects buildings and generates bounding boxes on the image that are then passed to an LVLM for damage classification and contextual interpretation. We evaluated our framework on two real-world benchmarks: RescueNet and FloodNet. In particular, the best combination under this framework accurately counts intact, partially damaged and completely destroyed buildings, surpassing isolated baselines by up to 2.1 R^2 points, while requiring only limited annotated data for the detection stage. Beyond reporting aggregate gains, we provide a detailed analysis of failure scenarios and edge cases, offering practical insights for practitioners and concrete directions for future work. Our source code and data are publicly available to the research community via the following repository: https://github.com/ungquanghuy-kddi/VLM_GDINO.git