Multiclass Semantic Segmentation of Wildland Fire Images Using Context-Aware Centralized Copy-Paste Data Augmentation
Authors: Joon Tai Kim, Nishanth Kunchala, Vishv Patel, Tianle Chen, Ziyu Dong, Daniel Ospina Acero, Roger Williams, Mrinal Kumar
Organizations: Mechanical and Aerospace Engineering Department, The Ohio State University, Columbus, OH 43210, USA · Department of Computer Science, Boston University, Boston, MA 02215, USA · Department of Wildland Resources, Utah State University, Logan, UT 84322, USA · Department of Electronic and Telecommunications Engineering, Universidad de Antioquia, Medell´ın, Colombia · School of Environment and Natural Resources, The Ohio State University, Columbus, OH 43210, USA
Producing accurate annotations for deep learning based image segmentation is both costly and labor intensive. This challenge is especially evident in wildland fire applications, where accurately labeled datasets are scarce due to the difficulty of collecting and annotating dynamic fire scenes. To address this problem, our previous work introduced the Centralized Copy-Paste Data Augmentation (CCPDA) method for semantic segmentation of wildland fire imagery, which generates artificial training samples by randomly pasting fire clusters from source images onto target images. However, random placement can produce contextually unrealistic scenes, such as fire burning on asphalt. In this paper, we present a context-aware strategy designed specifically to improve data quality and realism in small multiclass wildland fire datasets, ensuring that augmented samples remain contextually meaningful. The proposed method restricts fire placement to semantically valid target regions and selects the location whose Ash-Vegetation composition most closely matches the source context. This approach preserves existing fire regions in the target image, prevents unrealistic placements, and maintains contextual accuracy by generating images that resemble real wildland fire scenes. We evaluate the Context-Aware CCPDA strategy through numerical analysis and comparisons with other augmentation methods by a weighted sum-based multi-objective optimization (MOO) approach. The results confirm that the context-aware data augmentation strategy leads to improved segmentation performance and contextual realism, outperforming other augmentation procedures.
Figures & tables
Fig. 1 : The U-Net architecture used as the baseline model for semantic segmentation of wildland fire images. This figure is reused from [ 10 ] .
Fig. 2 : Flowchart for smoke dehazing method. This figure is reused from [ 10 ] .
Fig. 3 : A comparison of the original hazy RGB image (a) and the dehazed RGB image (b) achieved through a 95% haze-reduction procedure.
Fig. 4 : An illustration of the standard centralized copy-paste method for fire segmentation. The process involves: (1) identifying an original fire cluster (yellow) in the source image, (2) applying the centralized method to focus on the core fire area (red), and (3) pasting the centralized fire cluster onto a target image. This figure is reused from [ 10 ] .
Fig. 5 : Flowchart illustrating the proposed Context-Aware CCPDA method. The process identifies fire clusters, computes Ash and Vegetation statistics, and evaluates candidate paste locations in the target image based on compositional similarity and semantic constraints. Finally, the optimal location is selected; if none is identified, the bounding box is adaptively downsized and the process repeated.
Fig. 6 : A visual comparison of Non-Context-Aware and Context-Aware CCPDA results for a sample source and target image pair. Sub-figures (a) and (b) show the original source and target images, respectively, while (c) and (d) display zoomed-in augmented images from the upper-center region of (b), generated using the Non-Context-Aware and Context-Aware CCPDA methods, respectively.
Fig. 7 : A visualization of compositional matching for wildland fire image data augmentation. Green and yellow data points represent Ash –to- Vegetation percentage distributions from the source and target images, respectively. The inset illustrates the context-aware paste-location selection process for a single source fire cluster, where candidate target bounding boxes (red points) are compared with the source bounding box (single blue point), and the target bounding box with the closest compositional match is selected (single green star).
Learning Rate
Dropout Rate
Batch Size
F(x)
0.0005
0.3
4
0.83734
0.0005
0.2
16
0.83650
0.0005
0.2
4
0.83570
⋮
0.005
0.3
8
0.23578
0.01
0.1
4
0.23578
TABLE I : Selected hyperparameter configurations ranked by F(x) score. The first row, shown in bold, indicates the best-performing tuned configuration.
Method
Fire -FNR
Veg. -IoU
total-IoU
F(x)
Non-Augmented
18.04
67.40
54.45
0.7486
Rotation-by-18 ∘
14.17
71.14
60.54
0.7894
Non-Context-Aware Copy-Paste
8.24
71.52
60.91
0.8271
Context-Aware Copy-Paste
6.75
73.74
64.03
0.8458
TABLE II : Study 1 performance metrics (%) and weighted-sum scores across different augmentation methods.
(Study 1) Method
Fire [%]
(Study 2) Erosion %
Fire [%]
Non-Augmented
5.07
0%
8.80
Rotation-by-18 ∘
10.59
10%
8.31
N.C.A 1 Copy-Paste
9.30
20%
7.79
C.A 2 Copy-Paste
9.19
30%
7.38
Non-Augmented Data Pixel Total
655,360
Augmented Data Pixel Total
13,107,200
TABLE III : Comparison of Fire pixel percentages across Study 1 augmentation methods and Study 2 erosion levels for the expanded 26-image set, including total pixel counts for the corresponding non-augmented and augmented training datasets. 1 Non-Context-Aware, 2 Context-Aware.
Fig. 8 : A visual comparison of the semantic segmentation results of Study 1 for a sample test RGB image.
Fig. 9 : A visual comparison of Context-Aware CCPDA outputs using the first and a randomly selected optimal location for a sample training image. Sub-figure (b) shows the training image used as both the source and target image. Sub-figures (d) and (e) show the Context-Aware CCPDA outputs obtained by pasting the source fire clusters at the first optimal location and at a randomly selected location from the set of possible optimal locations in the target image, respectively. Sub-figures (a) and (c) provide enlarged views of the corresponding boxed regions in (d) and (e), respectively.
20-image set
26-image set
Method
Fire -FNR
Veg. -IoU
total-IoU
F(x)
Fire -FNR
Veg. -IoU
total-IoU
F(x)
Erosion-by-0%
5.22
66.98
56.16
0.8276
6.76
70.32
60.45
0.8323
Erosion-by-10%
4.00
66.79
57.93
0.8365
4.67
70.78
60.17
0.8460
Erosion-by-20%
7.59
67.99
57.29
0.8172
8.28
70.34
58.88
0.8213
Erosion-by-30%
9.04
68.15
57.09
0.8086
12.27
72.31
62.05
0.8059
TABLE IV : Study 2 performance metrics (%) and weighted-sum scores across erosion levels for the Context-Aware CCPDA method on the original 20-image set used to construct BURN 1 and the expanded 26-image set used to construct BURN 2.
Method
Fire -FNR
Veg. -IoU
total-IoU
F(x)
FCN
Non-Augmented
72.85
65.23
49.72
0.4024
Rotation-by-18 ∘
25.35
66.82
54.67
0.7026
Non-Context-Aware Copy-Paste
22.54
67.33
54.34
0.7208
Context-Aware Copy-Paste
21.72
66.19
52.99
0.7211
SegNet
TABLE V : Study 3 performance metrics (%) and weighted-sum scores across different augmentation methods on three semantic segmentation models: FCN, SegNet, and DeepLabV3+.
California High School, San Ramon, CA, 94583, USA · Department of Biological and Agricultural Engineering, University of California, Davis, Davis, CA, 95616, USA