BAWSeg: A UAV Multispectral Benchmark for Barley Weed Segmentation

Haitian Wang; Xinyu Wang; Muhammad Ibrahim; Dustin Severtson; Ajmal Mian

cs.CVMar 2, 2026

BAWSeg: A UAV Multispectral Benchmark for Barley Weed Segmentation

Authors: Haitian Wang, Xinyu Wang, Muhammad Ibrahim, Dustin Severtson, Ajmal Mian

Organizations: Department of Computer Science and Software Engineering, The University of Western Australia, 35 Stirling Highway, Crawley, WA 6009, Australia · Department of Primary Industries and Regional Development (DPIRD), Government of Western Australia, 1 Nash St., Perth, WA 6000, Australia

Abstract

Accurate weed mapping in cereal fields requires pixel-level segmentation from UAV imagery that remains reliable across fields, seasons, and illumination. Existing multispectral pipelines often depend on thresholded vegetation indices, which are brittle under radiometric drift and mixed crop--weed pixels, or on single-stream CNN and Transformer backbones that ingest stacked bands and indices, where radiance cues and normalized index cues interfere and reduce sensitivity to small weed clusters embedded in crop canopy. We propose VISA, a two-stream segmentation network that decouples these cues and fuses them at native resolution. The radiance stream learns from calibrated five-band reflectance using local residual convolutions, channel recalibration, spatial gating, and skip-connected decoding, which preserve fine textures, row boundaries, and small weed structures that are often weakened after ratio-based index compression. The index stream operates on vegetation-index maps with windowed self-attention to model local structure efficiently, state-space layers to propagate field-scale context without quadratic attention cost, and Slot Attention to form stable region descriptors that improve discrimination of sparse weeds under canopy mixing. To support supervised training and deployment-oriented evaluation, we introduce BAWSeg, a four-year UAV multispectral dataset collected over commercial barley paddocks in Western Australia, providing radiometrically calibrated blue, green, red, red edge, and near-infrared orthomosaics, derived vegetation indices, and dense crop, weed, and other labels with leakage-free block splits. On BAWSeg, VISA achieves 75.6% mIoU and 63.5% weed IoU with 22.8 M parameters, outperforming a multispectral SegFormer-B1 baseline by 1.2 mIoU and 1.9 weed IoU. Under cross-plot and cross-year protocols, VISA maintains 71.2% and 69.2% mIoU, respectively.

Figures & tables

Figure 1: Overview of UAV multispectral data collection and BAWSeg dataset construction. The pipeline includes flight acquisition with RTK and irradiance sensors, radiometric calibration, orthorectification, stitching, feature extraction, annotation, and resulting crop–weed–soil maps.

Figure 2: Planned Planned flight paths for the Kondinin experimental fields. Left, E8 polygon and survey lines planned in GSPro. Right, E2 polygon with the same overlap and altitude parameters. The parallel survey lines indicate the planned lawnmower flight trajectories within each field polygon. The inset shows a representative interior tile used for quality control of row orientation and texture.

Figure 3: Example of keypoint inliers for cross-band and inter-frame alignment. Green circles denote SIFT keypoints and blue lines denote matches retained after RANSAC.

Figure 4: From left to right, RGB orthomosaic, a single-band reflectance map, and the hard label map. Green denotes crop, red denotes weed, and white denotes other.

Figure 5: Overview of the proposed pipeline. Bottom: SRAB radiance stream with residual attention and a U-Net decoder. Top: vegetation-index stream with windowed self-attention, Mamba blocks, Slot Attention with mean-slot broadcast, and a scale-aligned refinement module. Here, the scale-aligned refinement module denotes the lightweight feature-refinement stage that aligns the index-stream representation with the radiance branch on the native $256\times 256$ grid before fusion. Right: fusion head that concatenates the two 64-channel feature maps at the native resolution to generate class logits.

Figure 6: Training dynamics of the proposed model. The upper panel shows loss, accuracy, mean IoU, precision, recall and F1 over 50 epochs. The lower panel shows precision versus recall for each epoch. Markers move towards the upper right region as training progresses and stabilise near the final operating point.

Year	Field	mIoU	Per Class IoU			Micro Metrics			Global Metrics
Year	Field	mIoU	Crop	Weed	Other	$P$	$R$	F1	OA	$\kappa$
2020	E2	0.748	0.842	0.621	0.781	0.838	0.846	0.842	0.943	0.784
2020	E8	0.751	0.844	0.623	0.786	0.840	0.848	0.844	0.944	0.786
2021	E2	0.754	0.845	0.626	0.791	0.842	0.850	0.846	0.945	0.789
2021	E8	0.759	0.848	0.630	0.800	0.844	0.852	0.848	0.946	0.791
2022	E2	0.762	0.850	0.636	0.801	0.846	0.854	0.850	0.947	0.795

Table 1: Within-plot segmentation performance per year and field on held-out test blocks. Mean IoU is estimated using

95\%

bootstrap confidence intervals. Precision, recall and F1 are micro-averaged over all classes.

Protocol	Train Set	Test Set	mIoU $\pm$ CI	Per Class IoU			Micro Metrics			Global Metrics
Protocol	Train Set	Test Set	mIoU $\pm$ CI	Crop	Weed	Other	$P$	$R$	F1	OA	$\kappa$
Within plot	E2 and E8, 2020 to 2023	E2 and E8, 2020 to 2023	$0.756\pm 0.004$	0.847	0.635	0.794	0.851	0.843	0.847	0.946	0.794
Cross plot	E2, 2020 to 2023	E8, 2020 to 2023	$0.712\pm 0.006$	0.840	0.576	0.724	0.838	0.830	0.834	0.937	0.768
Cross plot	E8, 2020 to 2023	E2, 2020 to 2023	$0.718\pm 0.006$	0.846	0.584	0.724	0.842	0.836	0.839	0.939	0.773
Cross year	2020 to 2022, E2 and E8	2023, E2 and E8	$0.692\pm 0.007$	0.832	0.544	0.700	0.828	0.816	0.822	0.936	0.752

Table 2: Segmentation performance under different evaluation protocols. Train and test sets are defined at the block level. Mean IoU is reported with

95\%

confidence intervals.

Method	Year/Venue	Input	mIoU $\pm$ CI	Crop IoU	Weed IoU	Others IoU	F1	OA	$\kappa$	Params (M)
RF + Indices Breiman (2001)	2021 ML	Indices	0.674 $\pm$ 0.010	0.812	0.532	0.679	0.734	0.907	0.703	0.2
SegNet RGB Badrinarayanan et al. (2017)	2019 TPAMI	RGB	0.694 $\pm$ 0.009	0.825	0.558	0.699	0.752	0.915	0.718	29.4
SegNet MSI Badrinarayanan et al. (2017)	2024 TPAMI	MSI	0.702 $\pm$ 0.009	0.831	0.567	0.708	0.759	0.918	0.724	29.5
U-Net RGB Ronneberger et al. (2015)	2023 MICCAI	RGB	0.712 $\pm$ 0.008	0.836	0.579	0.720	0.768	0.922	0.732	31.0
U-Net MSI Ronneberger et al. (2015)	2022 MICCAI	MSI	0.731 $\pm$ 0.007	0.842	0.594	0.758	0.783	0.930	0.748	31.1
UNet++ MSI Zhou et al. (2018)	2024 DLMIA	MSI	0.735 $\pm$ 0.007	0.844	0.603	0.759	0.787	0.932	0.751	34.2

Table 3: Comparison with classical baselines and modern segmentation networks under the within plot protocol on E2 and E8 from 2020 to 2023. MSI denotes the five calibrated multispectral bands and indices denote the five vegetation indices. Mean IoU is reported with

95\%

bootstrap confidence intervals over test blocks.

Variant	mIoU $\uparrow$	$\Delta$	Weed IoU $\uparrow$	Params (M) $\downarrow$	FLOPs (G) $\downarrow$	Mem (GB) $\downarrow$	FPS $\uparrow$
Module removal or replacement
Full model	$0.756\pm 0.004$	$0.000$	0.635	22.8	33.6	2.60	78
w/o VIMB (RAW only)	$0.731\pm 0.007$	$-0.025$	0.596	9.6	25.4	2.20	92
w/o relative bias in WSA	$0.749\pm 0.007$	$-0.007$	0.626	12.8	33.4	2.60	78
w/o Slot Attention	$0.744\pm 0.007$	$-0.012$	0.618	12.3	32.4	2.50	80
w/o mean-slot broadcast	$0.748\pm 0.007$	$-0.008$	0.622	12.3	32.4	2.50	80

Table 4: Ablation on the within‐plot protocol.

\Delta

is relative to the full model. Params in millions. FLOPs in billions at

256\times 256

. Mem in GB at FP16, batch size

1

. mIoU reports block-bootstrap 95% confidence intervals. Arrows indicate whether higher or lower values are preferable.

BAWSeg	Barley Weed Segmentation benchmark dataset
VISA	Vegetation-Index and Spectral Attention (two-stream segmentation network)
UAV	Unmanned Aerial Vehicle
RPAS	Remotely Piloted Aircraft System
RTK	Real-Time Kinematic
RGB	Red, Green, and Blue
NIR	Near-Infrared

Table 11

Remote Sensing Image Segmentation

Remote sensing image segmentation labels land cover and objects in satellite, aerial, radar, and multispectral imagery.

See all 220 papers

Apr 17, 2026

SWNet: A Cross-Spectral Network for Camouflaged Weed Detection

This paper presents SWNet, a bimodal end-to-end cross-spectral network specifically engineered for the detection of camouflaged weeds in dense agricultural environments. Plant camouflage, characterized by homochromatic blending where invasive species mimic the phenotypic traits of primary crops, poses a significant challenge for traditional computer vision systems. To overcome these limitations, SWNet utilizes a Pyramid Vision Transformer v2 backbone to capture long-range dependencies and a Bimodal Gated Fusion Module to dynamically integrate Visible and Near-Infrared information. By leveraging the physiological differences in chlorophyll reflectance captured in the NIR spectrum, the proposed architecture effectively discriminates targets that are otherwise indistinguishable in the visible range. Furthermore, an Edge-Aware Refinement module is employed to produce sharper object boundaries and reduce structural ambiguity. Experimental results on the Weeds-Banana dataset indicate that SWNet outperforms ten state-of-the-art methods. The study demonstrates that the integration of cross-spectral data and boundary-guided refinement is essential for high segmentation accuracy in complex crop canopies. The code is available on GitHub: https://cod-espol.github.io/SWNet/

Henry O. Velesaca, Luigi Miranda, Angel D. Sappa

ESPOL Polytechnic University, Campus Gustavo Galindo, 090902, Guayaquil, Ecuador · Software Engineering Department, Research Center for Information and Communication Technologies (CITIC-UGR), University of Granada, 18071, Granada, Spain · Computer Vision Center, Universitat Aut`onoma de Barcelona, 08193-Bellaterra, Barcelona, Spain

Agricultural Image Analysis Weed Detection Multispectral Object Detection Multimodal Fusion Camouflaged Object Detection

Ask PDF

Jun 4, 2026

Comparison of Deep Learning Frameworks For Rice Disease Mapping From UAV Multispectral Imaging

In this study, UAV multispectral imagery is used to segment the severity of bacterial leaf blight (BLB) in rice using convolutional neural networks (CNNs) and transformer-based models. The evaluated architectures include U-Net with a ResNet- 101 encoder, U-Net++ with EfficientNet-B3 and EfficientNetB7, DeepLabV3+, and SegFormer, all trained under a common pipeline with three input configurations (multispectral only, multispectral+NDVI, and multispectral+NDRE). Experiments are conducted using the publicly available BLB dataset with performance reported using mean IoU (mIoU), mean F1 (mF1), mean accuracy (mAcc), precision, and recall. U-Net++ with EfficientNet-B3 achieved the highest performance, with an mIoU of 97.62%. SegFormer obtained lower segmentation accuracy but comparable inference speed. Overall, the results indicate that lightweight CNN backbones remain more reliable for operational BLB monitoring while integration of vegetation indices provides small and consistent improvements. The study also highlights the value of standardised UAV datasets to compare disease mapping methods and encourages the use of CNN architectures for field implementation.

Yadav Raj Ghimire, Jagrati Talreja, Tewodros Syum Gebre +3

North Carolina A&T State University Greensboro, North Carolina, USA.

Image Segmentation Remote Sensing Image Segmentation Agricultural Remote Sensing U-Net UAV Remote Sensing

Ask PDF

Sep 4, 2026

AAMBERS-UAV: Acquisition-Aware Multimodal Backbone Evaluation and Ranking for UAV Weedy Rice Segmentation

UAV image collections contain spatially and temporally related frames, yet semantic-segmentation benchmarks commonly split them at image level. Such splitting can place samples from one acquisition in both model development and testing, obscuring transfer to a genuinely new survey. Using the 734-sample WeedyRice-RGBMS-DB, we fix a 124-image target-acquisition test set and compare two protocols with identical train, validation, and test counts: target-held-out, which excludes the target acquisition from development, and target-exposed, which admits its remaining images. SegFormer-B0 is evaluated with RGB, four-band multispectral (MS), and seven-channel RGB+MS input over two fixed-split seeds. RGB is strongest under complete acquisition holdout (

0.7317\pm0.0201

IoU), whereas RGB+MS becomes strongest after target exposure (

0.7822\pm0.0269

). A fixed-split U-Net/ResNet18 replication confirms positive exposure gains for all three inputs, but retains RGB as the best modality under both protocols. Acquisition exposure therefore increases measured performance across both evaluated backbones, while its effect on modality ranking is architecture-dependent. A supplied-split audit reveals strong near-sequential dependence, and corruption tests show that early fusion is substantially more sensitive to RGB--MS displacement than to moderate radiometric scaling. These results support acquisition-aware same-test evaluation as a necessary complement to ordinary image-level splitting in multimodal UAV benchmarks. The code and supporting the findings of this study will be publicly released upon acceptance of the paper.

Tarek Rahman, Nazim-E-Alam, Md Kishor Morol +1

United International University · ELITE Research Lab, New York, USA · American International University Bangladesh

Remote Sensing Image Segmentation Remote Sensing Benchmarks UAV Remote Sensing Multimodal Robustness

Ask PDF

Abstract

Remote Sensing Image Segmentation

Explore similar work

SWNet: A Cross-Spectral Network for Camouflaged Weed Detection

Comparison of Deep Learning Frameworks For Rice Disease Mapping From UAV Multispectral Imaging

AAMBERS-UAV: Acquisition-Aware Multimodal Backbone Evaluation and Ranking for UAV Weedy Rice Segmentation