BAWSeg: A UAV Multispectral Benchmark for Barley Weed Segmentation

Haitian Wang; Xinyu Wang; Muhammad Ibrahim; Dustin Severtson; Ajmal Mian

cs.CVMar 2, 2026

BAWSeg: A UAV Multispectral Benchmark for Barley Weed Segmentation

Authors: Haitian Wang, Xinyu Wang, Muhammad Ibrahim, Dustin Severtson, Ajmal Mian

Organizations: Department of Computer Science and Software Engineering, The University of Western Australia, 35 Stirling Highway, Crawley, WA 6009, Australia · Department of Primary Industries and Regional Development (DPIRD), Government of Western Australia, 1 Nash St., Perth, WA 6000, Australia

Abstract

Accurate weed mapping in cereal fields requires pixel-level segmentation from UAV imagery that remains reliable across fields, seasons, and illumination. Existing multispectral pipelines often depend on thresholded vegetation indices, which are brittle under radiometric drift and mixed crop--weed pixels, or on single-stream CNN and Transformer backbones that ingest stacked bands and indices, where radiance cues and normalized index cues interfere and reduce sensitivity to small weed clusters embedded in crop canopy. We propose VISA, a two-stream segmentation network that decouples these cues and fuses them at native resolution. The radiance stream learns from calibrated five-band reflectance using local residual convolutions, channel recalibration, spatial gating, and skip-connected decoding, which preserve fine textures, row boundaries, and small weed structures that are often weakened after ratio-based index compression. The index stream operates on vegetation-index maps with windowed self-attention to model local structure efficiently, state-space layers to propagate field-scale context without quadratic attention cost, and Slot Attention to form stable region descriptors that improve discrimination of sparse weeds under canopy mixing. To support supervised training and deployment-oriented evaluation, we introduce BAWSeg, a four-year UAV multispectral dataset collected over commercial barley paddocks in Western Australia, providing radiometrically calibrated blue, green, red, red edge, and near-infrared orthomosaics, derived vegetation indices, and dense crop, weed, and other labels with leakage-free block splits. On BAWSeg, VISA achieves 75.6% mIoU and 63.5% weed IoU with 22.8 M parameters, outperforming a multispectral SegFormer-B1 baseline by 1.2 mIoU and 1.9 weed IoU. Under cross-plot and cross-year protocols, VISA maintains 71.2% and 69.2% mIoU, respectively.

Figures & tables

Figure 1: Overview of UAV multispectral data collection and BAWSeg dataset construction. The pipeline includes flight acquisition with RTK and irradiance sensors, radiometric calibration, orthorectification, stitching, feature extraction, annotation, and resulting crop–weed–soil maps.

Figure 2: Planned Planned flight paths for the Kondinin experimental fields. Left, E8 polygon and survey lines planned in GSPro. Right, E2 polygon with the same overlap and altitude parameters. The parallel survey lines indicate the planned lawnmower flight trajectories within each field polygon. The inset shows a representative interior tile used for quality control of row orientation and texture.

Figure 3: Example of keypoint inliers for cross-band and inter-frame alignment. Green circles denote SIFT keypoints and blue lines denote matches retained after RANSAC.

Figure 4: From left to right, RGB orthomosaic, a single-band reflectance map, and the hard label map. Green denotes crop, red denotes weed, and white denotes other.

Figure 5: Overview of the proposed pipeline. Bottom: SRAB radiance stream with residual attention and a U-Net decoder. Top: vegetation-index stream with windowed self-attention, Mamba blocks, Slot Attention with mean-slot broadcast, and a scale-aligned refinement module. Here, the scale-aligned refinement module denotes the lightweight feature-refinement stage that aligns the index-stream representation with the radiance branch on the native $256\times 256$ grid before fusion. Right: fusion head that concatenates the two 64-channel feature maps at the native resolution to generate class logits.

Figure 6: Training dynamics of the proposed model. The upper panel shows loss, accuracy, mean IoU, precision, recall and F1 over 50 epochs. The lower panel shows precision versus recall for each epoch. Markers move towards the upper right region as training progresses and stabilise near the final operating point.

Year	Field	mIoU	Per Class IoU			Micro Metrics			Global Metrics
Year	Field	mIoU	Crop	Weed	Other	$P$	$R$	F1	OA	$\kappa$
2020	E2	0.748	0.842	0.621	0.781	0.838	0.846	0.842	0.943	0.784
2020	E8	0.751	0.844	0.623	0.786	0.840	0.848	0.844	0.944	0.786
2021	E2	0.754	0.845	0.626	0.791	0.842	0.850	0.846	0.945	0.789
2021	E8	0.759	0.848	0.630	0.800	0.844	0.852	0.848	0.946	0.791
2022	E2	0.762	0.850	0.636	0.801	0.846	0.854	0.850	0.947	0.795

Table 1: Within-plot segmentation performance per year and field on held-out test blocks. Mean IoU is estimated using

95\%

bootstrap confidence intervals. Precision, recall and F1 are micro-averaged over all classes.

Protocol	Train Set	Test Set	mIoU $\pm$ CI	Per Class IoU			Micro Metrics			Global Metrics
Protocol	Train Set	Test Set	mIoU $\pm$ CI	Crop	Weed	Other	$P$	$R$	F1	OA	$\kappa$
Within plot	E2 and E8, 2020 to 2023	E2 and E8, 2020 to 2023	$0.756\pm 0.004$	0.847	0.635	0.794	0.851	0.843	0.847	0.946	0.794
Cross plot	E2, 2020 to 2023	E8, 2020 to 2023	$0.712\pm 0.006$	0.840	0.576	0.724	0.838	0.830	0.834	0.937	0.768
Cross plot	E8, 2020 to 2023	E2, 2020 to 2023	$0.718\pm 0.006$	0.846	0.584	0.724	0.842	0.836	0.839	0.939	0.773
Cross year	2020 to 2022, E2 and E8	2023, E2 and E8	$0.692\pm 0.007$	0.832	0.544	0.700	0.828	0.816	0.822	0.936	0.752

Table 2: Segmentation performance under different evaluation protocols. Train and test sets are defined at the block level. Mean IoU is reported with

95\%

confidence intervals.

Method	Year/Venue	Input	mIoU $\pm$ CI	Crop IoU	Weed IoU	Others IoU	F1	OA	$\kappa$	Params (M)
RF + Indices Breiman (2001)	2021 ML	Indices	0.674 $\pm$ 0.010	0.812	0.532	0.679	0.734	0.907	0.703	0.2
SegNet RGB Badrinarayanan et al. (2017)	2019 TPAMI	RGB	0.694 $\pm$ 0.009	0.825	0.558	0.699	0.752	0.915	0.718	29.4
SegNet MSI Badrinarayanan et al. (2017)	2024 TPAMI	MSI	0.702 $\pm$ 0.009	0.831	0.567	0.708	0.759	0.918	0.724	29.5
U-Net RGB Ronneberger et al. (2015)	2023 MICCAI	RGB	0.712 $\pm$ 0.008	0.836	0.579	0.720	0.768	0.922	0.732	31.0
U-Net MSI Ronneberger et al. (2015)	2022 MICCAI	MSI	0.731 $\pm$ 0.007	0.842	0.594	0.758	0.783	0.930	0.748	31.1
UNet++ MSI Zhou et al. (2018)	2024 DLMIA	MSI	0.735 $\pm$ 0.007	0.844	0.603	0.759	0.787	0.932	0.751	34.2

Table 3: Comparison with classical baselines and modern segmentation networks under the within plot protocol on E2 and E8 from 2020 to 2023. MSI denotes the five calibrated multispectral bands and indices denote the five vegetation indices. Mean IoU is reported with

95\%

bootstrap confidence intervals over test blocks.

Variant	mIoU $\uparrow$	$\Delta$	Weed IoU $\uparrow$	Params (M) $\downarrow$	FLOPs (G) $\downarrow$	Mem (GB) $\downarrow$	FPS $\uparrow$
Module removal or replacement
Full model	$0.756\pm 0.004$	$0.000$	0.635	22.8	33.6	2.60	78
w/o VIMB (RAW only)	$0.731\pm 0.007$	$-0.025$	0.596	9.6	25.4	2.20	92
w/o relative bias in WSA	$0.749\pm 0.007$	$-0.007$	0.626	12.8	33.4	2.60	78
w/o Slot Attention	$0.744\pm 0.007$	$-0.012$	0.618	12.3	32.4	2.50	80
w/o mean-slot broadcast	$0.748\pm 0.007$	$-0.008$	0.622	12.3	32.4	2.50	80

Table 4: Ablation on the within‐plot protocol.

\Delta

is relative to the full model. Params in millions. FLOPs in billions at

256\times 256

. Mem in GB at FP16, batch size

1

. mIoU reports block-bootstrap 95% confidence intervals. Arrows indicate whether higher or lower values are preferable.

BAWSeg	Barley Weed Segmentation benchmark dataset
VISA	Vegetation-Index and Spectral Attention (two-stream segmentation network)
UAV	Unmanned Aerial Vehicle
RPAS	Remotely Piloted Aircraft System
RTK	Real-Time Kinematic
RGB	Red, Green, and Blue
NIR	Near-Infrared

Table 11

Remote Sensing Image Segmentation

Remote sensing image segmentation labels land cover and objects in satellite, aerial, radar, and multispectral imagery.

See all 220 papers

SWNet: A Cross-Spectral Network for Camouflaged Weed Detection
Apr 17, 2026Henry O. Velesaca, Luigi Miranda, Angel D. SappaAgricultural Image Analysis Weed Detection
ESPOL Polytechnic University, Campus Gustavo Galindo, 090902, Guayaquil, Ecuador · Software Engineering Department, Research Center for Information and Communication Technologies (CITIC-UGR), University of Granada, 18071, Granada, Spain · Computer Vision Center, Universitat Aut`onoma de Barcelona, 08193-Bellaterra, Barcelona, Spain
Comparison of Deep Learning Frameworks For Rice Disease Mapping From UAV Multispectral Imaging
Jun 4, 2026Yadav Raj Ghimire, Jagrati Talreja, Tewodros Syum Gebre +3Image Segmentation Remote Sensing Image Segmentation
North Carolina A&T State University Greensboro, North Carolina, USA.
AAMBERS-UAV: Acquisition-Aware Multimodal Backbone Evaluation and Ranking for UAV Weedy Rice Segmentation
Sep 4, 2026Tarek Rahman, Nazim-E-Alam, Md Kishor Morol +1Remote Sensing Image Segmentation Remote Sensing Benchmarks
United International University · ELITE Research Lab, New York, USA · American International University Bangladesh

Apr 17, 2026J/K move · Enter open · S save

SWNet: A Cross-Spectral Network for Camouflaged Weed Detection

Henry O. Velesaca, Luigi Miranda, Angel D. Sappa

ESPOL Polytechnic University, Campus Gustavo Galindo, 090902, Guayaquil, Ecuador · Software Engineering Department, Research Center for Information and Communication Technologies (CITIC-UGR), University of Granada, 18071, Granada, Spain · Computer Vision Center, Universitat Aut`onoma de Barcelona, 08193-Bellaterra, Barcelona, Spain

Agricultural Image Analysis Weed Detection Multispectral Object Detection Multimodal Fusion Camouflaged Object Detection

Open paper AskPDF arXiv

This paper presents SWNet, a bimodal end-to-end cross-spectral network specifically engineered for the detection of camouflaged weeds in dense agricultural environments. Plant camouflage, characterized by homochromatic blending where invasive species mimic the phenotypic traits of primary crops, poses a significant challenge for traditional computer vision systems. To overcome these limitations, SWNet utilizes a Pyramid Vision Transformer v2 backbone to capture long-range dependencies and a Bimodal Gated Fusion Module to dynamically integrate Visible and Near-Infrared information. By leveraging the physiological differences in chlorophyll reflectance captured in the NIR spectrum, the proposed architecture effectively discriminates targets that are otherwise indistinguishable in the visible range. Furthermore, an Edge-Aware Refinement module is employed to produce sharper object boundaries and reduce structural ambiguity. Experimental results on the Weeds-Banana dataset indicate that SWNet outperforms ten state-of-the-art methods. The study demonstrates that the integration of cross-spectral data and boundary-guided refinement is essential for high segmentation accuracy in complex crop canopies. The code is available on GitHub: https://cod-espol.github.io/SWNet/

Abstract

Remote Sensing Image Segmentation

Explore similar work

SWNet: A Cross-Spectral Network for Camouflaged Weed Detection

Comparison of Deep Learning Frameworks For Rice Disease Mapping From UAV Multispectral Imaging

AAMBERS-UAV: Acquisition-Aware Multimodal Backbone Evaluation and Ranking for UAV Weedy Rice Segmentation