AAMBERS-UAV: Acquisition-Aware Multimodal Backbone Evaluation and Ranking for UAV Weedy Rice Segmentation

Tarek Rahman; Nazim-E-Alam; Md Kishor Morol; Jannatun Noor

cs.CVSep 4, 2026

AAMBERS-UAV: Acquisition-Aware Multimodal Backbone Evaluation and Ranking for UAV Weedy Rice Segmentation

Authors: Tarek Rahman, Nazim-E-Alam, Md Kishor Morol, Jannatun Noor

Organizations: United International University · ELITE Research Lab, New York, USA · American International University Bangladesh

Abstract

UAV image collections contain spatially and temporally related frames, yet semantic-segmentation benchmarks commonly split them at image level. Such splitting can place samples from one acquisition in both model development and testing, obscuring transfer to a genuinely new survey. Using the 734-sample WeedyRice-RGBMS-DB, we fix a 124-image target-acquisition test set and compare two protocols with identical train, validation, and test counts: target-held-out, which excludes the target acquisition from development, and target-exposed, which admits its remaining images. SegFormer-B0 is evaluated with RGB, four-band multispectral (MS), and seven-channel RGB+MS input over two fixed-split seeds. RGB is strongest under complete acquisition holdout ( $0.7317\pm0.0201$ IoU), whereas RGB+MS becomes strongest after target exposure ( $0.7822\pm0.0269$ ). A fixed-split U-Net/ResNet18 replication confirms positive exposure gains for all three inputs, but retains RGB as the best modality under both protocols. Acquisition exposure therefore increases measured performance across both evaluated backbones, while its effect on modality ranking is architecture-dependent. A supplied-split audit reveals strong near-sequential dependence, and corruption tests show that early fusion is substantially more sensitive to RGB--MS displacement than to moderate radiometric scaling. These results support acquisition-aware same-test evaluation as a necessary complement to ordinary image-level splitting in multimodal UAV benchmarks. The code and supporting the findings of this study will be publicly released upon acceptance of the paper.

Figures & tables

Figure 1: Acquisition-level characteristics of WeedyRice-RGBMS-DB. The target acquisition differs from the source acquisitions in sample count, annotated weed coverage, and flight altitude, illustrating the acquisition-domain shift considered in this study.

Figure 2: Representative low-, medium-, and high-coverage samples with annotated weed coverage of 5.2%, 24.0%, and 72.4%, respectively. Columns show RGB, false-color NIR/Red Edge/Red, ground-truth mask, and mask overlay. The visual appearance and annotated prevalence vary substantially across the benchmark.

Figure 3: Dataset split and acquisition-aware development protocols. The fixed 124-image target test set is identical for both protocols; the remaining 123 target-acquisition images are available only in target-exposed development. Both protocols are evaluated on the same target test set across the three input configurations and both evaluated backbones.

Figure 4: Primary SegFormer protocol effect. In panel (a), thick lines show two-seed means and faint lines show the individual fixed-split seeds. In panel (b), colored points show per-seed exposure gains and hollow diamonds show the mean

\pm

seed standard deviation.

Measure	Validation / Test
Median same-date frame gap	1 frame
Within two training frames	98.65% / 100%
Median nearest train GPS	5.42 / 5.44 m

Table 1: Supplied-split dependence audit.

Protocol	Input	IoU	Dice	Precision	Recall	Specificity
Held-out	RGB	.7317 $\pm$ .0201	.8450 $\pm$ .0134	.8002 $\pm$ .0112	.8950 $\pm$ .0162	.6851 $\pm$ .0163
	MS	.7184 $\pm$ .0059	.8361 $\pm$ .0040	.7913 $\pm$ .0027	.8864 $\pm$ .0124	.6704 $\pm$ .0101
	RGB+MS	.7015 $\pm$ .0161	.8245 $\pm$ .0111	.7490 $\pm$ .0290	.9176 $\pm$ .0161	.5648 $\pm$ .0745
Exposed	RGB	.7666 $\pm$ .0328	.8677 $\pm$ .0210	.8259 $\pm$ .0351	.9144 $\pm$ .0037	.7269 $\pm$ .0653
	MS	.7348 $\pm$ .0142	.8471 $\pm$ .0094	.8287 $\pm$ .0045	.8663 $\pm$ .0149	.7476 $\pm$ .0036
	RGB+MS	.7822 $\pm$ .0269	.8777 $\pm$ .0170	.8479 $\pm$ .0216	.9097 $\pm$ .0116	.7697 $\pm$ .0355

Table 2: Full-resolution SegFormer performance on the identical 124-image target test set (mean

\pm

standard deviation over two fixed-split model seeds). Best IoU and Dice within each protocol are bold.

Backbone	Input	Held-out	Exposed	$\Delta$
SegFormer-B0	RGB	.7174	.7434	+.0260
	MS	.7142	.7248	+.0106
	RGB+MS	.7128	.7632	+.0504
U-Net/ResNet18	RGB	.7381	.7869	+.0489
	MS	.6937	.7403	+.0467
	RGB+MS	.7211	.7374	+.0164

Table 3: Fixed-split backbone replication at seed 17.

\Delta

is exposed minus held-out IoU.

Figure 5: Inference-only corruption of the seed-17 held-out SegFormer RGB+MS model. The dashed line marks clean IoU.

Figure 6: Coverage-stratified segmentation performance across low, medium, and high annotated weed-coverage groups. Points show mean image IoU, while whiskers indicate the reported cell range across the 12 primary SegFormer protocol–modality–seed cells.

Weed coverage	Images	Mean IoU	Cell range
$\leq 0.30$	8	0.564	0.442–0.675
0.30–0.60	62	0.662	0.598–0.737
$>0.60$	54	0.817	0.789–0.862

Table 4: Coverage-stratified image IoU for the primary SegFormer cells.

Figure 7: Backbone $\times$ exposure interaction on the identical target test set using the fixed split at seed 17. Target exposure improves all modality–backbone combinations, but the largest gain occurs for RGB+MS with SegFormer-B0 and for RGB with U-Net/ResNet18, demonstrating an architecture-dependent change in modality ranking.

Remote Sensing Image Segmentation

Remote sensing image segmentation labels land cover and objects in satellite, aerial, radar, and multispectral imagery.

See all 220 papers

BAWSeg: A UAV Multispectral Benchmark for Barley Weed Segmentation
Mar 2, 2026Haitian Wang, Xinyu Wang, Muhammad Ibrahim +2Remote Sensing Image Segmentation Agricultural Remote Sensing
Department of Computer Science and Software Engineering, The University of Western Australia, 35 Stirling Highway, Crawley, WA 6009, Australia · Department of Primary Industries and Regional Development (DPIRD), Government of Western Australia, 1 Nash St., Perth, WA 6000, Australia
PixDLM: A Dual-Path Multimodal Language Model for UAV Reasoning Segmentation
Apr 17, 2026Shuyan Ke, Yifan Mei, Changli Wu +4Image Segmentation Remote Sensing Image Segmentation
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, 361005, P.R. China · Shanghai Innovation Institute
Comparison of Deep Learning Frameworks For Rice Disease Mapping From UAV Multispectral Imaging
Jun 4, 2026Yadav Raj Ghimire, Jagrati Talreja, Tewodros Syum Gebre +3Image Segmentation Remote Sensing Image Segmentation
North Carolina A&T State University Greensboro, North Carolina, USA.

Mar 2, 2026J/K move · Enter open · S save

BAWSeg: A UAV Multispectral Benchmark for Barley Weed Segmentation

Haitian Wang, Xinyu Wang, Muhammad Ibrahim, Dustin Severtson+1

Department of Computer Science and Software Engineering, The University of Western Australia, 35 Stirling Highway, Crawley, WA 6009, Australia · Department of Primary Industries and Regional Development (DPIRD), Government of Western Australia, 1 Nash St., Perth, WA 6000, Australia

Remote Sensing Image Segmentation Agricultural Remote Sensing UAV Remote Sensing Remote Sensing Benchmarks

Open paper AskPDF arXiv

Accurate weed mapping in cereal fields requires pixel-level segmentation from UAV imagery that remains reliable across fields, seasons, and illumination. Existing multispectral pipelines often depend on thresholded vegetation indices, which are brittle under radiometric drift and mixed crop--weed pixels, or on single-stream CNN and Transformer backbones that ingest stacked bands and indices, where radiance cues and normalized index cues interfere and reduce sensitivity to small weed clusters embedded in crop canopy. We propose VISA, a two-stream segmentation network that decouples these cues and fuses them at native resolution. The radiance stream learns from calibrated five-band reflectance using local residual convolutions, channel recalibration, spatial gating, and skip-connected decoding, which preserve fine textures, row boundaries, and small weed structures that are often weakened after ratio-based index compression. The index stream operates on vegetation-index maps with windowed self-attention to model local structure efficiently, state-space layers to propagate field-scale context without quadratic attention cost, and Slot Attention to form stable region descriptors that improve discrimination of sparse weeds under canopy mixing. To support supervised training and deployment-oriented evaluation, we introduce BAWSeg, a four-year UAV multispectral dataset collected over commercial barley paddocks in Western Australia, providing radiometrically calibrated blue, green, red, red edge, and near-infrared orthomosaics, derived vegetation indices, and dense crop, weed, and other labels with leakage-free block splits. On BAWSeg, VISA achieves 75.6% mIoU and 63.5% weed IoU with 22.8 M parameters, outperforming a multispectral SegFormer-B1 baseline by 1.2 mIoU and 1.9 weed IoU. Under cross-plot and cross-year protocols, VISA maintains 71.2% and 69.2% mIoU, respectively.

Abstract

Remote Sensing Image Segmentation

Explore similar work

BAWSeg: A UAV Multispectral Benchmark for Barley Weed Segmentation

PixDLM: A Dual-Path Multimodal Language Model for UAV Reasoning Segmentation

Comparison of Deep Learning Frameworks For Rice Disease Mapping From UAV Multispectral Imaging