Organizations: School of Computing Sciences and Computer Engineering, The University of Southern Mississippi, Hattiesburg, MS 39406 USA. · Electrical Engineering Department, University of South Carolina, Columbia, SC, USA. · Computer Science Department, Faculty of Computers & Artificial Intelligence, Fayoum University, Fayoum, Egypt.
The deployment of lightweight segmentation models on drones for autonomous power line inspection presents a critical challenge: maintaining reliable performance under real-world conditions that differ from training data. Although compact architectures such as U-Net enable real-time onboard inference, their segmentation outputs can degrade unpredictably in adverse environments, raising safety concerns. In this work, we study the feasibility of using a large language model (LLM) as a semantic judge to assess the reliability of power line segmentation results produced by drone-mounted models. Rather than introducing a new inspection system, we formalize a watchdog scenario in which an offboard LLM evaluates segmentation overlays and examine whether such a judge can be trusted to behave consistently and perceptually coherently. To this end, we design two evaluation protocols that analyze the judge's repeatability and sensitivity. First, we assess repeatability by repeatedly querying the LLM with identical inputs and fixed prompts, measuring the stability of its quality scores and confidence estimates. Second, we evaluate perceptual sensitivity by introducing controlled visual corruptions (fog, rain, snow, shadow, and sunflare) and analyzing how the judge's outputs respond to progressive degradation in segmentation quality. Our results show that the LLM produces highly consistent categorical judgments under identical conditions while exhibiting appropriate declines in confidence as visual reliability deteriorates. Moreover, the judge remains responsive to perceptual cues such as missing or misidentified power lines, even under challenging conditions. These findings suggest that, when carefully constrained, an LLM can serve as a reliable semantic judge for monitoring segmentation quality in safety-critical aerial inspection tasks.
Figures & tables
Fig. 1 : System overview of the proposed LLM-as-Judge framework for powerline inspection. UAV RGB frames are segmented onboard and streamed as mask–image overlays to a ground station for compute offload. A multimodal LLM produces a discrete quality score si , confidence estimate ci , and textual explanation ei for real-time safety monitoring. Offline repeatability and scenario-based sensitivity analyses evaluate output stability and robustness under controlled corruptions.
Fig. 2 : Examples of clean images and synthetically corrupted variants used to construct the challenge set. Each column corresponds to a corruption type and each row shows a different image instance.
TABLE I : Repeatability metrics over 5 runs under different segmentation corruption conditions (schema-constrained).
corruption
severity
mean_ds
std_ds
ci95_ds_lo
ci95_ds_hi
mean_dc
std_dc
ci95_dc_lo
ci95_dc_hi
dz_score
dz_conf
fog
1
3.124
0.912
2.991
3.249
0.736
0.213
0.705
0.762
3.426
3.453
fog
2
3.143
0.949
3.005
3.263
0.741
0.211
0.711
0.768
3.311
3.509
fog
3
3.115
0.918
2.982
3.235
0.732
0.214
0.701
0.759
3.393
3.419
rain
1
0.465
0.828
0.350
0.571
0.052
0.149
0.030
0.071
0.562
0.346
rain
2
0.581
0.760
0.479
0.677
0.067
0.120
0.050
0.082
0.764
0.558
rain
3
0.806
0.871
0.687
0.922
0.109
0.166
0.085
0.131
0.926
0.654
TABLE II : Sensitivity Statistics of LLM-as-Judge Scores and Confidence Under Controlled Visual Corruptions
Fig. 3 : Visualization of sensitivity across corruption types.
Large language model reliability studies whether outputs remain correct, calibrated, robust, and controllable across prompts, contexts, and deployment conditions.
Vision-language models (VLMs) are often reported to outperform task-specific vision backbones for unmanned aerial vehicle (UAV) power-line defect assessment. We test that claim on ElecVQA-Bench, a 56,972-item benchmark derived from the public InsPLAD dataset, across six evaluation choices: partition, evaluated item set, label space, replication, input resolution, and side information. On a matched partition, a Swin Transformer and the strongest adapted VLM differ by only 0.03 points at binary screening. At seven-way defect typing, increasing the vision backbones from 224 px to the measured pixel budget of the VLM preprocessor narrows the gap against InternVL3.5-8B from +20.53 to -0.57 points for ResNet-50 and from +23.67 to +4.70 points for Swin-T. A pixel-budget audit shifts Qwen3-VL-8B macro recall by 10.78 points, yet a source-pixel-matched InternVL control still leaves Qwen ahead by 7.43 to 13.61 points while using 56% fewer visual tokens, so neither source pixels nor token budget explains the difference between the two VLMs. A two-seed global replication changes Qwen binary accuracy and seven-way macro recall by 0.86 and 1.02 points. After split-specific retraining, Qwen does not lead at crop or image level, and a 14-tower, three-seed replication reverses the sign across seeds, giving mean common-six macro recall of 0.9085 for Qwen against 0.9509 for ResNet-50. No split regime yields a family-level advantage that survives multiple-comparison correction. The study supports a benchmark-audit contribution rather than a general claim of VLM superiority.
Linghao Zhang, Siyu Xiang, Junwei Kuang +1
State Grid Sichuan Electric Power Research Institute, Chengdu 610041, China · Power System Security and Operation Key Laboratory of Sichuan Province
Reasoning segmentation has recently expanded from ground-level scenes to remote-sensing imagery, yet UAV data poses distinct challenges, including oblique viewpoints, ultra-high resolutions, and extreme scale variations. To address these issues, we formally define the UAV Reasoning Segmentation task and organize its semantic requirements into three dimensions: Spatial, Attribute, and Scene-level reasoning. Based on this formulation, we construct DRSeg, a large-scale benchmark for UAV reasoning segmentation, containing 10k high-resolution aerial images paired with Chain-of-Thought QA supervision across all three reasoning types. As a benchmark companion, we introduce PixDLM, a simple yet effective pixel-level multimodal language model that serves as a unified baseline for this task. Experiments on DRSeg establish strong baseline results and highlight the unique challenges of UAV reasoning segmentation, providing a solid foundation for future research.
Shuyan Ke, Yifan Mei, Changli Wu +4
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, 361005, P.R. China · Shanghai Innovation Institute
Large Language Models (LLMs) increasingly support Uncrewed Aerial Vehicle (UAV) swarm operations such as data collection scheduling, where the model reads structured sensor reports and decides which sensors to visit. An adversary who quietly manipulates those reports can redirect the swarm without modifying the model weights or the UAV. Defenses for this interface have been proposed architecturally but rarely implemented or evaluated. We implement and evaluate defense-in-depth at the perception-reasoning interface of LLM-Centric Agentic UAV Swarms. Five layers check the provenance of a report, whether its values are physically admissible, whether they agree with what swarm geometry and service history predict, whether the resulting schedule starves any sensor, and, when these fail, hand control to a deterministic scheduler that ignores the suspect input. We test each layer against an adversary strong enough to defeat the layer before it. For each of the three input-side layers, we derive in closed form how far a report can be distorted before that layer reacts, fixing each boundary from deployment parameters before any attack data is collected; across thirty matched simulation runs, predicted and measured boundaries agree. Separating attack detection from response is a well-established principle, and we quantify the cost of neglecting this distinction at the perception-reasoning interface. When the system rejects a report, it replaces it with the most recent accepted report. This prevents the adversary from controlling the UAV schedule, but it also increases cumulative cost by 79% and 74% for the two detectors, respectively, compared with the undefended system. The safety check does not detect any attacks, but it nevertheless reduces the attack-induced cost by 37.5%.