cs.CVAug 12, 2026
SaveAeroGround: A Comprehensive Benchmark for Aerial-Ground Collaborative Reasoning
Organizations: Shanghai Innovation Institute · College of Future Information Technology, Fudan University · The Chinese University of Hong Kong · College of Intelligent Robotics and Advanced Manufacturing, Fudan University · Chongqing Technology and Business University
Abstract
Vision-language models (VLMs) have been widely employed in understanding and reasoning tasks for unmanned aerial vehicles (UAVs). Existing UAV benchmarks primarily focus on aerial-view scenarios. However, whether current VLMs can perform well on understanding and reasoning tasks in aerial-ground collaborative scenarios which are practical in real-world applications like rescue and infrastructure inspection remains underexplored. To address this gap, we introduce AeroGround, a comprehensive benchmark for evaluating VLMs in aerial-ground collaborative reasoning. AeroGround is built upon a simulated aerial-ground dataset containing approximately 29,000 multimodal observation groups from diverse open environments, and provides 2,250 high-quality question-answering instances covering cross-view correspondence, spatial understanding, and reasoning. Experiments on 16 pretrained VLMs, together with two domain-adapted variants, reveal a substantial gap between current models and human performance: the best model achieves an average accuracy of 54.4%, whereas humans reach 93.3%. By systematically revealing the strengths and limitations of existing models in aerial-ground collaborative reasoning, AeroGround provides a foundation for developing more capable aerial-ground collaborative embodied intelligence systems.
Figures & tables
Figure 1: Overview of the aerial-ground dual-view benchmark. (a) Multi-view aerial-ground scene observations. (b) Examples of three directional subject types. (c) Three-level task taxonomy with 11 tasks.
Figure 2: Overview of the AeroGround benchmark task taxonomy. AeroGround is organized into three capability groups: object alignment, spatial understanding, and spatial reasoning, covering representative aerial–ground matching, localization, relation, and reasoning tasks.
| Obj. Align. | Spatial Underst. | Spatial Reason. | ||||||||||
| Model | Avg. | Single | Multi | Rel. | Dist. | Count | Ori. | Loc. | Occ. | ViewTr. | RevSeq. | Traj. |
| Reference | ||||||||||||
| Random Baseline | 24.6 | 23.6 | 0.0 | 35.0 | 23.2 | 15.6 | 23.3 | 22.7 | 34.4 | 32.8 | 22.5 | 23.0 |
| Human Level | 93.3 | 100.0 | 93.3 | 83.3 | 93.3 | 93.3 | 96.7 | 90.0 | 86.7 | 96.7 | 96.7 | 96.7 |
| Closed-source models | ||||||||||||
| GPT-5.4 | 40.1 | 37.4 | 14.7 | 68.9 | 48.8 | 39.1 | 30.0 | 38.0 | 42.2 | 33.9 | 41.2 | 26.0 |
Table 1: Main benchmark results. Entries are accuracies (%); higher is better. Avg. is computed over all questions. Bold and underline indicate the best and second-best task scores within each model group, respectively.
Figure 3: Directional subject analysis. (a) Chance-adjusted bias between Ground-to-Aerial and Aerial-to-Ground tasks. (b) Chance-adjusted accuracy across directional subject groups.
Figure 4: View configuration analysis. Accuracy across aerial-view modalities and front-view side relations.
Figure 5: Representative failure cases in aerial-ground dual-view reasoning, covering cross-view correspondence, visibility reasoning, reference-frame transformation, and scene registration.
Appendix figures & tables14 assets
Supplementary material from the paper’s appendix.
Appendix
| Scene type | Aerial | Ground | Total |
|---|---|---|---|
| City | 8,686 | 6,756 | 15,442 |
| Town | 5,664 | 4,499 | 10,163 |
| Factory | 248 | 825 | 1,073 |
| Desert | 491 | 355 | 846 |
| Forest | 246 | 1,407 | 1,653 |
| Total | 15,335 | 13,842 | 29,177 |
Table 2: Composition of the raw simulated AeroGround data pool by scene type.
| Capability level | Task category | Core question template |
| Object Alignment | Single Target Matching | Select the candidate target in [target view] that corresponds to the marked target in [source view]. |
| Multi-Target Matching | Determine the one-to-one correspondence from to the correct candidate labels. | |
| Spatial Understanding | Relative Distance Comparison | Order from nearest to farthest relative to [agent position]. |
| Multi-Target Relation | Order the comparison targets from nearest to farthest relative to reference target . | |
| Region Counting | Count how many [category] instances lie inside the marked aerial region. | |
| Relative Orientation Determination | Select the egocentric quadrant of [aerial/teammate direction] relative to the ground agent’s facing direction. |
Table 3: Task taxonomy and core question templates of the AeroGround Benchmark. Task names are consistent with Figure 1 (c).
Figure 6: Task category distribution of the AeroGround Benchmark.
Figure 7: Statistics of aerial–ground viewpoint relations in AeroGround.
| Output format | Parsing rule | Correctness criterion |
|---|---|---|
| Multiple-choice option | Extract one valid option letter from the allowed option set. | Correct only if the option equals the ground-truth option. |
| Multi-target matching template | Normalize the response and extract the complete one-to-one correspondence. | Correct only if the full mapping exactly matches the ground-truth mapping. |
| Invalid or ambiguous response | No valid answer, multiple conflicting answers, incomplete mapping, or unconstrained explanation. | Always scored as incorrect. |
Table 4: Answer parsing and scoring rules used in AeroGround.
| Item | Setting |
|---|---|
| Fine-tuning method | LoRA with SWIFT |
| Trainable modules | LoRA adapters on eligible linear layers |
| Frozen modules | Visual encoder and visual-language alignment layers |
| LoRA rank | 32 |
| LoRA scaling factor | 64 |
| LoRA dropout | 0.05 |
Table 5: Hyperparameters for supervised fine-tuning of Qwen3-VL models on AeroGround.
| Model | Aligned Sim. | Real Avg. | Single | Multi | Loc. | Ori. | Occ. | ViewTr. | Traj. |
|---|---|---|---|---|---|---|---|---|---|
| No. questions | – | 300 | 66 | 50 | 30 | 24 | 50 | 50 | 30 |
| Doubao-seed-2.0-pro | 46.8 | 45.7 | 59.1 | 30.0 | 40.0 | 41.7 | 56.0 | 50.0 | 26.7 |
| Gemini-3.1-Pro | 39.8 | 44.0 | 56.1 | 36.0 | 36.7 | 54.2 | 60.0 | 32.0 | 23.3 |
| Qwen3-VL-32B-Instruct | 29.7 | 34.3 | 50.0 | 12.0 | 56.7 | 25.0 | 38.0 | 34.0 | 16.7 |
| Qwen3-VL-8B-Instruct | 27.7 | 33.7 | 50.0 | 10.0 | 50.0 | 0.0 | 40.0 | 36.0 | 33.3 |
| Spatial-SSRL-Qwen3VL-4B | 27.6 | 31.7 | 50.0 | 2.0 | 26.7 | 25.0 | 44.0 | 30.0 | 33.3 |
Table 6: Results on the 300-question real-world pilot set. Aligned Sim. denotes the AeroGround score recomputed over the same seven tasks using the real-world task distribution. The remaining task columns report real-world accuracies (%). Bold and underline indicate the best and second-best overall scores, respectively.
Figure 8: Human–model task difficulty association. Each point denotes one task, comparing human accuracy with the accuracy of Doubao-seed-2.0-pro. The plot shows that model difficulty does not fully align with human difficulty, indicating a clear gap between human spatial understanding and current model capability.
Figure 9: Effect of aerial–ground spatial span on model accuracy. Spatial span is measured by the 3D distance between the aerial viewpoint and the ground viewpoint. Samples are grouped into coarse 20-meter distance bins.
Figure 10: Effect of target pixel area on model accuracy. We analyze five target-size-related tasks: single-object matching, multi-object matching, relative distance comparison, multi-object relation reasoning, and occluded object judgment. Left: the overall accuracy trend across target pixel area intervals. Right: task-level accuracy trends across the same intervals.
Figure 11: Task association analysis. The heatmap reports model-level Pearson correlations between task accuracies, revealing which task types tend to share similar model strengths and weaknesses.
| Obj. Align. | Spatial Underst. | Spatial Reason. | ||||||||||
| Model | Avg. | Single | Multi | Rel. | Dist. | Count | Ori. | Loc. | Occ. | ViewTr. | RevSeq. | Traj. |
| Reference | ||||||||||||
| Human Level | 91.1 | 100.0 | 93.3 | 74.3 | 91.3 | 92.1 | 95.7 | 87.1 | 79.7 | 95.1 | 95.7 | 95.7 |
| Closed-source models | ||||||||||||
| GPT-5.4 | 20.6 | 18.1 | 14.7 | 52.2 | 33.3 | 27.8 | 8.7 | 19.8 | 11.9 | 1.6 | 24.1 | 3.9 |
| Gemini-3.1-Pro | 30.8 | 17.5 | 25.9 | 67.5 | 62.5 | 9.2 | 8.7 | 33.6 | 3.4 | 56.8 | 16.1 | -2.6 |
Table 7: Chance-adjusted benchmark results after removing the effect of the random-choice baseline. Values are reported as CAA scores in percentage form, following the definition introduced in the evaluation protocol. A score of 0 corresponds to random-choice performance, while negative values indicate performance below the random baseline. Bold and underline indicate the best and second-best task scores within each model group, respectively.
| Asset | Creator / Owner | Usage | License / Terms |
| Unreal Engine | Epic Games | Simulation and rendering engine | Unreal Engine EULA |
| AirSim | Microsoft | Vehicle and camera simulation plugin/API | MIT License |
| Brushify - Urban Buildings Pack | Brushify Ltd. | Urban building and environment assets | Epic Content License Agreement / UE Marketplace Standard License |
| Downtown West Modular Pack | PurePolygons | Modular urban scene assets | Epic Content License Agreement / UE Marketplace Standard License |
| Factory Pack Vol.1 | MeikWModels | Industrial factory scene assets | CC BY 4.0 |
| Metaverse Villa City Modular | ErikGames | Modular villa/city scene assets | Epic Content License Agreement / UE Marketplace Standard License |
Table 8: Summary of major third-party assets, software, and datasets used in AeroGround.
Figure 12: Example interface used for human evaluation. Participants are shown the task instruction, candidate answers, paired aerial and ground-view observations, and an answer panel. Hidden reference information is not displayed during evaluation.