AeroIntent: A Synchronized Multimodal Benchmark for UAV Mission Intent Recognition

Yutong Chen; Jingxin Mao; Zhiwei Wei; Bing Li; Jianzhong Ni; Rongqing Zhang

cs.ROOct 19, 2026

AeroIntent: A Synchronized Multimodal Benchmark for UAV Mission Intent Recognition

Authors: Yutong Chen, Jingxin Mao, Zhiwei Wei, Bing Li, Jianzhong Ni, Rongqing Zhang

Organizations: Y. Chen, Z. Wei, and B. Li are with Tongji University. · J. Mao and R. Zhang are with The Hong Kong University of Science and Technology (Guangzhou). · J. Ni is with Nanchang Highly Electrical Appliances Co., Ltd.

Abstract

Reliable recognition of mission-level UAV intent from multimodal observations is essential for low-altitude airspace surveillance and security. Existing multimodal UAV datasets primarily support detection, tracking, and localization, leaving limited support for evaluating mission-intent recognition across held-out trajectory programs and incomplete radar histories. We introduce AeroIntent, a simulation-based benchmark comprising 990 synchronized multimodal sequences generated by 22 trajectory programs across five mission intents. The benchmark provides radar, RGB, infrared, and depth observations, together with program-disjoint, radar-gap, and observation-prefix evaluation protocols. Building on this benchmark, we propose AeroMIR, a structured Multimodal Intent Reasoner that integrates radar trajectories, derived behavioral features, selected visual evidence, and task knowledge for candidate-based UAV intent recognition. Under a strict program-disjoint protocol evaluating unseen tactical realizations, AeroMIR achieves a mean accuracy of 71.56% and a mean Macro-F1 of 69.84% across three seeds, outperforming the strongest trajectory-only baseline by 7.79 Macro-F1 points and a conventional radar-RGB late-fusion baseline by 19.32 points. AeroMIR also demonstrates strong performance under controlled radar gaps and early intent recognition settings. AeroIntent and AeroMIR provide a reproducible foundation for multimodal UAV intent recognition. The benchmark is available at: https://anonymous.4open.science/r/AeroIntent

Figures & tables

Fig. 1: Observation scenario and key challenges of UAV intent recognition. A fixed observation station combines radar measurements and RGB imagery to infer the intent of a UAV near a monitored building. The three research questions concern generalization to unseen trajectories expressing the same intent (Q1), robustness to incomplete radar observations (Q2), and early recognition from short observation prefixes (Q3).

Fig. 2: Overview of the AeroIntent benchmark. The four panels summarize scene diversity, airframe-payload configurations, mission-intent diversity, and synchronized multimodal observations. The benchmark covers three environments, 15 UAV configurations, and five mission intents implemented through 22 trajectory programs. The temporal panel illustrates RGB, infrared, depth, and echo-radar observations at five representative timestamps. The highlighted radar-gap interval indicates missing radar observations while visual observations remain available. All streams are saved at 10 Hz with shared frame IDs.

Intent	Programs	Train	Val	Test
Transit	6	180	45	45
Reconnaissance	6	180	45	45
Probe	3	45	45	45
Strike	5	135	45	45
Escape	2	36	9	45
Total	22	576	189	225

TABLE I: Program counts and sequence allocation per seed. The test contains one unseen program per class.

Fig. 3: Overview of the AeroMIR. Radar sequences, radar-derived behavioral features, and a gap-guided RGB frame are assembled with task definitions into a structured multimodal prompt. A vision-language model adapted with language LoRA scores five candidate intents using mean token log-likelihood, and the highest-scoring candidate is selected. Solid and dashed arrows indicate inference and training-only operations, respectively.

Method	Input	Accuracy	Macro-F1
MLP	R	$58.37\pm 9.27$	$53.60\pm 8.58$
LSTM	R	$53.48\pm 11.34$	$50.84\pm 9.12$
GRU	R	$60.00\pm 4.04$	$55.42\pm 3.06$
Attn-TCN-BiGRU [ 1 ]	R	$67.85\pm 5.10$	$62.05\pm 5.95$
BLAC-like [ 2 ]	R	$49.78\pm 8.20$	$45.96\pm 5.66$
B2L-lite [ 4 ]	R	$51.70\pm 2.30$	$48.08\pm 4.25$

TABLE II: Intent recognition under unseen tactical realizations (%, three-seed mean

\pm

population standard deviation). R: radar; V: RGB; B: radar-derived behavior. ZS uses no fine-tuning and receives ground-truth payload.

Fig. 4: Intent recognition under unseen tactical realizations. Mean row-normalized confusion matrices across three program-disjoint splits for eight representative baselines and AeroMIR. Rows denote ground-truth intents and columns denote predicted intents.

Fig. 5: Contribution of each evidence source. Candidate-CE ablation on the split generated with seed 42. Bars report Macro-F1, and filled circles indicate the included representations. a–f correspond to R, B, V, R+B, R+V, and R+B+V, respectively, where R denotes radar trajectories, B denotes radar-derived behavioral descriptors, and V denotes RGB evidence.

Fig. 6: Controlled radar-gap recognition. Three-seed mean Macro-F1 for eight baselines and AeroMIR under contiguous radar deletions covering 0–40% of the recording duration. The remaining radar observations are resampled to 30 slots, while RGB observations remain uncorrupted. AeroMIR and Radar–RGB Late Fusion apply gap-guided RGB selection to the remaining radar observations; the Lei-style zero-shot reference always uses the first RGB frame. Gap0 matches the main experiment.

Fig. 7: Early intent recognition. Three-seed mean Macro-F1 at 25%, 50%, 75%, and 100% observation-slot coverage for eight baselines and AeroMIR. Partial radar prefixes are resampled to 30 slots. AeroMIR and Radar–RGB Late Fusion apply gap-guided RGB selection within the observed prefix. The Lei-style zero-shot reference uses the first RGB frame of the full recording at every prefix.

Method	Gap40	Gap AUC	Early AUC
MLP	48.58	50.94	49.25
LSTM	43.39	45.78	45.56
GRU	47.63	51.21	48.22
Attn-TCN-BiGRU [ 1 ]	53.20	57.67	54.87
BLAC-like [ 2 ]	44.98	44.96	43.54
B2L-lite [ 4 ]	45.35	46.30	46.02

TABLE III: Temporal evidence summary (mean Macro-F1, %). AUC is the trapezoidal area normalized by the tested interval: gap0–40 or P25–100.

Multimodal Sensor Fusion

Multimodal sensor fusion combines data from different sensors to improve perception, tracking, localization, and prediction.

See all 418 papers

MulRobBench: A Decision-Level Benchmark for Safe and Security-Policy-Compliant Multimodal UAV Agents
Jul 26, 2026Belal S. Alsinglawi, Weizheng Wang, Junyi Wu +3Aerial Robotics AI Agent Security Benchmarks
College of Technological Innovation, Zayed University, Abu Dhabi, United Arab Emirates
Knowing the Self, Understanding the World: A Dual-Cognition Benchmark for UAV Spatio-temporal Reasoning with MLLMs
Jul 17, 2026Like Liu, Zhengzheng Xu, Haitao He +3Multimodal Grounding Aerial Robotics
Northwestern Polytechnical University Xi’an, Shaanxi, China
UAVBench and UAVIT-1M: Benchmarking and Enhancing MLLMs for Low-Altitude UAV Vision-Language Understanding
Mar 15, 2026Yang Zhan, Yuan YuanVision-Language Models VLM Evaluation
School of Artificial Intelligence, Optics, and Electronics (iOPEN), Northwestern Polytechnical University, Xi’an 710072, China.

Jul 26, 2026J/K move · Enter open · S save

MulRobBench: A Decision-Level Benchmark for Safe and Security-Policy-Compliant Multimodal UAV Agents

Belal S. Alsinglawi, Weizheng Wang, Junyi Wu, Lianhai Lin+2

College of Technological Innovation, Zayed University, Abu Dhabi, United Arab Emirates

Aerial Robotics AI Agent Security Benchmarks Robot Safety AI Agent Safety Benchmarks Robustness of VLA Models

Open paper AskPDF arXiv

In IoT-enabled smart-city settings, Uncrewed Aerial Vehicles (UAVs) are evolving from passive sensing platforms into cyber-physical decision makers that must respect operational rules under degraded observations and ambiguous language. Existing UAV and multimodal benchmarks cover aerial perception, navigation, collaboration, and task reasoning, but rarely test whether physical evidence, protocol constraints, and action risk stay coupled at critical decisions. We introduce MulRobBench, an offline, protocol-conditioned benchmark for Vision-Language-Action (VLA) UAV agents that links real UAV multimodal observations, protocol-level security-policy constraints, and action-level cyber-physical safety within an auditable decision contract. The evaluation set contains 3,024 samples spanning 17 task-taxonomy nodes and 12 metric scoring dimensions, organized around context understanding, multimodal evidence arbitration, degradation-aware reasoning, and risk-aware action planning. MulRobBench reports controlled semantic scores alongside strict structural diagnostics for policy compliance, formatting, unsafe actions, parsing, and dimension-level validity. Across 17 uniformly audited models, the best semantic protocol-decision score reaches 0.5141 and the best strict mean scoring-dimension accuracy reaches 0.1599. A matched 20-anchor modality-removal study changes 4-15 action selections per model, showing both visual and textual inputs influence decisions while the strongest input condition varies across metrics. Per-dimension and conditional analyses identify modality-trust selection, constraint extraction, strong glare, missing data, and high-entropy operator shorthand as principal sources of action instability. The central challenge is thus stable coupling of degraded evidence, security-policy constraints, and risk-bearing action, not isolated scene recognition.

Abstract

Multimodal Sensor Fusion

Explore similar work

MulRobBench: A Decision-Level Benchmark for Safe and Security-Policy-Compliant Multimodal UAV Agents

Knowing the Self, Understanding the World: A Dual-Cognition Benchmark for UAV Spatio-temporal Reasoning with MLLMs

UAVBench and UAVIT-1M: Benchmarking and Enhancing MLLMs for Low-Altitude UAV Vision-Language Understanding