Mobile robotic sensing systems must adapt to operator requirements, limited resources, changing connectivity, and observation opportunities shaped by physical motion. Evaluating runtime strategies therefore requires assessing whether their decisions fulfill mission objectives, beyond isolated perception accuracy or execution efficiency. We present EmbodiedIntentBench, a benchmark for intent-driven runtime adaptation in mobile robotic missions. It represents mission intent through structured contracts for coverage, temporal evidence, reporting, urgency, and resources, and couples simulated unmanned aerial vehicle (UAV) workloads with causal, resource-accounted execution. Computation decisions affect mission time, resource use, and which observations can contribute evidence. We evaluate fixed and adaptive perception strategies in multi-phase missions and under changing urgency, connectivity, and battery conditions. In the multi-phase Depot study, a configuration with 87.5% full-bank target recall across visible target observations completes only 1 of 16 missions, whereas a lower-recall configuration completes 14. In an ablation, ignoring execution-induced observation loss increases aggregate mission success and reverses the ordering of two runtime strategies. Replays using Jetson AGX Orin CPU and GPU profiles change the relative success of local and remote strategies. In selected Jetson GPU-profile replays, intent-driven adaptation reduces modeled task energy by up to 28% with unchanged mission success. EmbodiedIntentBench provides an extensible framework for evaluating runtime decisions against mission fulfillment under physical and resource constraints.
Figures & tables
Figure 1. Motivation and overview of EmbodiedIntentBench . (a) Operator-specified missions impose requirements for target coverage, evidence verification, timeliness, and resource use. (b) Local, split, and remote execution offer different computation and communication tradeoffs; the current evaluation considers whole-model local and remote execution. (c) Perception quality and execution latency influence mission evidence, as longer processing times can prevent subsequent observations from being processed. (d) EmbodiedIntentBench connects structured mission requirements with physically situated, resource-accounted execution and mission-level evaluation.
Figure 2. Architecture of EmbodiedIntentBench . (a) Intent specification: An operator request ( 1 ) is represented through a structured mission contract ( 2 ) defining evidence, temporal, reporting, and resource requirements. (b) Mission workloads and runtime decisions: Physically situated observations ( 3 ) and current system state inform a runtime policy ( 4 ) that selects a computation configuration. (c) Causal execution and resource accounting: A resource-accounted executor ( 5 ) applies the selected computation and its associated costs, while the execution clock and resource state ( 6 ) determine result availability and subsequent observation opportunities. (d) Mission evidence and evaluation: Admitted results contribute to evidence accumulation and reporting ( 7 ), while a mission evaluator ( 8 ) assesses fulfillment against the contract. Solid arrows indicate runtime execution and information flow, including state feedback; red dashed arrows indicate private ground truth accessible only to the evaluator.
Component
Required inputs
Extension mechanism
Mission specification
Structured goals, evidence, timing, reporting, and resource requirements.
Add mission contracts and corresponding evaluation rules.
Physical workload
Timestamped observations, vehicle poses, and private evaluation labels.
Import new recordings, environments, and trajectories.
Perception configuration
Prediction outputs, preprocessing metadata, and execution profiles.
Register new models and configuration-specific costs.
Runtime policy
Public contract, eligible observation, resource state, and selected history.
Implement fixed or adaptive computation-selection strategies.
Execution backend
Selected actions, processing stages, network conditions, and cost profiles.
Implement alternative execution and resource-accounting mechanisms.
Evidence and evaluation
Completed predictions, evidence history, mission contract, and evaluator truth.
Replace association and verification stages or introduce task-specific scorers.
Table 1. Implementation interfaces and extension mechanisms in EmbodiedIntentBench . Components can be replaced independently, subject to their input requirements and corresponding execution or evaluation rules.
Figure 3. Physical workloads and mission scenarios in EmbodiedIntentBench . (a) Wide-area Lake environment containing 24 people across eight sectors, used for the dynamic-urgency study. (b) Depot warehouse environment. (c) Recorded UAV observations illustrating variations in target visibility. (d) Illustrative wide-area sector circuit, shown for spatial context rather than as an evaluated trajectory. The multi-phase Lake study uses separate six-target corridor layouts, whereas dynamic-urgency recordings comprise local passes near focal sectors. Recorded trajectories remain fixed within each runtime-policy comparison.
Cohort
Sim. flights
Duration
Conditions
Policies
Runs
Varied factors
Lake multi-phase
6
120 s
24
21
504
Mission contract and initial battery; local perception selection.
Depot multi-phase
4
120 s
16
22
352
Mission contract and initial battery; local perception selection.
Lake dynamic urgency
6
42 s
72
9
648
Request urgency, modeled connectivity, and battery; local/remote model routing.
Table 2. Experimental cohorts and controlled factors. A condition combines one simulated recording with the specified mission and operating settings. Every policy is evaluated once per condition; the three cohorts are analyzed separately.
Figure 4. Perception quality and mission fulfillment across nine configurations in Depot. (a) Full-bank target recall and completed missions for fixed perception configurations, highlighting the discrepancy between Mask R-CNN union (0.875 recall; 1/16 missions) and tiled YOLO11s (0.807 recall; 14/16 missions). (b) Frozen predicted and observed full-bank target recall across configurations, illustrating variation in quality-estimation accuracy. Predicted recall is averaged over captures with visible-target counts as weights for offline analysis. Recall is evaluated over 680 visible target observations from 240 potential captures; mission success uses 16 resource-accounted execution conditions.
Fixed configuration
Recall
Success
Phases
Reserve
Mask R-CNN union
0.875
1/16
22/64
8/16
YOLO11s, full frame
0.818
12/16
58/64
16/16
YOLO11s, tiled
0.807
14/16
60/64
16/16
YOLO11s, tiled + 90∘
0.785
14/16
62/64
16/16
Table 3. Perception quality and mission fulfillment in Depot. Full-bank recall and mission success are reported alongside completed mission phases and battery-reserve compliance for representative fixed configurations. Recall uses 680 visible target observations across 240 potential captures; mission outcomes use 16 conditions, each comprising four required phases.
Figure 5. Runtime adaptation under changing urgency and resource conditions. (a) Mission success, late perception requests, and mean onboard task energy for urgency-aware and urgency-blind quality-and-risk routing across 48 matched conditions. (b) Mission success under favorable and degraded connectivity, comparing fixed remote execution with urgency-aware routing. (c) Mission success under ample and constrained battery conditions, comparing fixed local execution with urgency-aware routing. Connectivity and battery comparisons use 24 conditions per operating setting. Energy excludes flight consumption.
Figure 6. Impact of causal execution on mission outcomes and runtime strategy comparisons. (a) Mission successes under causal and no-skip execution across four inference-latency multipliers, using 648 matched policy–condition executions per setting. (b) Mission successes for urgency-aware quality routing and fixed remote Mask R-CNN under causal and no-skip execution. The relative advantage reverses from 41/72 versus 38/72 under causal execution to 45/72 versus 48/72 under no-skip execution. (c) Observation availability in an illustrative fixed-local Mask R-CNN execution. Causal processing admits 9 of 18 captures, compared with 18 under no-skip execution. Markers show capture processing, person-3 matches, and person-level report times.
Benchmark
Motion
Adapt.
View
Link
Energy
Evidence
Mission
MAVBench ( Boroujerdian et al., 2018 )
✓
–
✓
–
✓
–
–
Air Learning ( Krishnan et al., 2021a )
✓
–
✓
–
✓
–
–
OpenUAV ( Wang et al., 2024 )
✓
–
–
–
–
–
–
BEDI ( Guo et al., 2026 )
✓
–
–
–
–
–
–
UAVBench ( Ferrag et al., 2026 )
–
–
–
–
–
–
–
FloodReasonBench ( Bhattacharjya et al., 2026b )
–
–
–
–
✓
–
–
Table 4. Comparison with representative UAV, embodied, and robotic computing benchmarks. Motion: task-level evaluation involving simulated or physical robot motion; Adapt.: runtime inference scheduling or inference-configuration selection; View: inference timing affects observation availability or freshness; Link: network-aware computation placement; Energy: energy or power accounting; Evidence: temporal confirmation and reporting requirements; Mission: conjunctive evaluation of evidence, timing, reporting, and resource requirements. Checkmarks (✓) denote evaluated capabilities; dashes (–) indicate features outside the reported evaluation scope.
State Key Laboratory of Multimedia Information Processing, School of Computer2026 Science, Peking University · Institute for Brain and Intelligence, Fudan University · University of Science andJul Technology Beijing +2