Organizations: Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · Beijing Academy of Artificial Intelligence
Active embodied question answering requires agents to acquire evidence through viewpoint changes and physical interaction. The central challenge is to avoid undirected interaction: taking plausible actions without specifying what evidence they should reveal for answering the question. Action plans and accumulated histories alone can leave the informational purpose of an interaction implicit. We introduce EviSeek, a training-free framework that makes question-derived evidence demands explicit acquisition objectives, linking observation-conditioned actions and selectively retained evidence to the information the agent seeks. On 3,436 ESI-Bench questions, EviSeek improves accuracy over the Single-model agent by 7.1 percentage points with Qwen3-VL-32B and 3.7 points with Gemini 3.1 Pro, while improving visual-support-weighted accuracy and reducing mean token usage. These findings support making the informational purpose of interaction explicit, with action selection and evidence retention organized around what the agent needs to learn to answer the question.
Figures & tables
Figure 1: EviSeek connects question-derived evidence demands to acquisition goals, observation-conditioned actions, and retained evidence for answering. Unselected observations support control but are excluded from the evidence board.
Category
#Q
Qwen3-VL-32B
Gemini 3.1 Pro
Passive
Single-model
EviSeek
Passive
Single-model
EviSeek
Perceptual Grounding
750
58.0
62.3
78.5
75.9
80.0
84.9
Physical Structure
254
55.9
52.4
50.4
62.2
69.3
70.5
Physical Dynamics
150
32.7
65.3
53.3
40.0
64.0
68.0
Specular Reflection
300
49.3
44.0
44.3
47.7
54.0
49.7
Spatial Relations
781
27.9
28.3
46.2
27.8
36.9
46.6
Table 1: ESI-Bench accuracy (%) on 3,436 questions. #Q is task size; Overall is question-weighted. Bold marks the best result per backbone.
Figure 2: Correctness and visual-support decomposition on ESI-Bench (%). C and GW are support-weighted correct and incorrect mass; P and N are their residuals.
Method
Steps
Calls
Tokens
Images
Qwen3-VL-32B
Single-model
18.79
20.55
139,380
5.91
EviSeek
18.11
12.69
76,470
4.39
Gemini 3.1 Pro
Single-model
10.71
12.15
86,887
5.24
EviSeek
12.17
10.05
74,839
4.38
Table 2: Mean resource use across all 3,436 ESI-Bench questions: primitive steps, VLM calls, and total tokens per episode. Images denotes images per answer.
Variant
Acc
Δ Acc
EviSeek
52.7
—
w/o evidence demands
46.6
−6.1
w/o evidence board
47.6
−5.1
w/o selective retention
47.4
−5.3
Table 3: Qwen ablations on ESI-Bench: accuracy (Acc, %) and change from the corresponding online or replay reference ( Δ Acc, pp).
Method
C↑
C∗↑
Epath↑
dT↓
Blind
—
53.79
—
—
Single-model
19.48
40.09
18.66
6.42
Fine-EQA
49.92
67.10
33.15
5.60
EviSeek (nav.)
41.60
63.09
37.77
6.37
Table 4: EXPRESS-Bench with Gemini 3.1 Flash-Lite.
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Subcategory
#Q
Qwen
Gemini
Passive
Single-model
EviSeek
Passive
Single-model
EviSeek
Perceptual Grounding
View Hallucination
437
55.8
59.5
78.9
77.6
82.8
88.1
Partial Occlusion
95
22.1
50.5
86.3
54.7
74.7
85.3
Material Transparency
218
78.0
72.9
74.3
81.7
76.6
78.4
Physical Structure
Appendix
Table 5: Accuracy (%) across the 29 ESI-Bench subcategories. #Q is subcategory size; Overall is question-weighted. Bold marks the best result per backbone.
Subcategory
#Q
Qwen3-VL-32B
Gemini 3.1 Pro
Single-model
EviSeek
Single-model
EviSeek
Perceptual Grounding
View Hallucination
437
52.86
67.16
75.51
79.18
Partial Occlusion
95
37.89
35.26
64.21
60.53
Material Transparency
218
56.19
69.72
65.60
69.72
Physical Structure
Appendix
Table 6: Visual-support-weighted accuracy C (%) across ESI-Bench subcategories. Correct answers are weighted by answer-time visual support (0, 0.5, or 1). #Q is subcategory size; Overall is question-weighted.
Action
ESI
EXP
Locomotion
move_forward
✓
✓
move_backward
✓
—
move_left / move_right
✓
—
move_up / move_down
✓
—
Perception
Appendix
Table 7: Atomic actions supported by ESI-Bench (ESI) and EXPRESS-Bench (EXP). Checkmarks indicate availability; dashes indicate unsupported actions.
Intent
Description
Compiled actions
ESI
EXP
FACE
Orient toward a bearing
n×turn_left/right
✓
✓
APPROACH
Close in on a visible target
k×move_forward
✓
✓
GOTO
Reposition to a seen landmark
[turns]+k×move_forward
✓
✓
SWEEP
Scan the surroundings in a bounded fan
[turn_down]+n×turn_left/right
✓
✓
ORBIT
Shift azimuth around a visible target
n×(move_right,turn_left)
✓
—
RISE_TOPDOWN
Rise for an overhead layout view
3×move_up+n×turn_down
✓
—
Appendix
Table 8: Viewpoint intents and their compiled atomic actions in ESI-Bench (ESI) and EXPRESS-Bench (EXP). n and k are bounded repetition counts; brackets denote optional prefixes. Checkmarks indicate availability.
Baseline
Metric
Qwen3-VL-32B
Gemini 3.1 Pro
Δ
95% CI
n+/n−
p
Δ
95% CI
n+/n−
p
Passive
Acc
+12.19
[+10.30,+14.09]
783/364
<0.001
+12.98
[+11.20,+14.78]
742/296
<0.001
Single-model
Acc
+7.10
[+5.24,+8.99]
673/429
<0.001
+3.67
[+2.01,+5.36]
496/370
<0.001
Appendix
Table 9: EviSeek versus each baseline on 3,436 paired ESI-Bench questions. Δ is the accuracy gain (pp), with pointwise paired-bootstrap 95% CIs. n+/n− counts questions correct only for EviSeek/only for the baseline. Two-sided exact McNemar p values use Holm correction across the four comparisons.
Configuration
Acc
Tokens / step
Expl. steps
Single-model
46.71
6,127
9.3
EviSeek − intent
50.84
12,863
13.3
EviSeek
53.08
5,591
17.5
Appendix
Table 10: ESI-Bench intent ablation with Gemini 3.1 Flash-Lite. Acc is accuracy (%); tokens/step uses executed environment steps. Expl. steps is mean exploration length.
Figure 3: Spatial alignment: a top-down acquisition goal reveals that the three objects are collinear. Red frames show Single-model’s final views; green frames show retained EviSeek evidence. Colored bars mark acquisition subtasks.
Figure 4: Physical stacking: retained observations record the outcomes of testing candidate supports and top objects across manipulations. Red frames show Single-model’s final views; green frames show retained EviSeek evidence. Colored bars mark acquisition subtasks.
Figure 5: Contact judgment: Single-model stops at the initial view, while EviSeek inspects additional bearings before answering. Red frames show Single-model’s final views; green frames show retained EviSeek evidence. Colored bars mark acquisition subtasks.
Figure 6: Geometric configuration: a targeted overhead view reveals the arrangement of all three objects. Red frames show Single-model’s final views; green frames show retained EviSeek evidence. Colored bars mark acquisition subtasks.
Embodied question answering asks an agent to navigate, observe, and then answer questions about what it saw, yet answer accuracy alone cannot tell whether any of this happened. A plausible answer can come from the question, the room label, or the answer choices, and a correct answer does not show that the supporting observation was ever made. We present a multi-room benchmark whose 200 test episodes and 800 questions are stratified by route extent, with an evaluation ladder that holds the questions fixed while varying only the evidence supplied to the answerer, from text-only probes through closed-loop agents to replay of the reference trajectory. Questions are generated from symbolic records of rooms, visible objects, and frames, so every answer can be traced to the observations that support it. A closed-loop vision-language agent reaches no target in 200 episodes yet answers 45.5% of questions, within 0.1 points of the same model given no images; replaying the reference trajectory raises the same model to 57.4%, so the evidence exists along the route but is not gathered. A family-level audit then shows that access to evidence is not the same as use of it. Four of eight families are largely answerable from text, one cites frames that cannot establish its own answer, and the oracle conditions reach 59.5% on Rooms-visited by counting the supplied images. Evidence acquisition, answer coverage, and shortcut sensitivity must therefore be established before an accuracy gain can be credited to memory.
Shubh Garg, Divyansh Pandey, Niyati Rawal +2
Indian AI Research Organisation (IAIRO), GIFT City, Gujarat · Independent Researcher · BITS Pilani +1
Embodied Question Answering (EQA) combines visual scene understanding, goal-directed exploration, spatial and temporal reasoning under partial observability. A central challenge is to confine physical search to question-relevant subspaces while maintaining a compact, actionable memory of observations. Furthermore, for real-world deployment, fast inference time during exploration is crucial. We introduce FAST-EQA, a question-conditioned framework that (i) identifies likely visual targets, (ii) scores global regions of interest to guide navigation, and (iii) employs Chain-of-Thought (CoT) reasoning over visual memory to answer confidently. FAST-EQA maintains a bounded scene memory that stores a fixed-capacity set of region-target hypotheses and updates them online, enabling robust handling of both single and multi-target questions without unbounded growth. To expand coverage efficiently, a global exploration policy treats narrow openings and doors as high-value frontiers, complementing local target seeking with minimal computation. Together, these components focus the agent's attention, improve scene coverage, and improve answer reliability while running substantially faster than prior approaches. On HMEQA and EXPRESS-Bench, FAST-EQA achieves state-of-the-art performance, while performing competitively on OpenEQA and MT-HM3D.
Haochen Zhang, Nirav Savaliya, Faizan Siddiqui +1
Carnegie Mellon University, USA · Honda Research Institute USA
Aerial Embodied Question Answering (EQA) requires Unmanned Aerial Vehicles (UAVs) to actively perceive the environment and answer natural language questions. Existing outdoor EQA systems usually stop once the target enters the UAV's field of view, leaving the fine-grained viewpoint adjustment needed for evidence-seeking questions largely unresolved. To address this issue, we introduce FG-EQA, a fine-grained active perception EQA benchmark with more than 40K simulated trajectories and 1K real-world trajectories. Drawing inspiration from the ``waggle dance'' of scout bees, which iteratively adjust their flight paths to verify target information, we propose ScoutVLA, an evidence-driven Vision-Language-Action model for outdoor EQA. To emulate this active exploration behavior, ScoutVLA features a decoupled dual-expert architecture: a vision-language expert infers the semantic intent to identify missing evidence, while an independent action expert employs high-DoF flow matching to generate continuous viewpoint-refinement trajectories. To balance the competing demands of continuous control and semantic reasoning, we devise a decoupled training strategy with a knowledge insulation mechanism that prevents the action gradients from erasing the model's multimodal reasoning ability. Extensive simulated experiments and a qualitative real-world field study both verify the superiority of ScoutVLA over the state-of-the-art baselines, demonstrating a 10.48× higher average strict success rate and a 7.72× higher average QA correctness.
Wenhao Lu, Zhengqiu Zhu, Xiaofeng Wang +7
1National Key Laboratory of Digital Intelligent Modeling and Simulation, National University of Defense Technology · 2GigaAI