ARGOS: Who, Where, and When in Agentic Multi-Camera Person Search
Organizations: Korea Advanced Institute of Science and Technology (KAIST), Republic of Korea · University of Seoul, Republic of Korea · Korea Institute of Science and Technology (KIST), Republic of Korea
Abstract
Existing person search methods assume access to complete visual queries or exhaustive tracking, yet real-world witness accounts are vague, partial, and spread across cameras and time. We introduce ARGOS (Agentic Retrieval with Grounded Observational Search), a benchmark and agent framework that recasts multi-camera person search from one-shot retrieval on a complete query into interactive reasoning from partial clues. To our knowledge, ARGOS is the first interactive benchmark to couple witness dialogue with camera-network topology, requiring an agent to plan, question, and eliminate under information asymmetry. An ARGOS agent receives a vague witness statement and must decide what to ask, when to invoke spatial or temporal tools, and how to interpret ambiguous natural-language responses, all within a limited turn budget. To ground reasoning in physical constraints, the agent accesses a Spatio-Temporal Topology Graph (STTG) encoding camera connectivity and empirically validated transition times. The benchmark comprises 2{,}691 tasks across 14 real-world scenarios in three progressive tracks: semantic perception (Who, 989 tasks), spatial reasoning (Where, 550 tasks), and temporal reasoning (When, 1{,}152 tasks). We propose Turn-Weighted Success (TWS) as the primary metric, jointly measuring correctness and turn efficiency. Experiments with four LLM backbones show the benchmark is far from solved: the best agent achieves TWS of 0.383 (Track2) and 0.590 (Track3). Ablations confirm each component is essential: removing domain-specific tools drops Top-1 accuracy by up to 49.6 percentage points, and removing strategic reasoning halves TWS while barely affecting Top-1.
Figures & tables
| Track 1 ( Who ) | Track 2 ( Where ) | Track 3 ( When ) | |
| Task count | 989 | 550 | 1,152 |
| Interaction mode | Single-turn | Multi-turn | Multi-turn |
| Core capability | Semantic parsing | Spatial reasoning | Temporal reasoning |
| Avg. dialogue context | 5.6 turns | — | — |
| Avg. oracle turns ( ) | — | 2.02 | 1.89 |
| Primary metric | Top-1 Acc | TWS | TWS |
| Accessible | Not accessible |
| Gallery DB (24 attributes, read-only) | Candidate–camera mapping |
| Current candidate ID list | Ground-truth disambiguation path |
| Zone structure (sub-areas, cameras) | Which attributes the witness can answer |
| STTG statistics (min, median, max, std) | Per-candidate feasibility details |
| Available action types | Witness’s internal observable set |
| ID | Name | Input | Output | Tracks |
| T1 | query_gallery | candidate IDs | attribute records | 1,2,3 |
| T2 | analyze_dist | candidate IDs | counts, elim. power | 1,2,3 |
| T3 | get_zone_info | — | sub-areas, camera list | 2 |
| T4 | ask_witness | question type | NL response (string) | 2,3 |
| T5 | check_temporal | candidate IDs | filtered partition | 3 |
| T6 | filter_attr | attr, value, IDs | filtered IDs | 1,2,3 |
| Agent | SR@1 | SR@5 | Parse Acc. |
| Oracle | 100.0 | 100.0 | 100.0 |
| LLM ToolCall | 81.1 | 82.3 | 90.8 |
| LLM Direct | 73.3 | 73.3 | 93.0 |
| Rule-based | 32.2 | 48.0 | 66.0 |
| Track 2 (Spatial) | Track 3 (Temporal) | |||||||
| Backbone | TWS | Top-1 | SR t @5 | AvgT s | TWS | Top-1 | SR t @5 | AvgT s |
| Oracle | 1.000 | 100.0 | 100.0 | 2.05 | 1.000 | 100.0 | 100.0 | 1.88 |
| GPT-5.2 | 0.338 | 73.1 | 33.6 | 7.04 | 0.590 | 88.2 | 65.8 | 3.91 |
| GPT-4o | 0.323 | 74.5 | 31.6 | 7.49 | 0.567 | 80.6 | 60.8 | 3.91 |
| GPT-5-mini | 0.319 | 74.9 | 32.2 | 7.48 | 0.556 | 88.0 | 60.5 | 4.40 |
| Claude Sonnet 4 | 0.383 | 76.0 | 38.4 | 6.71 | 0.548 | 83.6 | 59.5 | 4.25 |
| Track 3 variant | Top-1 | TWS | AvgTurns |
| Temporal-first | 80.6 | 0.567 | 3.91 |
| Autonomous | 83.9 | 0.572 | 4.05 |
| w/o Temporal Tool | 31.0 | 0.054 | 13.44 |
| Track 2 | Track 3 | ||||||
| Agent | Removed | TWS | Top-1 | AvgT s | TWS | Top-1 | AvgT s |
| Oracle | (upper bound) | 1.000 | 100.0 | 2.05 | 1.000 | 100.0 | 1.88 |
| ARGOS Agent | (none) | 0.323 | 74.5 | 7.49 | 0.567 | 80.6 | 3.91 |
| w/o Strategy | LLM reasoning | 0.136 | 47.6 | 10.78 | 0.373 | 76.9 | 7.73 |
| w/o Spatial Tool | Spatial queries | 0.063 | 40.7 | 14.20 | — | ||
| w/o Temporal Tool | Temporal queries | — | 0.054 | 31.0 | 13.44 | ||
Agent Benchmarks
Agent benchmarks measure how well AI systems plan, use tools and memory, and complete tasks in interactive environments.
See all 121 papers