cs.CVOct 24, 2026

Fast Instinct, Slow Reason: A Dual-Speed Agentic Framework for Self-Configuring Multi-Object Tracking in Satellite Video

Authors: Fan He, Baining Wu, Yan Li, Xiaoyang Zeng

Organizations: State Key Laboratory of Integrated Chips and Systems, Fudan University, Shanghai 200433, China

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Multi-object tracking in satellite video underpins persistent monitoring, yet offline-tuned trackers lack the onboard intelligence and efficiency to adapt within an observation of tens of seconds. The root is that adaptation still means relabelling and retraining the detector, which cannot run on board in time. Tuning the tracker configuration strongly controls accuracy without touching the network, yet search needs labelled or proxy evaluations, and a language agent costs seconds and tokens per decision. We propose a dual-speed agentic framework pairing a fast instinct compiled offline with a slow reason serving the mission. Its algorithm, Arbitration of Unlabelled Shortlists from Prior Experience, or AUSPEX, learns a covering shortlist and a label-free ranker from labelled source tasks and commits one configuration per sequence. Its system, a Cascaded Onboard Runtime with Verified Uplinked Skills, or CORVUS, cascades reflex, skill, and agent layers, so that a verified skill commits 2.5 s into each observation while a language agent serves natural- language missions. On SatMTB and SatVideoDT with three recent detectors, AUSPEX raises IDF1 by 2.84 to 2.94 points and MOTA by 3.21 to 3.29 points on average without target labels, balances accuracy and labels best among sixteen methods, and loses at most 0.28 IDF1 points at 1.25 tracking passes. CORVUS executes four missions with execution accuracy up to 0.99 and target recall at the perception ceiling, spends no token on the routine decision, runs on Jetson Xavier NX, and hosts fifteen language models from six families without altering any commitment, demonstrating its extensibility and generalization.

Figures & tables

Fig. 1 : Motivation for adapting the tracker configuration on board. (a) An offboard loop cannot act within one observation, (b) the configuration tightly controls tracking scores, (c) this control is shared across datasets, detectors, and trackers, and (d) retraining for new scenes cannot run on board in time.
Fig. 2 : Overview of Auspex . (a) Offline, labelled source tasks yield a covering shortlist and a pairwise ranker on label-free statistics. (b) On board, the head of the shortlist tracks a short prefix in parallel, and the ranker commits one configuration for the whole sequence.
Fig. 3 : Overview of Corvus . (a) Onboard, link, and ground components, where the routine path commits θ\theta through the skill registry without passing the agent. (b) Timing of one sequence across the layers on a logarithmic axis, with ground segments not to scale and agent latencies from one measured run.
MissionInstruction aScenarioReasoning
AreaLock the three largest targetsCueing the most significant vessels and aircraftSort
TransientLock the three largest seen in two or more framesRejecting likely noise before cueingFilter
PersistenceLock the three largest seen in three or more frames even over a larger brief oneHolding stable targets for continuous monitoringOverride
GrowthLock the three whose area grows most from early to late framesWarning of approaching or expanding targetsCompare
TABLE I : Observation missions posed to the agent.
Method aYearCost bIDF1 gainMOTA gainNormalized hypervolume HH
LabelsPassesTokensLOTOLODOLOTOLODOLOTOLODO
No target labels
ISAC ∗ [ 48 ]20100001.04−- 0.710.94−- 0.840.340
MetaOD ∗ [ 49 ]2021000−- 2.87−- 6.27−- 3.19−- 6.3400
ZAP-HPO ∗ [ 50 ]2022000−- 0.50−- 3.08−- 1.60−- 4.0000
SQE ∗ [ 17 ]20200302.162.162.142.130.700.70
TABLE II : Average gain over the fixed configuration against state-of-the-art baselines.
Fig. 4 : Sources of the gain of Auspex in IDF1 and MOTA under LOTO and LODO. Blue marks IDF1 and green MOTA, plain bars or solid lines LOTO and hatched bars or dashed lines LODO. (b) Dark narrow bars give IDF1 and light bars MOTA, on an axis linear within half a point of zero and logarithmic beyond. (c) The lower panel stacks the coverage and arbitration regrets below the best single configuration, drawn dotted.
Fig. 5 : Trajectories on video 08 of SatVideoDT with SPAX detections over frames 294 to 443. Each column shows the full frame above the two regions outlined in blue and orange. Lines trace the last 150 frames and solid boxes mark the tracks in frame 443, both colored by identity, while yellow and red dashed boxes mark missed targets and false tracks and white stars mark identity switches. The regions are those in which the baselines miss the most targets that Auspex tracks, and the best configuration is chosen for this sequence with ground truth.
Variant aCost per sequence bIDF1 gainMOTA gainOracle IDF1 gain c
PassesTokensSwitchesLOTOLODOLOTOLODOLOTOLODO
Auspex3002.842.943.213.293.343.35
Commitment granularity
Task3002.782.883.143.333.083.08
Prefix, deferred1.25002.822.663.253.033.343.35
Prefix, immediate1.2500.87/0.792.572.462.982.813.343.35
TABLE III : Ablation of commitment granularity and commitment layer.
Fig. 6 : One observation of the persistence mission by Qwen3.5-9B on a SatMTB sequence, timed on the 10 fps frame clock. Each column shows the full frame above the regions outlined in blue and orange, at the commitment from the 25-frame prefix, at the return of the third scan with only the three scanned windows drawn, and at the submission with the locked targets numbered in order of selection and the three largest targets by ground truth in white dashed boxes. Lines and boxes are colored by identity.
Fig. 7 : Onboard deployment of Corvus on an NVIDIA Jetson Xavier NX, with the ground segment of Auspex on the host and the Ethernet link standing for the uplink and downlink.
ModelParams (B) aEX bR@3 bLat. (s)
TotalActiveAll95% CIGrowth
Qwen3.5 [ 77 ]0.90.90.140.07–0.210.040.116.6
Qwen3.5220.510.39–0.630.070.5314.6
Qwen3.5440.720.61–0.840.300.5411.1
Qwen3.5990.950.91–0.990.890.6326.8
Qwen3.527270.990.97–1.000.960.6726.2
TABLE IV : Ablation of language-model family and scale.
More in this topic

Multi-Object Tracking

Multi-object tracking (MOT) links detections across video frames to maintain object identities through motion, occlusion, and changing viewpoints.

See all 189 papers

Explore similar work

CardsList

  1. MVTrack: Ultrafast Appearance-Free Moving Object Tracking from Compressed Bitstreams

    Aug 11, 2026Iñaki Erregue, Kamal Nasrollahi, Sergio EscaleraVisual Object TrackingVideo Object Detection

    Universitat de Barcelona · Computer Vision Center · Milestone Systems +1

  2. Geometric-Topological Perception and Motion Prior for Real-Time Satellite Video Object Tracking

    Mar 8, 2026Zixiao Wen, Guangyao Zhou, Jiawei Li +4Visual Object TrackingSiamese Neural Networks

    Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing, 100094, China · Key Laboratory of Technology in Geo-Spatial Information Processing and Application System, Chinese Academy of Sciences, Beijing, 100190, China · School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences, Beijing, 101408, China

  3. ARC-SORT: Adaptive, Robust, Compensated Multi-Object Tracking with Interacting Multiple Models

    Oct 15, 2026Girish Kumar AdariMulti-Object Tracking

    Athlitix Inc.