This paper presents a multi-agent perception-action exploration alliance, dubbed A4VL, for efficient long-video reasoning. A4VL operates in a multi-round perception-action exploration loop with a selection of VLM agents. In each round, the team of agents performs video question-answer (VideoQA) via perception exploration followed by action exploration. During perception exploration, each agent learns to extract query-specific perception clue(s) from a few sampled frames and performs clue-based alignment to find the video block(s) that are most relevant to the query-specific event. During action exploration, A4VL performs video reasoning in three steps: (1) each agent produces its initial answer with rational, (2) all agents collaboratively scores one another through cross-reviews and relevance ranking, and (3) based on whether a satisfactory consensus is reached, the decision is made either to start a new round of perception-action deliberation by pruning (e.g., filtering out the lowest performing agent) and re-staging (e.g., new-clue and matching block based perception-action exploration), or to conclude by producing its final answer. The integration of the multi-agent alliance through multi-round perception-action exploration, coupled with event-driven partitioning and cue-guided block alignment, enables A4VL to effectively scale to real world long videos while preserving high quality video reasoning. Evaluation Results on five popular VideoQA benchmarks show that A4VL outperforms 18 existing representative VLMs and 11 recent methods optimized for long-video reasoning, while achieving significantly lower inference latency. Our code is released at https://github.com/git-disl/A4VL.
Figures & tables
Figure 1: Responses from A4VL and three of the strongest closed-sourced models.
Figure 2: Overview of A4VL architecture.
Figure 3: A4VL agent teaming workflow.
Figure 4: Example of A4VL on EgoSchema. Here all agents have wrong answers at the beginning, but later one of them turns to the correct answer and other agents are pruned.
Model
NeXT-QA
EgoSchema
LongVideoBench
MLVU
Video-MME
Short
Medium
Long
Average
GPT-4o* [ 25 ]
-
72.2
66.7
54.9
80.0/82.8
70.3/76.6
65.3/72.1
71.9/77.2
Gemini 1.5Pro* [ 37 ]
-
71.1
64.0
-
81.7/84.5
74.3/81.0
67.4/77.4
75.0/81.3
LLaVA-Video-7B-Qwen2 [ 55 ]
81.2
61.8
61.1
51.7
72.4/75.6
58.1/63.3
50.7/60.4
60.4/66.4
VideoLlama3-7B [ 49 ]
67.4
52.6
52.5
43.9
80.1/80.2
63.7/69.6
54.9/61.0
66.2/70.3
InternVL3.5-8B [ 41 ]
76.2
62.0
53.6
47.1
69.6/74.0
56.3/63.9
47.7/61.9
57.9/66.6
Table 1: A4VL compared with other methods on five benchmarks. For Video-MME, the result is in format w. subtitle/wo. subtitle. For Video-MME, the improvement row shows the improvement without subtitle, since A4VL is mainly designed to optimize visual reasoning. * near the model names mean we take the results from the model paper or benchmark leaderboard.
Benchmark
Models
NeXT-QA
Intern-78B
Intern-38B
QwenVL-72B
EgoSchema
Intern-78B
Intern-38B
QwenVL-72B
LongVideoBench
Intern-78B
Intern-38B
QwenVL-72B
MLVU
Intern-78B
Intern-38B
LLaVA-72B
Video-MME
Intern-78B
LLaVA-72B
QwenVL-72B
Table 2: Models used for each benchmark. Intern-78B refers to InternVL3-78B [ 41 ] , Intern-38B refers to InternVL3.5-38B [ 41 ] , QwenVL-72B refers to QwenVL-2.5-72B [ 38 ] , and LLaVA-72B refers to LLaVA-Video-72B-Qwen2 [ 55 ] .
Model
NeXT-QA
EgoSchema
MLVU
GPT-4o
23s
54s
127s
InternVL3-78B
15s
50s
204s
VideoAgent
20s
83s
175s
TraveLER
101s
94s
450s
A4VL
18s
37s
74s
Table 3: Average inference time per sample on four representative methods and A4VL.
Figure 5: Relationship between accuracy and maximum number of rounds.
#Rounds
1
2
3
NeXT-QA
1072
230
191
EgoSchema
303
101
96
LongVideoBench
566
341
430
MLVU
183
116
203
Video-MME
1289/1667
557/421
854/612
Table 4: Number of actual running rounds in A4VL. The data of Video-MME is shown in w/o subtitle / w. subtitle format.
Method
RRSampling
RESampling
ERSampling
Accuracy (%)
80.2
82.2
79.6
Time/Sample
35s
37s
37s
Table 5: Accuracy and latency of different sampling strategies on EgoSchema. R means random sampling and E means sampling from event-based blocks. The first character represents the sampling strategy for N1 perception frames, and the second character represents the sampling strategy for N2 action frames.
Method
MConsens
FConsens
Accuracy (%)
81.4
82.2
Time/Sample
26s
37s
Table 6: Results of different consensus conditions on EgoSchema. M means majority and F means full.
Method
NoPruneSum
NoPruneMaj
A4VL
Accuracy (%)
80.8
79.4
82.2
Time/Sample
60s
60s
37s
Table 7: Results of different pruning strategies on EgoSchema.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Algorithm 1 A4VL Algorithm.
Figure 6: Visualization of event-based video partitioning.
Figure 7: Example where the majority is wrong in the first round, but finally A4VL produces the correct answer. The example is taken from NeXT-QA.
Figure 8: Failure case where all agents achieve a wrong consensus in the first round. The example is taken from NeXT-QA.
Figure 9: Failure case where the majority is correct in the first round, but finally the final answer gets wrong. This example is taken from Video-MME (w/o sub).
Long-form video understanding often involves multiple questions about different aspects of the same recording. Yet existing video agents typically process each question through an isolated tool-use trajectory. This repeatedly restarts video exploration and memory construction, missing opportunities to acquire evidence jointly and progressively build a shared understanding that supports the complete question set. We introduce VAMR (Video Agent for Multi-Question Reasoning), which coordinates all questions about a video through one shared tool-use trajectory. At each round, a persistent policy model can invoke tools for one or more unresolved questions and submit answers for questions with sufficient evidence. Question-conditioned visual perception retrieves fine-grained clues for several questions in one call, while layered multi-question memory integrates reusable context into a shared video story and preserves separate evidence for individual questions. After supervised fine-tuning initializes this interaction protocol, we propose question-horizon policy optimization (\qhpo) to optimize shared trajectories in which questions progress and finish at different rounds. Specifically, a question-level critic estimates the value of each active question, while round alignment maps each question advantage to the rounds that directly serve it before the aligned advantages are aggregated to optimize the shared actor. Across LVBench, Video-Holmes, and LongVideoBench, VAMR achieves the highest accuracy overall and the fewest reasoning rounds among iterative methods. On LVBench, it reaches 62.1% accuracy, exceeding VideoARM by 4.3 points while reducing reasoning rounds and processed frames by 85.9% and 61.4%.
Runquan Gui, Hanzhu Chen, Zehao Wang +3
University of Science and Technology of China · Tencent
Despite rapid developments and widespread applications of MLLM agents, they still struggle with long-form video understanding (LVU) tasks, which are characterized by high information density and extended temporal spans. Recent research on LVU agents demonstrates that simple task decomposition and collaboration mechanisms are insufficient for long-chain reasoning tasks. Moreover, directly reducing the time context through embedding-based retrieval may lose key information of complex problems. In this paper, we propose Symphony, a multi-agent system, to alleviate these limitations. By emulating human cognition patterns, Symphony decomposes LVU into fine-grained subtasks and incorporates a deep reasoning collaboration mechanism enhanced by reflection, effectively improving the reasoning capability. Additionally, Symphony provides a VLM-based grounding approach to analyze LVU tasks and assess the relevance of video segments, which significantly enhances the ability to locate complex problems with implicit intentions and large temporal spans. Experimental results show that Symphony achieves state-of-the-art performance on LVBench, LongVideoBench, VideoMME, and MLVU, with a 5.0% improvement over the prior state-of-the-art method on LVBench. Code is available at https://github.com/Haiyang0226/Symphony.
Haiyang Yan, Hongyun Zhou, Peng Xu +2
Institute of Automation, Chinese Academy of Sciences · School of Future Technology, University of Chinese Academy of Sciences · Kuaishou Technology
Multimodal Large Language Models (MLLMs) have achieved remarkable progress on short video understanding yet remain limited on long videos due to the limited visual context window. Prevailing approaches rely on uniform frame sampling or recent coarse-to-fine agentic zooming, both of which struggle to localize sparse, decisive evidence in sufficiently long videos. We formulate long video understanding as a Sequential Evidence Acquisition (SEA) problem, in which an agent reads the video turn by turn along the temporal axis, deciding at each turn how fast to watch, what evidence to retain, when to revisit uncertain segments, and when to stop and answer. Inspired by this view, we propose VideoScout, a multi-turn reasoning agent that instantiates the SEA paradigm through adaptive reasoning pacing. Specifically, by dynamically controlling the viewing pace, VideoScout enables efficient traversal of long videos within a bounded visual context window, allowing the agent to access more video content while balancing content analysis depth with reading efficiency. To train VideoScout, we construct VideoScout-66K, a set of over 66K high-quality exploration turns from 10K answer-verified trajectories, and adopt a two-stage pipeline: cold-start supervised fine-tuning teaches the agent per-turn output format, while the Decoupled Clip and Dynamic sAmpling Policy Optimization (DAPO) algorithm performs trajectory-level reinforcement learning with a composite reward that jointly considers answer accuracy, output format compliance, and the temporal alignment between the agent's viewing progress and the teacher's answer timing measured by intersection-over-union (IoU). Extensive experiments on long video understanding and reasoning benchmarks demonstrate that our 7B model achieves strong performance compared with existing trained 7B agentic models.