VideoNexus: Video Perception Beyond the Reasoning Context
Organizations: The Chinese University of Hong Kong · KlingAI Research · The Hong Kong University of Science and Technology
Abstract
Understanding long videos requires locating evidence, resolving visual details, and integrating observations into an answer. Even when relevant evidence is present, performing these operations within long, dense visual sequences burdens both perception and reasoning. Video agents enable selective inspection, but returning frames to the planner still accumulates visual information in its reasoning context. Delegating perception reduces this burden, yet sparse sampling can miss decisive details, and per-frame descriptions can preserve context growth. We introduce VideoNexus, a training-free framework that separates detailed video perception from evidence-driven planning and reasoning. The planner controls where to inspect, what to resolve, and how densely to sample. Perception sub-agents process native audio-video in independent contexts and return compact, temporally grounded evidence for iterative search and verification. Using Gemini 3.8 Flash for both planning and perception, VideoNexus achieves 88.6% accuracy on full LVBench, exceeding same-backbone static ingestion and built-in agentic processing by 3.6 and 1.1 points, respectively, at 44% of the latter's measured API cost. A paired LVBench subset ablation shows lower planner input and higher accuracy. With Kimi K3 as the shared backbone, VideoNexus improves over the official Video-GAIA ReAct harness by 8.5 points in our evaluation. These results support compact evidence exchange for combining detailed observation with efficient planning and reasoning.
Figures & tables
| Method | All | L1 | L2 | Cul | Day | Eco | Geo | His | Tec |
| Official ReAct | 51.7 | 79.0 | 43.5 | 48.3 | 67.2 | 33.3 | 41.3 | 54.5 | 48.7 |
| VideoNexus | 60.2 | 77.4 | 55.0 | 55.2 | 67.2 | 50.0 | 50.8 | 68.2 | 64.1 |
| (a) Perception coupling: Gemini 3.8 Flash | ||||
| Local return | Acc. | Planner in | Planner out | $/q |
| Raw clip | 86.5 | 18.07 | 0.76 | 0.105 |
| Raw clip, no Transcribe | 87.0 | 8.92 | 0.61 | 0.153 |
| Text Finding | 89.0 | 12.33 | 0.57 | 0.110 |
| Global scan (frames per video) | Spacing | 1 frame | 2 frames | KIR 1 |
| VideoXAgent budget (60.8) | 66.6 s | 45.0 | 23.9 | 28.1 |
| VideoSeek overview (64) | 63.2 s | 45.9 | 24.5 | 28.7 |
| VideoSeek budget (92.3) | 43.8 s | 52.0 | 29.3 | 33.5 |
| Uniform 384 | 10.5 s | 72.8 | 54.6 | 49.4 |
| VideoNexus Overview ( 1024) | 4.0 s | 80.7 | 70.9 | 57.3 |
| Static 1 fps ( 2048) | 2.0 s | 82.8 | 78.5 | 59.4 |
| Model | Benchmark | Baseline | VideoNexus | |
| Gemini 3.8 Flash | LVBench | 85.0 | 88.6 | +3.6 |
| Gemini 3.0 Flash | LVBench | 74.0 | 80.4 | +6.4 |
| Kimi K3 | LVBench † | 73.0 | 79.5 | +6.5 |
| Kimi K3 | Video-GAIA | 51.7 | 60.2 | +8.5 |
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
| Tool | Planner input | Returned to Planner |
| Transcribe | – | timestamped ASR text |
| Overview | query | summary, 8 candidate intervals |
| Inspect | , fps, task | Finding ( section 3.3 ) |
| Optional (crops also used on LVBench) | ||
| Inspect Crops | time, regions, task | Finding on source-resolution crops |
| Web Search / Extract | queries, URLs | snippets, page summaries |
| Perceived | Returned | |||||
| Call | #calls | frames | tokens | tokens | per frame | |
| VideoNexus | ||||||
| Overview | 680 | 1,019 | 141,414 | 740 | 0.73 | 0.00 |
| inspect_frames | 4,376 | 61.6 | 5,046 | 401 | 6.5 | 0.18 |
| inspect_crops | 448 | – | – | 189 | – | – |
| VideoSeek [ 17 ] , same model and provider | ||||||
| Component | Variant | Acc. |
| Overview candidates | forced / optional / none | 80.0 / 77.0 / 76.0 |
| Overview content † | timeline + cand. / cand. only | 81.0 / 77.0 |
| Tool interface † | atomic / batched (2 runs) | 80.5 / 75.0 |
| Inspect resolution | high / medium / low | 80.0 / 80.0 / 81.5 |
| Inspect sampling | 2 fps / 1 fps | 81.5 / 81.0 |
| Inspect audio + ASR | without / with | 80.0 / 82.5 |
Long-Video QA
Long-video question answering develops methods for finding and reasoning over relevant visual and audio evidence across extended recordings.
See all 98 papers