NexSight: Verifiable Tasks for Fine-Grained Agentic Long-Video Understanding
Organizations: Zhejiang University · Tongji University · Shanghai Qiji Zhifeng Co., Ltd.
Abstract
Fine-grained questions about long videos ask how many times an action occurs, in what order events unfold, or exactly when a state changes. Answering them can require finding brief moments scattered across hours of footage. Agentic models that decide when and where to look promise to find these moments, yet this ability is hard both to measure and to learn. Sparse supporting evidence may establish that an action occurred without establishing its total count; verifying timeline-wide claims requires checking for missed occurrences and double counting. Answer-only supervision, in turn, does not explicitly supervise the search for evidence. We present NexSight, which builds verifiable tasks to both evaluate and train agentic long-video understanding. Its agentic pipeline, SightForge, accepts a task only if a verifier blind to the draft answer can reconstruct it from evidence densely reacquired from the raw video and counterexample checks fail to refute it. This yields SightLine, 310 evidence-traceable tasks from 108 long videos, evaluated through SightLoop, a shared multiscale harness supporting temporal exploration, local inspection, evidence integration, and revisiting while recording returned observations. On disjoint videos, solver trajectories on verified tasks become step-level supervision for observation decisions. The strongest model reaches only 54.52% on SightLine, yet 79.35% when given the annotated evidence. With under 6K examples, Sight-35B nearly doubles its base model's interactive accuracy (21.29% to 42.26%), outperforms the 397B model of its family, begins using explicit spatial inspection after post-training, which its base model never uses, and gains over 8 points on two external long-video benchmarks.
Figures & tables
| Model / deployment | Direct-64 | Agentic |
|---|---|---|
| Qwen3.5-35B-A3B-Base | 80 / 25.81 | 66 / 21.29 |
| Qwen3.5-35B-A3B-Inst | 103 / 33.23 | 87 / 28.06 |
| Sight-35B (ours) | 88 / 28.39 | 131 / 42.26 |
| Benchmark | Questions | 35B Base | Sight-35B | |
|---|---|---|---|---|
| Video-MME Long | 900 | 477 / 53.00 | 551 / 61.22 | +8.22 |
| LVBench subset | 123 | 55 / 44.72 | 65 / 52.85 | +8.13 |
Appendix figures & tables20 assets
Supplementary material from the paper’s appendix.
Appendix
| Content domain | Videos | Questions |
|---|---|---|
| Art Performance | 22 | 60 |
| Film / TV | 22 | 58 |
| Knowledge | 21 | 66 |
| Life / Vlog | 22 | 68 |
| Sports Competition | 21 | 58 |
| Total | 108 | 310 |
| Statistic | Value |
|---|---|
| Questions / videos | 310 / 108 |
| Domains / capabilities | 5 / 6 |
| Total video duration | 74.12 h |
| Evidence JPG references | 1,080 |
| Capability | Identifier | N |
|---|---|---|
| Temporal boundary | temporal_boundary | 50 |
| Cross-window count | cross_window_count | 32 |
| Unlocalized fact retrieval | unlocalized_fact_retrieval | 69 |
| Identity tracking | identity_tracking | 45 |
| Event order | event_order | 60 |
| State/attribute change | state_attribute_change | 54 |
| Checkpoint | Direct-64 | SightLoop Ledger On |
|---|---|---|
| Qwen3.5-35B-A3B-Base | 80 / 25.81 | 66 / 21.29 |
| Sight-35B (ours) | 88 / 28.39 | 131 / 42.26 |
| Model / checkpoint | Ledger | Acc. (%) | Model calls | Tool calls | Observed frames | Recorded tokens |
|---|---|---|---|---|---|---|
| GPT-5.6 Sol | On | 53.23 | 11.99 | 19.72 | 195.99 | 331,446 |
| GPT-5.6 Sol | Off | 54.52 | 9.64 | 17.82 | 211.20 | 252,835 |
| DeepSeek V4 Flash Vision | On | 49.68 | 11.26 | 14.63 | 149.08 | 268,386 |
| DeepSeek V4 Flash Vision | Off | 48.39 | 10.16 | 13.66 | 159.14 | 232,136 |
| Qwen3.5-397B-A17B | On | 31.61 | 16.67 | 14.06 | 274.19 | 506,441 |
| Qwen3.5-397B-A17B | Off | 34.84 | 15.08 | 12.16 | 248.36 | 408,774 |
| Model | Ledger | Valid / 310 | Semantic correct | Full-set Acc. (%) |
|---|---|---|---|---|
| GPT-5.6 Sol | On | 310/310 | 165 | 53.23 |
| GPT-5.6 Sol | Off | 310/310 | 169 | 54.52 |
| DeepSeek V4 Flash Vision | On | 310/310 | 154 | 49.68 |
| DeepSeek V4 Flash Vision | Off | 310/310 | 150 | 48.39 |
| Qwen3.5-397B-A17B | On | 303/310 | 98 | 31.61 |
| Qwen3.5-397B-A17B | Off | 299/310 | 108 | 34.84 |
| Model | Configuration | Correct / 310 | Accuracy (%) |
|---|---|---|---|
| GPT-5.6 Sol | Agentic Off | 169 | 54.52 |
| GPT-5.6 Sol | Agentic On | 165 | 53.23 |
| DeepSeek V4 Flash Vision | Agentic On | 154 | 49.68 |
| DeepSeek V4 Flash Vision | Agentic Off | 150 | 48.39 |
| GPT-5.6 Sol | Direct-128 | 135 | 43.55 |
| Nex-N2.5-Mini | Agentic Off | 133 | 42.90 |
| Model / deployment | Ledger | Overview | Temporal window | Frame extract. | Explicit spatial | Image subagent | Video / image OCR | |
|---|---|---|---|---|---|---|---|---|
| GPT-5.6 Sol | On | 310 | 1.04 | 11.69 | 1.50 | 0.25 | 0.13 | 0.44 / 0.27 |
| GPT-5.6 Sol | Off | 310 | 1.04 | 12.33 | 1.49 | 0.16 | 0.10 | 0.29 / 0.23 |
| DeepSeek V4 Flash Vision | On | 310 | 1.16 | 5.81 | 1.55 | 0.49 | 0.15 | 0.70 / 0.28 |
| DeepSeek V4 Flash Vision | Off | 310 | 1.13 | 6.91 | 1.68 | 0.50 | 0.17 | 0.79 / 0.26 |
| Nex-N2.5-Mini | On | 308 | 1.07 | 7.72 | 5.05 | 0.48 | 0.29 | 0.56 / 0.49 |
| Nex-N2.5-Mini | Off | 309 | 1.05 | 7.76 | 4.10 | 0.32 | 0.23 | 0.61 / 0.52 |
| Profile | Model calls | Tool calls | watch_clip calls | Time (min) |
|---|---|---|---|---|
| B1 | 8 | 16 | 10 | 12 |
| B2 | 10 | 22 | 14 | 18 |
| B3 | 14 | 36 | 24 | 25 |
| B4 | 14 | 36 | 24 | 60 |
| B5 | 14 | 36 | 24 | 120 |
| B6 | 24 | 60 | 32 | 70 |
| Condition | N | Valid | Unresolved | Correct | Acc. (%) | Hit (%) |
|---|---|---|---|---|---|---|
| Fixed | 310 | 309 | 0 | 112 | 36.13 | 19.84 |
| Random (seed 0) | 310 | 307 | 3 | 102 | 32.90 | 20.38 |
| Random (seed 1) | 310 | 310 | 1 | 97 | 31.29 | 17.47 |
| Random (seed 2) | 310 | 310 | 2 | 105 | 33.87 | 19.43 |
| Agentic | 310 | 310 | 1 | 127 | 40.97 | 40.63 |
| Condition | Physical calls | Unique frames | API images | Tokens | Repeat (%) | Overlap (%) |
|---|---|---|---|---|---|---|
| Fixed | 8.14 | 127.95 | 665.96 | 156,160 | 0.04 | 0.00 |
| Random (mean) | 8.10 | 127.91 | 661.97 | 155,960 | 0.07 | 3.14 |
| Agentic | 8.07 | 126.81 | 658.43 | 155,326 | 0.93 | 9.00 |
| Benchmark | Questions | 35B Base | Sight-35B | |
|---|---|---|---|---|
| LVBench subset | 123 | 37 / 30.08 | 46 / 37.40 | +7.32 |
| Video sources | Questions | Base correct | Sight-35B correct | Net gain | (pp) |
|---|---|---|---|---|---|
| Overlapping training sources | 300 | 173 | 191 | +18 | +6.00 |
| Disjoint from training sources | 600 | 304 | 360 | +56 | +9.33 |
| All | 900 | 477 | 551 | +74 | +8.22 |
| Video sources | Questions | Base correct | Sight-35B correct | Net gain |
|---|---|---|---|---|
| Overlapping training sources | 2 | 0 | 1 | +1 |
| Disjoint from training sources | 121 | 55 | 64 | +9 |
| All | 123 | 55 | 65 | +10 |
Long-Video Understanding
Long-video understanding enables models to retrieve, remember, and reason about events across extended visual and audio-visual sequences.
See all 162 papers