Organizations: Nanyang Technological University, Singapore · ByteDance Inc. · State Key Laboratory of Complex and Critical Software Environment, School of Computer Science and Engineering, Beihang University · Westlake University · Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences
Long instructional videos are increasingly used in medical education, industrial training, and procedural knowledge transfer, yet their evidence is often incomplete or unreliable: subtitles may be missing, ASR noisy, OCR sparse, and decisive actions briefly shown without narration. Existing VideoQA systems typically return labels or free-form text, making answer support difficult to inspect. We study Long Video Moment Question Answering (LVMQA), where a system returns timestamped answer-supporting evidence. We propose EP-LVMQA, an evidence-preserving coarse-to-fine framework comprising an Uncertainty-Aware Multimodal Evidence Timeline, a Coarse Evidence Planner, and an Evidence-Conditioned Fine Localizer. An evidence card transfers planner scores, matched cues, source-confidence proxies, modality-specific evidence, neighboring context, and a temporal prior between the latter two modules. On MedVidQA, EP-LVMQA achieves 26.1 [email protected] and 31.8 mIoU, improving the strongest non-ours results by 6.3 and 5.1 points. It raises Suff.@2 and Ver.@2 to 25.4 and 22.6, gains of 9.5 and 8.8 points. Under matched low-quality perturbations, average relative degradation is reduced from the strongest non-ours value of 14.6% to 11.1%. On Ego4D-NLQ and QVHighlights, EP-LVMQA remains broadly comparable to established task-specific localization methods.
Figures & tables
Figure 1: (a) Text-driven methods pass ASR/OCR-retrieved clips or scores to a clip-conditioned localizer, yielding a loose interval without provenance. (b) EP-LVMQA transfers multimodal evidence and source-aware metadata in an evidence card, enabling routed, prior-guided refinement and an inspectable primary evidence moment.
Figure 2: EP-LVMQA comprises three modules: (a) an Uncertainty-Aware Multimodal Evidence Timeline, (b) a Coarse Evidence Planner, and (c) an Evidence-Conditioned Fine Localizer. Question-Conditioned Evidence Cues condition the planner and localizer, which communicate through an Evidence-Preserving Candidate Interface implemented as an evidence card.
Table 1: Main comparison with explicit result provenance. MedVidQA is the target setting; Ego4D-NLQ and QVHighlights are compatibility checks. Prot.: Off.=reported/official reference, Rep.=our reproduction, M=matched feature-based evaluation, MP=matched-prompt Video-LLM evaluation. Off./Rep. means that localization is official while audit values come from reproduced saved predictions; these values are not combined into a matched superiority claim. All audit columns are measured strict-majority rates on the common blinded 300-query set. Bold and underlining denote the best and second-best listed value per metric, irrespective of protocol class.
Figure 3: Cross-benchmark performance on (a) Ego4D-NLQ and (b) QVHighlights, showing the top three methods for each metric. Ours denotes EP-LVMQA.
Method
Clean
ASR-50
OCR-50
HN-Cap
Conflict
Avg. drop ↓
Conflict Suff.@2
VPTSL/TAGV-style
18.2
11.4
16.6
13.2
12.0
26.9
9.1
Moment-DETR ( Lei et al. 2021 )
14.1
11.5
13.1
10.9
10.2
19.0
6.8
QD-DETR ( Moon et al. 2023b )
16.7
13.4
15.4
12.5
11.7
20.7
7.9
CG-DETR ( Moon et al. 2023a )
18.4
15.2
16.9
13.7
13.0
20.1
9.0
UniVTG ( Lin et al. 2023 )
19.8
16.7
18.2
15.0
14.4
18.8
10.6
CausalVTG ( Wang et al. 2025b )
19.2
16.5
17.9
15.8
15.4
14.6
11.9
Table 2: MedVidQA robustness in [email protected]. ASR/OCR-50 removes half the corresponding evidence; HN-Cap uses semantically similar non-overlapping caption negatives; Conflict retains ASR but replaces captions. Avg. drop is the mean relative loss across the four perturbations. All methods receive the same degraded input views.
Setting
Cand. R
Sel. dur.
Final R1
Cached time
@0.3 ↑
(%) ↓
@0.5 ↑
(s/query) ↓
A. Planner comparison under Top-5
Uniform sampling
31.8
11.6
11.9
0.54
Text-only planner
49.2
10.8
15.4
0.49
Visual-only planner
55.6
11.0
16.6
0.48
MomentSeeker Top-5 ( Yuan et al. 2025 )
61.9
12.3
18.2
0.69
Table 3: Planner and candidate-budget analysis on MedVidQA videos over 15 min. Block A fixes Top-5; Block B varies K . Cand. [email protected] is candidate recall; Sel. dur. is candidate-union/video duration. Cached time excludes offline extraction. † Oracle changes selection only; the fixed localizer receives no ground-truth boundaries.
Configuration
TL
Card
Src
Rtr
Cue
Ctx
Prior
R1
mIoU
Conflict
Suff.@2
Nec.@2
Ver.@2
@0.5
@0.7
Stage-level references
Standard MR reference
–
–
–
–
–
–
–
19.8
10.1
26.0
14.4
14.8
13.2
12.6
Coarse planner only
–
–
–
–
–
–
–
20.5
8.7
27.0
15.8
17.6
15.4
14.7
Component ablations
− TL
×
✓
×
×
✓
✓
✓
22.3
11.5
28.5
15.9
18.2
16.3
15.6
Table 4: Stage-level references and component ablations on MedVidQA. Checks/crosses mark enabled/removed components or downstream unavailability. TL is the evidence timeline; Src contains modality evidence, masks, and confidence; Rtr is the source router; Conflict is [email protected] under ASR–caption conflict. Suff.@2 and Ver.@2 are primary outcomes; Nec.@2 is auxiliary. Audit values use the common blinded set and strict-majority rule; all variants share the split, encoders, Top-5 budget, backbone, optimizer, and evaluation.
Figure 4: Frame-level comparisons under complete, missing, and conflicting evidence. EP-LVMQA is tighter or more resilient than TimeZero, retaining target overlap with incomplete or inconsistent evidence.