cs.CVNov 18, 2026

EgoJev: Finding Evidence in Egocentric Video by Reading VLM Decisions

Authors: Chunggi Lee, Hanspeter Pfister

Organizations: Harvard University

Abstract

Questions about long egocentric video often depend on one brief moment, such as the second a hand puts an object down. A vision-language model (VLM) that sees a few evenly spaced frames usually misses it. Training-free methods rank frames with a separate image-text model or ask the VLM to write a time. In a controlled test, the VLM itself locates the moment better when it chooses among labeled frames, and we read this choice from its output distribution. With the same candidates, this choice lands on the evidence more often than image-text scores or a written time span, and it compares all candidates in one decision. Based on this finding, we propose EgoJev, a training-free frame selector for wearable assistants. EgoJev reads where the evidence is from labeled thumbnails and looks again at the windows it points to. It places its few answer frames at the quantiles of the resulting distribution. One frozen VLM selects, answers and returns the evidence time, with no second model. A wearable device can encode its coarse thumbnails while it records. With Qwen3.5-4B and 8 answer frames, EgoJev raises QAEgo4D accuracy from 51.2% with uniform sampling to 62.0%, and it outperforms recent training-free frame selectors on QAEgo4D and MyEgo. A single-level version runs on a Mac and an iPhone.

Figures & tables

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Ego-Grounding for Personalized Question-Answering in Egocentric Videos

    Apr 2, 2026Junbin Xiao, Shenglang Zhang, Pengxiang Zhu +1Egocentric VQAQuestion Answering

  2. EVAThink: Evidence Sufficiency-Guided Training for Adaptive Thinking in Video Understanding

    Nov 10, 2026Changti Wu, Ruizhe Fan, Bin Wang +2Evidence-Grounded VQAReasoning for VLMs