Large Vision-Language Models (LVLMs) reason effectively about video content, but whether they perceive its event boundaries in a manner consistent with humans remains unclear. We first investigate this through Generic Event Boundary Detection (GEBD), which identifies event boundaries naturally perceived by humans in continuous video, without relying on predefined event categories. On Kinetics-GEBD dataset, LVLMs miss many boundaries when predicting directly from frames, but effectively refine candidates proposed by an unsupervised discovery method, achieving state-of-the-art unsupervised GEBD performance. We extend this discovery-refinement approach to hour-long egocentric video, where boundary discovery requires a broader semantic scale and whole-video LVLM refinement becomes impractical. We address these challenges through coarse-to-fine temporal discovery and localized LVLM accept/reject judgments. Human evaluation shows that the LVLM's agreement with group consensus falls within the range observed among human annotators, with disagreements concentrated on low-consensus cases. Using this pipeline, we construct EgoBoundary, a benchmark for event boundary detection in hour-scale egocentric video. Representative supervised and unsupervised methods degrade substantially on EgoBoundary, highlighting the challenge of long-horizon event boundary detection.
Figures & tables
Figure 1: Overview of the two key results. (a) Unsupervised discovery followed by LVLM refinement improves event boundary detection on Kinetics-GEBD. (b) Given local visual context around a candidate frame, the LVLM makes accept/reject judgments consistent with human annotators.
Figure 2: Overview of the unsupervised boundary discovery algorithm.
Figure 3: Coarse-to-fine discovery
Supervision
Method
Rel.Dis=0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
Avg
Supervised
BMN Lin et al. (2019)
0.186
0.204
0.213
0.220
0.226
0.230
0.233
0.237
0.239
0.241
0.223
BMN-StartEnd Lin et al. (2019)
0.491
0.589
0.627
0.648
0.660
0.668
0.674
0.678
0.681
0.683
0.640
TCN-TAPOS Shou et al. (2021)
0.464
0.560
0.602
0.628
0.645
0.659
0.669
0.676
0.682
0.687
0.627
TCN Lea et al. (2016)
0.588
0.657
0.679
0.691
0.698
0.703
0.706
0.708
0.710
0.712
0.685
PC Shou et al. (2021)
0.625
0.758
0.804
0.829
0.844
0.853
0.859
0.864
0.867
0.870
0.817
Temporal Perceiver Tan et al. (2023)
0.748
0.828
0.852
0.866
0.874
0.879
0.883
0.887
0.890
0.892
0.860
Table 1: Comparison with previous methods on Kinetics-GEBD. F1 scores are reported across Rel.Dis. thresholds.
Method
Precision
Recall
F1
Direct inference
0.8500
0.5540
0.6708
Unsupervised discovery
0.6871
0.7505
0.7174
Combination
0.6910
0.8077
0.7448
Table 2: Comparison of direct LVLM inference, unsupervised boundary discovery, and their combination on Kinetics-GEBD at Rel.Dis. =0.05 .
Rank
Judge
κ
1
Human A
0.534
2
Human B
0.510
3
Human C
0.508
4
Human D
0.477
5
Human E
0.398
6
LVLM
0.394
Table 3: Agreement with leave-one-out group consensus for the LVLM and human annotators.
Margin ≥
N
κ
Agree.
FR
2
87
0.394
70%
41%
4
69
0.551
78%
35%
6
58
0.597
81%
33%
8
39
0.667
85%
27%
10
23
0.721
87%
12%
Table 4: LVLM agreement as a function of human consensus. Margin denotes the absolute difference between accept and reject votes from ten human annotators.
Discovery
Initial
Rejected
Accepted
Coarse-to-fine
6,364
17.0%
83.0%
Fine only
6,364
22.3%
77.7%
Table 5: Comparison of coarse-to-fine and single-scale boundary discovery under the same candidate budget.
Method
#Bnd.
F1 @ tIoU
0.5
0.7
0.9
Discovery
265
0.7625
0.6188
0.4391
+ LVLM refinement
228
0.8276
0.7112
0.5259
Uniform ( n =discovery)
265
0.2515
0.0639
0.0040
Uniform ( n =refined)
228
0.2457
0.0474
0.0043
Uniform ( n =GT)
230
0.2833
0.0687
0.0043
Table 6: Comparison against manually annotated event boundaries. #Bnd. denotes the total number of predicted boundaries. Uniform baselines evenly partition each video using the same boundary counts as the corresponding predictions or ground truth.
Supervision
Method
Feat.
0.5
0.7
0.9
Supervised
BasicGEBD
G14
0.4783
0.3052
0.1823
S16
0.4723
0.2490
0.1307
Unsupervised
UBoCo
G14
0.3120
0.1544
0.0649
S16
0.2318
0.1181
0.0452
CoSeg-FFR
G14
0.2544
0.0983
0.0133
S16
0.2720
0.1017
0.0205
Table 7: Baseline performance on EgoBoundary using segment-level tIoU-F1.
Artifact
License / Terms
Kinetics-GEBD
CC BY-NC 4.0
EgoLife
S-Lab License 1.0
Perception Encoder
Apache License 2.0
Gemini 2.5 Pro
Google Gemini API Terms
Table 8: Summary of licenses and usage terms for external datasets and models.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 4: Qualitative examples of accepted event boundaries. The LVLM identifies meaningful transitions in location, activity, or social context, including moving outdoors, shifting attention to a group activity, and joining a social gathering. The proposed boundary is highlighted in red.
Figure 5: Qualitative examples of rejected event boundaries. Despite changes in visual appearance or camera viewpoint, the underlying activity remains consistent, such as walking along the same path or continuing a social gathering. The proposed boundary is highlighted in red.
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University · College of Computer and Data Science, Fuzhou University