This work presents a multi-label temporal event detection framework for video capsule endoscopy (VCE) that addresses the extreme class imbalance inherent in the Galar dataset by combining two principal contributions: an Angular Separation Loss on class prototypes and a Biological State Machine temporal decoder. The backbone remains BiomedCLIP, a biomedical vision-language foundation model. Three consecutive frames are fused through a Local Differencing Attention module that amplifies transient pathological signals by suppressing static temporal redundancy. An Anatomy Context Head then conditions pathological predictions on soft anatomical activations, exploiting the known spatial co-occurrence structure of GI findings. Learnable text-feature prompts and prototype-based logit augmentation are trained alongside an Angular Separation Loss that penalizes off-diagonal cosine similarity between class prototypes, preventing the prototype collapse that afflicts rare classes under extreme imbalance. To counteract the skewed label distribution, the training regime combines asymmetric focal loss, inverse-frequency weighted sampling, temporal Mixup, Exponential Moving Average, and per-class threshold calibration. The Biological State Machine decoder replaces naive gap merging with a physiologically grounded forward-only state transition over anatomy labels, eliminating the fragmentation artefact that produced hundreds of spurious anatomy events per video in the prior approach and reducing per-video anatomy output to 2--3 clinically realistic events. On the held-out RARE-VISION test set comprising three NaviCam examinations (161,025 frames), the updated pipeline achieves an overall temporal [email protected] of 0.3597 and [email protected] of 0.3399, representing a relative improvement of 46% and 44% respectively over the prior submission, with total inference completed in approximately 21 minutes on a single GPU.
Figures & tables
Figure 1: Overview of the proposed pipeline. Stage 1: Three consecutive VCE frames are processed using a BiomedCLIP ViT-B/16 backbone with LDA for temporal feature refinement. Stage 2: An Excitation Block and multi-head classifier jointly predict anatomy and pathology labels, enhanced using CoOp text features, learnable prototypes, and angular prototype augmentation. Stage 3: Training combines asymmetric focal loss, contrastive BCE, orthogonality regularization, Angular Separation Loss, EMA ( β=0.999 ), temporal Mixup ( α=0.3 ), and inverse-frequency weighted sampling for robust imbalanced learning. Stage 4: A BSM decoder performs temporal smoothing, forward-only GI transit enforcement, and event refinement to generate the final competition JSON with temporally localized multi-label VCE events.
Table 1: Per-video temporal mAP on the RARE-VISION test set (3 NaviCam examinations).
Label
AP
AUC
F1
Prec
Rec
Sup
mouth
0.543
0.947
0.625
0.517
0.789
19
esophagus
0.054
0.996
0.200
0.143
0.333
3
stomach
0.894
0.985
0.814
0.860
0.772
2,920
small intestine
0.893
0.970
0.850
0.803
0.903
14,689
colon
0.991
0.978
0.954
0.951
0.957
40,143
z-line
0.000
—
0.000
—
—
0
Table 2: Per-class validation metrics at the best checkpoint (epoch 3), after temperature calibration ( T∗=1.5 ) and per-class threshold optimization. “Sup” denotes the number of positive frames in the validation set.
Epoch
Train Loss
Val Loss
Val mAP
Macro F1
1
0.2850
0.0696
0.2373
0.2129
2
0.1757
0.0952
0.2421
0.2273
3
0.1457
0.1102
0.2438
0.2395
4
0.1241
0.1411
0.2388
0.2273
5
0.1135
0.1671
0.2337
0.2216
Table 3: Training dynamics over 5 epochs. Best checkpoint at epoch 3 is highlighted.
Video
Frames
Events
Class distribution
ukdd_navi_00051
44,878
22
stomach(1), blood(19), small int.(1), colon(1)
ukdd_navi_00068
53,220
23
stomach(1), blood(15), polyp(2), erosion(3), small int.(1), colon(1)
ukdd_navi_00076
62,927
24
small int.(1), blood(20), erosion(2), colon(1)
Table 4: Predicted event distribution per test video after BSM decoding.