Vision-Language Model-Based Phase-OTDR Event Classification
Organizations: Department of Computer Engineering, Isparta University of Applied Sciences, 32040 Isparta, Turkey · Department of Electrical and Electronics Engineering, Izmir Institute of Technology, 35430 Urla, Izmir, Turkey · Department of Biotechnology and Bioengineering, Izmir Institute of Technology, 35430 Urla, Izmir, Turkey
Abstract
Phase-sensitive optical time-domain reflectometry (Phase-OTDR) enables distributed vibration sensing, but reliable event classification has traditionally relied on fully supervised models that require extensive annotated data. This study examines the adaptability of a Vision-Language Model (VLM) for Phase-OTDR event recognition through few-shot learning. Raw samples are converted into Short-Time Fourier Transform (STFT) dB-magnitude spectrogram mosaics, color-mapped and resized to RGB images, and processed by a Contrastive Language-Image Pre-training (CLIP) image encoder; few-shot adaptation optimizes only the Tip-Adapter-F cache keys, leaving both encoders unchanged. Under a leakage-controlled protocol (train-side support/validation pool of 512 images per class, fixed held-out test of 120 per class, no test-set access during selection, 20 distinct support-sampling trials), the selected configuration (the ViT-L-14 Vision Transformer encoder with OpenAI weights, 128 shots per class) reaches 96.81 +/- 0.42% accuracy and a macro-averaged F1 score of 96.79 +/- 0.42% on the held-out test set; a label-efficient 16-shot setting, that is, 96 labeled support images at adaptation time across the six classes, already attains 90.17 +/- 0.81%. Tip-Adapter-F clearly outperforms both zero-shot CLIP and the standard training-free Tip-Adapter, and prompt and backbone analyses show that performance is stable across Vision Transformer encoders and insensitive to prompt wording at the selected operating point. Overall, CLIP visual features carry a transferable structure that a lightweight support cache turns into a label-efficient Phase-OTDR event classifier. The code and the corresponding image dataset are made publicly available on GitHub to support further research: https://github.com/miralab-ai/vlm-phase-otdr.
Figures & tables
| Method | Representation pre-training | Unlab. data | Shots / class | Trained components | Acc. |
| 2-D CNN [ 1 ] | none (random init.) | no | full split | whole network | |
| VBA-HA [ 20 ] | VGGish (audio) | no | 70 | VGGish head, BiLSTM, attention | |
| SS-BiMamba [ 18 ] | self-supervised on -OTDR | yes | 16 | encoder + MLP head; pseudo-labels | |
| Self-distill. [ 19 ] | self-supervised on -OTDR | yes | 15 | encoder + head | |
| This study | CLIP (web image–text), frozen | no | 16 | support-cache keys only | |
| This study | CLIP (web image–text), frozen | no | 128 | support-cache keys only |
| Class | Label | Benchmark recordings | Train-side pool | Held-out test | Subset total |
|---|---|---|---|---|---|
| Background | 0 | 2,946 | 512 | 120 | 632 |
| Digging | 1 | 2,512 | 512 | 120 | 632 |
| Knocking | 2 | 2,530 | 512 | 120 | 632 |
| Watering | 3 | 2,253 | 512 | 120 | 632 |
| Shaking | 4 | 2,728 | 512 | 120 | 632 |
| Walking | 5 | 2,450 | 512 | 120 | 632 |
| Setting | Stage 1 (backbone screening) | Stage 2 (hyperparameter tuning) |
|---|---|---|
| CLIP backbones | 8 candidates | Top 2 from Stage 1 |
| Prompt families | 4 | 4 |
| Shots per class | ||
| Fusion | validation-selected | , fixed |
| Training settings | lr ; 20 epochs; batch 32 | ; epochs ; batch size |
| Seeds |
| Backbone | Val macro-F1 | Val accuracy | |
|---|---|---|---|
| ViT-L-14/openai | 83.14% | 84.06% | 72 |
| ViT-H-14/laion2b | 82.10% | 82.88% | 72 |
| ViT-B-16/openai | 78.52% | 79.56% | 72 |
| ViT-B-32/openai | 77.50% | 78.53% | 72 |
| RN50/openai | 75.43% | 76.28% | 72 |
| RN50x4/openai | 73.57% | 75.03% | 72 |
| Stage | Accuracy | Macro-F1 |
|---|---|---|
| Zero-shot CLIP | 16.67% | 4.78% |
| Tip-Adapter | ||
| Tip-Adapter-F |
| Prompt family | Representative content |
|---|---|
| basic_clean | Three prompts per class built from a short descriptor : itself, “a clean spectrogram image of ” and “a fiber optic sensing image showing ”. |
| domain_visual (selected) | The four templates applied to the short descriptor (e.g. “digging-induced vibration pattern in a fiber optic sensing image”), plus a descriptive per-class paragraph; five prompts per class. |
| confusion_focused | The four templates applied to contrastive descriptors that state what each class is not (e.g. “dig class: irregular excavation activity, not a single knock, not smooth water, not periodic shaking, not clean footsteps”), plus the descriptor itself; five prompts per class. |
| shaking_focused | The four templates applied to descriptors written to separate Shaking from Watering and Walking, plus that descriptor and the confusion_focused descriptor; six prompts per class. |
| Prompt family | Test accuracy | Test macro-F1 |
|---|---|---|
| basic_clean | ||
| domain_visual (selected) | ||
| confusion_focused | ||
| shaking_focused |
| Source | Quantity | Value |
|---|---|---|
| Support draw (20 draws) | Standard deviation | pp |
| 95% CI of the mean | ||
| Finite test set ( ) | Binomial standard error | pp |
| 95% Wilson interval | ||
| Combined (single draw) | Standard uncertainty | pp |
| Per class ( ) | Recall standard error | – pp |