From zero-target transfer to target-site personalization: multi-source training across fall-detection models
Authors: Jaeseung Ryu, Jae-Hun Jung
Organizations: Industrial Data Science, Graduate School of Convergence, Pohang University of Science and Technology (POSTECH), Pohang, 37673, Republic of Korea. · Department of Mathematics and POSTECH Mathematical Institute for Data Science (MINDS), Pohang University of Science and Technology (POSTECH), Pohang, 37673, Republic of Korea.
Video-based fall detection could support timely assistance when a person cannot call for help. Unfamiliar cameras and scarce local labels challenge deployment. We examine whether joint training improves cross-dataset detection and subsequent local training. We analyze 2,697 clips from eight OmniFall datasets and one AI-Hub dataset. Positive clips contain a fall or collapse transition, while negative clips do not. We hold out each dataset in turn, training on the other eight. We evaluate three vision-language models (VLMs), Qwen, LLaVA, and InternVL, and three visual baselines, CLIP-temporal, VideoMAE, and Pose-TCN. We evaluate the stored source model without target data for training or selection. For each label budget, we update a separate copy using labeled clips from the held-out target dataset. We call this target-site personalization. We compare the Source-initialized model with Target-only, which skips source training but uses identical target clips and updates. Across nine targets, source training raises mean area under the receiver operating characteristic curve (AUROC) from frozen-model values of 0.821 to 0.912 for Qwen, 0.883 to 0.947 for LLaVA, and 0.939 to 0.962 for InternVL. With five target examples per class and 26 updates per run, 95% confidence intervals for the AUROC benefit over Target-only lie above zero for Qwen, CLIP-temporal, VideoMAE, and Pose-TCN. With full-target training, they lie above zero only for Qwen and LLaVA. Separate VLM analyses reveal threshold sensitivity and post-fall false detections. The benefit varies by model and label budget.
Figures & tables
Figure 1: Evaluation before and after target-site personalization. For each held-out family, source training produces a stored LOFO checkpoint. The checkpoint is evaluated without using target labels for training or model selection. Separate few-shot and full-target experiments then start from the same source checkpoint, forming the Source-initialized conditions. Each experiment includes a Target-only model initialized from the method’s base state. Few-shot training uses no validation labels. Full-target checkpoint selection uses the target validation partition. Target-test labels are used only to calculate evaluation metrics. Additional threshold and temporal analyses cover all three VLMs and are separate from the primary six-model comparison.
Family
Clips
Falls
All roots
Test
Test roots
Acquisition/domain role
AI-Hub a
892
446
9
260
2
Staged outdoor fixed CCTV, place, light, season, and weather variation
GMDCSA-24 b
217
78
4
39
1
Staged real homes, camera, background, lighting, and occlusion variation
Staged homes, background, appearance, distance, and angle variation
EDF d
225
84
5
47
1
Controlled non-occluded counterpart to OCCU
Table 1: Video families, target tests, and acquisition roles
Method
Optimizer
Learning rate
Regularization
Batch/update details
Full-target schedule
CLIP-temporal
Adam
10−3
Explicit L2 weight 10−4
Batch size 8
100 epochs, cosine schedule
VideoMAE
AdamW
5×10−5
Weight decay 0.05
Batch size 4, four-step gradient accumulation, gradient clipping at 1
5 epochs, cosine schedule
Pose-TCN
AdamW
10−3
Weight decay 10−3
Batch size 8 (few-shot) or 64 (full-target), gradient clipping at 5
50 epochs, cosine schedule
Table 2: Optimization settings for the visual baselines
Qwen2.5-VL
LLaVA-OneVision
InternVL3.5
Target
Frozen
LOFO
Δ
Frozen
LOFO
Δ
Frozen
LOFO
Δ
AI-Hub
0.6289
0.7355
+0.1066
0.7528
0.8251
+0.0723
0.8433
0.8511
+0.0078
GMDCSA-24
0.9511
0.9783
+0.0272
0.9212
1.0000
+0.0788
1.0000
0.9973
−0.0027
MCFD
0.7585
0.8898
+0.1312
0.7671
0.9503
+0.1832
0.8253
0.9208
+0.0955
OCCU
0.6088
0.8032
+0.1944
0.8588
0.9190
+0.0602
0.9456
0.9931
+0.0475
CAUCAFall
0.8742
0.9710
+0.0968
0.8968
0.9742
+0.0774
1.0000
0.9903
−0.0097
Table 3: Zero-target AUROC by held-out target family
Model
Adaptation/head
Trainable
AUROC
BA@ 0.5
MCC@ 0.5
CLIP frame mean
logistic
769
0.795
0.658
0.352
CLIP-temporal
logistic
3,073
0.929
0.819
0.684
VideoMAE
full-model fine-tuning
86.2 M
0.949
0.870
0.757
Pose-TCN
YOLO11x-pose + TCN
80,418
0.802
0.753
0.523
Qwen2.5-VL
frozen
0
0.821
0.710
0.438
LOFO LoRA
10.1 M
0.912
0.814
0.672
Table 4: Model performance with each target family held out
Figure 2: Absolute AUROC and paired source-initialization effects. For each model, the upper panel shows macro AUROC for Source-initialized and Target-only. The lower panel shows their difference with its pointwise 95% bootstrap interval and a zero-effect reference line. Intervals are displayed only on the difference scale. Point estimates average three run probabilities per clip before calculating each family metric. Few-shot estimates also average the three support-set metrics within each family. Every family has equal weight. Few-shot conditions share support labels and 26 updates. Full-target conditions use all target-training clips and validation-based selection. The horizontal positions denote separate experiments at different budgets, not sequential checkpoint updates. Full-target training also changes the training duration and validation policy.
Model
Zero-target LOFO source model
Source- initialized k=5
Source- initialized k=10
Source- initialized k=20
Source- initialized full-target
Qwen2.5-VL
0.9119
0.9237
0.9236
0.9378
0.9522
LLaVA-OneVision
0.9465
0.9425
0.9539
0.9614
0.9749
InternVL3.5
0.9621
0.9538
0.9657
0.9687
0.9746
CLIP-temporal
0.9294
0.9295
0.9347
0.9444
0.9513
VideoMAE
0.9493
0.9489
0.9588
0.9666
0.9741
Pose-TCN
0.8020
0.8318
0.8267
0.8479
0.8750
Table 5: AUROC of Source-initialized models at each target-label budget
Model
Zero- target
Δk=5 [95% CI]
Δk=10 [95% CI]
Δk=20 [95% CI]
Δ full [95% CI]
Full-target Source- initialized AUROC
Trainable
Qwen2.5-VL
0.912
+0.064 [+0.032, +0.104]
+0.037 [+0.020, +0.059]
+0.023 [+0.007, +0.042]
+0.020 [+0.004, +0.035]
0.952
10.1 M
LLaVA-OneVision
0.947
+0.016 [ −0.012 , +0.032]
+0.020 [+0.003, +0.034]
+0.013 [+0.002, +0.028]
+0.016 [+0.001, +0.032]
0.975
10.1 M
InternVL3.5
0.962
+0.007 [ −0.019 , +0.033]
+0.012 [ −0.007 , +0.031]
+0.008 [ −0.005 , +0.023]
+0.006 [ −0.004 , +0.012]
0.975
15.3 M
CLIP-temporal
0.929
+0.021 [+0.007, +0.037]
+0.015 [+0.001, +0.031]
+0.003 [ −0.008 , +0.011]
−0.002 [ −0.009 , +0.004]
0.951
3,073
VideoMAE
0.949
+0.057 [+0.017, +0.100]
+0.040 [+0.007, +0.078]
+0.038 [+0.010, +0.072]
+0.007 [ −0.001 , +0.018]
0.974
86.2 M
Pose-TCN
0.802
+0.109 [+0.041, +0.176]
+0.065 [+0.020, +0.105]
+0.048 [+0.010, +0.083]
+0.021 [ −0.010 , +0.053]
0.875
80,418
Table 6: Transfer to held-out target families and paired personalization across six models
Model
k /class
AUROC W/T/L
Δ BA [95% CI]
Δ MCC [95% CI]
Qwen2.5-VL
5
9/0/0
+0.067 [+0.031, +0.110]
+0.127 [+0.058, +0.205]
10
9/0/0
+0.035 [ −0.001 , +0.058]
+0.079 [+0.010, +0.125]
20
8/0/1
+0.028 [+0.005, +0.048]
+0.061 [+0.020, +0.098]
LLaVA-OneVision
5
8/0/1
+0.042 [+0.001, +0.082]
+0.091 [+0.016, +0.166]
10
8/0/1
+0.027 [ −0.009 , +0.045]
+0.057 [ −0.015 , +0.093]
20
5/3/1
+0.003 [ −0.017 , +0.029]
+0.005 [ −0.032 , +0.054]
Table 7: Effects of source initialization on few-shot BA, MCC, and AUROC across six models
Qwen2.5-VL
LLaVA-OneVision
InternVL3.5
Target
Target- only
Source- initialized
Δ
Target- only
Source- initialized
Δ
Target- only
Source- initialized
Δ
AI-Hub
0.7914
0.8035
+0.0121
0.8814
0.8732
−0.0081
0.8686
0.8720
+0.0034
GMDCSA-24
1.0000
1.0000
+0.0000
1.0000
1.0000
+0.0000
1.0000
0.9973
−0.0027
MCFD
0.8602
0.9224
+0.0621
0.9410
0.9922
+0.0512
0.9534
0.9814
+0.0280
OCCU
0.9954
1.0000
+0.0046
1.0000
1.0000
+0.0000
1.0000
1.0000
+0.0000
CAUCAFall
0.9871
0.9774
−0.0097
1.0000
1.0000
+0.0000
1.0000
1.0000
+0.0000
Table 8: Full-target VLM personalization by target family
CLIP-temporal
VideoMAE
Pose-TCN
Target
Target- only
Source- initialized
Δ
Target- only
Source- initialized
Δ
Target- only
Source- initialized
Δ
AI-Hub
0.8092
0.8028
−0.0064
0.8796
0.8916
+0.0120
0.5831
0.5803
−0.0028
GMDCSA-24
0.9864
0.9891
+0.0027
0.9946
1.0000
+0.0054
0.9538
0.9620
+0.0082
MCFD
0.9752
0.9658
−0.0093
0.9752
0.9876
+0.0124
0.6460
0.7484
+0.1025
OCCU
0.9861
0.9884
+0.0023
1.0000
0.9977
−0.0023
1.0000
0.9722
−0.0278
CAUCAFall
1.0000
0.9968
−0.0032
1.0000
1.0000
+0.0000
0.9968
1.0000
+0.0032
Table 9: Full-target visual-baseline personalization by target family
Figure 3: Relationships among dataset families in frozen CLIP feature space. (a) Recall of the family classifier on group-disjoint test groups after train/validation fitting. (b) Distances between family centroids using all unlabeled embeddings, including test images. (c) Fall-minus-non-fall vectors using all labels, including test labels. Panel (b) supplies the exploratory target-aware similarity grouping. Primary LOFO training and selection use neither (b) nor (c). The axis labels denote AI-Hub (AH), GMDCSA-24 (GM), MCFD (MC), OCCU (OC), CAUCAFall (CA), EDF (ED), Le2i (L2), OOPS-Fall (OO), and UP-Fall (UP).
Rule
Target information
BA
MCC
Fixed 0.5
none (fixed)
0.797
0.619
Mean-score
unlabeled target scores
0.837
0.684
Gaussian mixture (GMM)
unlabeled target scores
0.830
0.677
Otsu
unlabeled target scores
0.825
0.674
Oracle
test labels
0.882
0.766
Table 10: Selected threshold summaries across the 54 VLM–target–model-state combinations
Figure 4: Threshold and temporal analyses for the three VLMs. Panels (a)–(c) show in-sample recall against achieved FPR on 3,122 windows from the same 446 AI-Hub recordings used in the primary subset. Solid curves denote Frozen models and dashed curves denote the selected LOFO models. Large markers identify thresholds fitted to each model and state’s 2,676 labeled negatives at nominal 1% FPR. Panel (d) shows the eight-frame window-position sweep averaged over five selected target families. The horizontal axis gives the nominal position of the cached clip center within the selected input, not a verified fall timestamp. Both positive and negative windows change. Sweep AUROCs are not directly comparable with primary strided-input AUROCs.
Figure 5: Qualitative error sequences for Qwen2.5-VL. Each row shows the eight cached model-input frames (indices 0, 2, …, 14), with ground truth, scene category, frozen and LOFO scores, and the strict decision rule p>0.5 . Panels A and D use the primary three-run LOFO probability ensemble. Panels B and C use the prespecified AI-Hub LOFO run and show post-fall windows. Crops improve print visibility only. Predictions use full uncropped frames. Displayed heads were de-identified. Dataset-specific rights, attribution, and redistribution boundaries are reported in the Declarations.
Visual fall detection models are usually trained on small, staged datasets. Their real-world utility remains unclear; such data lacks diversity and evaluation protocols differ from paper to paper. We propose OmniFall, a unified benchmark of 15k videos (80 hours) with frame-level annotations in a single 16-class taxonomy. It spans three domains: OF-Staged unifies eight staged datasets with cross-subject and cross-view splits; OF-Synthetic adds 12k videos (17 h) with controlled demographic and environmental diversity; and OF-In-the-Wild provides a test-only set of genuine accident videos. We evaluate fine-tuned models as well as much larger zero-shot multimodal LLMs. On in-the-wild fall events, both do comparably well. The clinically critical fallen state is where they part: zero-shot models keep confusing fallen with lying, whereas models fine-tuned on synthetic data with explicit fallen-state scenes do substantially better. We release the unified annotations, the synthetic data, and the in-the-wild test set to foster the development of fall and fallen-state detectors for uncontrolled environments. Dataset: https://hf.co/datasets/simplexsigil2/omnifall
David Schneider, Zdravko Marinov, Moritz Mistol +6
Karlsruhe Institute of Technology · Mercedes-Benz Tech Innovation
Thermal cameras can watch over older people living alone without capturing faces, and multimodal large language models (LLMs) can judge a short video without task-specific training. We evaluate a production multimodal LLM, Gemini 3.8 Flash at temperature 0, as the decision stage of an event-triggered fall screener, used exactly as a deployed prototype uses it: one request per short thermal clip, with its sound track when there is one, and its predecessor Gemini 3.7 Flash on the same requests. On 438 staged falls and 563 non-fall windows that we located by hand in a public 32x24 thermal dataset, the deployed request called 69% of the held-out subjects' falls (67% over all subjects; bed falls excluded) and none of 499 non-fall windows, including people lying on the floor and getting up. Over 76 h of ordinary life in an occupied room, recorded at 160x120 with sound, it called none of 395 clips a fall (95% upper bound 1.2 false alarms per day) and caught 4 of the 9 staged falls the camera could see. One cautionary sentence in the prompt ("a person who goes out of view ... is not a fall") accounted for many of the misses: removing it raised sensitivity on the held-out subjects from 69% to 83% on 3.8 Flash and from 80% to 92% on Gemini 3.7 Flash, and in the home from 4 to 9 of the 9 visible falls, with no false call in ordinary living but at the price of calling deliberate lie-downs a fall. Sensitivity collapsed at 8x6 pixels (34%); identical requests disagreed on 4% of windows; and when the model called a fall, its explanation placed it within a second of the annotated landing 99% of the time. Two conventional detectors trained on the laboratory data caught nearly every held-out laboratory fall but raised 14.2 and 23.9 false alarms per day in the home. We release the frame-level fall annotations of the public dataset and every model answer on it.
Personalized fall detection models can significantly improve accuracy by adapting to individual motion patterns, yet their effectiveness is often limited by the scarcity of real-world fall data and the dominance of non-fall feedback samples. This imbalance biases the model toward routine activities and weakens its sensitivity to true fall events. To address this challenge, we propose a personalization framework that combines semi-supervised clustering with contrastive learning to identify and balance the most informative user feedback samples. The framework is evaluated under three retraining strategies, including Training from Scratch (TFS), Transfer Learning (TL), and Few-Shot Learning (FSL), to assess adaptability across learning paradigms. Real-time experiments with ten participants show that the TFS approach achieves the highest performance, with up to a 25% improvement over the baseline, while FSL achieves the second-highest performance with a 7% improvement, demonstrating the effectiveness of selective personalization for real-world deployment.
Awatif Yasmin, Tarek Mahmud, Sana Alamgeer +1
Texas State University · Texas A&M University - Kingsville