Oct 5, 2026, cs.CVJ/K move · Enter open · S save
Olivia Nocentini, Marta Lagomarsino, Gokhan Solak, Younggeol Cho+5
Human-Robot Interfaces and Interaction Laboratory, Istituto Italiano di Tecnologia, Via S.Quirico 19d, Genoa, Italy · Department of Bioengineering, Sir Michael Uren Hub Imperial College London 86 Wood Lane London W12 0BZ United Kingdom · Department of Technological Innovations and Safety of Plants, Products and Anthropic Settlements, Italian National Institute for Insurance against Accidents at Work (INAIL), Via R.Ferruzzi 38, Rome, Italy
Driver fatigue poses a significant challenge to railway safety, with traditional systems like the dead-man switch offering limited and basic alertness checks. This study presents a vision-based monitoring system that relies solely on a single front-facing RGB camera and a graph neural network to classify simulated train-driver states into alert, not-alert, and an emergency class comprising acted emergency-like behaviours. To optimize input representations for the model, an ablation study was performed, comparing three feature configurations: skeletal-only, facial-only, and a combination of both. Experimental results show that combining facial and skeletal features yields the highest accuracy (81%) for the three-class model under the light condition, outperforming models that use only facial or skeletal features. Furthermore, the combination of facial and skeletal features achieves 99% accuracy in the alert/not alert classification in light condition. Additionally, we introduced a controlled RGB video dataset containing alert, not alert, and acted emergency-like behaviours recorded under three illumination conditions. These contributions represent a step toward passive and non-contact train-driver state recognition based on facial and upper-body dynamics.