cs.HCNov 6, 2026

VeV: Video-Enhanced Vibration Data Augmentation For Structural Difference

Authors: Joonhee Lee, Jiale Zhang, Sungmin Lee, Ruiqing Wang, Jeffrey Shulkin, Yiwen Dong, Pei Zhang, JeongGil Ko

Organizations: Yonsei University, Seoul, South Korea · University of Michigan, Ann Arbor, MI, United States · University of Illinois Urbana-Champaign, IL, United States

Abstract

Surface-mounted vibration sensing provides non-invasive structural monitoring with high temporal fidelity across diverse ubiquitous computing scenarios such as smart farms, yet structure-borne wave models remain fragile to physical relocation and structural shifts. However, addressing this spatial data discontinuity remains limited by costly manual recalibration, insufficient physical grounding in data-driven synthesis, and substantial sim-to-real gaps in numerical digital twins. To resolve this spatial bottleneck without labor-intensive data recollection, we propose VeV, a physics-consistent spatial augmentation framework that synthesizes vibration responses at unmeasured target coordinates from sparse reference observations by coupling a monocular Vision-Derived Propagation Possibility Map for structural wave-guiding paths with a Visual Feature Map for transient source kinetics and implicit material dynamics. Evaluated on a 1,237 sensor-hour sow pig and piglet farm across disparate structures, VeV reduces waveform reconstruction MSE by over 57.5% compared to data-driven baselines and achieves up to 81.69% zero-shot classification accuracy, while reaching peak performance within just 30-50 calibration chunks to effectively eliminate cold-start retraining overhead and establish a scalable foundation extending to human-centric floor vibration sensing and ambient intelligence.

Figures & tables

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. VFM-SDM: A vision foundation model-based framework for training-free, marker-free, and calibration-free structural displacement measurement

    May 10, 2026Qingyu Xian, Hao Cheng, Berend Jan van der Zwaag +2Structural Health Monitoring

  2. Disentangling Damage from Operational Variability: A Label-Free Self-Supervised Representation Learning Framework for Output-Only Structural Damage Identification

    Apr 21, 2026Xudong Jian, Charikleia Stoura, Simon Scandella +1Disentangled Representation LearningSelf-Supervised Time Series Representation Learning

  3. Hybrid Spectro-Temporal Fusion Framework for Structural Health Monitoring

    Apr 17, 2026Jongyeop Kim, Jinki Kim, Doyun LeeFrequency-Domain Feature LearningMultiscale Time-Series Modeling