Organizations: Yonsei University, Seoul, South Korea · University of Michigan, Ann Arbor, MI, United States · University of Illinois Urbana-Champaign, IL, United States
Surface-mounted vibration sensing provides non-invasive structural monitoring with high temporal fidelity across diverse ubiquitous computing scenarios such as smart farms, yet structure-borne wave models remain fragile to physical relocation and structural shifts. However, addressing this spatial data discontinuity remains limited by costly manual recalibration, insufficient physical grounding in data-driven synthesis, and substantial sim-to-real gaps in numerical digital twins. To resolve this spatial bottleneck without labor-intensive data recollection, we propose VeV, a physics-consistent spatial augmentation framework that synthesizes vibration responses at unmeasured target coordinates from sparse reference observations by coupling a monocular Vision-Derived Propagation Possibility Map for structural wave-guiding paths with a Visual Feature Map for transient source kinetics and implicit material dynamics. Evaluated on a 1,237 sensor-hour sow pig and piglet farm across disparate structures, VeV reduces waveform reconstruction MSE by over 57.5% compared to data-driven baselines and achieves up to 81.69% zero-shot classification accuracy, while reaching peak performance within just 30-50 calibration chunks to effectively eliminate cold-start retraining overhead and establish a scalable foundation extending to human-centric floor vibration sensing and ambient intelligence.
Figures & tables
Figure 1. Illustration of VeV , from Observation to Deployment (Expend sensing). (Part of this illustration was generated using [ChatGPT 6 Astra ( OpenAI, 2026 ) ] to visualize the abstract workflow)
Figure 2. Illustration of intra-structural topology expansion and cross-structural multi-point migration. (This illustration was generated using [ChatGPT 6 Astra ( OpenAI, 2026 ) ] to visualize the abstract workflow)
Figure 3. Two types of crates with subtle differences, and visualization of sensor locations.
Vision Accuracy
Vibration Accuracy
Seen Crate
Unseen Crate
Arch
Size
Seen Crate
Unseen Crate
Seen Sensor Loc.
Unseen Sensor Loc.
Seen Sensor Loc.
Unseen Sensor Loc.
CNN
small
0.8465
0.8381
0.8027
0.3421
0.1802
0.1145
medium
0.8548
0.8471
0.7598
0.2854
0.0957
0.0712
large
0.7810
0.7911
0.7170
0.2519
0.0801
0.0583
RNN
small
0.7323
0.7350
0.8661
0.3812
0.1378
0.0924
Table 1. Performance Comparison: Vision vs. Vibration under Crate and Sensor Generalization.
Figure 4. 3D CAD and FEM simulation setup constructed based on the visual layout of the Unseen Crate in Figure 3(b) .
Augmentation Paradigm
Seen Crate
Unseen Crate
Seen Loc
Unseen Loc
Seen Loc
Unseen Loc
Classifier Upper Bound (Classifier w/ real data)
90.40%
91.42%
90.96%
89.87%
(1) Naive Data-Driven (Classifier w/ LSTM based synthetic data)
38.05%
28.35%
42.20%
21.94%
(2-a) Audio Encoder (Classifier w/ WavLM based synthetic data)
52.74%
53.34%
63.03%
54.33%
(2-b) Audio Encoder (Classifier w/ BEATs based synthetic data)
59.23%
61.73%
69.91%
63.07%
(3-a) Video-to-Audio (Classifier w/ MMAudio based synthetic data)
5.12%
4.97%
4.12%
4.96%
Table 2. Downstream classification accuracy across varying combinations of seen/unseen crates and sensor locations.
Figure 5. Overall architecture of VeV .
Crate
Crate Group
Seen Loc (2,3,5) [hours]
Unseen Loc (1,4) [hours]
Total [hours]
Crate_02
Seen
139.692
0.000
139.692
Crate_03
Seen
139.827
0.000
139.827
Crate_08
Seen
277.185
185.233
462.419
Crate_04
Unseen
68.455
0.000
68.455
Crate_06
Unseen
69.896
0.000
69.896
Crate_07
Unseen
214.073
142.661
356.734
Table 3. Crate Group Duration Summary
Figure 9
Method
Seen Crate + Seen Loc.
Seen Crate + Unseen Loc.
Unseen Crate + Seen Loc.
Unseen Crate + Unseen Loc.
Acc. (%) ↑
Micro-F1 ↑
Acc. (%) ↑
Micro-F1 ↑
Acc. (%) ↑
Micro-F1 ↑
Acc. (%) ↑
Micro-F1 ↑
Naive
38.05%
0.3470
28.35%
0.1909
42.20%
0.2554
21.94%
0.1916
Audio
59.23%
0.4243
61.73%
0.4288
69.91%
0.2075
63.07%
0.1923
FEM
0%
0.0
0%
0.0
0%
0.0
0%
0.0
Ours
81.69%
0.8180
78.54%
0.7437
72.16%
0.6852
71.69%
0.6749
Table 5. Downstream task performance evaluation across different crate and sensor location settings (Loc. = Sensor Location).
Figure 7. performance shift after finetune
Figure 8. Vibration waveform and optical flow magnitude to capture pig’s movement
Figure 9. Occlusion by environment.
Figure 10. LDA across methods on 4 different environments
Figure 11. Feature covariance eigenspectrum across principal axis indices (log scale).
Figure 12. Performance comparison under static image inputs across different environments.
Method
Seen Crate + Seen Loc.
Seen Crate + Unseen Loc.
Unseen Crate + Seen Loc.
Unseen Crate + Unseen Loc.
Acc. (%) ↑
Micro-F1 ↑
Acc. (%) ↑
Micro-F1 ↑
Acc. (%) ↑
Micro-F1 ↑
Acc. (%) ↑
Micro-F1 ↑
Proposed method ( VeV )
81.69%
0.8180
78.54%
0.7437
72.16%
0.6852
71.69%
0.6149
Visual Feature Map Only
64.80%
0.6321
62.54%
0.7107
58.71%
0.5752
59.20%
0.5471
Propagation Map Only
29.58%
0.2717
29.56%
0.1492
32.21%
0.2763
28.57%
0.2682
Table 6. Downstream task performance evaluation across different crate and sensor location settings (Loc. = Sensor Location).
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Architecture
Size
Vibration Params
Vision Params
CNN
small
141,445
99,589
medium
561,413
391,685
large
2,236,933
1,553,413
RNN
small
125,445
183,237
medium
793,093
1,019,269
large
4,341,765
5,236,485
Appendix
Table 7. Model Parameter Counts across Architectures and Scales.
Configuration Category
Design Specification
Value / Choice
Material Density ( ρ )
Structural Steel Baseline
7850 kg/m3
Young’s Modulus ( E )
Baseline Elasticity
200.0 GPa
Poisson’s Ratio ( ν )
Transverse Contraction
0.30
Rayleigh Damping ( α,β )
Mass/Stiffness Damping
0.05 s−1,1.0×10−5 s
Spatial Element Type
3D Tetrahedron
10-node quadratic tetrahedral
Polynomial Order ( p )
Quadratic Basis
p=2 (Mitigate shear locking)
Appendix
Table 8. FEM Simulation Configurations and Structural Steel Parameters
Reliable displacement measurement is fundamental for structural health monitoring and digital engineering workflows, as it provides direct structural response information. Vision-based measurement has emerged as a promising approach for low-cost, non-contact displacement monitoring. However, its deployment often remains constrained by task-specific model training or on-site preparation, such as marker installation or manual camera calibration. This study presents a Vision Foundation Model-based framework for Structural Displacement Measurement (VFM-SDM) that integrates VFM-inferred camera parameter estimation and point tracking to reconstruct multi-directional structural displacements via triangulation without task-specific training or on-site preparation, enabling efficient non-contact deployment in real-world applications. Structural geometry constraints are incorporated to suppress physically implausible deviations and improve estimation consistency. A multi-modal field dataset collected from an in-service pedestrian bridge is introduced alongside a unified benchmarking protocol to support reproducible evaluation. Representative results show low amplitude errors (NRMSErange: 0.11/0.12), strong temporal agreement (correlation coefficient: 0.86/0.88), and small peak-to-peak amplitude errors (RPPAE: 0.01/0.02) for vertical and lateral displacements, indicating robust performance under real-world conditions. The proposed framework advances automated, scalable displacement monitoring and lays the groundwork for VFM-enabled structural response measurements in digital twin and data-centric construction workflows.
Qingyu Xian, Hao Cheng, Berend Jan van der Zwaag +2
Pervasive Systems Research Group, Faculty of Electrical Engineering, Mathematics and Computer Science, University of Twente, Enschede, The Netherlands · Department of Earth Observation Science, Faculty of Geo-Information Science and Earth Observation (ITC), University of Twente, Enschede, The Netherlands · Department of Civil Engineering and Management, Faculty of Engineering Technology, University of Twente, Enschede, The Netherlands
Damage identification is a core task in structural health monitoring. In practice, however, its reliability is often compromised by confounding non-damage effects, such as variations in excitation and environmental conditions, which can induce changes comparable to or larger than those caused by structural damage. To address this challenge, this study proposes a self-supervised label-free disentangled representation learning framework for robust vibration-based structural damage identification. The proposed framework employs an autoencoder with two latent representations to learn directly from raw vibration acceleration signals. A self-supervised invariance regularization, implemented via Variance-Invariance-Covariance Regularization (VICReg), is imposed on one latent representation using baseline data where structural damage is assumed constant but operational and environmental conditions vary. In addition, a frequency-domain constraint is introduced to enforce agreement between the power spectral density reconstructed from the latent representation and that computed from the corresponding input time series. Together, these mechanisms promote disentanglement, enabling the learned representation to be sensitive to damage-related characteristics while remaining invariant to nuisance variability. The framework is trained in a fully end-to-end and label-free manner, requiring no prior information on damage, excitation, or environmental conditions, making it well-suited for real-world applications. Its effectiveness is validated on two distinct real-world vibration datasets, including a bridge and a gearbox. The results demonstrate robustness to operational variability, strong generalization capability, and good performance in both damage detection and quantification.
Xudong Jian, Charikleia Stoura, Simon Scandella +1
Department of Civil, Environmental and Geomatic Engineering, ETH Zürich, Zürich, Switzerland. · Department of Mechanical Engineering, Politecnico di Milano, Italy
Structural health monitoring plays a critical role in ensuring structural safety by analyzing vibration responses from engineering systems. This paper proposes a Spectro-Temporal Alignment framework and a Hybrid Spectro-Temporal Fusion framework that integrate arrival-time interval descriptors with spectral features to capture both fine-scale and coarse-scale vibration dynamics. Experiments conducted on data collected from an LDS V406 electrodynamic shaker demonstrate that the proposed spectro-temporal representations significantly outperform conventional input formulations. The results indicate that a temporal resolution (Δτ) of 0.008 of 0.02 favors traditional machine learning models, whereas a finer resolution (Δτ) of 0.008 effectively unlocks the performance potential of deep learning architectures. Beyond classification accuracy, a comprehensive stability analysis based on condensed indices, including mean performance, standard deviation, coefficient of variation, and balanced score, shows that the proposed hybrid framework consistently achieves higher accuracy with substantially lower variability compared to baseline and alignment-only approaches. Overall, these results demonstrate that the proposed framework provides a robust, accurate, and reliable solution for vibration-based structural health monitoring.
Jongyeop Kim, Jinki Kim, Doyun Lee
aDepartment of Information Technology, Georgia Southern University Statesboro, GA, USA · bDepartment of Mechanical Engineering, Georgia Southern University Statesboro, GA, USA · cDepartment of Civil Engineering and Construction, Georgia Southern University Statesboro, GA, USA