Learning-based Axial Video Motion Magnification
Organizations: Graduate School of AI, POSTECH · Department of Electrical Engineering, POSTECH · Institute for Convergence Research and Education in Advanced Technology, Yonsei University
Abstract
Video motion magnification amplifies invisible small motions to be perceptible, which provides humans with a spatially dense and holistic understanding of small motions in the scene of interest. This is based on the premise that magnifying small motions enhances the legibility of motions. In the real world, however, vibrating objects often possess convoluted systems that have complex natural frequencies, modes, and directions. Existing motion magnification often fails to improve legibility since the intricate motions still retain complex characteristics even after being magnified, which may distract us from analyzing them. In this work, we focus on improving legibility by proposing a new concept, axial motion magnification, which magnifies decomposed motions along the user-specified direction. Axial motion magnification can be applied to various applications where motions of specific axes are critical, by providing simplified and easily readable motion information. To achieve this, we propose a novel Motion Separation Module that enables to disentangle and magnify the motion representation along axes of interest. Furthermore, we build a new synthetic training dataset for the axial motion magnification task. Our proposed method improves the legibility of resulting motions along certain axes by adding a new feature: user controllability. Axial motion magnification is a more generalized concept; thus, our method can be directly adapted to the generic motion magnification and achieves favorable performance against competing methods.
Figures & tables
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
| Hyperparameters | Unit | Value |
|---|---|---|
| Vibration frequency | Hz | 20 |
| Peak amplitude of acceleration | 4.11 | |
| Camera-to-vibrator distance | m | 2 |
| Focal length | mm | 100 |
| Per-pixel sensor size | m | 5.86 |
| Method | Singh et al . [ 31 ] | Pan et al . [ 26 ] | STB-VMM [ 17 ] | DMM [ 24 ] | Ours | ||
|---|---|---|---|---|---|---|---|
| D1-model | D2-model | 1-recursive | 2-recursive | ||||
| FPS | 8.4 | 7.2 | 19.6 | 9.8 | 2.6 | 17.4 | 14.4 |