Can Music Generation Models Tap Along? Evaluating Generative Approaches to Beat Tracking
Organizations: Schulich School of Music, McGill University, Montréal, QC, Canada · Centre for Interdisciplinary Research in Music Media and Technology (CIRMMT), Montréal, Canada · Courant Institute, New York University, New York, USA · Mohamed bin Zayed University of Artificial Intelligence (MBZUAI), Abu Dhabi, UAE
Abstract
Existing studies have shown that music generative models can generate synchronized tracks that play alongside an input song. Can we extend this capability to develop a generative approach to beat tracking? To answer this question, we equip a pretrained MusicGen model with an audio-conditioning adapter to generate percussion tracks synchronized with the input music, from which we extract the beat positions. We train separate models with percussion targets of varying rhythmic complexity and compare them with two other generative models and conventional beat trackers. On GTZAN, our model achieves performance close to state-of-the-art beat trackers. Our model also demonstrates greater robustness: it shows the smallest drop in F-measure when drums are removed from the input at inference time. In certain genres, such as metal and disco, our model is also less likely to be misled by strong onsets or to miss beats with weak onsets. These results suggest that percussion generation has the potential to become a viable approach to beat tracking, and that its output is not merely a sequence of beat positions but an audible percussion track.
Figures & tables
| Method | Training songs | Params | F 70 | F 100 | F 150 | CMLt | AMLt | CMLc | AMLc | PQ |
| Ours (Click) | 1319 (49 h) | 10.4 M | 0.851 | 0.871 | 0.889 | 0.756 | 0.907 | 0.731 | 0.874 | 6.27 |
| Ours (Groove synth) | 1319 (49 h) | 10.4 M | 0.850 | 0.866 | 0.882 | 0.736 | 0.912 | 0.713 | 0.881 | 6.88 |
| Ours (Groove sampled) | 1319 (49 h) | 10.4 M | 0.849 | 0.863 | 0.878 | 0.727 | 0.907 | 0.704 | 0.877 | 5.93 |
| Stable Audio ControlNet | 1319 (49 h) | 561 M | 0.691 | 0.728 | 0.772 | 0.553 | 0.753 | 0.512 | 0.687 | 7.54 |
| STAGE | 214 (13 h) | 420 M | 0.335 | 0.443 | 0.609 | 0.244 | 0.482 | 0.201 | 0.387 | 7.09 |
| Beat This! | 4556 (167 h) | 20.3 M | 0.888 | 0.901 | 0.913 | 0.793 | 0.895 | 0.751 | 0.834 | – |