VioTech: Frame-Level Violin Playing Technique Detection with Multi-Scale Mixture-of-Experts
Organizations: Institute of Information Science, Academia Sinica, Taipei, Taiwan · Original Content Center, Gamania Inc., Taipei, Taiwan · Sony Computer Science Laboratories, Inc., Tokyo, Japan · National Taiwan University, Taipei, Taiwan
Abstract
Playing technique is crucial for expressive violin performance, yet few violin technique detection models exist. Current systems mainly rely on note-level pipelines that first estimate note boundaries and then classify techniques. This is especially problematic for bowed string instruments such as the violin, where soft onsets and continuous bow-string interaction render note segmentation unreliable. Here, we propose VioTech, a frame-level violin technique detection framework that predicts techniques directly at 10 ms resolution without explicit note segmentation. To handle the large temporal-scale variation among techniques, VioTech combines a frozen MERT encoder with a multi-scale Mixture-of-Experts (MoE) architecture and task-specific routing for tonal, articulation, and legato-related predictions. We also introduce VioTech120, the first manually annotated dataset of violin playing techniques constructed from real solo performance recordings, as well as an accompanying note-labeling user interface. Results demonstrate that VioTech achieved consistent improvements over prior methods and significantly enhanced cross-dataset generalization, establishing a new state-of-the-art in violin technique-aware transcription.
Figures & tables
| Model | Bach10 | URMP | ||||||
| P | R | F1 | F1 | P | R | F1 | F1 | |
| VioPTT [ 1 ] | 48.7 | 53.0 | 50.7 | 62.5 | 44.4 | 44.9 | 44.6 | 54.9 |
| VioTech w/o MoE | 54.8 | 58.0 | 56.1 | 65.2 | 39.1 | 57.5 | 46.3 | 59.7 |
| VioTech (Ours) | 60.9 | 65.0 | 62.8 | 79.1 | 45.6 | 60.4 | 51.9 | 69.3 |
Automatic Music Transcription
Automatic music transcription converts recorded music into symbolic notes, rhythms, instruments, and sometimes readable scores or tablature.
See all 39 papers