cs.CVNov 3, 2026

KeyWise: Efficient Video-Only Piano Transcription with a Lightweight Per-Key Architecture

Authors: Anastasios Chatzigiannis, Natalia Panagou, Maria Koziri

Organizations: Department of Informatics and Telecommunications, University of Thessaly, Lamia, Greece.

Abstract

Audio-based piano transcription is hampered by the sustain pedal and by noisy or missing soundtracks, problems that video-only transcription avoids, since a top-view video shows directly when each key is pressed and released. Most visual transcription networks, however, describe the whole keyboard with one global feature vector and must learn where each key is. In this paper, we propose KeyWise, a lightweight per-key architecture that exploits the known geometry of the keyboard instead of learning it. The rectified keyboard is divided into 88 equal tiles that serve as regions of interest (ROIs), and one small recurrent network, shared by all keys, follows each key over time. On PianoVAM, KeyWise reaches the onset accuracy of the state-of-the-art V2N (95.0% vs. 94.7% onset F1 at 50 ms and 97.8% vs. 97.0% at 100 ms) with 1.3 M instead of 28.2 M parameters, and it runs 4.7 times faster on the same GPU. At 100 ms, it is also significantly more accurate when the key releases (97.0% vs. 95.5% F1) or the velocities (89.4% vs. 84.5% F1) must be correct as well. On PianoYT, KeyWise pre-trained on PianoVAM reaches 69.4% onset F1 at 100 ms, comparable to the best published result, although it needs fine-tuning for new recording conditions.

Figures & tables

Explore similar work

CardsList
  1. Multi-Task Multi-Frame Visual Piano Transcription

    Aug 4, 2026Yonghyun Kim, Hoyeol Sohn, Juhan Nam +1Automatic Music TranscriptionTemporal Video Understanding

  2. A learning-based method of automatic music transcription

    Oct 29, 2026Phu-Anh-Tai Do, Huu-Hoang Nguyen, Duc-Hoang Nguyen +1Automatic Music TranscriptionAttention Mechanisms

  3. Statistical Models for Automatic Fingering-Annotated Piano Sheet Music Transcription

    Sep 23, 2026Daniel Penner, Abram HindleAutomatic Music Transcription