bFaaaP: An Inclusive, Head-Angle Piano-Pedal Interaction that Quantitatively Reproduces a Pianist's Intended Pedalling -- Foot-Free, for Acoustic and Electronic Pianos
Organizations: Shishido & Associates, Hachioji, Tokyo, Japan · SeemeData Labs, Inc., Nakano-ku, Tokyo, Japan · Shishido & Associates, Horinouchi, Hachioji, 192-0355, Tokyo, Japan · SeemeData Labs, Inc., 2-19-2 3F Nakano, Nakano-ku, 164-0001, Tokyo, Japan · Ootaki & Architects, Sayama, Saitama, Japan · Ootaki & Architects, Fujimi 1-19-15, Sayama, 350-1306, Saitama, Japan · Naru Science Software, Fujimi-shi, Saitama, Japan · Naru Science Software, Fujimi-shi, 354-0024, Saitama, Japan
Abstract
Expressive piano performance depends on the sustain (damper) pedal, operated by foot, excluding players who cannot readily use their feet: wheelchair users and others with lower-limb impairments, small children, and some elderly or disabled players. We present bFaaaP (barrier-Free assist as a Pedal), an inclusive, foot-free interaction that operates the pedal from the angle of the player's head: a smartphone tracks head pose with on-device augmented-reality (AR) face tracking and streams a compact command over Bluetooth Low Energy (BLE) to a pedal device. Supported by patent examination, our central claim is not the head-to-pedal architecture (anticipated by prior art) but a quantitative, user-tunable control law -- the patentable "key" to a natural, expressive result: the player presets a small angular dead-zone (offset 3-10 degrees) and a multiplier (10-50), which together fix a secondary, pre-adjustable response speed that reproduces the pianist's intended pedalling. An engineering trick decouples the fast AR sampling from the slower BLE rate. Two co-equal realizations share one controller: a non-destructive robotic actuator for acoustic pianos (Pro), anchored by a pneumatic "airback" (our coined term for an inflatable air-braced anchor) that absorbs the reaction force without modifying the instrument; and an electronic sustain switch for digital pianos (Switch). In a human-subject Auxiliary Pedal Effect Evaluation (APEE) with 15 participants, bFaaaP significantly increased sustained-tone energy (p<0.01) and was statistically indistinguishable from a player's own foot (p>0.05), with no significant difference across classes; one participant with a leg disability and a tracheostomy performed successfully. With nothing worn on the face and fast setup, bFaaaP has run in formal public concerts (2018-2025). We release the full hardware and software as open source.
Reinforcement learning can train bimanual dexterous hands to play piano in physics simulation with high note accuracy, but for high-DoF dexterous hands, relying solely on task rewards or IK inversion often leads to unnatural postures and joint overextension. We propose \textit{Adversarial Posture Regularization (APR)}. It avoids expensive, song-aligned expert demonstration data and instead uses a small amount of casual human playing data. By matching the distribution of the posture of the policy with the human prior through an adversarial objective, APR encourages more human-like hand shapes. Meanwhile, we collect and release unstructured hand motion data of piano playing using a consumer-grade Meta Quest 3, and retarget the key motion information to the Shadow Hand. Finally, we achieve significantly better performance than prior methods on all three human-likeness metrics (cPSI, BSE, and FAC) as well as in visual quality. Project repository: https://github.com/APRProject/APRPianist.
Expressive performance rendering (EPR) aims to generate realistic performances constrained on sequences of notes. However, flow matching audio editing models manipulate only synchronized music samples of the same duration, limiting their understanding of expressive timing. We introduce PianoKontext, a flow matching rendering model for classical piano music that generates variable-length performances in the latent space of a pretrained Music2Latent model. We synthesize MIDI scores into deadpan audio and employ Dynamic Time Warping (DTW) in the latent space to construct paired data for training. The aligned embeddings are concatenated in DiT blocks, allowing for a simple and effective learning of the dependencies between the score and performances. Audio samples are available at our demo page: https://realfolkcode.github.io/pianokontext_demo/.
We present DIY e-HandPan, a new open-source, low-cost and customizable handpan audio and MIDI protocol interface designed for musical performance, education and research. The proposed hardware is built from inexpensive electronic components and recycled materials using widely available fabrication techniques, making it accessible to makers, educators and researchers. The instrument can be implemented on two distinct MicroController Unit (MCU): Arduino or ESP32. The microcontroller captures strike velocity to provide expressive musical performance comparable to that of an acoustic handpan. In addition to real-time audio and MIDI generation, DIY e-HandPan integrates a bi-color LED guidance system capable of displaying musical sequences from MIDI files, providing an effective learning aid for beginners and educational activities. The modular architecture allows users to easily customize the number of notes, hardware configuration and embedded software according to specific applications. We present the complete hardware design, firmware and assembly instructions and we discuss the design choices and limitations, to evaluate the system in representative educational and musical performance scenarios. All design files, source code and documentation are released under an open-source license to improve the reproducibility and encourage further developments by the open hardware community.