cs.HCMar 15, 2026

Semi-Automatic Flute Robot and Its Acoustic Sensing

Authors: Hikari Kuriyama, Hiroaki Sonoda, Kouki Tomiyoshi, Gou Koutaki

Organizations: Graduate School of Science and Technology, Kumamoto University, Kumamoto 860-8555, Japan · Department of Computer Science and Electrical Engineering, Kumamoto University, Kumamoto 860-8555, Japan

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Flute performance requires mastery of complex fingering combinations and register-dependent embouchure control, particularly jet offset adjustment for low-register production. Existing haptic and semi-automated systems do not address both aspects simultaneously through mechanical actuation. To our knowledge, no prior system fully automates fingering while mechanically assisting low-register tone production without requiring embouchure control. We developed a semi-automatic flute robot with an automatic fingering mechanism: fourteen servo motors actuate all keys via wire-based and rack-and-pinion drives in response to MIDI input, enabling performers to produce complete musical pieces through airflow alone. A jet offset assist mechanism rotates the head joint by a calibrated 22∘22^\circ during low-register passages, shifting the jet offset toward a low-register configuration without modifying the instrument or embouchure. Fundamental frequency estimation confirmed correct pitch production across the chromatic range (C4--C7) and during musical performance. All key and lever movements were completed within 77.50ms, corresponding to tempo capacity exceeding standard requirements. Harmonic analysis (ΔSPL=SPL2−SPL3Δ\mathrm{SPL} = \mathrm{SPL}_2 - \mathrm{SPL}_3) showed a consistent increase in ΔΔSPL for all low-register notes when activated, consistent with the intended jet offset shift. Head joint rotation completed within 40.00ms. These results demonstrate mechanical feasibility of integrating automated fingering and register-dependent jet offset assistance under controlled conditions.

Figures & tables

Figure 1: The flute has three main sections: the head joint, middle joint, and foot joint. The head joint features a lip plate with a central embouchure hole. The body and foot joint are equipped with multiple keys and levers; pressing a key closes its tone hole, while pressing a lever opens the corresponding key.
Figure 2: Semi-automatic flute robot. (a) Overview of the system. Red highlights: automatic fingering mechanism (middle and foot joints). Blue highlights: jet offset assist mechanism (head and middle joints). (b) Control flow of the system. Both mechanisms are driven by servo motors controlled via MIDI messages from a PC.
Figure 3: Schematic of wire-based key actuation. The wire connects the key cover to the servo horn; motor rotation pulls the wire and presses the key.
Figure 4: Automatic fingering mechanism. (a) Front view: servo motors are mounted on an acrylic body, with the flute secured on top. (b) Rear view: a hand-placement grip is positioned to support a natural playing posture.
Figure 5: Key cover structure. (a) Cover attached to key. (b) Underside: three locking tabs for attachment; wire passes through one tab and is fixed with a crimp bead.
Figure 6: Rack-and-pinion mechanism for side-mounted keys. (a) Schematic of the conversion of motor rotation to linear motion. (b) Key actuation in use.
Figure 7: Jet offset: the displacement of the air jet center from the embouchure hole edge toward the inside of the flute (red arrow).
Figure 8: Principle of jet offset shift. The initial jet offset is represented by the blue line. Rotating the head joint by a predetermined angle shifts it to the red line.
Figure 9: Jet offset assist mechanism mounted near the middle-to-head joint junction. The servo motor drives the head joint via meshing gears on the servo horn and joint body.
Figure 10: Configuration around the circuit board.From right to left, the power connector, microcontroller, and motor driver are mounted. The servo motors are controlled by these components.
Figure 11: Score corresponding to the MIDI messages used in experiment [A1]. The piece is a chromatic scale from the lowest note C4 to the highest note C7.
Figure 12: f0f_{0} estimation for chromatic scale C4–C7. (a) Automatic fingering mechanism. (b) Manual performance (reference). Red line: target f0f_{0} ; blue dots: estimated f0f_{0} ; light red band: ±\pm 50 cent tolerance. ∙\bullet : unintended pitch; △\triangle : breath; □\square : unstable onset.
Figure 13: Spectrograms of chromatic scale C4–C7. (a) Automatic fingering mechanism: vertical streaks at fingering transitions indicate motor and key-click noise. (b) Manual performance (reference): no such components.
Figure 14: Score corresponding to the MIDI messages used in experiment [A2]. The piece consists of the first four measures of Mozart’s Turkish March .
Figure 15: f0f_{0} estimation for Mozart’s Turkish March . (a) Automatic fingering mechanism. (b) Manual performance (reference). Symbols as in Figure 12 .
TypeMotorKey/Lever
 Avg. [ms]SD [ms]Avg. [ms]SD [ms]
LowC key104.582.2468.332.04
C ♯\sharp key104.171.8657.503.12
D ♯\sharp lever57.501.6729.582.24
D key91.671.8670.421.25
D ♯\sharp trill lever72.081.9127.082.08
Table 1: Motor and key/lever movement times (mean ±\pm SD over 10 trials).
Note valueMax BPMOne measure [s] 1 1 1 For reference, a typical fast piece at BPM200 corresponds to approximately 1.2 s per measure.
Quarter note7740.31
Eighth note387
Sixteenth note194
Table 2: Theoretical maximum BPM of the automatic fingering mechanism, derived from the longest key movement time (E key: 77.50 ms). One-measure duration shown for reference (4/4 time).
Figure 16: Experimental setup for jet offset assist mechanism evaluation. Soundproof room with air pump placed outside; flow rate controlled by needle valve; airflow direction fixed by 3D-printed cap.
Figure 17: 3D-printed airflow cap. (a) Cap. (b) Mounted on lip plate. (c) Aperture dimensions. Rendered in Autodesk Fusion.
Figure 18: Acoustic verification of head joint configurations under identical airflow. (a) 15.97 L/min: middle-register configuration produces B ♭\flat 5; low-register configuration produces B ♭\flat 4. (b) 13.51 L/min: both produce B ♭\flat 4, but the low-register configuration shows larger amplitude.
Figure 19: Score for Experiment [B1]: first four measures of Kōjō no Tsuki (Rentarō Taki). Yellow regions: low register (Note Indices 1–2, 8–12). Gray region: middle register (Note Indices 3–7).
Figure 20: Δ\Delta SPL (2nd harmonic [dB] −- 3rd harmonic [dB]) per note, with and without the jet offset assist mechanism. Red: mechanism active. Blue: mechanism inactive. Yellow regions: low register. Gray region: middle register.
ActionDuration [ms]SD [ms]
Motor movement73.332.04
Head joint movement40.002.04
Table 3: Motor and head joint movement times (mean ±\pm SD over 10 trials).
Note valueMax BPMOne measure [s]
Quarter note15000.16
Eighth note750
Sixteenth note375
Table 4: Theoretical maximum BPM of the jet offset assist mechanism, derived from head joint rotation time (40.00 ms). One-measure duration shown for reference (4/4 time).
More in this topic

Robotics

Robotics studies the design and control of machines that perceive, move, and act in physical environments.

See all 751 papers

Explore similar work

CardsList
Sep 21, 2026

Phrase-Level Robotic Guqin Performance: Bimanual Motion Planning and Audio-Tactile Interaction Monitoring

Recent advances in humanoid robotics and embodied intelligence have enabled robots to perform increasingly complex manipulation tasks. However, musical instrument performance remains a formidable benchmark, demanding not only collision-free trajectory execution but also precise contact timing, asymmetric bimanual coordination, and target acoustic outcomes on physical instruments. The guqin, a seven-string fretless zither, presents unique manipulation challenges due to its millimetric string spacing, transient right-hand plucking, and sustained left-hand harmonic contacts. In this work, we present a physical heterogeneous dual-arm robotic system for phrase-level autonomous guqin performance. We formulate guqin playing as a hybrid discrete--continuous execution problem and develop a hierarchical planning framework that coordinates working finger assignment, configuration continuity, obstacle avoidance, and tight bimanual contact schedules across consecutive musical events. The system integrates vision-guided instrument localization, tactile-based harmonic contact monitoring, and auditory feedback-informed plucking parameter calibration. Real-world experiments on a 25-event phrase demonstrate that the system reliably executes coordinated open-string and seventh-hui harmonic sequences on a physical guqin, achieving 93.6% and 96.8% event correctness across repeated trials.

Zhen Wang, Zhiheng Chen, Tianyuan Bao +1

The Shenzhen Institute of Artificial Intelligence and Robotics for Society, Shenzhen, China · The Chinese University of Hong Kong - Shenzhen, Shenzhen, China

Bimanual Robotic ManipulationRoboticsRobotic ManipulationRobot Motion Planning
AskPDF
Aug 7, 2026

A Haptic Robot Finger Designed for Guqin Instrument Playing

With the rapid advancement of humanoid robotics and embodied intelligence technologies, numerous musical instrument-playing robots have emerged in recent years, such as pianos, chime bells, and taiko drums. These robots primarily employ open-loop positional control, rendering them incapable of operating instruments requiring dexterous hands and precise tactile perception, such as a violin, guitar, and guqin. This paper describes the design and validation of a high-precision tactile-sensing finger. By mimicking the shape of the fingertip and fingernail found on a human finger, we develop a biomimetic multimodal haptic fingertip and validate it on selected guqin string-contact tasks, including open-string and stopped-note comparisons, harmonic-tuning, and tactile-triggered bimanual coordination, using the guqin, a traditional Chinese musical instrument, as a challenging validation scenario rather than as a fully demonstrated robotic performance system. This research integrates tactile sensing with robotics technology, thereby contributing to applications in world heritage conservation and cultural dissemination.

Tianwei Zhang, Hanming Yan, Yang Yang. Ziya Wang

The Shenzhen Institute of Artificial Intelligence and Robotics for Society, Shenzhen, China · The Chinese University of Hong Kong - Shenzhen, Shenzhen, China · School of Physics and Optoelectronic Engineering, Shenzhen University, Shenzhen, China +1

Tactile SensingTactile Sensing for Robotic ManipulationRobotic Tactile Sensing
AskPDF
Apr 4, 2026

FlueBricks: A Construction Kit of Flute-like Instruments for Acoustic Reasoning

We present FlueBricks, a construction kit for acoustic reasoning via building and customizing flute-like instruments. By assembling generator, resonator, and connector modules that embody various aeroacoustic properties, users gain deeper understanding of how blowhole, tube length, and tone-hole placement alter onset, pitch, and timbre through hands-on experimentation. This forms a designer-player loop of configuring and playing to form, test, and refine acoustic behaviors-acoustic reasoning-shifting acoustic instruments from static artifacts to dynamic systems. To understand how users engage with this system, we conducted an exploratory study with 12 participants ranging from novices to professional musicians. During their explorations, we observed participants fluently switching between designer and player roles, scaffolding designs from familiar instruments, forming and refining their acoustic understanding of length, tone holes, and generator geometry, reinterpreting modules beyond their intended functions, and using their creations for performative acts such as pedagogical showing and musical expression. These collectively demonstrated FlueBricks's potential as a pedagogical tool for embodied acoustic reasoning.

Bo-Yu Chen, Chiao-Wei Huang, Lung-Pan Cheng

National Taiwan University, Taipei, Taiwan · Independent Researcher, Taipei, Taiwan

Audio ReasoningEmbodied ReasoningEducational Technology
AskPDF