Semi-Automatic Flute Robot and Its Acoustic Sensing

Hikari Kuriyama; Hiroaki Sonoda; Kouki Tomiyoshi; Gou Koutaki

cs.HCMar 15, 2026

Semi-Automatic Flute Robot and Its Acoustic Sensing

Authors: Hikari Kuriyama, Hiroaki Sonoda, Kouki Tomiyoshi, Gou Koutaki

Organizations: Graduate School of Science and Technology, Kumamoto University, Kumamoto 860-8555, Japan · Department of Computer Science and Electrical Engineering, Kumamoto University, Kumamoto 860-8555, Japan

Abstract

Flute performance requires mastery of complex fingering combinations and register-dependent embouchure control, particularly jet offset adjustment for low-register production. Existing haptic and semi-automated systems do not address both aspects simultaneously through mechanical actuation. To our knowledge, no prior system fully automates fingering while mechanically assisting low-register tone production without requiring embouchure control. We developed a semi-automatic flute robot with an automatic fingering mechanism: fourteen servo motors actuate all keys via wire-based and rack-and-pinion drives in response to MIDI input, enabling performers to produce complete musical pieces through airflow alone. A jet offset assist mechanism rotates the head joint by a calibrated $22^\circ$ during low-register passages, shifting the jet offset toward a low-register configuration without modifying the instrument or embouchure. Fundamental frequency estimation confirmed correct pitch production across the chromatic range (C4--C7) and during musical performance. All key and lever movements were completed within 77.50ms, corresponding to tempo capacity exceeding standard requirements. Harmonic analysis ( $Δ\mathrm{SPL} = \mathrm{SPL}_2 - \mathrm{SPL}_3$ ) showed a consistent increase in $Δ$ SPL for all low-register notes when activated, consistent with the intended jet offset shift. Head joint rotation completed within 40.00ms. These results demonstrate mechanical feasibility of integrating automated fingering and register-dependent jet offset assistance under controlled conditions.

Figures & tables

Figure 1: The flute has three main sections: the head joint, middle joint, and foot joint. The head joint features a lip plate with a central embouchure hole. The body and foot joint are equipped with multiple keys and levers; pressing a key closes its tone hole, while pressing a lever opens the corresponding key.

Panel of Figure 2 — Figure 2: Semi-automatic flute robot. (a) Overview of the system. Red highlights: automatic fingering mechanism (middle and foot joints). Blue highlights: jet offset assist mechanism (head and middle joints). (b) Control flow of the system. Both mechanisms are driven by servo motors controlled via MIDI messages from a PC.

Figure 3: Schematic of wire-based key actuation. The wire connects the key cover to the servo horn; motor rotation pulls the wire and presses the key.

Panel of Figure 4 — Figure 4: Automatic fingering mechanism. (a) Front view: servo motors are mounted on an acrylic body, with the flute secured on top. (b) Rear view: a hand-placement grip is positioned to support a natural playing posture.

Panel of Figure 5 — Figure 5: Key cover structure. (a) Cover attached to key. (b) Underside: three locking tabs for attachment; wire passes through one tab and is fixed with a crimp bead.

Panel of Figure 6 — Figure 6: Rack-and-pinion mechanism for side-mounted keys. (a) Schematic of the conversion of motor rotation to linear motion. (b) Key actuation in use.

Figure 7: Jet offset: the displacement of the air jet center from the embouchure hole edge toward the inside of the flute (red arrow).

Figure 8: Principle of jet offset shift. The initial jet offset is represented by the blue line. Rotating the head joint by a predetermined angle shifts it to the red line.

Figure 9: Jet offset assist mechanism mounted near the middle-to-head joint junction. The servo motor drives the head joint via meshing gears on the servo horn and joint body.

Figure 10: Configuration around the circuit board.From right to left, the power connector, microcontroller, and motor driver are mounted. The servo motors are controlled by these components.

Figure 11: Score corresponding to the MIDI messages used in experiment [A1]. The piece is a chromatic scale from the lowest note C4 to the highest note C7.

Panel of Figure 12 — Figure 12: $f_{0}$ estimation for chromatic scale C4–C7. (a) Automatic fingering mechanism. (b) Manual performance (reference). Red line: target $f_{0}$ ; blue dots: estimated $f_{0}$ ; light red band: $\pm$ 50 cent tolerance. $\bullet$ : unintended pitch; $\triangle$ : breath; $\square$ : unstable onset.

Panel of Figure 13 — Figure 13: Spectrograms of chromatic scale C4–C7. (a) Automatic fingering mechanism: vertical streaks at fingering transitions indicate motor and key-click noise. (b) Manual performance (reference): no such components.

Figure 14: Score corresponding to the MIDI messages used in experiment [A2]. The piece consists of the first four measures of Mozart’s Turkish March .

Panel of Figure 15 — Figure 15: $f_{0}$ estimation for Mozart’s Turkish March . (a) Automatic fingering mechanism. (b) Manual performance (reference). Symbols as in Figure 12 .

Type	Motor		Key/Lever
	Avg. [ms]	SD [ms]	Avg. [ms]	SD [ms]
LowC key	104.58	2.24	68.33	2.04
C $\sharp$ key	104.17	1.86	57.50	3.12
D $\sharp$ lever	57.50	1.67	29.58	2.24
D key	91.67	1.86	70.42	1.25
D $\sharp$ trill lever	72.08	1.91	27.08	2.08

Table 1: Motor and key/lever movement times (mean

\pm

SD over 10 trials).

Note value	Max BPM	One measure [s] 1 1 1 For reference, a typical fast piece at BPM200 corresponds to approximately 1.2 s per measure.
Quarter note	774	0.31
Eighth note	387
Sixteenth note	194

Table 2: Theoretical maximum BPM of the automatic fingering mechanism, derived from the longest key movement time (E key: 77.50 ms). One-measure duration shown for reference (4/4 time).

Figure 16: Experimental setup for jet offset assist mechanism evaluation. Soundproof room with air pump placed outside; flow rate controlled by needle valve; airflow direction fixed by 3D-printed cap.

Panel of Figure 17 — Figure 17: 3D-printed airflow cap. (a) Cap. (b) Mounted on lip plate. (c) Aperture dimensions. Rendered in Autodesk Fusion.

Panel of Figure 18 — Figure 18: Acoustic verification of head joint configurations under identical airflow. (a) 15.97 L/min: middle-register configuration produces B $\flat$ 5; low-register configuration produces B $\flat$ 4. (b) 13.51 L/min: both produce B $\flat$ 4, but the low-register configuration shows larger amplitude.

Figure 19: Score for Experiment [B1]: first four measures of Kōjō no Tsuki (Rentarō Taki). Yellow regions: low register (Note Indices 1–2, 8–12). Gray region: middle register (Note Indices 3–7).

Figure 20: $\Delta$ SPL (2nd harmonic [dB] $-$ 3rd harmonic [dB]) per note, with and without the jet offset assist mechanism. Red: mechanism active. Blue: mechanism inactive. Yellow regions: low register. Gray region: middle register.

Action	Duration [ms]	SD [ms]
Motor movement	73.33	2.04
Head joint movement	40.00	2.04

Table 3: Motor and head joint movement times (mean

\pm

SD over 10 trials).

Note value	Max BPM	One measure [s]
Quarter note	1500	0.16
Eighth note	750
Sixteenth note	375

Table 4: Theoretical maximum BPM of the jet offset assist mechanism, derived from head joint rotation time (40.00 ms). One-measure duration shown for reference (4/4 time).

Robotics

Robotics studies the design and control of machines that perceive, move, and act in physical environments.

See all 751 papers

Phrase-Level Robotic Guqin Performance: Bimanual Motion Planning and Audio-Tactile Interaction Monitoring
Sep 21, 2026Zhen Wang, Zhiheng Chen, Tianyuan Bao +1Bimanual Robotic Manipulation Robotics
The Shenzhen Institute of Artificial Intelligence and Robotics for Society, Shenzhen, China · The Chinese University of Hong Kong - Shenzhen, Shenzhen, China
A Haptic Robot Finger Designed for Guqin Instrument Playing
Aug 7, 2026Tianwei Zhang, Hanming Yan, Yang Yang. Ziya WangTactile Sensing Tactile Sensing for Robotic Manipulation
The Shenzhen Institute of Artificial Intelligence and Robotics for Society, Shenzhen, China · The Chinese University of Hong Kong - Shenzhen, Shenzhen, China · School of Physics and Optoelectronic Engineering, Shenzhen University, Shenzhen, China +1
FlueBricks: A Construction Kit of Flute-like Instruments for Acoustic Reasoning
Apr 4, 2026Bo-Yu Chen, Chiao-Wei Huang, Lung-Pan ChengAudio Reasoning Embodied Reasoning
National Taiwan University, Taipei, Taiwan · Independent Researcher, Taipei, Taiwan

Sep 21, 2026J/K move · Enter open · S save

Phrase-Level Robotic Guqin Performance: Bimanual Motion Planning and Audio-Tactile Interaction Monitoring

Zhen Wang, Zhiheng Chen, Tianyuan Bao, Tianwei Zhang

The Shenzhen Institute of Artificial Intelligence and Robotics for Society, Shenzhen, China · The Chinese University of Hong Kong - Shenzhen, Shenzhen, China

Bimanual Robotic Manipulation Robotics Robotic Manipulation Robot Motion Planning

Open paper AskPDF arXiv

Recent advances in humanoid robotics and embodied intelligence have enabled robots to perform increasingly complex manipulation tasks. However, musical instrument performance remains a formidable benchmark, demanding not only collision-free trajectory execution but also precise contact timing, asymmetric bimanual coordination, and target acoustic outcomes on physical instruments. The guqin, a seven-string fretless zither, presents unique manipulation challenges due to its millimetric string spacing, transient right-hand plucking, and sustained left-hand harmonic contacts. In this work, we present a physical heterogeneous dual-arm robotic system for phrase-level autonomous guqin performance. We formulate guqin playing as a hybrid discrete--continuous execution problem and develop a hierarchical planning framework that coordinates working finger assignment, configuration continuity, obstacle avoidance, and tight bimanual contact schedules across consecutive musical events. The system integrates vision-guided instrument localization, tactile-based harmonic contact monitoring, and auditory feedback-informed plucking parameter calibration. Real-world experiments on a 25-event phrase demonstrate that the system reliably executes coordinated open-string and seventh-hui harmonic sequences on a physical guqin, achieving 93.6% and 96.8% event correctness across repeated trials.

Abstract

Robotics

Explore similar work

Phrase-Level Robotic Guqin Performance: Bimanual Motion Planning and Audio-Tactile Interaction Monitoring

A Haptic Robot Finger Designed for Guqin Instrument Playing

FlueBricks: A Construction Kit of Flute-like Instruments for Acoustic Reasoning