SCISSOR: Score-Conditioned Instrument Source Separation for Orchestral Recordings

Yiheng Lu; Hao-Wen Dong

cs.SDSep 27, 2026

SCISSOR: Score-Conditioned Instrument Source Separation for Orchestral Recordings

Authors: Yiheng Lu, Hao-Wen Dong

Organizations: The Chinese University of Hong Kong, Shenzhen · University of Michigan

Abstract

Orchestral separation recovers instrument sections from mixtures in which shared pitches, harmonics, and timbres obscure source identity. An aligned score provides instrument labels, note pitches, and activity times. A score-informed approach appends piano rolls to audio features before mask prediction. We introduce SCISSOR (Score-Conditioned Instrument Source Separation for Orchestral Recordings), which uses the score to form a frame-wise query for each source. Each query matches a shared audio representation, and a softmax over instrument and background slots jointly assigns overlapping time-frequency evidence. The queries retain instrument identity even when notes are missing from the score. After training on SynthSOD and a small set of URMP and PHENICX-Anechoic recordings, SCISSOR achieves the highest average SDR on held-out real recordings. With SynthSOD-only training, it leads on SynthSOD and zero-shot PHENICX-Anechoic, and improves on its audio-only control on zero-shot URMP. SCISSOR also degrades less under score corruption than the evaluated score-based baselines.

Figures & tables

URMP ( $n=5$ )
System	Vn.	Va.	Vc.	Db.	Avg. [95% CI]
HTDemucs	$0.23$	$0.57$	$5.07$	$6.13$	$3.00$ [	$0.25$ ,	$5.09$ ]
BS-RoFormer	$2.46$	$0.86$	$3.31$	$5.32$	$2.99$ [	$1.00$ ,	$3.51$ ]
Tunturi et al., baseline	$1.91$	$1.43$	$5.52$	$5.89$	$3.69$ [	$1.63$ ,	$4.48$ ]
Tunturi et al., score-informed	$2.27$	$1.76$	$5.86$	$5.86$	$3.94$ [	$1.66$ ,	$5.38$ ]
Tunturi et al., score-only	$4.90$	$3.50$	$6.69$	$7.40$	$5.62$ [	$4.09$ ,	$7.17$ ]

Table 1: Museval SDR (dB) on held-out real recordings after SynthSOD and real-recording training. Values are estimates [95% CI]; the one-piece PHENICX test has no across-recording CI.

System	Score	SynthSOD ( $n=59$ )			PHENICX ( $n=4$ )			URMP ( $n=22$ )
Mixture (no separation)		$-7.45$ [	$-8.69$ ,	$-6.36$ ]	$-14.11$ [	$-17.80$ ,	$-8.26$ ]	$-5.23$ [	$-5.72$ ,	$-3.38$ ]
HTDemucs [ 12 ]		$4.17$ [	$2.43$ ,	$5.20$ ]	$-1.83$ [	$-5.74$ ,	$1.20$ ]	$1.93$ [	$0.75$ ,	$3.31$ ]
BS-RoFormer [ 7 ]		$3.39$ [	$2.30$ ,	$4.18$ ]	$-2.72$ [	$-5.33$ ,	$0.16$ ]	$1.54$ [	$0.84$ ,	$2.63$ ]
Tunturi et al., baseline [ 15 ]		$5.64$ [	$4.23$ ,	$6.27$ ]	$-0.15$ [	$-1.51$ ,	$2.35$ ]	$1.88$ [	$0.99$ ,	$2.69$ ]
Tunturi et al., score-informed	✓	$5.96$ [	$4.59$ ,	$6.53$ ]	$0.55$ [	$-0.69$ ,	$2.35$ ]	$3.35$ [	$2.17$ ,	$3.94$ ]
Tunturi et al., score-only	✓	$4.35$ [	$3.40$ ,	$5.01$ ]	$0.83$ [	$-2.43$ ,	$3.39$ ]	4.80 [	4.13,	5.96]

Table 2: Avg. museval SDR (dB) after SynthSOD-only training; estimates with 95% recording-bootstrap CIs.

Score error	Tunturi score-informed [ 15 ]	Tunturi score-only	SCISSOR
Clean score	$4.81$ [ $4.50$ , $5.11$ ]	$2.92$ [ $2.49$ , $3.33$ ]	$5.45$ [ $5.13$ , $5.76$ ]
Matched audio-only control	$4.58$ [ $4.29$ , $4.89$ ]	$4.58$ [ $4.29$ , $4.89$ ]	$4.98$ [ $4.67$ , $5.29$ ]
Onset jitter ( $\sigma=400$ ms)	$4.38$ [ $4.10$ , $4.67$ ]	$2.15$ [ $1.79$ , $2.50$ ]	$5.23$ [ $4.91$ , $5.53$ ]
Global offset ( $1$ s)	$3.92$ [ $3.64$ , $4.21$ ]	$1.78$ [ $1.45$ , $2.10$ ]	$4.97$ [ $4.67$ , $5.27$ ]
Tempo scaling ( $10\%$ )	$3.35$ [ $3.10$ , $3.61$ ]	$0.99$ [ $0.75$ , $1.22$ ]	$4.72$ [ $4.42$ , $5.01$ ]
Pitch substitution ( $50\%$ )	$4.34$ [ $4.04$ , $4.63$ ]	$2.10$ [ $1.70$ , $2.46$ ]	$5.29$ [ $4.98$ , $5.59$ ]

Table 3: Whole-piece SDR (dB) under score corruption on 22 SynthSOD test pieces; estimates with 95% piece-bootstrap CIs.

Explore similar work

On a Separate Note: Robust Score-Informed Note Separation with a Two-Stream TFC-TDF U-Net and Adaptive Set Ownership
Sep 24, 2026Benjamin Shiue-Hal Chou, Purvish Jajal, Nicholas John Eliopoulos +5Speech Separation Separation
Purdue University · Loyola University Chicago · University of Michigan
From Image to Music Language: A Two-Stage Structure Decoding Approach for Complex Polyphonic OMR
Apr 22, 2026Nan Xu, Shiheng Li, Shengchao HouHuman Annotations
FindLab
TUTTI: Toward generalizable audio-to-score transcription via fully synthesized data
Sep 1, 2026Jianhuai Hu, Yashan Wang, Shangda Wu +7Text-To-Music Transcription
Central Conservatory of Music · Independent Researcher · Wuhan Conservatory of Music +2

Sep 24, 2026, cs.SDJ/K move · Enter open · S save

On a Separate Note: Robust Score-Informed Note Separation with a Two-Stream TFC-TDF U-Net and Adaptive Set Ownership

Benjamin Shiue-Hal Chou, Purvish Jajal, Nicholas John Eliopoulos, James C. Davis+4

Purdue University · Loyola University Chicago · University of Michigan

Speech Separation Separation Notes U-Net Discontinuous Gating

Open paper AskPDF arXiv

Score-informed note separation seeks to extract the performed waveform of all individual notes, often from a polyphonic recording. Existing deep learning systems generally only target instrument-level stems. We present, to our knowledge, the first deep learning approach to score-informed note separation, NoteSep. NoteSep extracts the queried notes by applying an extraction stage model, NoteGrab, once per note. Conditioned on pitch, onset, and offset, NoteGrab separates harmonic and percussive components in two U-Nets linked by bidirectional cross-attention; selective harmonic gating suppresses lower-octave interference while preserving percussive attacks. Finally, a joint separation stage applies Adaptive Set Ownership (ASO) to compare concurrent NoteGrab estimates and reallocate mixture energy. We curate SCNS-Train (25,729 mixtures and 743,920 targets) for training and SCNS-Eval (16 instruments, disjoint scores and libraries) for evaluation. On SCNS-Eval, NoteSep reaches a median SI-SDR of 7.39~~dB, compared with 2.49~~dB for our strongest baseline. See the demo page at https://benschou.com/notesep.

Abstract

Explore similar work

On a Separate Note: Robust Score-Informed Note Separation with a Two-Stream TFC-TDF U-Net and Adaptive Set Ownership

From Image to Music Language: A Two-Stage Structure Decoding Approach for Complex Polyphonic OMR

TUTTI: Toward generalizable audio-to-score transcription via fully synthesized data