Efficient Long-Sequence Diffusion Modeling for Symbolic Music Generation

Jinhan Xu; Xing Tang; Houpeng Yang; Haoran Zhang; Shenghua Yuan; Jiatao Chen; Tianming Xi; Jing Wang

cs.SDFeb 28, 2026

Efficient Long-Sequence Diffusion Modeling for Symbolic Music Generation

Authors: Jinhan Xu, Xing Tang, Houpeng Yang, Haoran Zhang, Shenghua Yuan, Jiatao Chen, Tianming Xi, Jing Wang,

Organizations: School of Computer Science and Artificial Intelligence, Wuhan University of Technology, Wuhan, China 430070 · School of Computer Science, Hubei University of Technology, Wuhan, China 430068 · School of Information Engineering, Hubei University of Economics, Wuhan, China 430205; Hubei Key Laboratory of Digital Finance Innovation, Wuhan, China 430205

Abstract

Symbolic music generation is a challenging task in multimedia generation, involving long sequences with hierarchical temporal structures, long-range dependencies, and fine-grained local details. Though recent diffusion-based models produce high quality generations, they tend to suffer from high training and inference costs with long symbolic sequences due to iterative denoising and sequence-length-related costs. To deal with such problem, we put forth a diffusing strategy named SMDIM to combine efficient global structure construction and light local refinement. SMDIM uses structured state space models to capture long range musical context at near linear cost, and selectively refines local musical details via a hybrid refinement scheme. Experiments performed on a wide range of symbolic music datasets which encompass various Western classical music, popular music and traditional folk music show that the SMDIM model outperforms the other state-of-the-art approaches on both the generation quality and the computational efficiency, and it has robust generalization to underexplored musical styles. These results show that SMDIM offers a principled solution for long-sequence symbolic music generation, including associated attributes that accompany the sequences. We provide a project webpage with audio examples and supplementary materials at https://3328702107.github.io/smdim-music/.

Figures & tables

Figure 2: The diagram shows SMDIM on the left and its core component, the MFA Block on the right. SMDIM processes input sequences hierarchically, while the MFA Block combines Mamba, FeedForward, and self-attention layers to balance scalability and precision. The [Mask] tokens represent noise, which is transformed into music symbols through denoising, resulting in coherent musical sequences.

Method	Backbone	Paradigm	Rep.	Diff. Ref.	G–L Des.	Loc. Rec.
MusicTr [ 41 ]	Transformer	AR	MIDI	✗	✗	✗
GETMusic [ 42 ]	Transformer	AR/S2S	Score	✗	✗	✗
SCHmUBERT [ 28 ]	Transformer	Diff.	VAE	✗	✗	✗
MusicMamba [ 43 ]	SSM (Mamba)	AR	REMI	✗	✗	✗
SMDIM (Ours)	Hybrid	Diff.(Ref)	Token	✓	✓	✓

Table 1: Comparison of architectural and generative designs for long-sequence music generation. (Diff.: Diffusion; AR: Autoregressive; G–L: Global–Local)

Dataset	Method	Used Pitch	IOI	Pitch Hist	Pitch Range	Velocity	Note Duration	Note Density	Avg
Maestro	GT	$0.905\pm 0.127$	$0.902\pm 0.025$	$0.955\pm 0.009$	$0.909\pm 0.007$	$0.910\pm 0.281$	$0.896\pm 0.016$	$0.917\pm 0.015$	$0.913\pm 0.009$
	MusicTr	$0.631\pm 0.092$	$\underline{0.836\pm 0.033}$	$\underline{0.868\pm 0.054}$	$0.813\pm 0.070$	$0.604\pm 0.048$	$0.350\pm 0.058$	$\underline{0.894\pm 0.012}$	$0.755\pm 0.016$
	SCHmUBERT	$0.429\pm 0.059$	$0.502\pm 0.051$	$0.701\pm 0.030$	$0.678\pm 0.053$	$0.417\pm 0.074$	$0.885\pm 0.015$	$0.773\pm 0.046$	$0.626\pm 0.027$
	Polyffusion	$\mathbf{0.803\pm 0.020}$	$0.831\pm 0.024$	$0.770\pm 0.040$	$0.814\pm 0.017$	$0.424\pm 0.068$	$0.786\pm 0.021$	$0.823\pm 0.027$	$0.750\pm 0.013$
	MusicMamba	$0.551\pm 0.027$	$0.707\pm 0.019$	$0.669\pm 0.012$	$\underline{0.846\pm 0.022}$	$\mathbf{0.915\pm 0.010}$	$\mathbf{0.936\pm 0.007}$	$\mathbf{0.897\pm 0.015}$	$\underline{0.789\pm 0.010}$
	GETMusic	$0.542\pm 0.100$	$0.799\pm 0.045$	$\mathbf{0.949\pm 0.020}$	$0.683\pm 0.049$	$\underline{0.883\pm 0.026}$	$0.857\pm 0.032$	$0.539\pm 0.059$	$0.750\pm 0.016$

Table 2: Objective evaluation of unconditional generation. The OA for 7 music attributes and the average OA are reported. The highest and second highest OA excluding GT are bolded and underlined respectively.

\mathcal{L}_{\mathrm{vb}}=D_{\mathrm{KL}}\!\left(q(\mathbf{x}_{T}|\mathbf{x}_{0})\,\|\,p(\mathbf{x}_{T})\right)

Algorithm 1 Training Process for SMDIM

Component	Setting/Parameter
Framework	PyTorch
Hardware	2 × NVIDIA RTX 4090
Training duration	12 hours
Training steps	200,000
Optimizer	AdamW
Initial learning rate	$5\times 10^{-4}$

Table 3: Training Configuration and Hyperparameters.

Method	Model	Dataset	Representation
MusicTr	transformer	Maestro	MIDI-like
PolyDiff	Diffusion	POP909	Piano roll
SCHmUBERT	Diffusion	LMD	MusicVAE
GETMusic	Diffusion	Musescore	GetScore
MusicMamba	Mamba	FolkDB	REMI-M

Table 4: Baselines for music generation.

Dataset	Method	Used Pitch	IOI	Pitch Hist	Pitch Range	Velocity	Note Duration	Note Density	Avg
Maestro	GT	$0.905\pm 0.127$	$0.902\pm 0.025$	$0.955\pm 0.009$	$0.909\pm 0.007$	$0.910\pm 0.281$	$0.896\pm 0.016$	$0.917\pm 0.015$	$0.913\pm 0.009$
	Mamba Block	$\underline{\mathbf{0.795\pm 0.033}}$	$0.151\pm 0.016$	$\mathbf{0.826\pm 0.191}$	$\underline{0.856\pm 0.026}$	$0.789\pm 0.051$	$0.413\pm 0.073$	$0.448\pm 0.062$	$0.605\pm 0.033$
	Transformer Block	$0.662\pm 0.068$	$\mathbf{0.911\pm 0.017}$	$0.762\pm 0.030$	$0.729\pm 0.066$	$\mathbf{0.906\pm 0.021}$	$0.888\pm 0.018$	$\underline{0.876\pm 0.021}$	$0.819\pm 0.024$
	AFM	$0.792\pm 0.049$	$0.832\pm 0.030$	$\underline{0.826\pm 0.027}$	$0.851\pm 0.045$	$0.843\pm 0.031$	$0.820\pm 0.026$	$0.868\pm 0.035$	$\underline{0.833\pm 0.020}$
	FMA	$0.724\pm 0.076$	$0.891\pm 0.018$	$0.737\pm 0.029$	$0.849\pm 0.043$	$0.848\pm 0.034$	$\underline{0.891\pm 0.016}$	$0.873\pm 0.021$	$0.830\pm 0.021$
	MFA (2SA)	$0.805\pm 0.048$	$0.803\pm 0.037$	$0.812\pm 0.028$	$0.889\pm 0.035$	$0.751\pm 0.046$	$0.849\pm 0.037$	$0.843\pm 0.025$	$0.822\pm 0.026$

Table 5: Ablation experiments for the design of SMDIM with various architectures. The order of layers significantly impacts the performance. The highest and second-highest OA excluding GT are bolded and underlined, respectively.

Figure 3: Ablation results on GFLOPs of SMDIM model vs. SCHmUBERT model at different input seq. lengths. All GFLOPs are calculated by thop package for comparison.

Method	Step Time (s)	GPU Memory (GB)
SCHmUBERT	0.54	35
MFA (2SA)	0.63	24
Ours	0.35	21

Table 6: Comparison of per-step latency and GPU memory usage with SCHmUBERT and MFA (2SA) on 2048-token sequences. Our method achieves the lowest latency and memory footprint.

Figure 4: Average Overlapping Area (OA) of music generated by SMDIM under different output sequence lengths.

ID	Grp.	Profession / Major	Status	Yrs	Area of Expertise
P01	Theo.	Composition	Asst. Prof.	17	Harmony, Counterpoint
P02	Theo.	Music Theory	PhD Cand.	14	Form, Tonal Harmony
P03	Theo.	Screen Scoring	Composer	11	Orchestration, Development
P04	Theo.	Jazz Studies	Master Std.	10	Jazz Harmony, Improv.
P05	Theo.	Comp. & Arranging	Lecturer	19	Arrangement, Aural Skills
P06	Perf.	Piano Performance	Musician	15	Classical, Articulation

Table 7: Demographic profiles of the 15 professional evaluators.

Models	Emotional	Melodic	Rhythmic	Structural
	Consistency	Fluency	Stability	Clarity
MusicTr	7.68	7.87	7.37	6.99
SCHmUBERT	7.72	7.85	7.38	7.03
PolyFusion	7.63	7.83	7.34	6.98
MusicMamba	7.69	7.85	7.29	7.00
Ours	7.66	7.90 ∗	7.30	7.09 ∗

Table 8: Subjective listening study results of the 4 evaluation aspects (1-10 scale). Statistically significant improvements over the best baseline are marked with * (

p<0.05

, paired two-sided t-test).

Panel of Figure 5 — Figure 5: Representative failure cases generated by the proposed model. (a)–(b) Failure cases exhibiting extreme pitch ranges and overly dense vertical note stacking, resulting in musically implausible symbolic structures. The regions highlighted in red and blue indicate representative local errors, including abnormal pitch outliers and excessive simultaneous note activations. (c) A failure case from the final portion of a generated sequence, illustrating structural degradation and weakened global musical organization over time.

Symbolic Music Generation

Symbolic music generation creates and edits music as structured notes, rhythms, chords, and scores rather than directly as audio.

See all 58 papers

Fast Diffusion-GAN Model for Generating Symbolic Music with Emotions
Oct 21, 2023Jincheng Zhang, György Fazekas, Charalampos SaitisTimbre-Conditioned Music Generation Music Generation
Centre for Digital Music, Queen Mary University of London, UK
Diff-Symbo: Text-Controlled Long-Duration Symbolic Music Generation Using Autoregressive Latent Diffusion Model
Aug 5, 2026Zhiwei Lin, Jun Chen, Boshi Tang +5Text-to-Music Generation Latent Diffusion Models
Shenzhen International Graduate School, Tsinghua University, Shenzhen, China · Huawei Technologies Co., Ltd., Shenzhen, China · The Chinese University of Hong Kong, Hong Kong SAR, China
Anchored Cyclic Generation: A Novel Paradigm for Long-Sequence Symbolic Music Generation
Apr 7, 2026Boyu Cao, Lekai Qian, Dehan Li +3Symbolic Music Generation Music Generation
South China University of Technology, School of Future Technology

Oct 21, 2023J/K move · Enter open · S save

Fast Diffusion-GAN Model for Generating Symbolic Music with Emotions

Jincheng Zhang, György Fazekas, Charalampos Saitis

Centre for Digital Music, Queen Mary University of London, UK

Timbre-Conditioned Music Generation Music Generation Symbolic Music Generation Diffusion Models Controllable Music Generation Few-Step Diffusion Sampling

Open paper AskPDF arXiv

Diffusion models have shown promising results for a wide range of generative tasks with continuous data, such as image and audio synthesis. However, little progress has been made on using diffusion models to generate symbolic music since this class of generative models struggles with discrete data while its iterative sampling process is computationally expensive. In this work, we propose a diffusion model integrated with a Generative Adversarial Network, aiming to mitigate the slow sampling drawback of diffusion models applied to symbolic music generation. We enable the use of diffusion model by representing symbolic music as image-like pianorolls with velocity values instead of discrete event tokens and then used a Variational Autoencoder to obtain their latent representations to train the proposed diffusion model. Our results demonstrate the successful control of our diffusion model to generate symbolic music with a desired emotion. Our model uses merely 15 denoising diffusion steps to produce high-quality music, while most diffusion models for symbolic music generation require hundreds or thousands of steps.

Abstract

Symbolic Music Generation

Explore similar work

Fast Diffusion-GAN Model for Generating Symbolic Music with Emotions

Diff-Symbo: Text-Controlled Long-Duration Symbolic Music Generation Using Autoregressive Latent Diffusion Model

Anchored Cyclic Generation: A Novel Paradigm for Long-Sequence Symbolic Music Generation