Efficient Long-Sequence Diffusion Modeling for Symbolic Music Generation

Jinhan Xu; Xing Tang; Houpeng Yang; Haoran Zhang; Shenghua Yuan; Jiatao Chen; Tianming Xi; Jing Wang

cs.SDFeb 28, 2026

Efficient Long-Sequence Diffusion Modeling for Symbolic Music Generation

Authors: Jinhan Xu, Xing Tang, Houpeng Yang, Haoran Zhang, Shenghua Yuan, Jiatao Chen, Tianming Xi, Jing Wang,

Organizations: School of Computer Science and Artificial Intelligence, Wuhan University of Technology, Wuhan, China 430070 · School of Computer Science, Hubei University of Technology, Wuhan, China 430068 · School of Information Engineering, Hubei University of Economics, Wuhan, China 430205; Hubei Key Laboratory of Digital Finance Innovation, Wuhan, China 430205

Abstract

Symbolic music generation is a challenging task in multimedia generation, involving long sequences with hierarchical temporal structures, long-range dependencies, and fine-grained local details. Though recent diffusion-based models produce high quality generations, they tend to suffer from high training and inference costs with long symbolic sequences due to iterative denoising and sequence-length-related costs. To deal with such problem, we put forth a diffusing strategy named SMDIM to combine efficient global structure construction and light local refinement. SMDIM uses structured state space models to capture long range musical context at near linear cost, and selectively refines local musical details via a hybrid refinement scheme. Experiments performed on a wide range of symbolic music datasets which encompass various Western classical music, popular music and traditional folk music show that the SMDIM model outperforms the other state-of-the-art approaches on both the generation quality and the computational efficiency, and it has robust generalization to underexplored musical styles. These results show that SMDIM offers a principled solution for long-sequence symbolic music generation, including associated attributes that accompany the sequences. We provide a project webpage with audio examples and supplementary materials at https://3328702107.github.io/smdim-music/.

Figures & tables

Figure 2: The diagram shows SMDIM on the left and its core component, the MFA Block on the right. SMDIM processes input sequences hierarchically, while the MFA Block combines Mamba, FeedForward, and self-attention layers to balance scalability and precision. The [Mask] tokens represent noise, which is transformed into music symbols through denoising, resulting in coherent musical sequences.

Method	Backbone	Paradigm	Rep.	Diff. Ref.	G–L Des.	Loc. Rec.
MusicTr [ 41 ]	Transformer	AR	MIDI	✗	✗	✗
GETMusic [ 42 ]	Transformer	AR/S2S	Score	✗	✗	✗
SCHmUBERT [ 28 ]	Transformer	Diff.	VAE	✗	✗	✗
MusicMamba [ 43 ]	SSM (Mamba)	AR	REMI	✗	✗	✗
SMDIM (Ours)	Hybrid	Diff.(Ref)	Token	✓	✓	✓

Table 1: Comparison of architectural and generative designs for long-sequence music generation. (Diff.: Diffusion; AR: Autoregressive; G–L: Global–Local)

Dataset	Method	Used Pitch	IOI	Pitch Hist	Pitch Range	Velocity	Note Duration	Note Density	Avg
Maestro	GT	$0.905\pm 0.127$	$0.902\pm 0.025$	$0.955\pm 0.009$	$0.909\pm 0.007$	$0.910\pm 0.281$	$0.896\pm 0.016$	$0.917\pm 0.015$	$0.913\pm 0.009$
	MusicTr	$0.631\pm 0.092$	$\underline{0.836\pm 0.033}$	$\underline{0.868\pm 0.054}$	$0.813\pm 0.070$	$0.604\pm 0.048$	$0.350\pm 0.058$	$\underline{0.894\pm 0.012}$	$0.755\pm 0.016$
	SCHmUBERT	$0.429\pm 0.059$	$0.502\pm 0.051$	$0.701\pm 0.030$	$0.678\pm 0.053$	$0.417\pm 0.074$	$0.885\pm 0.015$	$0.773\pm 0.046$	$0.626\pm 0.027$
	Polyffusion	$\mathbf{0.803\pm 0.020}$	$0.831\pm 0.024$	$0.770\pm 0.040$	$0.814\pm 0.017$	$0.424\pm 0.068$	$0.786\pm 0.021$	$0.823\pm 0.027$	$0.750\pm 0.013$
	MusicMamba	$0.551\pm 0.027$	$0.707\pm 0.019$	$0.669\pm 0.012$	$\underline{0.846\pm 0.022}$	$\mathbf{0.915\pm 0.010}$	$\mathbf{0.936\pm 0.007}$	$\mathbf{0.897\pm 0.015}$	$\underline{0.789\pm 0.010}$
	GETMusic	$0.542\pm 0.100$	$0.799\pm 0.045$	$\mathbf{0.949\pm 0.020}$	$0.683\pm 0.049$	$\underline{0.883\pm 0.026}$	$0.857\pm 0.032$	$0.539\pm 0.059$	$0.750\pm 0.016$

Table 2: Objective evaluation of unconditional generation. The OA for 7 music attributes and the average OA are reported. The highest and second highest OA excluding GT are bolded and underlined respectively.

\mathcal{L}_{\mathrm{vb}}=D_{\mathrm{KL}}\!\left(q(\mathbf{x}_{T}|\mathbf{x}_{0})\,\|\,p(\mathbf{x}_{T})\right)

Algorithm 1 Training Process for SMDIM

Component	Setting/Parameter
Framework	PyTorch
Hardware	2 × NVIDIA RTX 4090
Training duration	12 hours
Training steps	200,000
Optimizer	AdamW
Initial learning rate	$5\times 10^{-4}$

Table 3: Training Configuration and Hyperparameters.

Method	Model	Dataset	Representation
MusicTr	transformer	Maestro	MIDI-like
PolyDiff	Diffusion	POP909	Piano roll
SCHmUBERT	Diffusion	LMD	MusicVAE
GETMusic	Diffusion	Musescore	GetScore
MusicMamba	Mamba	FolkDB	REMI-M

Table 4: Baselines for music generation.

Dataset	Method	Used Pitch	IOI	Pitch Hist	Pitch Range	Velocity	Note Duration	Note Density	Avg
Maestro	GT	$0.905\pm 0.127$	$0.902\pm 0.025$	$0.955\pm 0.009$	$0.909\pm 0.007$	$0.910\pm 0.281$	$0.896\pm 0.016$	$0.917\pm 0.015$	$0.913\pm 0.009$
	Mamba Block	$\underline{\mathbf{0.795\pm 0.033}}$	$0.151\pm 0.016$	$\mathbf{0.826\pm 0.191}$	$\underline{0.856\pm 0.026}$	$0.789\pm 0.051$	$0.413\pm 0.073$	$0.448\pm 0.062$	$0.605\pm 0.033$
	Transformer Block	$0.662\pm 0.068$	$\mathbf{0.911\pm 0.017}$	$0.762\pm 0.030$	$0.729\pm 0.066$	$\mathbf{0.906\pm 0.021}$	$0.888\pm 0.018$	$\underline{0.876\pm 0.021}$	$0.819\pm 0.024$
	AFM	$0.792\pm 0.049$	$0.832\pm 0.030$	$\underline{0.826\pm 0.027}$	$0.851\pm 0.045$	$0.843\pm 0.031$	$0.820\pm 0.026$	$0.868\pm 0.035$	$\underline{0.833\pm 0.020}$
	FMA	$0.724\pm 0.076$	$0.891\pm 0.018$	$0.737\pm 0.029$	$0.849\pm 0.043$	$0.848\pm 0.034$	$\underline{0.891\pm 0.016}$	$0.873\pm 0.021$	$0.830\pm 0.021$
	MFA (2SA)	$0.805\pm 0.048$	$0.803\pm 0.037$	$0.812\pm 0.028$	$0.889\pm 0.035$	$0.751\pm 0.046$	$0.849\pm 0.037$	$0.843\pm 0.025$	$0.822\pm 0.026$

Table 5: Ablation experiments for the design of SMDIM with various architectures. The order of layers significantly impacts the performance. The highest and second-highest OA excluding GT are bolded and underlined, respectively.

Figure 3: Ablation results on GFLOPs of SMDIM model vs. SCHmUBERT model at different input seq. lengths. All GFLOPs are calculated by thop package for comparison.

Method	Step Time (s)	GPU Memory (GB)
SCHmUBERT	0.54	35
MFA (2SA)	0.63	24
Ours	0.35	21

Table 6: Comparison of per-step latency and GPU memory usage with SCHmUBERT and MFA (2SA) on 2048-token sequences. Our method achieves the lowest latency and memory footprint.

Figure 4: Average Overlapping Area (OA) of music generated by SMDIM under different output sequence lengths.

ID	Grp.	Profession / Major	Status	Yrs	Area of Expertise
P01	Theo.	Composition	Asst. Prof.	17	Harmony, Counterpoint
P02	Theo.	Music Theory	PhD Cand.	14	Form, Tonal Harmony
P03	Theo.	Screen Scoring	Composer	11	Orchestration, Development
P04	Theo.	Jazz Studies	Master Std.	10	Jazz Harmony, Improv.
P05	Theo.	Comp. & Arranging	Lecturer	19	Arrangement, Aural Skills
P06	Perf.	Piano Performance	Musician	15	Classical, Articulation

Table 7: Demographic profiles of the 15 professional evaluators.

Models	Emotional	Melodic	Rhythmic	Structural
	Consistency	Fluency	Stability	Clarity
MusicTr	7.68	7.87	7.37	6.99
SCHmUBERT	7.72	7.85	7.38	7.03
PolyFusion	7.63	7.83	7.34	6.98
MusicMamba	7.69	7.85	7.29	7.00
Ours	7.66	7.90 ∗	7.30	7.09 ∗

Table 8: Subjective listening study results of the 4 evaluation aspects (1-10 scale). Statistically significant improvements over the best baseline are marked with * (

p<0.05

, paired two-sided t-test).

Panel of Figure 5 — Figure 5: Representative failure cases generated by the proposed model. (a)–(b) Failure cases exhibiting extreme pitch ranges and overly dense vertical note stacking, resulting in musically implausible symbolic structures. The regions highlighted in red and blue indicate representative local errors, including abnormal pitch outliers and excessive simultaneous note activations. (c) A failure case from the final portion of a generated sequence, illustrating structural degradation and weakened global musical organization over time.

Symbolic Music Generation

Symbolic music generation creates and edits music as structured notes, rhythms, chords, and scores rather than directly as audio.

See all 58 papers

Oct 21, 2023

Fast Diffusion-GAN Model for Generating Symbolic Music with Emotions

Diffusion models have shown promising results for a wide range of generative tasks with continuous data, such as image and audio synthesis. However, little progress has been made on using diffusion models to generate symbolic music since this class of generative models struggles with discrete data while its iterative sampling process is computationally expensive. In this work, we propose a diffusion model integrated with a Generative Adversarial Network, aiming to mitigate the slow sampling drawback of diffusion models applied to symbolic music generation. We enable the use of diffusion model by representing symbolic music as image-like pianorolls with velocity values instead of discrete event tokens and then used a Variational Autoencoder to obtain their latent representations to train the proposed diffusion model. Our results demonstrate the successful control of our diffusion model to generate symbolic music with a desired emotion. Our model uses merely 15 denoising diffusion steps to produce high-quality music, while most diffusion models for symbolic music generation require hundreds or thousands of steps.

Jincheng Zhang, György Fazekas, Charalampos Saitis

Centre for Digital Music, Queen Mary University of London, UK

Timbre-Conditioned Music Generation Music Generation Symbolic Music Generation Diffusion Models Controllable Music Generation Few-Step Diffusion Sampling

Ask PDF

Aug 5, 2026

Diff-Symbo: Text-Controlled Long-Duration Symbolic Music Generation Using Autoregressive Latent Diffusion Model

Text-controlled symbolic music generation has recently gained research attention due to its versatile, flexible and straightforward approach to music composition. However, previous approaches tend to generate symbolic music with compromising quality, diversity, controllability and limited duration. In this paper, we present Diff-Symbo, an innovative method that uses latent diffusion model (LDM) to generate high-quality, diverse and long-duration symbolic music. To address the lack of text-symbolic music dataset, we develop a comprehensive dataset with 19,345 text templates by employing large language model. Furthermore, we design a music information encoder to reduce the training overhead while extracting more effective control representations. Given textual descriptions, our proposed method leverages LDM to improve the quality and diversity of music generation. Our method also improves the duration and the compositional consistency of music generation through an autoregressive approach. Experimental results show significant improvements of Diff-Symbo in text controllability, duration, and the quality of generated music compared to the baseline models such as GPT-4, MuseCoco and Multitrack Music Transformer (MMT). As one of the pioneer models in this field, Diff-Symbo paves the way towards controllable and high-quality symbolic music composition based on LDM, offering valuable contributions to both music amateurs and practitioners.

Zhiwei Lin, Jun Chen, Boshi Tang +5

Shenzhen International Graduate School, Tsinghua University, Shenzhen, China · Huawei Technologies Co., Ltd., Shenzhen, China · The Chinese University of Hong Kong, Hong Kong SAR, China

Text-to-Music Generation Latent Diffusion Models Controllable Music Generation Symbolic Music Generation Music Generation

Ask PDF

Apr 7, 2026

Anchored Cyclic Generation: A Novel Paradigm for Long-Sequence Symbolic Music Generation

Generating long sequences with structural coherence remains a fundamental challenge for autoregressive models across sequential generation tasks. In symbolic music generation, this challenge is particularly pronounced, as existing methods are constrained by the inherent severe error accumulation problem of autoregressive models, leading to poor performance in music quality and structural integrity. In this paper, we propose the Anchored Cyclic Generation (ACG) paradigm, which relies on anchor features from already identified music to guide subsequent generation during the autoregressive process, effectively mitigating error accumulation in autoregressive methods. Based on the ACG paradigm, we further propose the Hierarchical Anchored Cyclic Generation (Hi-ACG) framework, which employs a systematic global-to-local generation strategy and is highly compatible with our specifically designed piano token, an efficient musical representation. The experimental results demonstrate that compared to traditional autoregressive models, the ACG paradigm achieves reduces cosine distance by an average of 34.7% between predicted feature vectors and ground-truth semantic vectors. In long-sequence symbolic music generation tasks, the Hi-ACG framework significantly outperforms existing mainstream methods in both subjective and objective evaluations. Furthermore, the framework exhibits excellent task generalization capabilities, achieving superior performance in related tasks such as music completion.

Boyu Cao, Lekai Qian, Dehan Li +3

South China University of Technology, School of Future Technology

Symbolic Music Generation Music Generation Autoregressive Generation

Ask PDF

Abstract

Symbolic Music Generation

Explore similar work

Fast Diffusion-GAN Model for Generating Symbolic Music with Emotions

Diff-Symbo: Text-Controlled Long-Duration Symbolic Music Generation Using Autoregressive Latent Diffusion Model

Anchored Cyclic Generation: A Novel Paradigm for Long-Sequence Symbolic Music Generation