SongEcho: Towards Cover Song Generation via Instance-Adaptive Element-wise Linear Modulation
Authors: Sifei Li, Yang Li, Zizhou Wang, Yuxin Zhang, Fuzhang Wu, Oliver Deussen, Tong-Yee Lee, Weiming Dong
Organizations: MAIS, Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · ISRC, Institute of Software, Chinese Academy of Sciences · University of Konstanz · National Cheng-Kung University
Cover songs constitute a vital aspect of musical culture, preserving the core melody of an original composition while reinterpreting it to infuse novel emotional depth and thematic emphasis. Although prior research has explored the reinterpretation of instrumental music through melody-conditioned text-to-music models, the task of cover song generation remains largely unaddressed. In this work, we reformulate our cover song generation as a conditional generation, which simultaneously generates new vocals and accompaniment conditioned on the original vocal melody and text prompts. To this end, we present SongEcho, which leverages Instance-Adaptive Element-wise Linear Modulation (IA-EiLM), a framework that incorporates controllable generation by improving both conditioning injection mechanism and conditional representation. To enhance the conditioning injection mechanism, we extend Feature-wise Linear Modulation (FiLM) to an Element-wise Linear Modulation (EiLM), to facilitate precise temporal alignment in melody control. For conditional representations, we propose Instance-Adaptive Condition Refinement (IACR), which refines conditioning features by interacting with the hidden states of the generative model, yielding instance-adaptive conditioning. Additionally, to address the scarcity of large-scale, open-source full-song datasets, we construct Suno70k, a high-quality AI song dataset enriched with comprehensive annotations. Experimental results across multiple datasets demonstrate that our approach generates superior cover songs compared to existing methods, while requiring fewer than 30% of the trainable parameters. The code, dataset, and demos are available at https://github.com/lsfhuihuiff/SongEcho_ICLR2026.
Figures & tables
Figure 1: Differences between other condition injection mechanisms and our approach. EiLM eliminates the need for separate learning of temporal alignment in (a) while offering more flexible modulation than (b). “ ⊕ ” represents element-wise addition, and “ ⊙ ” represents element-wise multiplication.
Figure 2: We employ a Diffusion Transformer (DiT) as the song generation backbone with a novel conditioning method, “IA-EiLM”, for vocal melody control. A Pitch Extractor and Melody Encoder extract melody features, denoted as “ m ”. The IA-EiLM module, integrated into each Transformer block, comprises two components: IACR and EiLM. “IACR” facilitates interaction between “ m ” and hidden states “ hi ”, refining melody condition, while “EiLM” modulates “ hi ” into “ him ” with modulation parameters “ γi ” and “ βi ”, derived from the refined melody condition.
RPA ↑
RCA ↑
OA ↑
CLAP ↑
FD ↓
KL ↓
PER ↓
TP ↓
ACE-Step ( Gong et al., 2025 )
-
-
-
0.2930
73.53
0.2670
0.4168
-
ACE-Step+SA ControlNet ( Hou et al., 2025 )
0.6209
0.6440
0.6858
0.2875
105.95
0.2019
0.3714
1.6B
ACE-Step+SA ControlNet+LoRA ( Hou et al., 2025 )
0.6214
0.6431
0.6833
0.2892
99.19
0.1850
0.3734
331M
ACE-Step+MuseControlLite ( Tsai et al., 2025 )
0.5205
0.5346
0.5940
0.2977
72.04
0.2151
0.4194
189M
SongEcho (Ours)
0.7080
0.7339
0.6952
0.3243
42.06
0.1123
0.2951
49.1M
Table 1: Quantitative evaluation results on Suno70k test set. “TP” represents trainable parameters. The best results are in highlighted bold and the second best ones are underlined (same in the following tables).
RPA ↑
RCA ↑
OA ↑
CLAP ↑
FD ↓
KL ↓
PER ↓
ACE-Step ( Gong et al. (2025) )
-
-
-
0.2800
70.54
0.3478
0.3899
ACE-Step+SA ControlNet ( Hou et al., 2025 )
0.6078
0.6336
0.6759
0.2477
110.73
0.2479
0.3874
ACE-Step+SA ControlNet+LoRA ( Hou et al. (2025) )
0.6143
0.6361
0.6741
0.2536
97.60
0.2407
0.4114
ACE-Step+MuseControlLite ( Tsai et al. (2025) )
0.5164
0.5275
0.6025
0.2462
68.73
0.2764
0.4758
SongEcho (Ours)
0.7066
0.7333
0.7001
0.2674
40.37
0.2117
0.3091
Table 2: Quantitative evaluation results on Suno70k test set with swapped tags.
RPA ↑
RCA ↑
OA ↑
CLAP ↑
FD ↓
KL ↓
PER ↓
ACE-Step ( Gong et al. (2025) )
-
-
-
0.2590
71.56
0.3305
0.4510
ACE-Step+SA ControlNet ( Hou et al., 2025 )
0.6463
0.6600
0.6934
0.2666
114.18
0.4069
0.5234
ACE-Step+SA ControlNet+LoRA ( Hou et al. (2025) )
0.6335
0.6465
0.6837
0.2583
104.76
0.3112
0.5901
ACE-Step+MuseControlLite ( Tsai et al. (2025) )
0.5421
0.5498
0.6208
0.2600
90.19
0.3913
0.5760
SongEcho (Ours)
0.7164
0.7326
0.7097
0.2824
51.98
0.1933
0.4487
Table 3: Quantitative evaluation results on SongEval ( Yao et al., 2025 ) .
w/ Music Background
w/o Music Background
MF ↑
TA ↑
AQ ↑
OP ↑
MF ↑
TA ↑
AQ ↑
OP ↑
ACE-Step+SA ControlNet+LoRA ( Hou et al., 2025 )
3.056
3.285
3.085
3.104
3.133
3.636
3.182
3.160
ACE-Step+MuseControlLite ( Tsai et al., 2025 )
2.630
3.026
2.581
2.622
2.689
3.333
2.591
2.622
SongEcho (Ours)
3.644
3.800
3.756
3.819
3.884
4.160
3.916
3.942
Table 4: Mean opinion scores (1–5) comparing melody fidelity (MF), text adherence (TA), Audio Quality (AQ) and overall preference (OP).
RPA ↑
RCA ↑
OA ↑
CLAP ↑
FD ↓
KL ↓
PER ↓
w/ EA, w/o IACR
0.6336
0.6476
0.6683
0.3014
73.83
0.1689
0.3276
w/ EiLM, w/o IACR
0.6799
0.7000
0.6793
0.2999
75.28
0.1569
0.3166
IA-EiLM → Self-Attn
0.6190
0.6429
0.6303
0.3195
47.34
0.1434
0.3462
100 Training Samples
0.4677
0.4889
0.4812
0.2854
71.85
0.1402
0.4159
1000 Training Samples
0.6505
0.6775
0.6559
0.3115
48.59
0.1135
0.2871
SongEcho (Ours)
0.7080
0.7339
0.6952
0.3243
42.06
0.1123
0.2951
Table 5: Ablation study of our method. “w/ EA” represents replace EiLM with element-wise addition, and “IA-EiLM → Self-Attn” indicates that we insert our IA-EiLM module before the self-attention layer in each Transformer block.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 3: We visualize the F0 contour extracted from the song, along with the word-level and phoneme-level timestamps produced by the Montreal Forced Aligner (MFA) ( McAuliffe et al., 2017 ) . The full lyrics used in the example are: “I had a horse named Hair, we weren’t going nowhere.”
Figure 4: Lyrics transcribed by Whisper ( Radford et al., 2023 ) with All-in-One ( Yao et al., 2025 ) for SongEval ( Yao et al., 2025 ) exhibit punctuation errors, including run-on sentences (orange), multiple clauses per line (blue), and incorrect sentence breaks (green), whereas Suno70k’s native lyrics ensure each phrase is on a separate line.
RPA ↑
RCA ↑
OA ↑
CLAP ↑
FD ↓
KL ↓
PER ↓
Suno70k
λ=15.0
0.5205
0.5346
0.5940
0.2977
72.04
0.2151
0.4194
λtext=15.0,λmelody=7.0
0.4896
0.5042
0.5557
0.3159
55.82
0.1820
0.4837
λtext=7.0,λmelody=15.0
0.5536
0.5653
0.6351
0.2942
73.59
0.2763
0.6159
Suno70k+Swapped Tags
λ=15.0
0.5164
0.5275
0.6025
0.2462
68.73
0.2764
0.4758
λtext=15.0,λmelody=7.0
0.4798
0.4924
0.5669
0.2704
54.11
0.3087
0.5446
λtext=7.0,λmelody=15.0
0.5578
0.5682
0.6459
0.2290
74.39
0.3445
0.6387
Appendix
Table 6: Quantitative evaluation results of three inference guidance strategies for the ACE-Step+MuseControlLite baseline ( Tsai et al., 2025 ) .
Dataset
Suno70k
SongEval
Metrics
Coherence ↑
Musicality ↑
Memorability ↑
Clarity ↑
Naturalness ↑
Coherence ↑
Musicality ↑
Memorability ↑
Clarity ↑
Naturalness ↑
ACE-Step
3.144
2.873
2.991
2.957
2.817
3.389
3.149
3.254
3.180
3.059
SA ControlNet
3.237
2.958
3.058
3.018
2.909
3.317
3.061
3.131
3.072
2.953
MuseControlLite
2.871
2.591
2.667
2.607
2.592
2.914
2.686
2.705
2.648
2.617
SongEcho (Ours)
3.776
3.485
3.644
3.534
3.440
3.941
3.698
3.834
3.680
3.590
Appendix
Table 7: Comparison of SongEval ( Yao et al., 2025 ) Aesthetics Metrics Across Methods.
Figure 5: Attention map visualization of MuseControlLite ( Tsai et al., 2025 ) under full-audio conditioning. The clear diagonal pattern indicates that the post-softmax attention matrix approximates an identity matrix.
A cover song re-renders an existing piece, most commonly by preserving its tonal content -- melody and chord progression -- while reshaping other musical attributes. This suggests a natural formulation for generative cover synthesis with pretrained music language models: explicitly control tonal content, while specifying lyrics and style through text. Existing approaches condition on frame-level features, tightly coupling generation to the reference's timing and structure, and limiting flexibility such as tempo variation, structural rearrangement, or partial conditioning. To this end, we present MuLaCover, a cover generation model that conditions a pretrained text-to-song foundation model on an editable symbolic lead sheet. Our design enables alignment-free control: generated outputs preserve the tonal signature of the source without frame-level alignment, allowing flexible timing, structure, and segment-level generation. We further introduce a training curriculum with partially mismatched audio-symbolic pairs, improving diversity and robustness. We evaluate MuLaCover with objective metrics, standard subjective ratings, and in-depth expert interviews that provide fine-grained diagnostic insights. Experiments show that MuLaCover achieves state-of-the-art performance among open-source systems and is competitive with leading commercial models such as Suno-v5.5. Audio samples and code are available at our project page: https://mulalabs.ai/mulacover/
Lynn Yi, Zihan Xiong, Junjie Cao +17
MBZUAI · Peking University · MuLa Labs, Vera Praxis Lab +2
Cover song generation (CSG) should preserve the melodic and linguistic content of a reference song while recreating the remaining musical components. The state-of-the-art model SongEcho utilizes F0 sequences and voiced/unvoiced (V/UV) tags for conditioning; however, implicit linguistic information from V/UV tags cannot guarantee lyric accuracy, leading to a high phoneme error rate (PER). Inspired by singing voice synthesis (SVS), we propose MPEcho, which integrates a phoneme encoder and a length regulator (LR) into the SongEcho framework. By providing explicit phoneme-level conditioning and precise temporal boundaries, MPEcho significantly reduces PER. To enable this, we developed Phonsa, a Whisper-based automatic transcription model that provides high-precision phoneme-level annotations for singing voices, overcoming the scarcity of high-quality audio-phoneme pairs. Experimental results validate the effectiveness of Phonsa for alignment and MPEcho for end-to-end CSG. The audio samples, code and weights can be accessed from https://lonian6.github.io/MPEcho.github.io/.
In this report, we present a unified song generation framework capable of producing high-quality full-length music from lyrics, text descriptions, and musical attributes. The proposed framework supports three tasks: Lyrics-to-Song Generation, which generates complete songs from text descriptions, lyrics, and musical attributes; Instrumental Music Generation, which creates music without vocals; and Cover Song Generation, which reinterprets existing songs with different styles while preserving their melodic content. Architecturally, our system consists of four main components: a semantic-aware tokenizer, hybird-LM, FullDiT, and a two-level melody module. The tokenizer encodes audio into 8-codebook RVQ tokens for efficient discrete music representation. Based on these tokens, hybird-LM performs hierarchical autoregressive audio-token modeling for full-song generation. To improve audio fidelity, FullDiT performs full-song flow matching in a continuous VAE latent space conditioned on codec tokens, lyrics, and text captions. For cover song generation, the melody module extracts and discretizes melody cues from reference audio to guide generation while preserving the original melodic content. Finally, we investigate DPO, GRPO, and OPD as reward-based post-training strategies for hybird-LM and apply flow-based GRPO to FullDiT to improve musicality and rendering quality. Experimental results on a multilingual automatic benchmark, complemented by the Artificial Analysis Music with Vocals leaderboard, show that the proposed framework achieves competitive performance in the evaluated settings.