SongEcho: Towards Cover Song Generation via Instance-Adaptive Element-wise Linear Modulation

Sifei Li; Yang Li; Zizhou Wang; Yuxin Zhang; Fuzhang Wu; Oliver Deussen; Tong-Yee Lee; Weiming Dong

cs.SDFeb 23, 2026

SongEcho: Towards Cover Song Generation via Instance-Adaptive Element-wise Linear Modulation

Authors: Sifei Li, Yang Li, Zizhou Wang, Yuxin Zhang, Fuzhang Wu, Oliver Deussen, Tong-Yee Lee, Weiming Dong

Organizations: MAIS, Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · ISRC, Institute of Software, Chinese Academy of Sciences · University of Konstanz ·

Abstract

Cover songs constitute a vital aspect of musical culture, preserving the core melody of an original composition while reinterpreting it to infuse novel emotional depth and thematic emphasis. Although prior research has explored the reinterpretation of instrumental music through melody-conditioned text-to-music models, the task of cover song generation remains largely unaddressed. In this work, we reformulate our cover song generation as a conditional generation, which simultaneously generates new vocals and accompaniment conditioned on the original vocal melody and text prompts. To this end, we present SongEcho, which leverages Instance-Adaptive Element-wise Linear Modulation (IA-EiLM), a framework that incorporates controllable generation by improving both conditioning injection mechanism and conditional representation. To enhance the conditioning injection mechanism, we extend Feature-wise Linear Modulation (FiLM) to an Element-wise Linear Modulation (EiLM), to facilitate precise temporal alignment in melody control. For conditional representations, we propose Instance-Adaptive Condition Refinement (IACR), which refines conditioning features by interacting with the hidden states of the generative model, yielding instance-adaptive conditioning. Additionally, to address the scarcity of large-scale, open-source full-song datasets, we construct Suno70k, a high-quality AI song dataset enriched with comprehensive annotations. Experimental results across multiple datasets demonstrate that our approach generates superior cover songs compared to existing methods, while requiring fewer than 30% of the trainable parameters. The code, dataset, and demos are available at https://github.com/lsfhuihuiff/SongEcho_ICLR2026.

Figures & tables

Figure 1: Differences between other condition injection mechanisms and our approach. EiLM eliminates the need for separate learning of temporal alignment in (a) while offering more flexible modulation than (b). “ $\oplus$ ” represents element-wise addition, and “ $\odot$ ” represents element-wise multiplication.

Figure 2: We employ a Diffusion Transformer (DiT) as the song generation backbone with a novel conditioning method, “IA-EiLM”, for vocal melody control. A Pitch Extractor and Melody Encoder extract melody features, denoted as “ $m$ ”. The IA-EiLM module, integrated into each Transformer block, comprises two components: IACR and EiLM. “IACR” facilitates interaction between “ $m$ ” and hidden states “ $h_{i}$ ”, refining melody condition, while “EiLM” modulates “ $h_{i}$ ” into “ $h_{i}^{m}$ ” with modulation parameters “ $\gamma_{i}$ ” and “ $\beta_{i}$ ”, derived from the refined melody condition.

	RPA $\uparrow$	RCA $\uparrow$	OA $\uparrow$	CLAP $\uparrow$	FD $\downarrow$	KL $\downarrow$	PER $\downarrow$	TP $\downarrow$
ACE-Step ( Gong et al., 2025 )	-	-	-	0.2930	73.53	0.2670	0.4168	-
ACE-Step+SA ControlNet ( Hou et al., 2025 )	0.6209	0.6440	0.6858	0.2875	105.95	0.2019	0.3714	1.6B
ACE-Step+SA ControlNet+LoRA ( Hou et al., 2025 )	0.6214	0.6431	0.6833	0.2892	99.19	0.1850	0.3734	331M
ACE-Step+MuseControlLite ( Tsai et al., 2025 )	0.5205	0.5346	0.5940	0.2977	72.04	0.2151	0.4194	189M
SongEcho (Ours)	0.7080	0.7339	0.6952	0.3243	42.06	0.1123	0.2951	49.1M

Table 1: Quantitative evaluation results on Suno70k test set. “TP” represents trainable parameters. The best results are in highlighted bold and the second best ones are underlined (same in the following tables).

	RPA $\uparrow$	RCA $\uparrow$	OA $\uparrow$	CLAP $\uparrow$	FD $\downarrow$	KL $\downarrow$	PER $\downarrow$
ACE-Step ( Gong et al. (2025) )	-	-	-	0.2800	70.54	0.3478	0.3899
ACE-Step+SA ControlNet ( Hou et al., 2025 )	0.6078	0.6336	0.6759	0.2477	110.73	0.2479	0.3874
ACE-Step+SA ControlNet+LoRA ( Hou et al. (2025) )	0.6143	0.6361	0.6741	0.2536	97.60	0.2407	0.4114
ACE-Step+MuseControlLite ( Tsai et al. (2025) )	0.5164	0.5275	0.6025	0.2462	68.73	0.2764	0.4758
SongEcho (Ours)	0.7066	0.7333	0.7001	0.2674	40.37	0.2117	0.3091

Table 2: Quantitative evaluation results on Suno70k test set with swapped tags.

	RPA $\uparrow$	RCA $\uparrow$	OA $\uparrow$	CLAP $\uparrow$	FD $\downarrow$	KL $\downarrow$	PER $\downarrow$
ACE-Step ( Gong et al. (2025) )	-	-	-	0.2590	71.56	0.3305	0.4510
ACE-Step+SA ControlNet ( Hou et al., 2025 )	0.6463	0.6600	0.6934	0.2666	114.18	0.4069	0.5234
ACE-Step+SA ControlNet+LoRA ( Hou et al. (2025) )	0.6335	0.6465	0.6837	0.2583	104.76	0.3112	0.5901
ACE-Step+MuseControlLite ( Tsai et al. (2025) )	0.5421	0.5498	0.6208	0.2600	90.19	0.3913	0.5760
SongEcho (Ours)	0.7164	0.7326	0.7097	0.2824	51.98	0.1933	0.4487

Table 3: Quantitative evaluation results on SongEval ( Yao et al., 2025 ) .

	w/ Music Background				w/o Music Background
	MF $\uparrow$	TA $\uparrow$	AQ $\uparrow$	OP $\uparrow$	MF $\uparrow$	TA $\uparrow$	AQ $\uparrow$	OP $\uparrow$
ACE-Step+SA ControlNet+LoRA ( Hou et al., 2025 )	3.056	3.285	3.085	3.104	3.133	3.636	3.182	3.160
ACE-Step+MuseControlLite ( Tsai et al., 2025 )	2.630	3.026	2.581	2.622	2.689	3.333	2.591	2.622
SongEcho (Ours)	3.644	3.800	3.756	3.819	3.884	4.160	3.916	3.942

Table 4: Mean opinion scores (1–5) comparing melody fidelity (MF), text adherence (TA), Audio Quality (AQ) and overall preference (OP).

	RPA $\uparrow$	RCA $\uparrow$	OA $\uparrow$	CLAP $\uparrow$	FD $\downarrow$	KL $\downarrow$	PER $\downarrow$
w/ EA, w/o IACR	0.6336	0.6476	0.6683	0.3014	73.83	0.1689	0.3276
w/ EiLM, w/o IACR	0.6799	0.7000	0.6793	0.2999	75.28	0.1569	0.3166
IA-EiLM $\rightarrow$ Self-Attn	0.6190	0.6429	0.6303	0.3195	47.34	0.1434	0.3462
100 Training Samples	0.4677	0.4889	0.4812	0.2854	71.85	0.1402	0.4159
1000 Training Samples	0.6505	0.6775	0.6559	0.3115	48.59	0.1135	0.2871
SongEcho (Ours)	0.7080	0.7339	0.6952	0.3243	42.06	0.1123	0.2951

Table 5: Ablation study of our method. “w/ EA” represents replace EiLM with element-wise addition, and “IA-EiLM

\rightarrow

Self-Attn” indicates that we insert our IA-EiLM module before the self-attention layer in each Transformer block.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 3: We visualize the F0 contour extracted from the song, along with the word-level and phoneme-level timestamps produced by the Montreal Forced Aligner (MFA) ( McAuliffe et al., 2017 ) . The full lyrics used in the example are: “I had a horse named Hair, we weren’t going nowhere.”

Figure 4: Lyrics transcribed by Whisper ( Radford et al., 2023 ) with All-in-One ( Yao et al., 2025 ) for SongEval ( Yao et al., 2025 ) exhibit punctuation errors, including run-on sentences (orange), multiple clauses per line (blue), and incorrect sentence breaks (green), whereas Suno70k’s native lyrics ensure each phrase is on a separate line.

		RPA $\uparrow$	RCA $\uparrow$	OA $\uparrow$	CLAP $\uparrow$	FD $\downarrow$	KL $\downarrow$	PER $\downarrow$
Suno70k	$\lambda=15.0$	0.5205	0.5346	0.5940	0.2977	72.04	0.2151	0.4194
	$\lambda_{text}=15.0,\lambda_{melody}=7.0$	0.4896	0.5042	0.5557	0.3159	55.82	0.1820	0.4837
	$\lambda_{text}=7.0,\lambda_{melody}=15.0$	0.5536	0.5653	0.6351	0.2942	73.59	0.2763	0.6159
Suno70k+Swapped Tags	$\lambda=15.0$	0.5164	0.5275	0.6025	0.2462	68.73	0.2764	0.4758
	$\lambda_{text}=15.0,\lambda_{melody}=7.0$	0.4798	0.4924	0.5669	0.2704	54.11	0.3087	0.5446
	$\lambda_{text}=7.0,\lambda_{melody}=15.0$	0.5578	0.5682	0.6459	0.2290	74.39	0.3445	0.6387

Appendix

Table 6: Quantitative evaluation results of three inference guidance strategies for the ACE-Step+MuseControlLite baseline ( Tsai et al., 2025 ) .

Dataset	Suno70k					SongEval
Metrics	Coherence $\uparrow$	Musicality $\uparrow$	Memorability $\uparrow$	Clarity $\uparrow$	Naturalness $\uparrow$	Coherence $\uparrow$	Musicality $\uparrow$	Memorability $\uparrow$	Clarity $\uparrow$	Naturalness $\uparrow$
ACE-Step	3.144	2.873	2.991	2.957	2.817	3.389	3.149	3.254	3.180	3.059
SA ControlNet	3.237	2.958	3.058	3.018	2.909	3.317	3.061	3.131	3.072	2.953
MuseControlLite	2.871	2.591	2.667	2.607	2.592	2.914	2.686	2.705	2.648	2.617
SongEcho (Ours)	3.776	3.485	3.644	3.534	3.440	3.941	3.698	3.834	3.680	3.590

Appendix

Table 7: Comparison of SongEval ( Yao et al., 2025 ) Aesthetics Metrics Across Methods.

Figure 5: Attention map visualization of MuseControlLite ( Tsai et al., 2025 ) under full-audio conditioning. The clear diagonal pattern indicates that the post-softmax attention matrix approximates an identity matrix.

Music Generation

Music generation studies computational methods for composing, synthesizing, and editing music from symbolic, audio, textual, and multimodal inputs.

See all 107 papers

MuLaCover: Flexible Cover Song Generation via Symbolic Tonal Control
Oct 19, 2026Lynn Yi, Zihan Xiong, Junjie Cao +17Music Generation Controllable Music Generation
MBZUAI · Peking University · MuLa Labs, Vera Praxis Lab +2
MPEcho: A Melody and Phoneme-Aware Generative Framework for Controllable Cover Song Generation
Jul 29, 2026Wei-Jaw Lee, Hsuan-Yu Yeh, Ting-Yi Hu +3Controllable Music Generation Music Generation
GICE, National Taiwan University
Pushing the Frontier of Full-Song Generation: Hierarchical Autoregressive Planning Meets Flow-Matching Rendering
Jul 22, 2026Junyu Dai, Xinyue Fan, Weiqin Li +14Text-to-Music Generation Music Generation
Alibaba Token Foundry

Oct 19, 2026J/K move · Enter open · S save

MuLaCover: Flexible Cover Song Generation via Symbolic Tonal Control

Lynn Yi, Zihan Xiong, Junjie Cao, Xiaolong Weng+16

MBZUAI · Peking University · MuLa Labs, Vera Praxis Lab+2

Music Generation Controllable Music Generation

Open paper AskPDF arXiv

A cover song re-renders an existing piece, most commonly by preserving its tonal content -- melody and chord progression -- while reshaping other musical attributes. This suggests a natural formulation for generative cover synthesis with pretrained music language models: explicitly control tonal content, while specifying lyrics and style through text. Existing approaches condition on frame-level features, tightly coupling generation to the reference's timing and structure, and limiting flexibility such as tempo variation, structural rearrangement, or partial conditioning. To this end, we present MuLaCover, a cover generation model that conditions a pretrained text-to-song foundation model on an editable symbolic lead sheet. Our design enables alignment-free control: generated outputs preserve the tonal signature of the source without frame-level alignment, allowing flexible timing, structure, and segment-level generation. We further introduce a training curriculum with partially mismatched audio-symbolic pairs, improving diversity and robustness. We evaluate MuLaCover with objective metrics, standard subjective ratings, and in-depth expert interviews that provide fine-grained diagnostic insights. Experiments show that MuLaCover achieves state-of-the-art performance among open-source systems and is competitive with leading commercial models such as Suno-v5.5. Audio samples and code are available at our project page: https://mulalabs.ai/mulacover/

Abstract

Music Generation

Explore similar work

MuLaCover: Flexible Cover Song Generation via Symbolic Tonal Control

MPEcho: A Melody and Phoneme-Aware Generative Framework for Controllable Cover Song Generation

Pushing the Frontier of Full-Song Generation: Hierarchical Autoregressive Planning Meets Flow-Matching Rendering