While arousal can be reliably predicted from musical audio, emotional valence remains difficult to model due to its subjective and context-dependent nature. In this work, we investigate the potential of in-context learning (ICL) with large language models to advance continuous valence and arousal prediction. Rather than predicting emotion scores directly, our framework leverages contextual demonstrations from an auxiliary dataset to refine the initial predictions produced by a small, task-specific base model. We systematically evaluate a text-based ICL variant using lyrics and generated audio captions, alongside a multimodal variant that incorporates raw audio signals. Using the entire MER60 dataset strictly as a held-out test set, our experiments demonstrate substantial performance gains, particularly for valence. Text-based ICL improves the valence R2 from 0.110 to 0.427 with 80 demonstration shots, while the multimodal variant achieves 0.346 with only 10 shots. These findings suggest that training-free ICL offers an effective inference-time strategy for current language models to predict continuous music emotion.
Figures & tables
Figure 1: Overview of the proposed framework. A base audio model produces initial valence–arousal (VA) predictions, which are refined via in-context learning using either a text-only LLM (bottom) or a multimodal LLM (top).
Figure 2: Prompt structure of text and multimodal ICL, where (vbase,abase)=b denotes the prediction of the base model, and (vgt,agt)=y the ground-truth VA values for a clip.
Model
Size
Training Data
DEAM [ 2 ]
CH818 [ 12 ]
MER60 [ 27 ]
RV2
RA2
RV2
RA2
RV2
RA2
SVR-based [ 11 ]
< 1M
CH818
N/A
N/A
0.250
0.770
0.270
0.800
VGGish [ 3 ]
62M
DEAM
0.520
0.611
0.122
0.256
–0.071
0.424
Music2Emo [ 15 ]
100M
DEAM + EmoMusic + PMemo
0.518
0.623
0.204
0.276
–0.438
0.511
Base model (ours)
91M
DEAM
0.542
0.529
–0.024
0.099
–0.127
0.660
Base model (ours)
91M
CH818
–0.062
0.168
0.430
0.782
0.110
0.783
Table 1: Comparison of baseline models under within- and cross-dataset settings.
Table 2: Text-based ICL results (mean±STD over five runs).
Recent advances in text-to-music models have enabled coherent music generation from text prompts, yet fine-grained emotional control remains unresolved. We introduce LARA-Gen, a framework for continuous emotion control that aligns the internal hidden states with an external music understanding model through Latent Affective Representation Alignment (LARA), enabling effective training. In addition, we design an emotion control module based on a continuous valence-arousal space, disentangling emotional attributes from textual content and bypassing the bottlenecks of text-based prompting. Furthermore, we establish a benchmark with a curated test set and a robust Emotion Predictor, facilitating objective evaluation of emotional controllability in music generation. Extensive experiments demonstrate that LARA-Gen achieves continuous, fine-grained control of emotion and significantly outperforms baselines in both emotion adherence and music quality. Generated samples are available at https://anonymous2232330.github.io/laragen-web/.
Jiahao Mei, Shimeng Li, Xuenan Xu +7
X-LANCE Lab, Shanghai Jiao Tong University, China · Shanghai AI Lab, China · Shanghai Mental Health Center, Shanghai Jiao Tong University School of Medicine, China
This paper investigates whether music large language models (MusicLLMs) can be aligned for emotion regression. While MusicLLMs have shown strong performance in music information retrieval tasks, their ability to predict arousal and valence scores remains limited, since emotion regression has not been an explicit training objective. To examine whether MusicLLMs can be aligned with emotion, we train MusicLLMs on emotion regression and compare two strategies: instruction tuning and feedback-driven alignment. Our experiments show that task-aware instruction tuning enables MusicLLMs to predict emotion levels to some extent, although the accuracy remains limited. Applying feedback-driven alignment with a verifiable numerical reward substantially improves performance on both arousal and valence over instruction tuning alone. We further show that our approach improves emotion regression performance while maintaining MusicQA capability.
Recent generative music models offer increasingly fine-grained control through text and audio conditioning, yet how faithfully they follow intended emotional cues remains an open question. We address this gap with a unified evaluation pipeline for emotion-following in generated music. Using all 1000 tracks in GTZAN, we extract semantic audio descriptions with DashengLM, an audio captioning model, and estimate source-track valence and arousal with Music2Emotion, a music emotion recognition model. We construct affect-aware text prompts by combining descriptions with top-ranked emotion tags and generate 30-second outputs with three systems, Stable Audio Open, MusicGen, and InspireMusic, evaluating both text- and audio-conditioned generation. To measure emotion-following, we compute valence and arousal on generated audio and compare them with the source tracks using absolute error and Euclidean distance in valence-arousal space. Text-conditioned generation consistently outperforms audio conditioning, with MusicGen (text) and InspireMusic (text) achieving the best performance, while audio-conditioned variants prove less stable. We further find that valence is preserved more reliably than arousal and that emotion-following varies substantially across genres. These findings underscore the importance of evaluating affective controllability directly rather than relying solely on general quality or prompt-relevance metrics.