Known Music, Unknown Response: Leakage-Audited Dynamic Arousal Forecasting With Scheduled Future Music

Bin Wen; Dai-Qiao Zhang; Tien-Ping Tan

cs.SDOct 16, 2026

Known Music, Unknown Response: Leakage-Audited Dynamic Arousal Forecasting With Scheduled Future Music

Authors: Bin Wen, Dai-Qiao Zhang, Tien-Ping Tan

Organizations: School of Computer Sciences, Universiti Sains Malaysia, 11800, Pulau Pinang, Malaysia · School of Mathematical Sciences, Universiti Sains Malaysia, 11800, Pulau Pinang, Malaysia

Abstract

Dynamic music emotion recognition usually estimates concurrent or lag-aligned labels from audio that has already been observed. We instead forecast the change in consensus arousal when the upcoming music is scheduled but all later affect labels remain hidden. Evaluation holds out complete songs, encodes each eligible 4-s chunk independently, predicts arousal innovation, and compares the correct future with displaced segments from the same song. On PMEmo and a 1,494-song DEAM evaluation pool excluded from engineering development, a complete past/future MuQ representation raises concordance correlation coefficient (CCC) by 0.389--0.507 over recent affect and remains 0.268--0.382 above a calibrated nonlinear state model. A future-only trajectory contains no past or current audio and still exceeds that state model by 0.150--0.249. The earliest legal future chunk, which ends 5.5--11 s before the target, also remains informative. Correct placement exceeds an overlapping 5-s shift by 0.100--0.209 and the acoustic-nearest eligible same-song segment by 0.078--0.105, with positive song-cluster lower bounds. These results show that both scheduled content and its within-song position carry forecast information under the stated observation rules. The study provides a leakage-controlled reference for known-future music forecasting without treating predictive substitutions as causal interventions.

Figures & tables

Figure 1: Forecast setting. Affect labels after origin

t

are hidden although the scheduled waveform is known. A valid chunk lies completely inside the permitted future interval. Negative controls preserve the current state and song identity while replacing only the scheduled segment.

Task	Target	Audio available	Future response labels	Main identification question
Static MER	Song/excerpt emotion	Complete target excerpt	Not applicable	What emotion does the excerpt express?
Dynamic MER	Concurrent/lagged emotion	Target-era acoustic context	Often used for training alignment	What emotion is expressed now?
Past-only affect forecast	Future level/change	History through origin	Hidden	What follows from state and observed history?
This work	Future arousal innovation	History plus scheduled future support	Hidden after origin	Does the known future, and its placement, add forecast information?
Causal recommendation	Individual outcome under an action	Candidate/action description	Counterfactual unobserved	Which action changes an individual’s outcome?

Table 1: Task Taxonomy and Information Available at Prediction Time

Term	Meaning
$t$	Forecast origin; labels are observed only through this time
$H$	Forecast horizon
$\delta$	Response gap between legal music support and target
$\mathbf{s}_{t}$	Affect history available at the origin
$a_{t,H}$	Scheduled waveform interval permitted at prediction time
$\Delta\mathbf{y}_{t,H}$	Change from the current to the later affect label

Table 2: Notation and Terms Used in the Forecast Setting

Threat	Failure mode	Design response
Emotion inertia	Copy current level	Predict innovation; strong state controls
Track identity	Memorize song/style	Hold out complete songs
Encoder leakage	Attend beyond support	Independent complete 4-s chunks
Coarse content	Any segment works	Same-song local and nearest controls
Window dependence	Narrow uncertainty	Song-cluster bootstrap

Table 3: Primary Threats and Experimental Responses

Data	Horizon	Songs	Windows	Win./song
PMEmo	10 s	611	19,336	31.6
PMEmo	15 s	481	13,677	28.4
DEAM holdout	10 s	1,494	74,181	49.7
DEAM holdout	15 s	1,494	59,241	39.7

Table 4: Evaluation Data After Complete-Support Filtering

Component	Setting
Audio support	4-s chunks; 1-s stride
State ridge	Standardized; $\alpha=10$
MuQ head	Train-fold PCA-256; ridge $\alpha=100$
Outer/inner splits	$3\times 5$ / 4; song-disjoint
Response delay	2 s
Bootstrap	5,000 song-cluster draws

Table 5: Fixed Model and Evaluation Settings

Song	Cutoff	Mean cos.	Min cos.	Rel. L2
1	19.22 s	0.923	0.366	0.350
10	19.22 s	0.890	0.307	0.445
100	24.03 s	0.899	0.294	0.429

Table 6: Suffix-Perturbation Audit of Full-Song MuQ Encoding

Data	$H$	State	MuQ mean	MuQ relation	Relation–state [adj. 97.5% CI]
PMEmo	10 s	0.028	0.315	0.417	+0.389 [0.342, 0.434]
PMEmo	15 s	0.006	0.335	0.453	+0.446 [0.370, 0.514]
DEAM holdout	10 s	0.041	0.317	0.458	+0.416 [0.391, 0.442]
DEAM holdout	15 s	0.058	0.402	0.564	+0.507 [0.475, 0.536]

Table 7: Protocol-Matched Forecasts of Arousal Innovation

Data	$H$	State ridge	HGB raw/cal.	Past only	Future only	Full relation	Future–past [LB]
PMEmo	10 s	0.028	0.075/0.159	0.158	0.310	0.427	+0.151 [0.112]
PMEmo	15 s	0.006	0.081/0.127	0.163	0.323	0.462	+0.160 [0.093]
DEAM holdout	10 s	0.041	0.094/0.161	0.128	0.339	0.465	+0.211 [0.190]
DEAM holdout	15 s	0.058	0.129/0.180	0.168	0.429	0.562	+0.261 [0.236]

Table 8: Strong-State and Past/Future Attribution. “HGB raw” is the direct-innovation output used to form residual targets; “cal.” applies an inner-fold location–scale map only for the CCC state comparison.

Data	$H$	State HGB (cal.)	Earliest 4 s	Latest 4 s	Full-support mean	Early–state [LB]
PMEmo	10 s	0.159	0.243	0.285	0.319	+0.084 [0.036]
PMEmo	15 s	0.127	0.205	0.306	0.339	+0.078 [0.026]
DEAM holdout	10 s	0.161	0.241	0.290	0.338	+0.080 [0.046]
DEAM holdout	15 s	0.180	0.224	0.352	0.422	+0.044 [0.005]

Table 9: Candidate-Only Localization Within the Legal Future Support (CCC)

		MAE			RMSE
Data	$H$	HGB raw	Future only	Full relation	HGB raw	Future only	Full relation
PMEmo	10 s	0.02531	0.02525	0.02438	0.03681	0.03502	0.03312
PMEmo	15 s	0.03174	0.03212	0.03032	0.04545	0.04382	0.04052
DEAM holdout	10 s	0.05562	0.05497	0.05260	0.09283	0.08497	0.08004
DEAM holdout	15 s	0.07713	0.07383	0.06941	0.12250	0.10816	0.10019

Table 10: Absolute-Error Audit for the Strong-State Attribution Pipeline

Figure 2: Main empirical hierarchy. (a) Candidate-only future audio is well above state, and the complete past/future relation is higher still. (b) Correct placement remains superior after the future is replaced by a strongly overlapping local shift or by the acoustic-nearest eligible same-song segment.

Data	$H$	Local shift	Overlap	Local margin [LB]	NN cosine	NN margin [LB]
PMEmo	10 s	4.86 s	0.392	+0.135 [0.108]	0.947	+0.105 [0.078]
PMEmo	15 s	4.83 s	0.629	+0.100 [0.071]	0.979	+0.086 [0.063]
DEAM holdout	10 s	5.00 s	0.375	+0.209 [0.185]	0.939	+0.087 [0.074]
DEAM holdout	15 s	5.00 s	0.615	+0.153 [0.131]	0.973	+0.078 [0.067]

Table 11: Hard Same-Song Temporal Controls

Data	$H$	Repeat SD	Fold range	Positive
PMEmo	10 s	0.0043	[0.086, 0.192]	15/15
PMEmo	15 s	0.0035	[0.075, 0.144]	15/15
DEAM	10 s	0.0026	[0.152, 0.241]	15/15
DEAM	15 s	0.0014	[0.089, 0.175]	15/15

Table 12: Repeat/Fold Stability of the Local-Placement Margin

Figure 3: MAE stratified by quartiles of absolute true arousal innovation. The state model is difficult to beat for near-zero changes, whereas correctly placed audio provides the largest benefit in Q4. Wrong-time audio retains broad song information but is consistently weaker than the correct future for large changes.

Emotion Recognition

Emotion recognition uses language, voice, face, body movement, and physiological signals to infer emotional categories or dimensions such as valence and arousal.

See all 129 papers

Predicting Music Valence and Arousal via Textual and Multimodal In-Context Learning in Large Language Models
Oct 22, 2026Chi-En Dai, Chih-Pin Tan, Yi-Hsuan YangValence-Arousal Modeling Emotion Recognition
Graduate Institute of Communication Engineering, National Taiwan University, Taiwan
MeloTune: On-Device Arousal Learning and Peer-to-Peer Mood Coupling for Proactive Music Curation
Apr 12, 2026Hongwei XuMusic Recommendation Recommender Systems
Aligning MusicLLM with Emotion using Instruction Tuning and Feedback-Driven Alignment
Jun 23, 2026Takuya Hasumi, Welly NaptaliLLM Alignment AI Alignment
LY Corporation

Oct 22, 2026J/K move · Enter open · S save

Predicting Music Valence and Arousal via Textual and Multimodal In-Context Learning in Large Language Models

Chi-En Dai, Chih-Pin Tan, Yi-Hsuan Yang

Graduate Institute of Communication Engineering, National Taiwan University, Taiwan

Valence-Arousal Modeling Emotion Recognition Multimodal ICL Multimodal Large Language Models

Open paper AskPDF arXiv

While arousal can be reliably predicted from musical audio, emotional valence remains difficult to model due to its subjective and context-dependent nature. In this work, we investigate the potential of in-context learning (ICL) with large language models to advance continuous valence and arousal prediction. Rather than predicting emotion scores directly, our framework leverages contextual demonstrations from an auxiliary dataset to refine the initial predictions produced by a small, task-specific base model. We systematically evaluate a text-based ICL variant using lyrics and generated audio captions, alongside a multimodal variant that incorporates raw audio signals. Using the entire MER60 dataset strictly as a held-out test set, our experiments demonstrate substantial performance gains, particularly for valence. Text-based ICL improves the valence

R^2

from 0.110 to 0.427 with 80 demonstration shots, while the multimodal variant achieves 0.346 with only 10 shots. These findings suggest that training-free ICL offers an effective inference-time strategy for current language models to predict continuous music emotion.

Abstract

Emotion Recognition

Explore similar work

Predicting Music Valence and Arousal via Textual and Multimodal In-Context Learning in Large Language Models

MeloTune: On-Device Arousal Learning and Peer-to-Peer Mood Coupling for Proactive Music Curation

Aligning MusicLLM with Emotion using Instruction Tuning and Feedback-Driven Alignment