Known Music, Unknown Response: Leakage-Audited Dynamic Arousal Forecasting With Scheduled Future Music
Authors: Bin Wen, Dai-Qiao Zhang, Tien-Ping Tan
Organizations: School of Computer Sciences, Universiti Sains Malaysia, 11800, Pulau Pinang, Malaysia · School of Mathematical Sciences, Universiti Sains Malaysia, 11800, Pulau Pinang, Malaysia
Dynamic music emotion recognition usually estimates concurrent or lag-aligned labels from audio that has already been observed. We instead forecast the change in consensus arousal when the upcoming music is scheduled but all later affect labels remain hidden. Evaluation holds out complete songs, encodes each eligible 4-s chunk independently, predicts arousal innovation, and compares the correct future with displaced segments from the same song. On PMEmo and a 1,494-song DEAM evaluation pool excluded from engineering development, a complete past/future MuQ representation raises concordance correlation coefficient (CCC) by 0.389--0.507 over recent affect and remains 0.268--0.382 above a calibrated nonlinear state model. A future-only trajectory contains no past or current audio and still exceeds that state model by 0.150--0.249. The earliest legal future chunk, which ends 5.5--11 s before the target, also remains informative. Correct placement exceeds an overlapping 5-s shift by 0.100--0.209 and the acoustic-nearest eligible same-song segment by 0.078--0.105, with positive song-cluster lower bounds. These results show that both scheduled content and its within-song position carry forecast information under the stated observation rules. The study provides a leakage-controlled reference for known-future music forecasting without treating predictive substitutions as causal interventions.
Figures & tables
Figure 1: Forecast setting. Affect labels after origin t are hidden although the scheduled waveform is known. A valid chunk lies completely inside the permitted future interval. Negative controls preserve the current state and song identity while replacing only the scheduled segment.
Task
Target
Audio available
Future response labels
Main identification question
Static MER
Song/excerpt emotion
Complete target excerpt
Not applicable
What emotion does the excerpt express?
Dynamic MER
Concurrent/lagged emotion
Target-era acoustic context
Often used for training alignment
What emotion is expressed now?
Past-only affect forecast
Future level/change
History through origin
Hidden
What follows from state and observed history?
This work
Future arousal innovation
History plus scheduled future support
Hidden after origin
Does the known future, and its placement, add forecast information?
Causal recommendation
Individual outcome under an action
Candidate/action description
Counterfactual unobserved
Which action changes an individual’s outcome?
Table 1: Task Taxonomy and Information Available at Prediction Time
Term
Meaning
t
Forecast origin; labels are observed only through this time
H
Forecast horizon
δ
Response gap between legal music support and target
st
Affect history available at the origin
at,H
Scheduled waveform interval permitted at prediction time
Δyt,H
Change from the current to the later affect label
Table 2: Notation and Terms Used in the Forecast Setting
Threat
Failure mode
Design response
Emotion inertia
Copy current level
Predict innovation; strong state controls
Track identity
Memorize song/style
Hold out complete songs
Encoder leakage
Attend beyond support
Independent complete 4-s chunks
Coarse content
Any segment works
Same-song local and nearest controls
Window dependence
Narrow uncertainty
Song-cluster bootstrap
Table 3: Primary Threats and Experimental Responses
Data
Horizon
Songs
Windows
Win./song
PMEmo
10 s
611
19,336
31.6
PMEmo
15 s
481
13,677
28.4
DEAM holdout
10 s
1,494
74,181
49.7
DEAM holdout
15 s
1,494
59,241
39.7
Table 4: Evaluation Data After Complete-Support Filtering
Component
Setting
Audio support
4-s chunks; 1-s stride
State ridge
Standardized; α=10
MuQ head
Train-fold PCA-256; ridge α=100
Outer/inner splits
3×5 / 4; song-disjoint
Response delay
2 s
Bootstrap
5,000 song-cluster draws
Table 5: Fixed Model and Evaluation Settings
Song
Cutoff
Mean cos.
Min cos.
Rel. L2
1
19.22 s
0.923
0.366
0.350
10
19.22 s
0.890
0.307
0.445
100
24.03 s
0.899
0.294
0.429
Table 6: Suffix-Perturbation Audit of Full-Song MuQ Encoding
Data
H
State
MuQ mean
MuQ relation
Relation–state [adj. 97.5% CI]
PMEmo
10 s
0.028
0.315
0.417
+0.389 [0.342, 0.434]
PMEmo
15 s
0.006
0.335
0.453
+0.446 [0.370, 0.514]
DEAM holdout
10 s
0.041
0.317
0.458
+0.416 [0.391, 0.442]
DEAM holdout
15 s
0.058
0.402
0.564
+0.507 [0.475, 0.536]
Table 7: Protocol-Matched Forecasts of Arousal Innovation
Data
H
State ridge
HGB raw/cal.
Past only
Future only
Full relation
Future–past [LB]
PMEmo
10 s
0.028
0.075/0.159
0.158
0.310
0.427
+0.151 [0.112]
PMEmo
15 s
0.006
0.081/0.127
0.163
0.323
0.462
+0.160 [0.093]
DEAM holdout
10 s
0.041
0.094/0.161
0.128
0.339
0.465
+0.211 [0.190]
DEAM holdout
15 s
0.058
0.129/0.180
0.168
0.429
0.562
+0.261 [0.236]
Table 8: Strong-State and Past/Future Attribution. “HGB raw” is the direct-innovation output used to form residual targets; “cal.” applies an inner-fold location–scale map only for the CCC state comparison.
Data
H
State HGB (cal.)
Earliest 4 s
Latest 4 s
Full-support mean
Early–state [LB]
PMEmo
10 s
0.159
0.243
0.285
0.319
+0.084 [0.036]
PMEmo
15 s
0.127
0.205
0.306
0.339
+0.078 [0.026]
DEAM holdout
10 s
0.161
0.241
0.290
0.338
+0.080 [0.046]
DEAM holdout
15 s
0.180
0.224
0.352
0.422
+0.044 [0.005]
Table 9: Candidate-Only Localization Within the Legal Future Support (CCC)
MAE
RMSE
Data
H
HGB raw
Future only
Full relation
HGB raw
Future only
Full relation
PMEmo
10 s
0.02531
0.02525
0.02438
0.03681
0.03502
0.03312
PMEmo
15 s
0.03174
0.03212
0.03032
0.04545
0.04382
0.04052
DEAM holdout
10 s
0.05562
0.05497
0.05260
0.09283
0.08497
0.08004
DEAM holdout
15 s
0.07713
0.07383
0.06941
0.12250
0.10816
0.10019
Table 10: Absolute-Error Audit for the Strong-State Attribution Pipeline
Figure 2: Main empirical hierarchy. (a) Candidate-only future audio is well above state, and the complete past/future relation is higher still. (b) Correct placement remains superior after the future is replaced by a strongly overlapping local shift or by the acoustic-nearest eligible same-song segment.
Data
H
Local shift
Overlap
Local margin [LB]
NN cosine
NN margin [LB]
PMEmo
10 s
4.86 s
0.392
+0.135 [0.108]
0.947
+0.105 [0.078]
PMEmo
15 s
4.83 s
0.629
+0.100 [0.071]
0.979
+0.086 [0.063]
DEAM holdout
10 s
5.00 s
0.375
+0.209 [0.185]
0.939
+0.087 [0.074]
DEAM holdout
15 s
5.00 s
0.615
+0.153 [0.131]
0.973
+0.078 [0.067]
Table 11: Hard Same-Song Temporal Controls
Data
H
Repeat SD
Fold range
Positive
PMEmo
10 s
0.0043
[0.086, 0.192]
15/15
PMEmo
15 s
0.0035
[0.075, 0.144]
15/15
DEAM
10 s
0.0026
[0.152, 0.241]
15/15
DEAM
15 s
0.0014
[0.089, 0.175]
15/15
Table 12: Repeat/Fold Stability of the Local-Placement Margin
Figure 3: MAE stratified by quartiles of absolute true arousal innovation. The state model is difficult to beat for near-zero changes, whereas correctly placed audio provides the largest benefit in Q4. Wrong-time audio retains broad song information but is consistently weaker than the correct future for large changes.
Emotion recognition uses language, voice, face, body movement, and physiological signals to infer emotional categories or dimensions such as valence and arousal.
While arousal can be reliably predicted from musical audio, emotional valence remains difficult to model due to its subjective and context-dependent nature. In this work, we investigate the potential of in-context learning (ICL) with large language models to advance continuous valence and arousal prediction. Rather than predicting emotion scores directly, our framework leverages contextual demonstrations from an auxiliary dataset to refine the initial predictions produced by a small, task-specific base model. We systematically evaluate a text-based ICL variant using lyrics and generated audio captions, alongside a multimodal variant that incorporates raw audio signals. Using the entire MER60 dataset strictly as a held-out test set, our experiments demonstrate substantial performance gains, particularly for valence. Text-based ICL improves the valence R2 from 0.110 to 0.427 with 80 demonstration shots, while the multimodal variant achieves 0.346 with only 10 shots. These findings suggest that training-free ICL offers an effective inference-time strategy for current language models to predict continuous music emotion.
Chi-En Dai, Chih-Pin Tan, Yi-Hsuan Yang
Graduate Institute of Communication Engineering, National Taiwan University, Taiwan
MeloTune is an iPhone-deployed music agent that instantiates the Mesh Memory Protocol (MMP) and Symbolic-Vector Attention Fusion (SVAF) as a production system for affect-aware music curation with peer-to-peer mood coupling. Each device runs two closed-form continuous-time (CfC) networks: a private listener-level CfC that predicts a short-horizon affective trajectory on Russell's circumplex and drives proactive curation, and a shared mesh-runtime CfC at MMP Layer 6 that integrates Cognitive Memory Blocks (CMBs) from co-listening peers. CfC hidden states never cross the wire; only structured CMBs do. A Personal Arousal Function (PAF) replaces the standard linear mapping from audio intensity to psychological arousal with a per-listener learned adjustment, trained from behavioral signals (skip, completion, favorite, volume) and from drift between user-declared mood and machine inference. The same track receives different arousal predictions for different listeners. The model (94,552 parameters) achieves trajectory MAE 0.414, pattern accuracy 96.6%, and intent accuracy 69.4% on held-out validation. PAF evidence from a live deployment session (46 observations across 11 genres) demonstrates that the learning loop operates end-to-end, with pop reaching full confidence after 22 observations. All inference runs on-device via CoreML. To our knowledge, this is the first production deployment of MMP/SVAF on consumer mobile hardware. The accompanying SDK (sym-swift v0.3.78, SYMCore v0.3.7) enforces strict protocol conformance. Music is the case study; the substrate is the contribution.
This paper investigates whether music large language models (MusicLLMs) can be aligned for emotion regression. While MusicLLMs have shown strong performance in music information retrieval tasks, their ability to predict arousal and valence scores remains limited, since emotion regression has not been an explicit training objective. To examine whether MusicLLMs can be aligned with emotion, we train MusicLLMs on emotion regression and compare two strategies: instruction tuning and feedback-driven alignment. Our experiments show that task-aware instruction tuning enables MusicLLMs to predict emotion levels to some extent, although the accuracy remains limited. Applying feedback-driven alignment with a verifiable numerical reward substantially improves performance on both arousal and valence over instruction tuning alone. We further show that our approach improves emotion regression performance while maintaining MusicQA capability.