Evaluating AI Musical Creativity: Towards a Quantitative Multi-Dimensional Framework Based on Value, Novelty, and Surprise
Organizations: Department of Arts and Sciences, University College London, Malet Place, London, WC1E 7JG, United Kingdom. · Department of Computer Science and Engineering, University of Bologna, Viale del Risorgimento, 2, Bologna, 40136, Italy. · Centre for Artificial Intelligence, Department of Computer Science, University College London, 66-72 Gower Street, London, WC1E 6EA, United Kingdom.
Abstract
Despite the increasing sophistication of generative Artificial Intelligence (AI), research on the evaluation of AI-generated musical creativity remains limited. To address this gap, this paper proposes a quantitative framework for the formalization and evaluation of AI musical creativity. Building on Margaret Boden's theory, which conceptualizes creativity in terms of value, novelty, and surprise, the framework adapts and operationalizes these three dimensions for the musical domain. On the hypothesis that certain aspects of musical creativity can be assessed with a degree of objectivity, a domain-specific set of quantitative metrics for evaluating the creativity of AI-generated music is introduced. Rather than reducing musical creativity to a single unifying measure, this article proposes an ensemble of complementary metrics that capture its different dimensions, recognizing the complexity of musical artifacts and the difficulty of defining meaningful multi-objective compositional criteria. The framework has two implications. First, it gives researchers and developers a principled methodology and quantitative tools for empirically evaluating creative outputs, enabling more informed model development and reducing unnecessary computational costs. Second, it lays the groundwork for future generative models to optimize directly for creativity by adopting these metrics as learning objectives.
Figures & tables
| Component Configuration | Interpretation and Effect on M-Creativity |
|---|---|
| All components above norm ( , , ) | Represents ideal creativity. The musical output is valuable, original, and surprising, typically perceived as highly creative and impactful. |
| One component far above norm (e.g., , while others ) | A very strong component may compensate for average or weak others. Creativity can still be perceived as high, especially if the dominant trait is M-value. |
| One component below norm (e.g., , others ) | Creativity may be reduced despite novelty and surprise. The work might be interesting but aesthetically weak, dissonant, or incoherent. |
| Two components below norm (e.g., , ) | If only M-value is above norm, output may be technically sound but uninspired. Creativity is typically below average. |
| All components below norm ( , , ) | Represents a failure of creativity. The result is likely predictable, unoriginal, and musically unengaging. |
| Balanced trade-off or synergy (e.g., , , ) | Moderate or near-norm value in one component can be offset by strong performance in others. Creative synergy emerges when two components jointly elevate the perception of novelty and impact. |
| Component | Description |
|---|---|
| Musical Structure ( Ockelford, 2009 ; Scherer and Zentner, 2001 ) | Formal elements such as motifs, harmony, phrasing, and chord progressions shape emotional arcs and guide listener expectations. The I–V–vi–IV progression is a widely used and emotionally impactful example. |
| Neuropsychobiological Processing ( Peretz et al., 2004 ) | Music perception is rooted in how the brain processes sound. Neural circuits involved in pitch and rhythm overlap with those used in language. Responses vary based on experience, emotion, and training ( Sarasso et al., 2019 ) . |
| Human Musical Development ( Welch, 2005 ; Malloch, 1999 ) | Musical understanding arises from biological predisposition and cultural exposure. Even without training, listeners acquire intuitive knowledge of musical norms. |
| Sociocultural Context ( Himonides and Welch, 2005 ; Durrant, 2017 ) | Cultural backgrounds shape what is valued in music. Different genres emphasize different features (e.g., timbre in jazz, rhythm in rap), forming genre-specific aesthetic expectations. |
| Listener Characteristics ( Sloboda and O’Neill, 2001 ) | Personal factors such as emotion, personality, and memory shape perception. The same piece may be valued differently depending on the listener’s context. |
| Production of Acoustic Signals ( Sundberg, 1987 ; Juslin, 2001 ; Gabrielsson, 2003 ) | Vocal performance is shaped by control of breath, vocal fold tension, and resonance, which affect tone and expressiveness. |
| Metric | Formula | Interpretation |
|---|---|---|
| M-value | ||
| Mean Emotional Distance | Average emotional intensity over time | |
| Emotional Variability | Expressivity: variation in emotional strength | |
| Emotional Volatility | Emotional fluctuation over time | |
| Emotional Inflection Count | Count of sign changes in or | Complexity of emotional arc |
| M-surprise | ||
Music Generation Evaluation
Music Generation Evaluation measures the quality, musical structure, prompt alignment, controllability, and robustness of generated music.
See all 43 papers