DJustify: Reason-and-Verify DJ Planning for Pop Transitions and Story-Driven Sets
Authors: Wei-Han Hsu, Li-Jie Lin, Po-Hsuan Lai, Po-Hsiang Huang, Jeng-Yue Liu, Li Su, Yi-Hsuan Yang
Organizations: Data Science Degree Program, National Taiwan University and Academia Sinica, Taipei, Taiwan · Institute of Information Science, Academia Sinica, Taipei, Taiwan · National Taiwan University, Taipei · Taiwan AI Labs, Taipei · Carnegie Mellon University, USA · AI Center of Research Excellence (AI-CoRE), National Taiwan University, Taipei, Taiwan
Automatic DJ systems mostly target electronic dance music, whose regular structure makes rule-based mixing relatively easier. Vocal-rich popular music is harder: which track to play next, where to cue it, and which transition to use are tied together, so fixed rules handle them poorly. We present DJustify, a pop AI-DJ that puts an LLM in a reason--propose--verify--repair loop and lets it choose the next track, cue points, and mixing technique together. The LLM samples several transition plans; an external acoustic checker then scores the rendered audio and triggers repair when a check fails. Sampling keeps several candidate transitions per track pair instead of one fixed mix. In blind listening tests our transitions beat Mosaikbox, a reimplemented crossfade, and a cue baseline that reuses the crossfade's cues; that last comparison shows most of the margin comes from cue placement. The same planner also picks and orders a set along an emotion arc: this improves story coherence over a no-story ablation, and within a set story and transition ratings show no measurable gap from a human DJ. The checker mainly adds reliability, not higher average quality: it repairs rare but audible errors such as cut vocals, while the planner stays non-deterministic.
Figures & tables
Figure 1: AI-DJ overview. Top: a planner jointly selects the next track, cue points, and technique; the rendered transition is scored by acoustic checks (vocal/energy/structure/pairing) and failures drive repair. Bottom: a story prompt guides track selection and ordering into a valence–arousal arc, invoking the same loop to build a set.
Gate
Detected failure → corrective action
Vocal
Cut mid-phrase → shift the cue to the phrase end
Vocal
Vocals collide in the overlap → shorten it or cut
Energy
Silent gap or energy hole → adjust length or technique
Structure
Early cut in A or late entry in B→ move the cue
Structure
Entry stays in B ’s intro → re-cue to the verse or chorus
Pairing
Valence–arousal mismatch → re-select track B
Table 1: The checker: rendered-audio gates and corrective actions.
Figure 2: Transition quality in Test1 ( N=103 ). Bars show mean ratings; error bars are 95% bootstrap confidence intervals. Ours significantly outperforms Mosaikbox, Bittner’17/Spotify, and Ours (Bittner cues); replacing our selected cues with Bittner’17 cues reduces the mean score from 5.6 to 4.8.
Figure 3: Output diversity over 20 fixed track pairs: for each pair we sample the planner 10 times ( T=0.6 ) and count distinct transitions (differing in technique, cut, or entry). The four settings are proposals without/with reasoning and rendered outputs with the checker (reasoning off) and the full system (Ours). A deterministic rule gives one (dashed line); reasoning raises diversity and repair adds a little.
Variant
Reasoning
Checker
Renderability
Ours − C − R
–
–
91%
Ours − C
✓
–
98%
Ours
✓
✓
99%
Table 2: Renderability of planner variants, 100 sampled seeds.
Figure 4: Story-driven set quality (Test2, N=99 ), mean story and transition ratings. Story is rated above No-story on story ( p=.002 ) and its transitions beat the crossfade ( p=.043 , uncorrected).
Mixing requires not only applying processing but also assessing the resulting mix. Audio-language models can provide conversa- tional mixing advice, but existing evaluations do not separately test whether they can judge specific mixing issues. Since professional supervision for such judgments is scarce, we construct a curriculum of 11 vocal-mix issues at multiple severity levels and use it to train MixJudge, a language-queryable audio critic. MixJudge shows its ability to retrieve mixing issues, distinguish opposite issues, and order the severity of an issue. Moreover, its judgments transfer to menu selection: given a set of processing candidates, MixJudge often favors versions acoustically closer to a held-out professional version, demonstrating how explicit mixing judgment can support comparison and downstream mixing decisions.
Kuan-Hsun Ho, Yuan-Chiao Cheng, Yen-Tung Yeh +1
Independent Researcher · Music Informatics Group, Georgia Institute of Technology · Music and AI Lab, National Taiwan University +1
We introduce StepAudio 3 Music, a large-scale, long-form music generation model that supports explicit musical planning and open-domain text-controlled generation. The StepAudio Music Tokenizer represents audio as a 50-Hz stream from a 65536-entry single codebook, using semantically informed self-supervised and multi-task training to preserve musical structure and reconstruction-relevant information. A flow-matching diffusion Transformer (DiT) predicts continuous StepAudio VAE latents, which our VAE decoder converts into 48-kHz audio. This discrete-continuous design is guided by comparisons of single-codebook VQ, Semantic and Acoustic RVQ, and different DiT configurations. For explicit planning, a Mixture-of-Experts autoregressive model uses ABC notation to produce an intermediate arrangement plan (ABC-CoT) before predicting music tokens, making harmony, rhythm, and melodic structure part of the generation context. A progressive training curriculum and supervised fine-tuning support song and instrumental generation, accompaniment generation from dry vocals, and cover-song synthesis for up to 5 minutes and 30 seconds. With reinforcement learning via direct preference optimization (DPO), the final model achieves the highest AudioBox Content Enjoyment, Content Usefulness, and Production Quality scores and the highest MuQ-MuLan similarity among the evaluated systems, with competitive SongBench results. On the preliminary Artificial Analysis Music Arena Vocals leaderboard, it obtains a Quality Elo of 1105, behind only Suno V5.5 and Mureka and ahead of Suno V5, MiniMax models, and other systems. Audio demonstrations are available at https://stepaudiollm.github.io/step-audio-3-music.
Chengli Feng, Zhiyue Wu, Jiahao Song +10
StepFun · The Chinese University of Hong Kong · University of California San Diego +1
Music popularity prediction has attracted growing research interest, with relevance to artists, platforms, and recommendation systems. However, the explosive rise of AI-generated music platforms has created an entirely new and largely unexplored landscape, where a surge of songs is produced and consumed daily without the traditional markers of artist reputation or label backing. Key, yet unexplored in this pursuit is aesthetic quality. We propose APEX, the first large-scale multi-task learning framework for AI-generated music, trained on over 211k songs (10k hours of audio) from Suno and Udio, that jointly predicts engagement-based popularity signals - streams and likes scores - alongside five perceptual aesthetic quality dimensions from frozen audio embeddings extracted from MERT, a self-supervised music understanding model. Aesthetic quality and popularity capture complementary aspects of music that together prove valuable: in an out-of-distribution evaluation on the Music Arena dataset, comprising pairwise human preference battles across eleven generative music systems unseen during training, including aesthetic features consistently improves preference prediction, demonstrating strong generalisation of the learned representations across generative architectures.
Jaavid Aktar Husain, Dorien Herremans
AMAAI Lab, Singapore University of Technology and Design