Controllable Music Generation

Follow this topic Ask about this topic

Momentum

10 papers in the last four weeks, up 43% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 66

All topics

Dance to Music Generation leveraging Pre-training with Unpaired data and Contrastive Alignment
Jul 12, 2026Ryota Kimura, Sangheon Park, Natalia Polouliakh +1Controllable Music Generation Cross-Modal Alignment
Sony Computer Science Laboratories · Keio University · Georgia Institute of Technology
A Quantized Native Runtime for On-Device Semantic Audio Generation
Jul 9, 2026Matteo Spanio, Antonio RodàText-to-Audio Generation Controllable Music Generation
Centro di Sonologia Computazionale (CSC) · Dept. of Information Engineering, University of Padova
Designing Maintainable Hybrid Generative Systems: A Quantum-Inspired Approach to Automated Music Harmony Generation
Jul 7, 2026Josef PavlicekControllable Music Generation Symbolic Music Generation
Czech Technical University in Prague / Faculty of Information technology / Department of Software Engineering Prague/ Czech Republic
A Text-Steerable Instrument for Sketching Procedural Soundscapes via Language Models
Jul 1, 2026Prabal GuptaText-to-Music Generation Controllable Music Generation
Rama Labs, Kitchener, Canada · Rama Labs Kitchener, Ontario, Canada
LeVo 2: Stable and Melodious Song Generation via Hierarchical Representation Modeling and Progressive Post-Training
Jun 29, 2026Shun Lei, Huaicheng Zhang, Dapeng Wu +8Text-to-Music Generation Controllable Music Generation
Tsinghua-CUHK Joint Research Center for Media Sciences, Technologies and Systems, Shenzhen International Graduate School, Tsinghua University, Shenzhen, China · Wuhan University, Wuhan, China · Hong Kong Polytechnic University, Hong Kong, China +1
Real-Time Interactive Music Generation via Data-Free Streaming Consistency Distillation
Jun 23, 2026Baisen Wang, Chenxi Bao, Qisong HanText-to-Music Generation Controllable Music Generation
Work done in ZhuoLab. · DynamiX · Cardiff University
Libretto: Giving LLM Agents a Sense of Musical Structure
Jun 21, 2026Yichen XuControllable Music Generation Symbolic Music Generation
University of California, Berkeley
Attractive and Repulsive Pattern Control in Sequence Generation
Jun 19, 2026Francois PachetMarkov Models Controllable Music Generation
Closing the Loop: PID Feedback Control for Interpretable Activation Steering in Symbolic Music Generation
Jun 17, 2026Ioannis Prokopiou, Pantelis Vikatos, Maximos Kaliakatsos-Papakostas +2Transformer Interpretability Controllable Music Generation
Athens University of Economics and Business, Athens, Greece · Orfium Research, Athens, Greece · Hellenic Mediterranean University, Chania, Greece +2
AdaTT: Text-Guided Instrument Timbre Transfer with Target-Adaptive Structural Control
Jun 14, 2026Dabin Kim, Junwon Lee, Juhan NamControllable Music Generation Audio Diffusion Models
Graduate School of Cultural Technology and 2Graduate School of AI, KAIST, South Korea
AudioX-Turbo: A Unified Framework for Efficient Anything-to-Audio Generation
Jun 10, 2026Zeyue Tian, Lei Ke, Zhaoyang Liu +8Text-to-Audio Generation Diffusion Model Distillation
The Hong Kong University of Science and Technology · Noiz AI · Tsinghua University +1
PianoKontext: Expressive Performance Rendering from Deadpan Context
Jun 10, 2026Dmitrii GavrilevControllable Music Generation
Applied AI Institute
Towards Unified Song Generation and Singing Voice Conversion with Accompaniment Co-Generation
Jun 5, 2026Ziyu Zhang, Chunyu Qiang, Xiaopeng Wang +10Controllable Music Generation Music Generation
Kuaishou Technology, China · Beijing Institute of Technology, China · Institute of Automation, Chinese Academy of Sciences, China
Exploring LLMs for South Asian Music Understanding and Generation
Jun 3, 2026Faria Binte Kader, Mohtasim Hadi Rafi, Shah Wasif Sajjad +1Music Generation Evaluation LLM Evaluation
University of Central Florida · Florida, USA · Auburn University +1
A Minimalist Brain-Computer Musical Interface for Real-Time Emotion-Driven Sonification: System Design and Preliminary Evaluation
May 31, 2026Pablo A. Monroy-D'Croz, Rafael Ramirez-Melendez, Julian Cespedes-GuevaraBrain-Computer Interfaces Controllable Music Generation
Universitat Pompeu Fabra, Barcelona, Spain · Universidad Icesi, Cali, Colombia
SegTune: Structured and Fine-Grained Control for Song Generation
May 31, 2026Yuejiao Wang, Zihao Ji, Pengfei Cai +6Text-to-Music Generation Language Model-Based Control
Kling Team, Kuaishou Technology · University of Science and Technology of China, Hefei, Anhui, China · Peking University, Beijing, China
Latent Space Disentanglement via Activation Steering for Interpretable Attribute Control in Symbolic Music Generation
May 29, 2026Ioannis Prokopiou, Pantelis Vikatos, Maximos Kaliakatsos-Papakostas +2Disentangled Representation Learning Controllable Music Generation
Athens University of Economics and Business Orfium Athens, Greece · Innovation Lab Orfium Athens, Greece · Department of Music Technology and Acoustics Hellenic Mediterranean University Rethymno, Greece +2
AnchorSteer: Self-Discovered Concept Injection for Structure-Preserving Music Editing
May 29, 2026Chih-Heng Chang, Keng-Seng Ho, Chih-Yu Tsai +3Audio Editing Controllable Music Generation
National Taiwan University Taipei, Taiwan
DEMON: Diffusion Engine for Musical Orchestrated Noise
May 27, 2026Ryan FosdickControllable Music Generation Audio Diffusion Models
Daydream
Live Music Diffusion Models: Efficient Fine-Tuning and Post-Training of Interactive Diffusion Music Generators
May 21, 2026Zachary Novack, Stephen Brade, Haven Kim +8Diffusion Model Alignment Controllable Music Generation
UC San Diego · MIT · Adobe
Break-the-Beat! Controllable MIDI-to-Drum Audio Synthesis
May 14, 2026Shuyang Cui, Zhi Zhong, Qiyu Wu +9Controllable Music Generation Music Generation
Sony Group Corporation · Sony AI
Text2Score: Generating Sheet Music From Textual Prompts
May 13, 2026Keshav Bhandari, Sungkyun Chang, Abhinaba Roy +4Text-to-Music Generation Music Generation Evaluation
Queen Mary University of London, UK · EmotionWave, Korea · Singapore University of Technology and Design, Singapore +1
Polyphonia: Zero-Shot Timbre Transfer in Polyphonic Music with Acoustic-Informed Attention Calibration
May 11, 2026Haowen Li, Tianxiang Li, Yi Yang +2Audio Editing Controllable Music Generation
School of Future Technology, South China University of Technology, Guangzhou, China
Remix the Timbre: Diffusion-Based Style Transfer Across Polyphonic Stems
May 10, 2026Leduo Chen, Junchuan Zhao, Shengchen LiDiffusion Transformer Controllable Music Generation
XJTLU
A Decomposed Retrieval-Edit-Rerank Framework for Chord Generation
May 8, 2026Qiqi He, Dichucheng Li, Xiaoheng Sun +1Controllable Music Generation Symbolic Music Generation
NetEase Cloud Music Shanghai, China · Individual Researcher Hong Kong, China
MindMelody: A Closed-Loop EEG-Driven System for Personalized Music Intervention
May 2, 2026Yimeng Zhang, Yueru Sun, Haoyu Gu +1Controllable Music Generation Music Generation
South China University of Technology, Guangzhou, China
SymphonyGen: 3D Hierarchical Orchestral Generation with Controllable Harmony Skeleton
Apr 28, 2026Xuzheng He, Nan Nan, Zhilin Wang +7Controllable Music Generation Symbolic Music Generation
Department of AI Music and Music Information Technology, Central Conservatory of Music · Frontier Institute of Science and Technology, and Interdisciplinary Research Center of Frontier Science and Technology, Xi’an Jiaotong University · University of Science and Technology of China +1
Huí Sù: Co-constructing a Dual Feedback Apparatus
Apr 28, 2026Yichen Wang, Charles Patrick MartinControllable Music Generation Music Generation
The Australian National University, Canberra, ACT, Australia
Aligning Language Models for Lyric-to-Melody Generation with Rule-Based Musical Constraints
Apr 20, 2026Hao Meng, Siyuan Zheng, Shuran Zhou +2Text-to-Music Generation LLM Alignment
Zuoyebang Education Technology, Beijing, China
Latent Fourier Transform
Apr 20, 2026Mason Wang, Cheng-Zhi Anna HuangControllable Music Generation Frequency-Domain Feature Learning
CSAIL

Jul 12, 2026, cs.SDJ/K move · Enter open · S save

Dance to Music Generation leveraging Pre-training with Unpaired data and Contrastive Alignment

Ryota Kimura, Sangheon Park, Natalia Polouliakh, Taketo Akama

Sony Computer Science Laboratories · Keio University · Georgia Institute of Technology

Controllable Music Generation Cross-Modal Alignment

Open paper AskPDF arXiv

Dance-to-music generation is a promising task for applications such as choreography support and automatic accompaniment, where temporal coordination between body movement and sound is essential. In particular, using human joint positions as the motion representation is attractive because they explicitly capture body dynamics while being lightweight, privacy-preserving, and easy to integrate with motion capture and pose-estimation pipelines. A central challenge in this setting, however, is the scarcity of high-quality paired dance-music data, since collecting accurately synchronized pairs is costly and often constrained by copyright and performance rights. This makes it difficult to train end-to-end models solely from paired data. To address this issue, we propose a dance-conditioned music generation framework that efficiently exploits both unpaired and paired data. Our method combines pretrained unimodal encoders for motion and music, beat-guided contrastive pretraining to align their feature spaces, and a ControlNet-style conditioning module on top of a pretrained text-to-audio diffusion model. Experiments on AIST++ demonstrate that the proposed techniques improve both dance-music alignment and audio quality, as confirmed by quantitative and qualitative evaluations. Compared to a state-of-the-art method, our approach achieves superior dance alignment performance and competitive audio quality. Code is available at https://github.com/kmraven/AudioLDM-ControlNet .

Momentum

Related topics

Latest papers 66