cs.CVJul 15, 2026

Bring Music The Horizon: Music-Driven 360^\circ Video Generation

Authors: Kai Hsu TsaiYong Wei FuHung I YangYu-Chih Chen

Organizations: Department of Computer Science, National Yang Ming Chiao Tung University, Hsinchu 300093, Taiwan

Abstract

Music visualization offers a powerful way to enhance listeners' understanding and experience of music by translating auditory signals into visual forms. However, most existing approaches either rely heavily on lyrics or generate flat, non-immersive videos similar to conventional music videos, which limits their ability to convey the emotional dynamics of music and provide an immersive listening experience. We propose Bring Music The Horizon, an emotion-aware pipeline for music-driven 360^\circ video generation. Given an input song, our work first estimates its emotional trajectory by predicting valence-arousal values at the level of every four bars. These values are then converted into emotion-aware visual guidance using EmotiCrafter, and these guidance vectors can be manipulated by the SEGA framework, which provides fine-grained semantic control for keyframe generation. Finally, image-to-video models are applied to the generated keyframes to synthesize temporally continuous 360^\circ videos for immersive music visualization. Our pipeline generates 360^\circ music visualization videos that reflect the emotional progression and temporal structure of the input song. We demonstrate its capability using songs from different genres and provide qualitative comparisons with From-Sound-To-Sight, a representative audio-to-visual generation baseline, on our project page at https://etoile-et-toi-mp3.github.io/BMTH_Project_Page/.

Explore similar work

CardsList
  1. VIBE: Video Instruction-aligned Background music gEneration

    Aug 31, 2026Aryan Vijay Bhosale, Vaibhavi Lokegaonkar, Vishnu Raj +5Music GenerationVibe Coding