cs.SDMar 2, 2026

ViTex: Visual Texture Control for Multi-Track Symbolic Music Generation via Discrete Diffusion Models

Authors: Xiaoyu Yi, Qi He, Gus Xia, Ziyu Wang

Organizations: School of EECS, Peking University · Music X Lab, MBZUAI · Courant Institute of Mathematical Sciences, New York University

Abstract

In automatic music generation, a central challenge is to design controls that enable meaningful human-machine interaction. Existing systems often rely on extrinsic inputs such as text prompts or metadata, which do not allow humans to directly shape the composition. While prior work has explored intrinsic controls such as chords or hierarchical structure, these approaches mainly address piano or vocal-accompaniment settings, leaving multitrack symbolic music largely underexplored. We identify instrumentation, the choice of instruments and their roles, as a natural dimension of control in multi-track composition, and propose ViTex, a visual representation of instrumental texture. In ViTex, color encodes instrument choice, spatial position represents pitch and time, and stroke properties capture local textures. Building on this representation, we develop a discrete diffusion model conditioned on ViTex and chord progressions to generate 8-measure multi-track symbolic music, enabling explicit texture-level control while maintaining strong unconditional generation quality. The demo page and code are avaliable at https://vitex2025.github.io/.

Figures & tables

Explore similar work

CardsList
  1. Diff-Symbo: Text-Controlled Long-Duration Symbolic Music Generation Using Autoregressive Latent Diffusion Model

    Aug 5, 2026Zhiwei Lin, Jun Chen, Boshi Tang +5Text-to-Music GenerationLatent Diffusion Models

  2. MusicLayout: Explicit Structural Planning for Controllable Text-to-Music Generation

    Aug 10, 2026Shuyu Li, Kejun Zhang, Jiahe Lei +5Text-to-Music Generation

  3. Getting Motif-ated: Controllable AI Compositions from Injected Motif Prompts

    Sep 26, 2026Chao Peter Yang, Cynthia Rudin, Yue Jiang +2Controllable Music GenerationSymbolic Music Generation