cs.SDNov 9, 2026

BARS: Beat-Adaptive Rap Synthesis via Flow Matching

Authors: Taemin Cho, Hector Martel, Enyan Koh

Organizations: BandLab Technologies, Singapore

Abstract

We present BARS (Beat-Adaptive Rap Synthesis), a non-autoregressive rapping voice generation model based on Flow Matching, conditioned on a beat track (instrumental backing), lyrics, and a short reference voice for zero-shot timbre control. Unlike prior autoregressive rap synthesis, where rap pace and output duration are strictly determined by the beat, BARS enables independent control over the rap tempo, allowing flexible-length generation without explicit duration prediction or phoneme-level alignment. We encode lyrics using a pretrained ByT5 encoder with a lightweight Transformer adapter trained jointly with the model, without relying on speech-pretrained text encoders, enabling natural handling of non-standard orthography (e.g., slang, abbreviations) and numerals common in rap lyrics. To address alignment instabilities, such as token skipping or repetition, under strong beat conditioning, we propose Asymmetric Scaled RoPE for cross-attention, which introduces a positional structure to facilitate sequential lyric-audio alignment learning. Furthermore, the model supports partial re-generation via span masking, enabling lyric-level editing while preserving the surrounding vocal style. Experiments demonstrate that the proposed model achieves high lyric fidelity, natural rap flow that is rhythmically coherent with the beat, and robust tempo control across a wide range of rapping styles and speeds.

Figures & tables

Explore similar work

CardsList
  1. DiffSynth-Music: Audio-Conditioned KV-Cache Adapters for Controllable Music Generation

    Sep 11, 2026Zhongjie Duan, Shengchuan Gao, Hong Zhang +1Controllable Music GenerationAudio Diffusion Models

  2. VocalRender: Score-Native Singing Voice Synthesis for Real-World Composition

    Jul 30, 2026Yukun Chen, Tianrui Wang, Zhaoxi Mu +2Singing Voice SynthesisAudio Diffusion Models