cs.SDNov 18, 2026

Constructing Dynamic Conversations with Neural Back-channel Generation and Voice-line Scheduling

Authors: Yifang Pan, Chris Landreth, Karan Singh

Organizations: University of Toronto, Toronto, Canada · JALI Research, Toronto, Canada

Abstract

We present a novel approach to the problem of combining two voice-line channels of dyadic conversational audio into a unified conversation with controllable back-channeling and turn-taking dynamics. In voice performances for the game industry, the two tracks of voice-lines are typically recorded or synthesized in isolation. Our method takes two such unsynchronized tracks and uses a neural scheduler to sequence them into a single dynamic conversation, modifying both tracks by inserting or removing pauses and introducing voice overlap as appropriate. We further propose a back-channel predictor that is able to predict opportunity for back-channels for a given conversational turn's audio and content, which the scheduler can then incorporate. We evaluate our scheduler and predictor independently and in combination by: various quantitative comparison; a qualitative and perceptual comparison to prior art and naive baselines; a compelling set of results showing animator control over the diversity, extent and nature of back-channeling and turn dynamics.

Figures & tables

Explore similar work

CardsList
  1. Controlling Backchannels in Streamable Full-duplex Models

    Sep 24, 2026Maike Züfle, Peter Polák, Sefik Emre Eskimez +3Spoken Dialogue SystemsSpeech Language Models

  2. Learn2Chat: Rethinking Dyadic Talking Heads via Interaction-Modulated Monologic Priors

    Jul 11, 2026Zikai Huang, Siyue Chen, Xuemiao Xu +4Disentangled Representation LearningHuman Motion Generation