cs.CVNov 18, 2026

Gaspard: Expressive unified speech and gesture synthesis with continuous autoregressive flow-matching and decoupled classifier-free guidance

Authors: Téo Guichoux, Théodor Lemerle, Catherine Pelachaud, Nicolas Obin

Organizations: ISIR · IRCAM · CNRS Sorbonne University, Paris, France

Abstract

Human communication involves highly coordinated multimodal signals such as speech and co-speech gestures. Existing approaches for generating both modalities commonly follow a cascaded pipeline, where speech is generated first and subsequently used to drive gesture generation; speech generation is therefore independent of gestures. We introduce Gaspard, a unified speech--gesture generation model based on continuous autoregressive generation with next-step flow-matching predictions, and trained on Seamless Interaction, a large-scale multimodal dataset. The heterogeneous temporal properties of speech and gesture motivate modality-specific prediction heads and decoupled classifier-free guidance over the two generation streams. This guidance rebalances the influence of speech and gesture at inference time, improving expressivity while mitigating dynamics collapse in continuous autoregressive gesture generation. Objective and perceptual evaluations show that Gaspard is a strong gesture generation model against unimodal baselines and that it produces highly synchronous speech and gestures compared to a matched cascaded system.

Figures & tables

Explore similar work

CardsList
  1. GestureFAR: Streaming Co-Speech Gesture Generation with Flow Autoregression

    Sep 18, 2026Pinxin Liu, Haiyang Liu, Jiahao Luo +3Co-Speech Gesture GenerationFlow-Based Generative Modeling

  2. Full-Duplex Interaction with Speech, Expressions and Gestures

    Oct 15, 2026Siyin Wang, Wenyi Yu, Terumi Chiba +5Facial Expression SynthesisMultimodal Generation

  3. SemConFlow: Semantic Grounding of Holistic Co-Speech Gesture Generation with Contrastive Flow-Matching

    Mar 27, 2026Lanmiao Liu, Esam Ghaleb, Aslı Özyürek +1Multimodal GroundingFlow Matching