Co-Speech Gesture Generation

The field studies how to generate gestures and other nonverbal behavior that fit a speaker’s words, prosody, emotion, and conversational context, for avatars and robots. Common approaches include diffusion and flow-matching models, causal autoregressive motion-token models for streaming, and semantic planning or retrieval that separates communicative meaning from rhythmic movement and personal style. Systems are evaluated on datasets such as BEAT2 and in human studies, with growing attention to real-time interaction, dyadic behavior, and physical robot constraints. A central challenge is that realistic motion does not necessarily convey the right meaning, respond to a partner, or align with speech, and automatic metrics do not always predict human judgments of these qualities.