Co-Speech Gesture Generation

Overview

The field studies how to generate gestures and other nonverbal behavior that fit a speaker’s words, prosody, emotion, and conversational context, for avatars and robots. Common approaches include diffusion and flow-matching models, causal autoregressive motion-token models for streaming, and semantic planning or retrieval that separates communicative meaning from rhythmic movement and personal style. Systems are evaluated on datasets such as BEAT2 and in human studies, with growing attention to real-time interaction, dyadic behavior, and physical robot constraints. A central challenge is that realistic motion does not necessarily convey the right meaning, respond to a partner, or align with speech, and automatic metrics do not always predict human judgments of these qualities.

Latest papers

CardsList

55 papers5 papers in the last four weeks, down 64% on the four weeks before. 0.0% of all new papers.

  1. CFGComposer: Compositional Audio-Text-to-Motion Generation via Copula Score Correction

    Nov 11, 2026Pengyang Shi, Jinlu Zhang, Chunze Lin +2Co-Speech Gesture GenerationText-to-Motion Generation

  2. FlashMotion: Ultra-Fast Long-Term 3D Motion Generation

    Nov 10, 2026Kaixing Yang, Xulong Tang, Ronghui Li +10Human Motion GenerationText-to-Motion Generation

  3. Rhythmic-Semantic Co-Speech Gestures for Emotional Dialogue in Robots

    Nov 10, 2026Christian Arzate Cruz, Edwin C. Montiel-Vázquez, Stefanos Gkikas +2Co-Speech Gesture GenerationSocial Robotics

  4. TalkFlow: Structured Latent Flow Matching for Speech Driven Holistic Motion Generation

    Oct 29, 2026Xinmu Wang, Fumi Honda, Xiang Gao +3Human Motion GenerationCo-Speech Gesture Generation

  5. SynerGest: Temporal-Spatial Synergistic Gesture Generation for Dyadic Conversations

    Oct 16, 2026Chengxiang Guo, Jian Wang, Lanmiao Liu +3Contrastive Flow MatchingCo-Speech Gesture Generation

  6. Full-Duplex Interaction with Speech, Expressions and Gestures

    Oct 15, 2026Siyin Wang, Wenyi Yu, Terumi Chiba +5Facial Expression SynthesisMultimodal Generation

  7. Kandinsky S2V 1.0: Audio-Driven Avatar Video Generation

    Oct 13, 2026Nikolay Semenov, Artem Riabinin, Sergey Bassenko +8Audio-Video GenerationAudio-Driven Facial Animation

  8. PAGE: Parts-Aware Guidance for Co-Speech Gesture Portrait Video Generation

    Oct 9, 2026Yabiao Wang, Jiafu Wu, Yicheng Xu +7Video GenerationCo-Speech Gesture Generation

  9. Perceptually Grounded and Semantics-Aware Evaluation for Holistic Co-Speech Gesture Generation

    Oct 8, 2026Nick Milkin, Lanmiao Liu, Esam Ghaleb +2Co-Speech Gesture GenerationGenerative AI Evaluation

  10. Talk, Render, Act: Integrating Social Gesture and Digital Face with Synchronized Speech for Conversational Humanoid Robot

    Oct 5, 2026Jin Jiang, Kun Li, Jiancong Ma +1Robot Motion GenerationCo-Speech Gesture Generation

  11. A Benchmark for Spatially Grounded Gesture Generation

    Oct 2, 2026Anna Deichler, Rishabh Dabral, Fethiye Irmak Dogan +2Co-Speech Gesture GenerationMultimodal Grounding

  12. ECHO-G: Embodied Co-speech Humanoid mOtion Generation

    Sep 30, 2026Yizhao Li, Pusen Gao, Ming Wang +3Text-to-Motion GenerationHuman Motion Generation

  13. GestAdapt: Workspace-Conditioned Co-Speech Gesture Generation for Humanoid Robots

    Sep 29, 2026Bosong Ding, Xianglin Zhang, Miao Xin +2Robot Motion GenerationStyle-Conditioned Motion Generation

  14. When Semantics Matter: Reliability-Aware Semantic-Rhythm Control for Co-Speech Gesture Generation

    Sep 29, 2026Zhirui Xing, Ming Meng, Kaige Li +2Co-Speech Gesture Generation

  15. DualTrack: Synchronized speech-gesture generation via symmetric coupling of pretrained priors

    Sep 29, 2026Yuanzhuo Hu, Zehan Liu, Xiaoyi Qin +1Human Motion GenerationCo-Speech Gesture Generation

  16. Do Motion Tokenizers for Co-Speech Gesture Generation Encode Gesture Semantics?

    Sep 28, 2026Varsha Suresh, Divij Jain, Jia Liu +2Co-Speech Gesture GenerationMotion Representation Learning

  17. SocialHumanoid: Towards Expressive Humanoid Behavior via One-Step Co-Speech Motion Generation

    Sep 27, 2026Chengqun Yang, Tengjie Zhu, Liang Xu +10Human Motion GenerationCo-Speech Gesture Generation

  18. MIRA: Real-Time Full-Duplex Human-Robot Interaction for Embodied Companions

    Sep 21, 2026Lijian Lin, Ye Zhu, Fan Zhang +5Co-Speech Gesture GenerationHuman-Robot Interaction

  19. EmoPose: Vision-Language Model Guided Emotion-Aware Gesture Generation for Humanoid Robots

    Sep 20, 2026Daojie Peng, Bingtao Wang, Fulong Ma +3Large Language Model-Based Robot PlanningHumanoid Robot Control

  20. AVTR-1: Open Stack for Real-Time Interactive Avatars

    Sep 19, 2026Artem Kravtsov, Dmitrii Ziganshin, Vsevolod Poletaev +4granger causalityReal-Time Interactive Video Generation

  21. GestureFAR: Streaming Co-Speech Gesture Generation with Flow Autoregression

    Sep 18, 2026Pinxin Liu, Haiyang Liu, Jiahao Luo +3Co-Speech Gesture GenerationFlow-Based Generative Modeling

  22. Co-Speech with You: Training-Free Personalization of Robot Co-Speech Gestures

    Sep 12, 2026Bosong Ding, Selma Ancel, Giacomo Spigler +1Co-Speech Gesture GenerationSocial Robotics

  23. Multi-Modal Controlled Coherent Motion Generation

    Sep 10, 2026Yifei Liu, Qiong Cao, Hongwei Yi +2Human Motion Generation3D Avatar Generation

  24. Puppeteer: Object-Grounded Posture-Aware Co-Speech Gesture Generation

    Aug 31, 2026Vida Adeli, Soroush Mehraban, Jacob Rommann +3Motion Diffusion ModelsHuman Motion Generation

  25. Motion-Omni: End-to-End Joint Speech and Full-Body Motion for Spoken Dialogue

    Aug 28, 2026Chengqian Ma, Wei Tao, Haoyu Zhang +1Multimodal GenerationCo-Speech Gesture Generation

  26. RoboGesture: Real-Time Semantic-aligned Co-Speech Gestures Generation for Humanoid Interaction

    Aug 27, 2026Zifan Wang, Ziang Ren, Pengyang Shi +7Co-Speech Gesture GenerationRobot Motion Generation

  27. InteractGesture: Progressive Chunk Guidance for Continuous Streaming Co-Speech Gesture Control

    Aug 26, 2026Ekkasit Pinyoanuntapong, Ajinkya Deogade, Paul Streli +5Co-Speech Gesture GenerationHuman Motion Generation

  28. When Words and Bodies Disagree: Realizing Verbal and Nonverbal Mismatch

    Aug 24, 2026Parisa Ghanad Torshizi, Stacy MarsellaHuman Behavior SimulationCo-Speech Gesture Generation