cs.RONov 10, 2026

Rhythmic-Semantic Co-Speech Gestures for Emotional Dialogue in Robots

Authors: Christian Arzate Cruz, Edwin C. Montiel-Vázquez, Stefanos Gkikas, Miguel Mejia Tobar, Randy Gomez

Organizations: Honda Research Institute Japan, Wako, Saitama, Japan · School of Engineering and Sciences, Tecnologico de Monterrey, Monterrey, Mexico

Abstract

Co-speech gestures can increase engagement and improve speech understanding. Yet, most data-driven robot systems generate either rhythmic beat-like motion or semantically meaningful iconic gestures, but not both. We present an emotion-aware rhythmic-semantic co-speech gesture system that generates iconic and beat behaviors in real time from text and the robot's internal emotional state. We formalize the task of beat-iconic gesture generation using emotion and propose lightweight models that predict iconic timing and intensity, then fill the remaining spans with emotion-conditioned beats at an appropriate intensity level. We train on the BEAT2 dataset and deploy the system on the social robot Haru. In an online user study with 60 participants, our method shows stronger speech-gesture alignment and is preferred over baseline methods.

Figures & tables

Explore similar work

CardsList
  1. Efficient Emotion-Aware Iconic Gesture Prediction for Robot Co-Speech

    Apr 13, 2026Edwin C. Montiel-Vazquez, Christian Arzate Cruz, Stefanos Gkikas +3Co-Speech Gesture GenerationTransformer

  2. When Semantics Matter: Reliability-Aware Semantic-Rhythm Control for Co-Speech Gesture Generation

    Sep 29, 2026Zhirui Xing, Ming Meng, Kaige Li +2Co-Speech Gesture Generation

  3. RoboGesture: Real-Time Semantic-aligned Co-Speech Gestures Generation for Humanoid Interaction

    Aug 27, 2026Zifan Wang, Ziang Ren, Pengyang Shi +7Co-Speech Gesture GenerationRobot Motion Generation