Multimodal Behavior

Multimodal behavior research focuses on understanding and modeling how humans express themselves through multiple channels simultaneously (e.g., speech, gestures, facial expressions). Current research emphasizes developing computational models, often employing transformer-based architectures and diffusion models, to analyze and synthesize this complex behavior, particularly within social interactions and collaborative tasks. This work aims to improve human-computer interaction, particularly in robotics and virtual agents, by creating more natural and intuitive interfaces, and also offers insights into human cognition and social dynamics through objective behavioral analysis. The ultimate goal is to build systems capable of understanding and responding appropriately to the rich tapestry of human communication.