cs.SDNov 11, 2026

CFGComposer: Compositional Audio-Text-to-Motion Generation via Copula Score Correction

Authors: Pengyang Shi, Jinlu Zhang, Chunze Lin, Yizhou Wang, Li Yi

Organizations: Tsinghua University · Galbot · CFCS, School of Computer Science, Peking University · State Key Laboratory of General Artificial Intelligence · 

Abstract

Generating whole-body gestures that are simultaneously rhythmically synchronized with speech audio and semantically aligned with text descriptions is a fundamental challenge in embodied AI and social robotics. We tackle this Audio-Text-to-Motion (AT2M) problem by composing two independently-trained diffusion models at inference time and refine bias in joint inference by modeling the copula score sIs_I. A rigorous theoretical analysis reveals that naive epsilon blending corresponds to a Product-of-Experts formulation that omits the copula score sIs_I: the cross-modal interaction between audio and text that neither marginal model can capture. We propose CFGComposer: a jointly-trained AT2M diffusion model that estimates sIs_I via a CFG analogy and injects it as an εε-space correction. Experiments on BEAT demonstrate best-in-class semantic fidelity (SRGR 0.99160.9916, FID 27.1827.18) with GT-level beat alignment (BA 0.18260.1826 vs. GT 0.17430.1743), and 76-83% win rate in blind pairwise human evaluation against all prior methods. Physical deployment on a Unitree G1 humanoid confirms direct sim-to-real transfer.

Figures & tables

Explore similar work

CardsList
  1. ECHO-G: Embodied Co-speech Humanoid mOtion Generation

    Sep 30, 2026Yizhao Li, Pusen Gao, Ming Wang +3Text-to-Motion GenerationHuman Motion Generation

  2. Plan, Don't Pose: Long Composite Motion Generation with Text-Aligned BFM

    May 28, 2026Nikolay Shvetsov, Maksim Bobrin, Nazar Buzun +2Text-to-Motion GenerationHuman Motion Generation