RDM: Recurrent Diffusion Model for Human Motion Generation

Mirgahney Mohamed; Harry Jake Cunningham; Riccardo Barbano; Marc P. Deisenroth; Lourdes Agapito

cs.CVJun 11, 2024

RDM: Recurrent Diffusion Model for Human Motion Generation

Authors: Mirgahney Mohamed, Harry Jake Cunningham, Riccardo Barbano, Marc P. Deisenroth, Lourdes Agapito

Organizations: Department of Computer Science, University College London

Abstract

Human motion generation is a challenging task due to its high dimensionality and the difficulty of generating fine-grained motions. Diffusion methods have been proposed due to their high sample quality and expressiveness. Early approaches treat the entire sequence as a whole, which is computationally expensive and restricts sequence length. In contrast, autoregressive diffusion models generate longer sequences. However, their reliance on fully denoising previous frames complicates training and inference. Consequently, we propose RDM, a new recurrent diffusion formulation inspired by Recurrent Neural Networks (RNNs). RDM explicitly conditions the diffusion process on preceding noisy frames, avoiding the cost of full denoising. To overcome the challenge of maintaining its probabilistic nature, we employ Normalizing Flows to model recurrent connections. Our evaluations demonstrate RDM's effectiveness: it achieves comparable performance to autoregressive baselines and generates long sequences that remain aligned with the text. RDM also skips diffusion steps during inference, reducing computational cost by up to 18x over autoregressive baselines. Project page: https://www.mirgahney.com/rfdm.io/.

Figures & tables

Figure 1: Temporal diffusion generation. Volume diffusion (a) generates sequences limited by the horizon ( $L$ ), shown by the dashed line; On the other hand, autoregressive (b) and recurrent diffusion (c) extend sequences beyond the horizon ( $L+r$ ). However, recurrent diffusion produces better-aligned sequences (green circles) with “dribbling with a basketball”.

Figure 2: Temporal diffusion models inference. a) Volume diffusion treats the entire sequence as a monolithic input and diffuses it as a whole. b) Autoregressive diffusion conditions only the reverse process on previously estimated frames while ignoring the forward process. c) Recurrent diffusion leverages a recurrent formulation and explicitly conditions both forward and reverse processes on previous noisy frames.

Figure 3: Method overview: RDM extends diffusion to temporal sequences via a 2D dependency grid across diffusion steps $t$ and temporal steps $i$ , using invertible Normalizing Flows to keep the recurrent transformations probabilistically tractable. Three regimes compose the grid: flow-only ( $t=0$ ) transforms between clean segments; diffusion-only ( $i=0$ ) is standard diffusion on the first segment; diffusion-flow ( $t{>}0,i{>}0$ ) jointly conditions on the preceding noisy state $x^{i}_{t-1}$ and the previous segment’s diffused state $x^{i-1}_{t}$ (see Sec. 4 for the full formulation).

Figure 4: Qualitative results on extended generation on the HumanML3D dataset [ Guo et al.(2022a)Guo, Zou, Zuo, Wang, Ji, Li, and Cheng ] . We evaluate extended generation with the prompt “A person is dribbling with a basketball”, generating motion up to $245$ frames (right of the dashed line); green/red circles mark correct/broken dribbling frames. Within the training horizon (left of the dashed line), MD-49 dribbles correctly at first but loses the motion near the boundary, raising its arms overhead, and fails beyond the horizon. MD-28 dribbles correctly for longer, but its hands stop dribbling once past the horizon. In contrast, RDM-28 keeps dribbling correctly throughout.

Framework	Method	R-Precision ( $\uparrow$ )			FID ( $\downarrow$ )	MultiModal Dist ( $\downarrow$ )	MultiModality ( $\uparrow$ )
Framework	Method	Top 1	Top 2	Top 3	FID ( $\downarrow$ )	MultiModal Dist ( $\downarrow$ )	MultiModality ( $\uparrow$ )
	Real motion	$0.511^{\pm.003}$	$0.703^{\pm.002}$	$0.797^{\pm.003}$	$0.002^{\pm.000}$	$2.974^{\pm.008}$	-
VAE	TEMOS [ Petrovich et al.(2022)Petrovich, Black, and Varol ]	$0.424^{\pm.002}$	$0.612^{\pm.002}$	$0.722^{\pm.002}$	$3.734^{\pm.028}$	$3.703^{\pm.008}$	$0.368^{\pm.018}$
	T2M [ Guo et al.(2022b)Guo, Zou, Zuo, Wang, Ji, Li, and Cheng ]	$0.457^{\pm.002}$	$0.639^{\pm.003}$	$0.740^{\pm.003}$	$1.067^{\pm.002}$	$3.340^{\pm.008}$	$2.090^{\pm.083}$
	MoMask [ Guo et al.(2024)Guo, Mu, Javed, Wang, and Cheng ]	$0.521^{\pm.002}$	$0.713^{\pm.002}$	$0.807^{\pm.002}$	$0.045^{\pm.002}$	$2.958^{\pm.008}$	$1.241^{\pm.040}$
Volume Diffusion	MDM [ Tevet et al.(2023)Tevet, Raab, Gordon, Shafir, Cohen-or, and Bermano ]	$0.320^{\pm.005}$	$0.498^{\pm.004}$	$0.611^{\pm.007}$	$0.544^{\pm.044}$	$5.566^{\pm.027}$	$2.799^{\pm.072}$

Table 1: Quantitative results on the HumanML3D [ Guo et al.(2022a)Guo, Zou, Zuo, Wang, Ji, Li, and Cheng ] test set. All methods use ground truth motion length. Evaluations were run 20 times with

95\%

CI. Rows are colour-coded as best , second best , and third best . Our recurrent methods (RDM-49/28) outperform segment-based baselines significantly and are comparable to CLoSD (DiP) [ Tevet et al.(2025)Tevet, Raab, Cohan, Reda, Luo, Peng, Bermano, and van de Panne ] .

Framework	Method	R-Precision ( $\uparrow$ )			FID ( $\downarrow$ )	MultiModal Dist ( $\downarrow$ )	MultiModality ( $\uparrow$ )
Framework	Method	Top 1	Top 2	Top 3	FID ( $\downarrow$ )	MultiModal Dist ( $\downarrow$ )	MultiModality ( $\uparrow$ )
	Real motion	$0.5138$	$0.6957$	$0.7937$	$0.002^{\pm.000}$	$2.974^{\pm.008}$	-
VAE	TEMOS [ Petrovich et al.(2022)Petrovich, Black, and Varol ]	$0.353^{\pm.006}$	$0.561^{\pm.007}$	$0.687^{\pm.005}$	$3.717^{\pm.051}$	$3.417^{\pm.019}$	$0.532^{\pm.034}$
	T2M [ Guo et al.(2022b)Guo, Zou, Zuo, Wang, Ji, Li, and Cheng ]	$0.370^{\pm.005}$	$0.569^{\pm.007}$	$0.693^{\pm.007}$	$2.770^{\pm.109}$	$3.401^{\pm.008}$	$1.482^{\pm.065}$
	MoMask [ Guo et al.(2024)Guo, Mu, Javed, Wang, and Cheng ]	$0.433^{\pm.007}$	$0.656^{\pm.005}$	$0.781^{\pm.005}$	$0.204^{\pm.011}$	$2.779^{\pm.022}$	$1.131^{\pm.043}$
Volume Diffusion	MDM [ Tevet et al.(2023)Tevet, Raab, Gordon, Shafir, Cohen-or, and Bermano ]	$0.164^{\pm.004}$	$0.291^{\pm.004}$	$0.396^{\pm.004}$	$0.497^{\pm.021}$	$9.191^{\pm.022}$	$1.907^{\pm.214}$

Table 2: Quantitative results on the KIT-ML [ Plappert et al.(2016)Plappert, Mandery, and Asfour ] test set. All methods use ground truth motion length. Evaluations were run 20 times with

95\%

CI. Rows are colour-coded as best , second best , and third best . Our recurrent methods (RDM-49/28) significantly outperform segment-based baselines. MD-x baselines degrade with more segments, because the dataset features shorter sequences (max

196

frames), which harms future segment prediction. Volume diffusion models generally outperform segment-based methods.

Framework	Method	Total Inference Time (s) $\downarrow$						FLOPs (T) $\downarrow$				Parameters
		DDIM				DDPM	DDIM				DDPM
		10	50	100	200	1000	10	50	100	200	1000
Volume Diffusion	MotionDiffuse [ Zhang et al.(2022)Zhang, Cai, Pan, Hong, Guo, Yang, and Liu ]	$-$	$2.71^{\pm 0.23}$	$4.92^{\pm.04}$	$9.79^{\pm.08}$	$52.07^{\pm 2.06}$	$-$	$64.64$	$128.44$	$256.02$	$1276.71$	$x\in\mathbb{R}^{196\times 251}$
Segment-based diffusion	CLoSD (DiP) [ Tevet et al.(2025)Tevet, Raab, Cohan, Reda, Luo, Peng, Bermano, and van de Panne ]	$1.77^{\pm.037}$	$8.73^{\pm.03}$	$17.5^{\pm.09}$	$34.87{\pm.04}$	$(93.18^{\pm.16})$	$45.89$	$229.44$	$458.88$	$917.76$	$(2617.93)$	$x\in\mathbb{R}^{40\times 251}$
	MD-28	$-$	$5.40^{\pm.08}$	$11.39^{\pm.80}$	$21.34{\pm.09}$	$113.20^{\pm 4.63}$	$-$	$83.41$	$165.97$	$331.08$	$1652.01$	$x\in\mathbb{R}^{28\times 251}$
	RDM-49 (ours)	$0.21^{\pm.007}$	$1.95^{\pm 1.71}$	$2^{\pm 1.19}$	$3.26^{\pm 1.51}$	$10.89^{\pm 2.11}$	$0.275$	$0.282$	$0.458$	$0.809$	$3.62$	$x\in\mathbb{R}^{49\times 251}$

Table 3: Inference time comparison. Rows are colour-coded as best , second best , and third best . Our method skips diffusion steps via staircase sampling , reducing inference time and FLOPs compared to volume methods that process the entire sequence. RDM is more efficient than segment-based baselines (MD-x/CLoSD [ Tevet et al.(2025)Tevet, Raab, Cohan, Reda, Luo, Peng, Bermano, and van de Panne ] ) because it does not fully denoise previous frames to generate new ones.

Figure 5: Computational efficiency comparison between RDM and CLoSD (DiP) [ Tevet et al.(2025)Tevet, Raab, Cohan, Reda, Luo, Peng, Bermano, and van de Panne ] . (left) Inference time vs. sequence length for RDM with 4, 7, 14 segments (DDIM-50 steps) and CLoSD. Error bars show

\pm 3

standard deviations. (right) Speedup analysis dividing CLoSD inference time by RDM’s. RDM-49/28 outperforms CLoSD by

(3.51\text{ to }18.11)\times

across sequence lengths, with higher speedups for shorter sequences. Speedup varies with segment configuration: RDM-49 is faster than RDM-28, RDM-14 is slowest.

Figure 6: Qualitative comparison on HumanML3D [ Guo et al.(2022a)Guo, Zou, Zuo, Wang, Ji, Li, and Cheng ] . Additional stacked-SMPL qualitative examples comparing volume-diffusion methods — MotionDiffuse [ Zhang et al.(2022)Zhang, Cai, Pan, Hong, Guo, Yang, and Liu ] , and Light-T2M [ Zeng et al.(2025)Zeng, Huang, Wu, and Zheng ] — against segment-based methods — MD-28 and RDM-28 — showing all five produce motions aligned with the input prompts.

Figure 7: FID stability beyond the training horizon (lower is better; note the broken $y$ -axis). RDM-28 stays consistently below CLoSD (DiP) and MD-28 as generation extends well past the $196$ -frame training horizon.

Figure 8: User study analysis results. Semantic quality in percentage denotes how well the motion visually corresponds to the input sentences. Motion quality in percentage shows the overall quality of motion in terms of naturalness. Smoothness in percentage shows how good the overall motion smoothness and coherence are. (A higher value is better.) Our method’s score, denoted in RDM-28, has the highest percentage compared to the baseline. RDM-196 receives the largest share of participant preference (left).

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 9: Diffusion-flow symmetries. By design, we can arrive at diffusion-flow samples following different routes. Some routes are more efficient than others as they allow for skipping diffusion sampling steps, but all are mathematically equivalent. 2D grid symmetries. Normalizing flows properties gives us two grids ( forward-flow and inverse-flow ). Both are mathematically equivalent, but the forward-flow is more numerically stable.

Method	R-Precision ( $\uparrow$ )			FID ( $\downarrow$ )	MultiModal Dist ( $\downarrow$ )	MultiModality ( $\uparrow$ )
Method	Top 1	Top 2	Top 3	FID ( $\downarrow$ )	MultiModal Dist ( $\downarrow$ )	MultiModality ( $\uparrow$ )
Real motions	$0.5138$	$0.6957$	$0.7937$	$0.002^{\pm.000}$	$2.974^{\pm.008}$	-
Disentangle	$0.3022^{\pm.0091}$	$0.4954^{\pm.0084}$	$0.6161^{\pm.0063}$	$6.2045^{\pm.1114}$	$4.4230^{\pm.05}$	$1.6361^{\pm.1407}$
Entangle-996	$0.405^{\pm.002}$	$0.593^{\pm.003}$	$0.702^{\pm.002}$	$0.402^{\pm.018}$	$3.651^{\pm.008}$	$1.199^{\pm.04}$
Entangle-900	$0.334^{\pm.012}$	$0.537^{\pm.005}$	$0.667^{\pm.01}$	$3.824^{\pm.233}$	$2.761^{\pm.019}$	$1.284^{\pm.078}$
Entangle-800	$0.335^{\pm.005}$	$0.539^{\pm.006}$	$0.674^{\pm.002}$	$3.648^{\pm.221}$	$3.821^{\pm.06}$	$1.241^{\pm.115}$

Appendix

Table 4: Quantitative results on the KIT-ML [ Plappert et al.(2016)Plappert, Mandery, and Asfour ] test set. All methods use ground truth motion length. Evaluations were conducted

20

times with

95\%

CI reported. Rows are colour-coded as best , second best , and third best . RDM’s staircase sampling trade-off between computational complexity and sampling quality. We found that starting the sampling at segment length provides the best result compared to starting earlier (Entangle-x, where x indicates the staircase sampling start) or “disentangled", where we sample the first segment with diffusion-only and roll out the rest with flow-only . We use RDM-28 for all experiments.

Method (RDM-28)	Total Inference Time (s) $\downarrow$	FLOPs (T) $\downarrow$	Parameters
	DDPM	DDPM
	1000	1000
Disentangle	$11.10^{\pm.59}$	$2.31$	$x\in\mathbb{R}^{49\times 251}$
Entangle-997	$11.7^{\pm 2.02}$	$2.32$	$x\in\mathbb{R}^{49\times 251}$
Entangle-900	$13.37^{\pm.31}$	$4.06$	$x\in\mathbb{R}^{49\times 251}$
Entangle-800	$18.02^{\pm 4.14}$	$5.5$	$x\in\mathbb{R}^{49\times 251}$

Appendix

Table 5: Inference time comparison. All methods use ground truth motion length. Evaluations were conducted

20

times with

95\%

CI reported. Rows are colour-coded as best , second best , and third best . RDM’s staircase sampling trade-off between computational complexity and sampling quality. Although “disentangled sampling" (sampling the first segment with diffusion-only and the rest with flow-only ) has the lowest cost, it yields the worst results compared to Entangle-x, which starts staircase sampling at point x.

Panel of Figure 10 — Figure 10: Computational efficiency comparison between RDM and CLoSD (DiP) [ Tevet et al.(2025)Tevet, Raab, Cohan, Reda, Luo, Peng, Bermano, and van de Panne ] . (left) Inference time versus sequence length for RDM with three segment numbers (4, 7, 14 segments, DDIM-10 steps) and CLoSD (DiP). Error bars represent $\pm 3$ standard deviation. (right) Speedup analysis showing CLoSD (DiP) inference time divided by RDM inference time. RDM consistently outperforms CLoSD (DiP) by a significant margin across all sequence lengths, with higher speedups for shorter sequences. The speedup depends on the segment configuration: RDM-49 is more efficient than RDM-28, and RDM-14 is the least efficient.

Panel of Figure 11 — Figure 11: Computational efficiency comparison between RDM and CLoSD (DiP) [ Tevet et al.(2025)Tevet, Raab, Cohan, Reda, Luo, Peng, Bermano, and van de Panne ] . (left) Inference time versus sequence length for RDM with three segment numbers (4, 7, 14 segments, DDIM-100 steps) and CLoSD (DiP). Error bars represent $\pm 3$ standard deviation. (right) Speedup analysis showing CLoSD (DiP) inference time divided by RDM inference time. RDM consistently outperforms CLoSD (DiP) by a significant margin across all sequence lengths, with higher speedups for shorter sequences. The speedup depends on the segment configuration: RDM-49 is more efficient than RDM-28, and RDM-14 is the least efficient.

Explore similar work

Feb 26, 2026

Causal Motion Diffusion Models for Autoregressive Motion Generation

Recent advances in motion diffusion models have substantially improved the realism of human motion synthesis. However, existing approaches either rely on full-sequence diffusion models with bidirectional generation, which limits temporal causality and real-time applicability, or autoregressive models that suffer from instability and cumulative errors. In this work, we present Causal Motion Diffusion Models (CMDM), a unified framework for autoregressive motion generation based on a causal diffusion transformer that operates in a semantically aligned latent space. CMDM builds upon a Motion-Language-Aligned Causal VAE (MAC-VAE), which encodes motion sequences into temporally causal latent representations. On top of this latent representation, an autoregressive diffusion transformer is trained using causal diffusion forcing to perform temporally ordered denoising across motion frames. To achieve fast inference, we introduce a frame-wise sampling schedule with causal uncertainty, where each subsequent frame is predicted from partially denoised previous frames. The resulting framework supports high-quality text-to-motion generation, streaming synthesis, and long-horizon motion generation at interactive rates. Experiments on HumanML3D and SnapMoGen demonstrate that CMDM outperforms existing diffusion and autoregressive models in both semantic fidelity and temporal smoothness, while substantially reducing inference latency.

Qing Yu, Akihisa Watanabe, Kent Fujiwara

LY Corporation · Waseda University

Human Motion Generation Diffusion Models for Motion Generation Autoregressive Diffusion Text-to-Motion Generation Diffusion Models Diffusion Model Inference Acceleration

Ask PDF

Jan 21, 2026

Reconstruction-Anchored Diffusion Model for Text-to-Motion Generation

Diffusion models have seen widespread adoption for text-driven human motion generation and related tasks due to their impressive generative capabilities and flexibility. However, current motion diffusion models face two major limitations: a representational gap caused by pre-trained text encoders that lack motion-specific information, and error propagation during the iterative denoising process. This paper introduces Reconstruction-Anchored Diffusion Model (RAM) to address these challenges. First, RAM leverages a motion latent space as intermediate supervision for text-to-motion generation. To this end, RAM co-trains a motion reconstruction branch with two key objective functions: self-regularization to enhance the discrimination of the motion space and motion-centric latent alignment to enable accurate mapping from text to the motion latent space. Second, we propose Reconstructive Error Guidance (REG), a testing-stage guidance mechanism that exploits the motion diffusion model's inherent self-correction ability to mitigate error propagation. At each denoising step, REG uses the motion reconstruction branch to reconstruct the previous estimate, reproducing the prior error patterns. By amplifying the residual between the current prediction and the reconstructed estimate, REG highlights the improvements in the current prediction. Extensive experiments demonstrate that RAM achieves significant improvements and state-of-the-art performance. Our code will be released.

Yifei Liu, Changxing Ding, Ling Guo +2

South China University of Technology · Joy Future Academy

Diffusion Model Guidance Text-to-Motion Generation Human Motion Generation Diffusion Models for Motion Generation

Ask PDF

May 28, 2026

DC-Motion: Decoupling Structure and Details via Discrete-Continuous Tokens for Human Motion Generation

Text-to-motion generation requires modeling both global action structure and fine-grained motion dynamics from natural language. Existing approaches typically rely on either continuous diffusion models or vector-quantized discrete representations. Diffusion models generate smooth motions but lack explicit compositional structure for temporal planning, while discrete token-based methods improve controllability but compress motion into finite codebooks, losing fine-grained dynamics. We argue that this limitation stems from a representation mismatch: action semantics such as intent, phase transitions, and temporal layout are inherently discrete and compositional, whereas joint trajectories and motion dynamics are continuous and locally correlated. To address this, we propose DC-Motion, a discrete-continuous factorized framework for human motion generation. DC-Motion decomposes motion into discrete structural tokens capturing global action layout and continuous residual latents modeling fine-grained dynamics. A text-conditioned structure generator predicts discrete tokens via iterative masked modeling, and a diffusion-based residual generator produces continuous motion conditioned on the structure. Experiments on HumanML3D and KIT-ML demonstrate that DC-Motion achieves strong performance in both FID and R-Precision, outperforming representative diffusion-based and discrete-token baselines.

Hequan Wang, Xuean Chen, Jiaxu Zhang +2

State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan 430072, China · Geophysical Exploration Brigade of Hubei Provincial Geological Bureau, Wuhan 430058, China · Information Systems Technology and Design Pillar, Singapore University of Technology and Design, 487372, Singapore

Text-to-Motion Generation Human Motion Generation Generative Modeling Diffusion Models for Motion Generation Conditional Diffusion Models

Ask PDF