RDM: Recurrent Diffusion Model for Human Motion Generation

Mirgahney Mohamed; Harry Jake Cunningham; Riccardo Barbano; Marc P. Deisenroth; Lourdes Agapito

cs.CVJun 11, 2024

RDM: Recurrent Diffusion Model for Human Motion Generation

Authors: Mirgahney Mohamed, Harry Jake Cunningham, Riccardo Barbano, Marc P. Deisenroth, Lourdes Agapito

Organizations: Department of Computer Science, University College London

Abstract

Human motion generation is a challenging task due to its high dimensionality and the difficulty of generating fine-grained motions. Diffusion methods have been proposed due to their high sample quality and expressiveness. Early approaches treat the entire sequence as a whole, which is computationally expensive and restricts sequence length. In contrast, autoregressive diffusion models generate longer sequences. However, their reliance on fully denoising previous frames complicates training and inference. Consequently, we propose RDM, a new recurrent diffusion formulation inspired by Recurrent Neural Networks (RNNs). RDM explicitly conditions the diffusion process on preceding noisy frames, avoiding the cost of full denoising. To overcome the challenge of maintaining its probabilistic nature, we employ Normalizing Flows to model recurrent connections. Our evaluations demonstrate RDM's effectiveness: it achieves comparable performance to autoregressive baselines and generates long sequences that remain aligned with the text. RDM also skips diffusion steps during inference, reducing computational cost by up to 18x over autoregressive baselines. Project page: https://www.mirgahney.com/rfdm.io/.

Figures & tables

Figure 1: Temporal diffusion generation. Volume diffusion (a) generates sequences limited by the horizon ( $L$ ), shown by the dashed line; On the other hand, autoregressive (b) and recurrent diffusion (c) extend sequences beyond the horizon ( $L+r$ ). However, recurrent diffusion produces better-aligned sequences (green circles) with “dribbling with a basketball”.

Figure 2: Temporal diffusion models inference. a) Volume diffusion treats the entire sequence as a monolithic input and diffuses it as a whole. b) Autoregressive diffusion conditions only the reverse process on previously estimated frames while ignoring the forward process. c) Recurrent diffusion leverages a recurrent formulation and explicitly conditions both forward and reverse processes on previous noisy frames.

Figure 3: Method overview: RDM extends diffusion to temporal sequences via a 2D dependency grid across diffusion steps $t$ and temporal steps $i$ , using invertible Normalizing Flows to keep the recurrent transformations probabilistically tractable. Three regimes compose the grid: flow-only ( $t=0$ ) transforms between clean segments; diffusion-only ( $i=0$ ) is standard diffusion on the first segment; diffusion-flow ( $t{>}0,i{>}0$ ) jointly conditions on the preceding noisy state $x^{i}_{t-1}$ and the previous segment’s diffused state $x^{i-1}_{t}$ (see Sec. 4 for the full formulation).

Figure 4: Qualitative results on extended generation on the HumanML3D dataset [ Guo et al.(2022a)Guo, Zou, Zuo, Wang, Ji, Li, and Cheng ] . We evaluate extended generation with the prompt “A person is dribbling with a basketball”, generating motion up to $245$ frames (right of the dashed line); green/red circles mark correct/broken dribbling frames. Within the training horizon (left of the dashed line), MD-49 dribbles correctly at first but loses the motion near the boundary, raising its arms overhead, and fails beyond the horizon. MD-28 dribbles correctly for longer, but its hands stop dribbling once past the horizon. In contrast, RDM-28 keeps dribbling correctly throughout.

Framework	Method	R-Precision ( $\uparrow$ )			FID ( $\downarrow$ )	MultiModal Dist ( $\downarrow$ )	MultiModality ( $\uparrow$ )
Framework	Method	Top 1	Top 2	Top 3	FID ( $\downarrow$ )	MultiModal Dist ( $\downarrow$ )	MultiModality ( $\uparrow$ )
	Real motion	$0.511^{\pm.003}$	$0.703^{\pm.002}$	$0.797^{\pm.003}$	$0.002^{\pm.000}$	$2.974^{\pm.008}$	-
VAE	TEMOS [ Petrovich et al.(2022)Petrovich, Black, and Varol ]	$0.424^{\pm.002}$	$0.612^{\pm.002}$	$0.722^{\pm.002}$	$3.734^{\pm.028}$	$3.703^{\pm.008}$	$0.368^{\pm.018}$
	T2M [ Guo et al.(2022b)Guo, Zou, Zuo, Wang, Ji, Li, and Cheng ]	$0.457^{\pm.002}$	$0.639^{\pm.003}$	$0.740^{\pm.003}$	$1.067^{\pm.002}$	$3.340^{\pm.008}$	$2.090^{\pm.083}$
	MoMask [ Guo et al.(2024)Guo, Mu, Javed, Wang, and Cheng ]	$0.521^{\pm.002}$	$0.713^{\pm.002}$	$0.807^{\pm.002}$	$0.045^{\pm.002}$	$2.958^{\pm.008}$	$1.241^{\pm.040}$
Volume Diffusion	MDM [ Tevet et al.(2023)Tevet, Raab, Gordon, Shafir, Cohen-or, and Bermano ]	$0.320^{\pm.005}$	$0.498^{\pm.004}$	$0.611^{\pm.007}$	$0.544^{\pm.044}$	$5.566^{\pm.027}$	$2.799^{\pm.072}$

Table 1: Quantitative results on the HumanML3D [ Guo et al.(2022a)Guo, Zou, Zuo, Wang, Ji, Li, and Cheng ] test set. All methods use ground truth motion length. Evaluations were run 20 times with

95\%

CI. Rows are colour-coded as best , second best , and third best . Our recurrent methods (RDM-49/28) outperform segment-based baselines significantly and are comparable to CLoSD (DiP) [ Tevet et al.(2025)Tevet, Raab, Cohan, Reda, Luo, Peng, Bermano, and van de Panne ] .

Framework	Method	R-Precision ( $\uparrow$ )			FID ( $\downarrow$ )	MultiModal Dist ( $\downarrow$ )	MultiModality ( $\uparrow$ )
Framework	Method	Top 1	Top 2	Top 3	FID ( $\downarrow$ )	MultiModal Dist ( $\downarrow$ )	MultiModality ( $\uparrow$ )
	Real motion	$0.5138$	$0.6957$	$0.7937$	$0.002^{\pm.000}$	$2.974^{\pm.008}$	-
VAE	TEMOS [ Petrovich et al.(2022)Petrovich, Black, and Varol ]	$0.353^{\pm.006}$	$0.561^{\pm.007}$	$0.687^{\pm.005}$	$3.717^{\pm.051}$	$3.417^{\pm.019}$	$0.532^{\pm.034}$
	T2M [ Guo et al.(2022b)Guo, Zou, Zuo, Wang, Ji, Li, and Cheng ]	$0.370^{\pm.005}$	$0.569^{\pm.007}$	$0.693^{\pm.007}$	$2.770^{\pm.109}$	$3.401^{\pm.008}$	$1.482^{\pm.065}$
	MoMask [ Guo et al.(2024)Guo, Mu, Javed, Wang, and Cheng ]	$0.433^{\pm.007}$	$0.656^{\pm.005}$	$0.781^{\pm.005}$	$0.204^{\pm.011}$	$2.779^{\pm.022}$	$1.131^{\pm.043}$
Volume Diffusion	MDM [ Tevet et al.(2023)Tevet, Raab, Gordon, Shafir, Cohen-or, and Bermano ]	$0.164^{\pm.004}$	$0.291^{\pm.004}$	$0.396^{\pm.004}$	$0.497^{\pm.021}$	$9.191^{\pm.022}$	$1.907^{\pm.214}$

Table 2: Quantitative results on the KIT-ML [ Plappert et al.(2016)Plappert, Mandery, and Asfour ] test set. All methods use ground truth motion length. Evaluations were run 20 times with

95\%

CI. Rows are colour-coded as best , second best , and third best . Our recurrent methods (RDM-49/28) significantly outperform segment-based baselines. MD-x baselines degrade with more segments, because the dataset features shorter sequences (max

196

frames), which harms future segment prediction. Volume diffusion models generally outperform segment-based methods.

Framework	Method	Total Inference Time (s) $\downarrow$						FLOPs (T) $\downarrow$				Parameters
		DDIM				DDPM	DDIM				DDPM
		10	50	100	200	1000	10	50	100	200	1000
Volume Diffusion	MotionDiffuse [ Zhang et al.(2022)Zhang, Cai, Pan, Hong, Guo, Yang, and Liu ]	$-$	$2.71^{\pm 0.23}$	$4.92^{\pm.04}$	$9.79^{\pm.08}$	$52.07^{\pm 2.06}$	$-$	$64.64$	$128.44$	$256.02$	$1276.71$	$x\in\mathbb{R}^{196\times 251}$
Segment-based diffusion	CLoSD (DiP) [ Tevet et al.(2025)Tevet, Raab, Cohan, Reda, Luo, Peng, Bermano, and van de Panne ]	$1.77^{\pm.037}$	$8.73^{\pm.03}$	$17.5^{\pm.09}$	$34.87{\pm.04}$	$(93.18^{\pm.16})$	$45.89$	$229.44$	$458.88$	$917.76$	$(2617.93)$	$x\in\mathbb{R}^{40\times 251}$
	MD-28	$-$	$5.40^{\pm.08}$	$11.39^{\pm.80}$	$21.34{\pm.09}$	$113.20^{\pm 4.63}$	$-$	$83.41$	$165.97$	$331.08$	$1652.01$	$x\in\mathbb{R}^{28\times 251}$
	RDM-49 (ours)	$0.21^{\pm.007}$	$1.95^{\pm 1.71}$	$2^{\pm 1.19}$	$3.26^{\pm 1.51}$	$10.89^{\pm 2.11}$	$0.275$	$0.282$	$0.458$	$0.809$	$3.62$	$x\in\mathbb{R}^{49\times 251}$

Table 3: Inference time comparison. Rows are colour-coded as best , second best , and third best . Our method skips diffusion steps via staircase sampling , reducing inference time and FLOPs compared to volume methods that process the entire sequence. RDM is more efficient than segment-based baselines (MD-x/CLoSD [ Tevet et al.(2025)Tevet, Raab, Cohan, Reda, Luo, Peng, Bermano, and van de Panne ] ) because it does not fully denoise previous frames to generate new ones.

Figure 5: Computational efficiency comparison between RDM and CLoSD (DiP) [ Tevet et al.(2025)Tevet, Raab, Cohan, Reda, Luo, Peng, Bermano, and van de Panne ] . (left) Inference time vs. sequence length for RDM with 4, 7, 14 segments (DDIM-50 steps) and CLoSD. Error bars show

\pm 3

standard deviations. (right) Speedup analysis dividing CLoSD inference time by RDM’s. RDM-49/28 outperforms CLoSD by

(3.51\text{ to }18.11)\times

across sequence lengths, with higher speedups for shorter sequences. Speedup varies with segment configuration: RDM-49 is faster than RDM-28, RDM-14 is slowest.

Figure 6: Qualitative comparison on HumanML3D [ Guo et al.(2022a)Guo, Zou, Zuo, Wang, Ji, Li, and Cheng ] . Additional stacked-SMPL qualitative examples comparing volume-diffusion methods — MotionDiffuse [ Zhang et al.(2022)Zhang, Cai, Pan, Hong, Guo, Yang, and Liu ] , and Light-T2M [ Zeng et al.(2025)Zeng, Huang, Wu, and Zheng ] — against segment-based methods — MD-28 and RDM-28 — showing all five produce motions aligned with the input prompts.

Figure 7: FID stability beyond the training horizon (lower is better; note the broken $y$ -axis). RDM-28 stays consistently below CLoSD (DiP) and MD-28 as generation extends well past the $196$ -frame training horizon.

Figure 8: User study analysis results. Semantic quality in percentage denotes how well the motion visually corresponds to the input sentences. Motion quality in percentage shows the overall quality of motion in terms of naturalness. Smoothness in percentage shows how good the overall motion smoothness and coherence are. (A higher value is better.) Our method’s score, denoted in RDM-28, has the highest percentage compared to the baseline. RDM-196 receives the largest share of participant preference (left).

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 9: Diffusion-flow symmetries. By design, we can arrive at diffusion-flow samples following different routes. Some routes are more efficient than others as they allow for skipping diffusion sampling steps, but all are mathematically equivalent. 2D grid symmetries. Normalizing flows properties gives us two grids ( forward-flow and inverse-flow ). Both are mathematically equivalent, but the forward-flow is more numerically stable.

Method	R-Precision ( $\uparrow$ )			FID ( $\downarrow$ )	MultiModal Dist ( $\downarrow$ )	MultiModality ( $\uparrow$ )
Method	Top 1	Top 2	Top 3	FID ( $\downarrow$ )	MultiModal Dist ( $\downarrow$ )	MultiModality ( $\uparrow$ )
Real motions	$0.5138$	$0.6957$	$0.7937$	$0.002^{\pm.000}$	$2.974^{\pm.008}$	-
Disentangle	$0.3022^{\pm.0091}$	$0.4954^{\pm.0084}$	$0.6161^{\pm.0063}$	$6.2045^{\pm.1114}$	$4.4230^{\pm.05}$	$1.6361^{\pm.1407}$
Entangle-996	$0.405^{\pm.002}$	$0.593^{\pm.003}$	$0.702^{\pm.002}$	$0.402^{\pm.018}$	$3.651^{\pm.008}$	$1.199^{\pm.04}$
Entangle-900	$0.334^{\pm.012}$	$0.537^{\pm.005}$	$0.667^{\pm.01}$	$3.824^{\pm.233}$	$2.761^{\pm.019}$	$1.284^{\pm.078}$
Entangle-800	$0.335^{\pm.005}$	$0.539^{\pm.006}$	$0.674^{\pm.002}$	$3.648^{\pm.221}$	$3.821^{\pm.06}$	$1.241^{\pm.115}$

Appendix

Table 4: Quantitative results on the KIT-ML [ Plappert et al.(2016)Plappert, Mandery, and Asfour ] test set. All methods use ground truth motion length. Evaluations were conducted

20

times with

95\%

CI reported. Rows are colour-coded as best , second best , and third best . RDM’s staircase sampling trade-off between computational complexity and sampling quality. We found that starting the sampling at segment length provides the best result compared to starting earlier (Entangle-x, where x indicates the staircase sampling start) or “disentangled", where we sample the first segment with diffusion-only and roll out the rest with flow-only . We use RDM-28 for all experiments.

Method (RDM-28)	Total Inference Time (s) $\downarrow$	FLOPs (T) $\downarrow$	Parameters
	DDPM	DDPM
	1000	1000
Disentangle	$11.10^{\pm.59}$	$2.31$	$x\in\mathbb{R}^{49\times 251}$
Entangle-997	$11.7^{\pm 2.02}$	$2.32$	$x\in\mathbb{R}^{49\times 251}$
Entangle-900	$13.37^{\pm.31}$	$4.06$	$x\in\mathbb{R}^{49\times 251}$
Entangle-800	$18.02^{\pm 4.14}$	$5.5$	$x\in\mathbb{R}^{49\times 251}$

Appendix

Table 5: Inference time comparison. All methods use ground truth motion length. Evaluations were conducted

20

times with

95\%

CI reported. Rows are colour-coded as best , second best , and third best . RDM’s staircase sampling trade-off between computational complexity and sampling quality. Although “disentangled sampling" (sampling the first segment with diffusion-only and the rest with flow-only ) has the lowest cost, it yields the worst results compared to Entangle-x, which starts staircase sampling at point x.

Panel of Figure 10 — Figure 10: Computational efficiency comparison between RDM and CLoSD (DiP) [ Tevet et al.(2025)Tevet, Raab, Cohan, Reda, Luo, Peng, Bermano, and van de Panne ] . (left) Inference time versus sequence length for RDM with three segment numbers (4, 7, 14 segments, DDIM-10 steps) and CLoSD (DiP). Error bars represent $\pm 3$ standard deviation. (right) Speedup analysis showing CLoSD (DiP) inference time divided by RDM inference time. RDM consistently outperforms CLoSD (DiP) by a significant margin across all sequence lengths, with higher speedups for shorter sequences. The speedup depends on the segment configuration: RDM-49 is more efficient than RDM-28, and RDM-14 is the least efficient.

Panel of Figure 11 — Figure 11: Computational efficiency comparison between RDM and CLoSD (DiP) [ Tevet et al.(2025)Tevet, Raab, Cohan, Reda, Luo, Peng, Bermano, and van de Panne ] . (left) Inference time versus sequence length for RDM with three segment numbers (4, 7, 14 segments, DDIM-100 steps) and CLoSD (DiP). Error bars represent $\pm 3$ standard deviation. (right) Speedup analysis showing CLoSD (DiP) inference time divided by RDM inference time. RDM consistently outperforms CLoSD (DiP) by a significant margin across all sequence lengths, with higher speedups for shorter sequences. The speedup depends on the segment configuration: RDM-49 is more efficient than RDM-28, and RDM-14 is the least efficient.

Explore similar work

Causal Motion Diffusion Models for Autoregressive Motion Generation
Feb 26, 2026Qing Yu, Akihisa Watanabe, Kent FujiwaraHuman Motion Generation Diffusion Models for Motion Generation
LY Corporation · Waseda University
Reconstruction-Anchored Diffusion Model for Text-to-Motion Generation
Jan 21, 2026Yifei Liu, Changxing Ding, Ling Guo +2Diffusion Model Guidance Text-to-Motion Generation
South China University of Technology · Joy Future Academy
DC-Motion: Decoupling Structure and Details via Discrete-Continuous Tokens for Human Motion Generation
May 28, 2026Hequan Wang, Xuean Chen, Jiaxu Zhang +2Text-to-Motion Generation Human Motion Generation
State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan 430072, China · Geophysical Exploration Brigade of Hubei Provincial Geological Bureau, Wuhan 430058, China · Information Systems Technology and Design Pillar, Singapore University of Technology and Design, 487372, Singapore

Feb 26, 2026J/K move · Enter open · S save

Causal Motion Diffusion Models for Autoregressive Motion Generation

Qing Yu, Akihisa Watanabe, Kent Fujiwara

LY Corporation · Waseda University

Human Motion Generation Diffusion Models for Motion Generation Autoregressive Diffusion Text-to-Motion Generation Diffusion Models Diffusion Model Inference Acceleration

Open paper AskPDF arXiv

Recent advances in motion diffusion models have substantially improved the realism of human motion synthesis. However, existing approaches either rely on full-sequence diffusion models with bidirectional generation, which limits temporal causality and real-time applicability, or autoregressive models that suffer from instability and cumulative errors. In this work, we present Causal Motion Diffusion Models (CMDM), a unified framework for autoregressive motion generation based on a causal diffusion transformer that operates in a semantically aligned latent space. CMDM builds upon a Motion-Language-Aligned Causal VAE (MAC-VAE), which encodes motion sequences into temporally causal latent representations. On top of this latent representation, an autoregressive diffusion transformer is trained using causal diffusion forcing to perform temporally ordered denoising across motion frames. To achieve fast inference, we introduce a frame-wise sampling schedule with causal uncertainty, where each subsequent frame is predicted from partially denoised previous frames. The resulting framework supports high-quality text-to-motion generation, streaming synthesis, and long-horizon motion generation at interactive rates. Experiments on HumanML3D and SnapMoGen demonstrate that CMDM outperforms existing diffusion and autoregressive models in both semantic fidelity and temporal smoothness, while substantially reducing inference latency.

Abstract

Explore similar work

Causal Motion Diffusion Models for Autoregressive Motion Generation

Reconstruction-Anchored Diffusion Model for Text-to-Motion Generation

DC-Motion: Decoupling Structure and Details via Discrete-Continuous Tokens for Human Motion Generation