Möbius: A Unified Motion-Language Model Bridging Understanding and Generation

Jiazhi Shu; Chengfeng Zhao; Jinyi Wang; Yubo Zhao; Zhiyang Dou; Ronghui Li; Yu Lu; Xinwei Jiang

cs.CVNov 3, 2026

Möbius: A Unified Motion-Language Model Bridging Understanding and Generation

Authors: Jiazhi Shu, Chengfeng Zhao, Jinyi Wang, Yubo Zhao, Zhiyang Dou, Ronghui Li, Yu Lu, Xinwei Jiang,

Organizations: The Hong Kong University of Science and Technology · Tencent Games Motus · Massachusetts Institute of Technology

Abstract

We aim to build a truly unified motion model that can understand, generate, and edit human motion within a single framework. Existing motion UMMs remain only partially unified: motion understanding is handled by a language model, while generation is often delegated to a separate diffusion module operating over learned motion latents. We introduce Möbius, which brings the entire diffusion process into the same multimodal representation space used for motion understanding. Clean motion, noisy motion, and text jointly participate in shared attention, while role-specific parameters process understanding and generation streams. Importantly, Möbius operates directly on continuous frame-level raw motion, requiring neither discrete tokenization nor a pretrained motion VAE. This simple formulation naturally supports motion captioning, text-to-motion generation, temporal completion, and instruction-guided editing within a single model. On HumanML3D, Möbius achieves state-of-the-art performance among unified motion models. Scaling to 1.7B parameters and 1,700 hours of motion data further demonstrates that our framework is both effective and scalable.

Figures & tables

Figure 1: Multi-turn motion creation with Möbius . A motion sequence is progressively generated, completed, edited, and understood through natural language in multiple turns.

Figure 2: Architectural comparison of unified motion–language modeling paradigms. (a) Language-based motion models discretize motion into tokens and model motion and text within a shared LLM. (b) Continuous motion–language models use the LLM to produce conditioning representations, while motion is generated by an external diffusion module. (c) Möbius brings the denoising process into the unified multimodal backbone, allowing noisy motion and language to interact throughout generation.

Figure 3: Overview of Möbius . Motion understanding and generation share a unified multimodal backbone. Text tokens, clean motion frames, and noisy target frames with flow time $t$ participate in shared attention at every layer, while role-specific parameters handle normalization, projections, and feed-forward transformations. The text head autoregressively predicts captions, and the motion head recovers target frames in parallel for generation, temporal completion, and editing. The central matrices illustrate attention visibility for motion understanding and generation.

Figure 4: Qualitative results on motion understanding and generation. (a) Text-to-motion generation results of HY-Motion ( Wen et al., 2025 ) (yellow), Möbius (blue), and MotionMillion ( Fan et al., 2025 ) (purple). (b) Motion-to-text understanding compared with MotionGPT3 ( Zhu et al., 2025 ) . (c) Instruction-based editing: source (gray), Möbius (blue), and MotionLab ( Guo et al., 2025 ) (green). (d) Temporal completion: prediction, backcasting, in-betweening, and keyframe infilling; known and generated frames are orange and blue, respectively.

Trained on HumanML3D
Method	Task	R@1 $\uparrow$	R@2 $\uparrow$	R@3 $\uparrow$	FID $\downarrow$	MMDist $\downarrow$	Diversity $\rightarrow$
GT	–	0.7703	0.9030	0.9442	0.0000	14.8785	27.7705
T2M-GPT ( Zhang et al., 2023 )	Gen.	0.5516	0.7056	0.7788	25.4913	19.0912	25.5949
MoMask ( Guo et al., 2024 )	Gen.	0.6404	0.7974	0.8609	21.0729	18.1216	25.9789
MDM ( Tevet et al., 2022 )	Gen.	0.5208	0.6937	0.7701	35.5169	19.4246	25.3383
MotionStreamer ( Xiao et al., 2025 )	Gen.	0.6303	0.7865	0.8498	12.2110	16.5810	27.4637

Table 1: Comparison of text-to-motion generation on HumanML3D ( Guo et al., 2022a ) . “Task" indicates whether a model is generation-only (Gen.) or unified for motion understanding and generation (Uni.). We use the evaluator from MotionStreamer ( Xiao et al., 2025 ) .

Method	BLEU-1 $\uparrow$	BLEU-4 $\uparrow$	ROUGE-L $\uparrow$	CIDEr $\uparrow$	BERT $\uparrow$	R@1 $\uparrow$	R@2 $\uparrow$	R@3 $\uparrow$	MMDist $\downarrow$
GT	1.0000	1.0000	1.0000	4.5165	1.0000	0.5068	0.7064	0.8041	2.9784
TM2T ( Guo et al., 2022b )	0.4912	0.0760	0.3808	0.1719	0.3215	0.5180	0.7178	0.8079	2.9584
MotionGPT ( Jiang et al., 2023 )	0.4605	0.1210	0.3849	0.3409	0.3388	0.5087	0.6880	0.7776	3.1121
MotionGPT3 ( Zhu et al., 2025 )	0.5383	0.1605	0.4165	0.4077	0.3425	0.5333	0.7178	0.8043	2.8533
UniMuMo ( Yang et al., 2024 )	0.4488	0.1271	0.3560	0.3114	0.4517	0.5162	0.7032	0.7984	2.9658
Ours 0.6B (ML3D only)	0.5664	0.2032	0.4636	0.5754	0.4541	0.6056	0.7936	0.8761	2.4693

Table 2: Comparison of motion captioning on HumanML3D ( Guo et al., 2022a ) . We follow the TM2T evaluation protocol ( Guo et al., 2022b ) using raw captions.

Method	FID $\downarrow$	R@1 $\uparrow$	R@2 $\uparrow$	R@3 $\uparrow$	AvgR $\downarrow$	MMDist $\downarrow$	Diversity $\rightarrow$
GT	0.0000	1.0000	1.0000	1.0000	1.0000	0.0000	8.3500
Source motion (no edit)	0.1066	0.4640	0.6525	0.7502	3.5100	2.2817	8.2000
MotionFix (TMED) ( Athanasiou et al., 2024 )	0.5282	0.3248	0.4877	0.5982	5.3600	3.1974	7.8900
MotionLab ( Guo et al., 2025 )	0.1339	0.4610	0.6308	0.7315	3.7500	2.4112	8.1800
Ours (0.6B) †	0.0955	0.4936	0.6821	0.7828	2.8026	1.8749	8.4468

Table 3: Instruction-based motion editing results on MotionFix ( Athanasiou et al., 2024 ) . † denotes our editing variant trained with additional MotionFix pairs in Phase II. The no-edit source and GT are shown as references and excluded from ranking.

				Understanding (M2T)				Generation (T2M)
ID	Routing	Encoding	Training	R@1 $\uparrow$	BLEU-1 $\uparrow$	CIDEr $\uparrow$	BERT $\uparrow$	R@1 $\uparrow$	R@3 $\uparrow$	FID $\downarrow$	MMDist $\downarrow$
GT				0.5068	1.0000	4.5165	1.0000	0.7703	0.9442	0.0000	14.8785
1	Dense	VQ	Und. $\to$ Gen.	0.5695	0.5593	0.4568	0.4296	0.5627	0.7741	6.0912	16.4993
2	Role	VAE	Und. $\to$ Gen.	0.5846	0.5619	0.5522	0.4404	0.8251	0.9655	5.9970	13.8139
3	Modality	Frame	Und. only	0.5675	0.5222	0.4525	0.4134	–	–	–	–
4	Modality	Frame	Und. $\to$ Gen.	0.0328	0.0776	0.0040	-0.3768	0.7512	0.9250	18.0517	15.3010

Table 4: Ablation study. We study motion encoding, routing, and understanding–generation training. † allows denoising to access only the final-layer context representations; ‡ randomly initializes the denoising parameters instead of transferring those learned during understanding. Their M2T scores equal those of the full model and are not ranked. Bold and underline denote the best and second-best variants, excluding GT.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Component	Möbius -0.6B	Möbius -1.7B
Backbone	Qwen3-0.6B	Qwen3-1.7B
Vocabulary size	151,673	151,673
Total parameters	1.047B	3.148B
Motion representation	HY201, 30 FPS	HY201, 30 FPS
Transformer layers	28	28
Hidden dimension	1,024	2,048

Appendix

Table S1: Architecture specification of Möbius .

Hyper-parameter	Value	Hyper-parameter	Value
Training schedule		Generation objective
M2T training steps	20,000	Prediction target	$x_{0}$
Generation training steps	200,000	Timestep sampling	Uniform
Warmup steps (each stage)	2,000	Motion dimension	201
Token budget / GPU	65,536	Text condition dropout	0.1
Gradient accumulation	1	Geometric loss	Enabled

Appendix

Table S2: Training and inference hyper-parameters of Möbius -1.7B. We first train motion-to-text (M2T) for 20,000 steps, followed by 200,000 steps of generation-only fine-tuning. Both stages use 2,000 warmup steps. Training batches are dynamically packed by token count. Inference uses EMA weights.

Task	Context / output	Observed motion
T2M	$C\rightarrow B_{t}$	None
M2T	$[C,B_{S}]\rightarrow Y$	Full source
Editing	$[C,B_{S}]\rightarrow B_{t}$	Full source
Prediction	$[C,B_{S}]\rightarrow B_{t}$	Prefix $\Omega$
Backcasting	$[C,B_{S}]\rightarrow B_{t}$	Suffix $\Omega$
In-betweening	$[C,B_{S}]\rightarrow B_{t}$	Prefix and suffix

Appendix

Table S3: Unified task construction.

\Omega

denotes observed temporal indices. For temporal tasks, observations use the target timeline.

Setting	Text-to-motion	Motion-to-text
Sampler / decoder	Euler with clean-endpoint-to-velocity conversion	Autoregressive greedy decoding
Steps / time grid	50; uniform on $[0,1]$ ; timestep shift $1.0$	Not applicable
CFG scale	$3.0$	Not applicable
Initialization / input	Standard Gaussian noise	Observed clean motion
Motion length	Reference length, clipped to $[2,300]$ frames at 30 FPS	Input motion length
Response limit / stopping	Not applicable	128 new tokens; stop at the configured EOS token

Appendix

Table S4: T2M and M2T inference-pipeline settings. The response-token limit is the M2T script default. Editing and temporal-inpainting conditions are specified separately in their task protocols.

Model	Task	Batch	Latency (s)	Clips/s	Memory (GiB)	Tokens/clip
0.6B	T2M	1	4.988	0.20	1.99	—
0.6B	T2M	8	6.462	1.24	2.07	—
0.6B	M2T	1	0.319	3.13	2.00	10.0
0.6B	M2T	8	0.898	8.91	2.16	12.4
1.7B	T2M	1	5.072	0.20	5.91	—
1.7B	T2M	8	6.547	1.22	6.00	—

Appendix

Table S5: Inference efficiency. Each input has 150 motion frames (5 seconds at 30 FPS). Latency is the mean per batch over five measured calls after two warmup calls. Memory is peak PyTorch allocated memory, including model weights.

Metric	Definition and interpretation
R@ $k$ $\uparrow$	Fraction of queries whose paired target ranks among the top $k$ candidates in the task-specific embedding space. Ranks are computed within the candidate set of the corresponding protocol.
FID $\downarrow$	Fréchet distance between Gaussian fits to generated and real motion embeddings; measures distributional agreement.
MMDist / MM-D $\downarrow$	Mean paired text–motion embedding distance for T2M/M2T; the editing table uses motion–motion distance. This is not maximum mean discrepancy.
M2M-Dist $\downarrow$	Mean distance between generated and target motion embeddings for editing.
AvgR $\downarrow$	Mean rank of the paired target in editing retrieval; depends on candidate-set size.
Diversity $\rightarrow$	Mean embedding distance between sampled motion pairs; compare with the real-data reference.

Appendix

Table S6: Meaning of the evaluation metrics.

\uparrow

and

\downarrow

indicate preferred directions;

\rightarrow

indicates closeness to the real-data value.

Figure S1: Additional text-to-motion results. Motion sequences are shown with their conditioning descriptions to illustrate prompt adherence.

Figure S2: Additional motion-to-text examples. The motion sequences and corresponding captions illustrate motion understanding at the example level.

	Base	Ours
Q1. How does the body’s height change when stepping up versus stepping down?
Answer	When stepping up, the body moves upward as the person lifts the leg and pushes onto the higher surface. When stepping down, the body moves downward as the person lowers the foot to the lower surface and transfers their weight onto it.	The body’s center of mass rises when stepping up and lowers when stepping down, following the change in support height.
Q2. What happens to the knee and hip joints when a person squats down?
Answer	As a person squats down, both the knees and hips bend to lower the body. The knees move forward while the hips move backward, allowing the person to descend while maintaining balance.	Both the knees and hips flex as the body lowers, with increasing joint flexion toward the bottom of the squat.
Q3. Why is it harder to maintain balance when hopping on one foot than when jumping on both feet?
Answer	Hopping on one foot is more difficult because the body has only one foot supporting its weight. This provides less stability and requires more coordination to keep the body upright, whereas using both feet gives a more stable base of support.	One-foot hopping has a smaller support area, requiring tighter control of the center of mass to remain balanced over the supporting foot.

Appendix

Table S7: Qualitative comparison on motion-knowledge QA and motion-language instruction generation. All questions are text-only and require no motion input.

Variant	R@1 $\uparrow$	R@2 $\uparrow$	R@3 $\uparrow$	FID $\downarrow$	MMDist $\downarrow$
Main-table model	0.8209	0.9397	0.9670	6.1291	13.9973
Multi-task model	0.8056	0.9221	0.9583	6.6746	14.0824

Appendix

Table S8: T2M performance of the main-table and multi-task 0.6B variants. The first row reproduces Table 1 ; the second uses the multi-task 20k EMA checkpoint. FID is reported in the evaluator’s raw embedding space, as in the main table. M2T scores are unchanged.

Motion Prediction
Method	MPJPE $\downarrow$	[P]-MPJPE $\downarrow$	FID $\downarrow$	R1 $\uparrow$	R2 $\uparrow$	R3 $\uparrow$	MM-D $\downarrow$
GT	0.000	0.000	0.000	0.448	0.646	0.767	3.466
CondMDI ( Cohan et al., 2024 )	11.439	5.882	3.213	0.383	0.565	0.667	3.927
Kimodo ( Rempe et al., 2026 )	8.781	1.368	1.480	0.337	0.503	0.616	4.532
HY-Motion (1B) ( Wen et al., 2025 )	9.372	0.856	3.185	0.401	0.592	0.701	4.148
MotionLab ( Guo et al., 2025 )	9.829	4.086	1.108	0.436	0.626	0.733	3.479

Appendix

Table S9: Motion temporal completion on HumanML3D. [P]-MPJPE measures the preservation error on conditioned frames. Lower MPJPE, [P]-MPJPE, FID, and MM-D are better, while higher R-Precision is better.

Human Motion Generation

Human motion generation creates articulated movement from language, audio, video, spatial constraints, or interaction goals.

See all 296 papers

UniMotion: A Unified Framework for Motion-Text-Vision Understanding and Generation
Mar 23, 2026Ziyi Wang, Xinshun Wang, Shuang Chen +2Cross-Modal Learning Unified Multimodal Models
State Key Laboratory of General Artificial Intelligence, Peking University, Shenzhen Graduate School, China · Donghua University, China · School of Automation Science and Engineering, South China University of Technology
MUGEN: A Unified Framework for Efficient Motion Understanding and Generation
Jul 30, 2026Zhankai Ye, Yukai Jin, Bingyang Wei +5Representation Learning Human Motion Generation
1Florida State University · 2Texas Christian University · University of Miami +1
UMO: Unified In-Context Learning Unlocks Motion Foundation Model Priors
Mar 16, 2026Xiaoyan Cong, Zekun Li, Zhiyang Dou +9Human Motion Generation Motion Diffusion Models
Brown University · Massachusetts Institute of Technology · Max-Planck Institute for Intelligent Systems +2

Mar 23, 2026J/K move · Enter open · S save

UniMotion: A Unified Framework for Motion-Text-Vision Understanding and Generation

Ziyi Wang, Xinshun Wang, Shuang Chen, Yang Cong+1

State Key Laboratory of General Artificial Intelligence, Peking University, Shenzhen Graduate School, China · Donghua University, China · School of Automation Science and Engineering, South China University of Technology

Cross-Modal Learning Unified Multimodal Models Cross-Modal Alignment Human Motion Generation Human Motion Understanding Multimodal Foundation Models+1

Open paper AskPDF arXiv

We present UniMotion, to our knowledge the first unified framework for simultaneous understanding and generation of human motion, natural language, and RGB images within a single architecture. Existing unified models handle only restricted modality subsets (e.g., Motion-Text or static Pose-Image) and predominantly rely on discrete tokenization, which introduces quantization errors and disrupts temporal continuity. UniMotion overcomes both limitations through a core principle: treating motion as a first-class continuous modality on equal footing with RGB. A novel Cross-Modal Aligned Motion VAE (CMA-VAE) and symmetric dual-path embedders construct parallel continuous pathways for Motion and RGB within a shared LLM backbone. To inject visual-semantic priors into motion representations without requiring images at inference, we propose Dual-Posterior KL Alignment (DPA), which distills a vision-fused encoder's richer posterior into the motion-only encoder. To address the cold-start problem -- where text supervision alone is too sparse to calibrate the newly introduced motion pathway -- we further propose Latent Reconstruction Alignment (LRA), a self-supervised pre-training strategy that uses dense motion latents as unambiguous conditions to co-calibrate the embedder, backbone, and flow head, establishing a stable motion-aware foundation for all downstream tasks. UniMotion achieves state-of-the-art performance across seven tasks spanning any-to-any understanding, generation, and editing among the three modalities, with especially strong advantages on cross-modal compositional tasks.

Abstract

Human Motion Generation

Explore similar work

UniMotion: A Unified Framework for Motion-Text-Vision Understanding and Generation

MUGEN: A Unified Framework for Efficient Motion Understanding and Generation

UMO: Unified In-Context Learning Unlocks Motion Foundation Model Priors