ProjFlow: Projection Sampling with Flow Matching for Zero-Shot Exact Spatial Motion Control

Akihisa Watanabe; Qing Yu; Edgar Simo-Serra; Kent Fujiwara

cs.CVFeb 26, 2026

ProjFlow: Projection Sampling with Flow Matching for Zero-Shot Exact Spatial Motion Control

Authors: Akihisa Watanabe, Qing Yu, Edgar Simo-Serra, Kent Fujiwara

Organizations: Waseda University · LY Corporation

Abstract

Generating human motion with precise spatial control is a challenging problem. Existing approaches often require task-specific training or slow optimization, and enforcing hard constraints frequently disrupts motion naturalness. Building on the observation that many animation tasks can be formulated as a linear inverse problem, we introduce ProjFlow, a training-free sampler that achieves zero-shot, exact satisfaction of linear spatial constraints while preserving motion realism. Our key advance is a novel kinematics-aware metric that encodes skeletal topology. This metric allows the sampler to enforce hard constraints by distributing corrections coherently across the entire skeleton, avoiding the unnatural artifacts of naive projection. Furthermore, for sparse inputs, such as filling in long gaps between a few keyframes, we introduce a time-varying formulation using pseudo-observations that fade during sampling. Extensive experiments on representative applications, motion inpainting, and 2D-to-3D lifting, demonstrate that ProjFlow achieves exact constraint satisfaction and matches or improves realism over zero-shot baselines, while remaining competitive with training-based controllers.

Figures & tables

Figure 2: Overview of the Projection Sampling Step. At each timestep $t$ : (1) predict the clean endpoint $\hat{{\bm{x}}}_{1}$ from ${\bm{x}}_{t}$ using the learned velocity $v_{\theta}({\bm{x}}_{t},t)$ ; (2) enforce the linear–Gaussian measurements ${\bm{y}}=A{\bm{x}}+{\bm{\epsilon}}$ by computing a correction $\Delta{\bm{x}}_{1}^{\star}$ that projects $\hat{{\bm{x}}}_{1}$ to the measurement set under the kinematics-aware metric $R$ . This metric encodes skeletal topology and spreads updates coherently along the kinematic tree. The measurement covariance $\Sigma$ modulates the pull toward the observations; smaller values yield stronger attraction and recover hard constraints as $\Sigma\to 0$ . (3) Finally, stochastically recompose the corrected endpoint to obtain the next state ${\bm{x}}_{t+\Delta t}$ .

Figure 3: Pseudo-observations for motion inpainting. Sparse observations are interpolated to guide intermediate frames. This guidance is controlled by two mechanisms: Dynamic Masking activates a time-scheduled neighborhood, and Adaptive Variance treats original observations as hard constraints and the interpolated guides as soft constraints.

Figure 4: Text-conditioned pelvis-trajectory control. Given the prompt “ a person runs forward in an S-shaped path ” and a pelvis control signal, we compare OmniControl [ 63 ] , MaskControl [ 45 ] , and ProjFlow (ours). The rendered motions and the trajectory plots both visualize the generated pelvis trajectory ( orange ) overlaid on the target control signal ( gray dotted line ).

Controlling Joint	Methods	Zero-shot?	FID $\downarrow$	R-Precision	Diversity $\rightarrow$	Foot Skating	Traj. err. $\downarrow$	Loc. err. $\downarrow$	Avg. err. $\downarrow$
Controlling Joint	Methods	Zero-shot?	FID $\downarrow$	Top 3	Diversity $\rightarrow$	Ratio. $\downarrow$	Traj. err. $\downarrow$	Loc. err. $\downarrow$	Avg. err. $\downarrow$
	GT	-	$0.000$	$0.795$	$10.455$	-	$0.000$	$0.000$	$0.000$
Pelvis	MDM [ 57 ]	✓	$1.792$	$0.673$	$9.131$	$0.1019$	$0.4022$	$0.3076$	$0.5959$
	PriorMDM [ 51 ]	✗	$0.393$	$0.707$	$9.847$	$0.0897$	$0.3457$	$0.2132$	$0.4417$
	GMD [ 21 ]	✓	$0.238$	$0.763$	$10.011$	$0.1009$	$0.0931$	$0.0321$	$0.1439$
	OmniControl [ 63 ]	✗	$0.081$	$0.789$	$10.323$	$\underline{0.0547}$	$0.0387$	$0.0096$	$0.0338$

Table 1: Quantitative text-conditioned motion generation with spatial control signals and upper-body editing on HumanML3D [ 15 ] . In the first section, methods are trained and evaluated solely on pelvis controls. In the middle section, methods are trained on all joints and evaluated separately on each controlled joint. Only average results are reported for brevity. We include details in the supplementary material. The last section presents upper-body editing results. bold face / underline indicates the best/2 nd results.

Figure 5: 2D-to-3D hand-trajectory lifting with text conditioning. The input condition includes the text prompt “ a person draws a heart with their hand while walking ,” an initial 2D keypose, and a left-wrist 2D trajectory shaped like a heart. Sketch2Anim [ 68 ] fails to reproduce the heart path precisely, the shape collapses, and the subject does not exhibit walking motion. In contrast, ProjFlow follows the heart-shaped wrist trajectory accurately while maintaining a natural walking motion throughout the sequence.

Condition	Method	Realism		Control Accuracy				Text-Motion Matching
Condition	Method	FID $\downarrow$	Foot Skating $\downarrow$	MPJPE-2D $\downarrow$	MPJPE-3D $\downarrow$	Avg. Err.-2D $\downarrow$	Avg. Err.-3D $\downarrow$	Matching $\downarrow$	R-precision (Top-3) $\uparrow$
Average	Motion Retrieval	0.690	0.064	0.057	0.076	0.290	0.410	4.060	0.640
	Lift-and-Control	0.979	0.089	0.054	0.071	0.261	0.340	3.297	0.752
	Direct 2D-to-Motion	2.553	0.112	0.040	0.055	0.193	0.275	3.723	0.687
	Sketch2Anim [ 68 ]	0.525	0.103	0.036	0.048	0.087	0.134	3.077	0.802
	ACMDM-S-PS22+ProjFlow	0.349	0.146	0.000	0.042	0.000	0.331	3.363	0.748

Table 2: Quantitative analysis of ProjFlow and three baseline models proposed in Sketch2Anim [ 68 ] on the HumanML3D [ 15 ] . Evaluation metrics on motion realism, control accuracy, and text-motion match are presented. Following OmniControl [ 63 ] , we report both the average error of all joints (Average) and their random combination (Cross). bold face / underline indicates the best/2 nd results.

Variant	FID $\downarrow$	R-Prec.	Div. $\rightarrow$	Foot $\downarrow$	Traj. $\downarrow$	Loc. $\downarrow$	Avg. $\downarrow$
ProjFlow (Full)	0.097	0.779	10.651	0.0603	0.0000	0.0000	0.0000
Euclid. ( $R{=}I$ )	1.152	0.740	10.107	0.0595	0.0000	0.0000	0.0000
No noise ( $\eta_{t}{=}0$ )	3.429	0.707	9.307	0.0863	0.0000	0.0000	0.0000
Plain masking	0.880	0.748	10.187	0.0632	0.0000	0.0000	0.0000

Table 3: Ablation studies of ProjFlow.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Symbol	Type / shape	Note
$A$	$\mathbb{R}^{m\times d}$	linear operator
${\bm{y}}$	$\mathbb{R}^{m}$	measurements
${\bm{x}}_{1}$	$\mathbb{R}^{d}$	clean motion endpoint
$\hat{{\bm{x}}}_{1}$	$\mathbb{R}^{d}$	estimate of ${\bm{x}}_{1}$
$\Sigma$	$\mathbb{R}^{m\times m}$	PSD covariance
$R$	$\mathbb{R}^{d\times d}$	PD metric / precision

Appendix

Table 4: Notation used in the supplementary derivations.

Block	Name	Symbol	Value
Kinematics-aware metric	joint coupling weight	$w_{\text{kin}}$	10.0
Kinematics-aware metric	ridge	$\lambda$	1.0
Dynamic Masking	min radius (frames)	$l_{\min}$	3
Dynamic Masking	max radius (frames)	$l_{\max}$	10
Adaptive Variance	time base	$\tau_{\min}$	0.1
	strength	$c_{0}$	3.0

Appendix

Table 5: Hyperparameters for motion inpainting.

Figure 6: Average inference time per 196-frame sample .

Figure 7: Ablation of ProjFlow components vs. control intensity on motion inpainting. We compare the full model ( Full ) against variants that (i) replace the kinematics-aware metric with a Euclidean metric ( Euclid ), (ii) remove the stochastic noise-mixing step ( Without Noise ), or (iii) disable pseudo-observations and rely only on hard keyframes ( Plain Masking ).

Controlling Joint	Methods	FID $\downarrow$	R-Precision	Diversity $\rightarrow$	Foot Skating	Traj. err. $\downarrow$	Loc. err. $\downarrow$	Avg. err. $\downarrow$
Controlling Joint	Methods	FID $\downarrow$	Top 3	Diversity $\rightarrow$	Ratio. $\downarrow$	Traj. err. $\downarrow$	Loc. err. $\downarrow$	Avg. err. $\downarrow$
	GT	$0.000$	$0.795$	$10.455$	-	$0.0000$	$0.0000$	$0.0000$
Pelvis	ACMDM - S - PS22+ProjFlow (Full)	0.107	0.784	10.645	0.0630	0.0000	0.0000	0.0000
	ACMDM - S - PS22+ProjFlow (Euclid)	4.360	0.686	8.953	0.0550	0.0000	0.0000	0.0000
	ACMDM - S - PS22+ProjFlow (w/o noise)	2.439	0.734	9.666	0.0960	0.0000	0.0000	0.0000
	ACMDM - S - PS22+ProjFlow (Plain Masking)	2.091	0.726	9.838	0.0658	0.0000	0.0000	0.0000

Appendix

Table 6: Ablation of ACMDM-S-PS22+ProjFlow on HumanML3D. Methods are evaluated on all joints and reported per controlled joint. bold face / underline indicates the best/2 nd results if applied.

Controlling Joint	Methods	Zero-shot?	FID $\downarrow$	R-Precision	Diversity $\rightarrow$	Foot Skating	Traj. err. $\downarrow$	Loc. err. $\downarrow$	Avg. err. $\downarrow$
Controlling Joint	Methods	Zero-shot?	FID $\downarrow$	Top 3	Diversity $\rightarrow$	Ratio. $\downarrow$	Traj. err. $\downarrow$	Loc. err. $\downarrow$	Avg. err. $\downarrow$
	GT	$-$	$0.000$	$0.795$	$10.455$	-	$0.000$	$0.000$	$0.000$
Train On Pelvis	MDM [ 57 ]	✓	$1.792$	$0.673$	$9.131$	$0.1019$	$0.4022$	$0.3076$	$0.5959$
	PriorMDM [ 51 ]	✗	$0.393$	$0.707$	$9.847$	$0.0897$	$0.3457$	$0.2132$	$0.4417$
	GMD [ 21 ]	✓	$0.238$	$0.763$	$10.011$	$0.1009$	$0.0931$	$0.0321$	$0.1439$
	OmniContol [ 63 ]	✗	$0.081$	$0.789$	$10.323$	$\underline{0.0547}$	$0.0387$	$0.0096$	$0.0338$

Appendix

Table 7: Quantitative text-conditioned motion generation with spatial control signals and upper-body editing on HumanML3D. In the first section, methods are trained and evaluated solely on pelvis controls. In the middle section, methods are trained on all joints and evaluated separately on each controlled joint. bold face / underline indicates the best/2 nd results.

Controlling Joint	Methods	Zero-shot?	FID $\downarrow$	R-Precision	Diversity $\rightarrow$	Foot Skating	Traj. err. $\downarrow$	Loc. err. $\downarrow$	Avg. err. $\downarrow$
Controlling Joint	Methods	Zero-shot?	FID $\downarrow$	Top 3	Diversity $\rightarrow$	Ratio. $\downarrow$	Traj. err. $\downarrow$	Loc. err. $\downarrow$	Avg. err. $\downarrow$
	GT	-	$0.002$	$0.797$	$9.503$	-	$0.000$	$0.000$	$0.000$
Train On Pelvis	MDM [ 57 ]	✓	$0.698$	$0.602$	$9.197$	$0.1019$	$40.22$	$30.76$	$59.59$
	PriorMDM [ 51 ]	✗	$0.475$	$0.583$	$9.156$	$0.0897$	$34.57$	$21.32$	$44.17$
	GMD [ 21 ]	✓	$0.576$	$0.665$	$9.206$	$0.1009$	$9.31$	$3.21$	$14.39$
	OmniControl [ 63 ]	✗	$0.218$	$0.687$	$9.422$	$\mathbf{0.0547}$	$\underline{3.87}$	$\underline{0.96}$	$3.38$

Appendix

Table 8: Quantitative text-conditioned motion generation with spatial control signals and upper-body editing on HumanML3D [ 15 ] . The first section covers pelvis-only control; the middle section shows the average for all joints. The last section presents upper-body editing results. bold face / underline indicates the best/2 nd results.

Human Motion Generation

Human motion generation creates articulated movement from language, audio, video, spatial constraints, or interaction goals.

See all 291 papers

MotionSpaceFlow: Representation-Aware Flow Matching in Direct Motion Space
Sep 28, 2026Qing Yu, Kent FujiwaraFlow Matching Text-to-Motion Generation
LY Corporation, Tokyo, Japan
Constraint-Aware Flow Matching: Decision Aligned End-to-End Training for Constrained Sampling
May 12, 2026Jacob K. Christopher, James E. Warner, Ferdinando FiorettoConstrained Generative Modeling Flow Matching
University of Virginia · NASA Langley Research Center
Training-free Controllable Human Motion Generation under Heterogeneous Constraints
Jul 2, 2026Xiaofei Hui, Bo Yan, Haoxuan Qu +2Constrained Generative Modeling Human Motion Generation
Lancaster University, United Kingdom · Shenzhen International Graduate School, Tsinghua University, China

Sep 28, 2026J/K move · Enter open · S save

MotionSpaceFlow: Representation-Aware Flow Matching in Direct Motion Space

Qing Yu, Kent Fujiwara

LY Corporation, Tokyo, Japan

Flow Matching Text-to-Motion Generation Human Motion Generation Flow-Based Generative Modeling Motion Representation Learning Diffusion Models for Motion Generation

Open paper AskPDF arXiv

Recent advances in diffusion and flow models have substantially improved text-driven human motion generation. Yet most methods generate in low-dimensional, temporally downsampled latent spaces learned primarily for reconstruction, a bottleneck that can limit generation quality and preclude direct manipulation of individual frames and joints. We introduce MotionSpaceFlow (MSFlow), a representation-aware flow-matching framework that predicts clean motion directly in continuous motion space without a learned encoder or decoder. To account for the anisotropic structure of direct motion representations, we propose representation-aware noise scaling and show how the initial Gaussian source scale governs the covariance of intermediate probability-path marginals. We further introduce a Representation-Aware Multimodal Diffusion Transformer (RA-MMDiT), which jointly updates token-level language and full-resolution motion features through joint attention while adapting temporal information flow to the motion representation: causal attention for incremental features defined by frame-to-frame changes, and bidirectional attention for global features such as absolute joint coordinates. Across different datasets and motion representations, MSFlow achieves state-of-the-art text-to-motion performance. Its global representation variant additionally enables zero-shot, inference-time control over any joint or frame through projection sampling without control-conditioned training, delivering leading motion quality with exact constraint satisfaction.

Abstract

Human Motion Generation

Explore similar work

MotionSpaceFlow: Representation-Aware Flow Matching in Direct Motion Space

Constraint-Aware Flow Matching: Decision Aligned End-to-End Training for Constrained Sampling

Training-free Controllable Human Motion Generation under Heterogeneous Constraints