Self-consistency has proven to be an effective technique for improving LLM performance on natural language reasoning tasks in a lightweight, unsupervised manner. In this work, we study how to adapt self-consistency to visual domains. Specifically, we consider the generation and verification of LLM-produced motion graphics trajectories. Given a prompt (e.g., "Move the circle in a spiral path"), we first sample diverse motion trajectories from an LLM, and then identify groups of consistent trajectories via clustering. Our key insight is to model the family of shapes associated with a prompt as a prototype trajectory paired with a group of geometric transformations (e.g., rigid, similarity, and affine). Two trajectories can then be considered consistent if one can be transformed into the other under the warps allowable by the transformation group. We propose an algorithm that automatically recovers a shape family, using hierarchical relationships between a set of candidate transformation groups. Our approach improves the accuracy of LLM-based trajectory generation by 4-6%. We further extend our method to support verification, observing 11% precision gains over VLM baselines. Our code and dataset are available at https://majiaju.io/trajectory-self-consistency .
Figures & tables
Figure 2: We define a shape family F(o,W) with a prototype trajectory o and a Lie transformation group W . The shape family is then the set of all trajectories that can be warped into one another w(o) by all warps w∈W (left). In this example, any of the figure 8’s in the family can serve as the prototype since they can all be warped into one another. We can check whether a trajectory ti is a member of the family by checking if there is a warp w∈W that can transform ti into o (right).
Figure 3: Hierarchy of Lie transformation groups and shape families they induce. Each node represents a transformation group and depicts a prototype square-shaped trajectory o as well as other trajectories w(o) within the corresponding shape family.
Figure 4: We present clustering results on LLM-generated samples from three example prompts in our dataset. Each trajectory has a ground truth label on the upper right. Clusters colored green are the chosen largest clusters, and we note that their sizes vary, sometimes less than half of the total number of trajectories (middle). In the pentagon case, failed samples (Cluster 2–4) appear visually distinct from the true ones, while some of the failed deltoids (Cluster 4–8) look closer to the correct one with their triangular forms. The rightmost parabola is a skewed one that would have been grouped into the largest cluster under the affine warp.
Method
Decision Criteria
GPT-4.1
GPT-5
LLM-Direct
—
62.1
79.1
Ours
Majority-Consensus
68.0
83.3
Ours
Hierarchical-Consistency
66.7
82.6
Ours
Oracle
68.5
83.5
Table 1: Motion trajectory generation experiment (see Sec. 5.2 ). We evaluate the accuracy of LLM-generated motion trajectories (GPT-4.1 and GPT-5) under different decoding strategies. Under different decision criteria for selecting a transformation group W , our self-consistency approach improves upon the baseline alternative of directly generating a single sample (LLM-Direct).
Method
Decision Criteria
Precision
Recall
F1 Score
GPT-4.1
—
62.0
96.9
75.6
GPT-5
—
74.0
84.7
79.0
Ours
Majority-Consensus
85.8
66.1
74.6
Ours
Hierarchical-Consistency
80.5
89.0
84.6
Ours
Oracle
87.9
83.3
85.6
Table 2: Motion trajectory verification experiment (see Sec. 5.3 ). We compare our self-consistency-based approach for verifying whether a trajectory matches an input prompt against the alternative of using a VLM (GPT-4.1 or GPT-5).
Figure 5: F1 scores across different numbers of sampled trajectories N . Decision criteria performances stabilize after N=10 .
Appendix figures & tables18 assets
Supplementary material from the paper’s appendix.
Appendix
Wrgd-ref
Wrgd
Wsim
Wsim-ref
Wsim-ani
Waff
Total
105
44
38
23
12
2
224
Appendix
Table S1 : Number of prompts per warp in our benchmark.
Method
Decision Criteria
GPT-4.1
GPT-5
LLM-Direct
—
62.1
79.1
Ours
Majority-Consensus
68.0
83.3
Ours
Most-Restrictive
66.5
82.5
Ours
Hierarchical-Consistency
66.7
82.6
Ours
Least-Restrictive
64.6
82.1
Ours
Oracle
68.5
83.5
Appendix
Table S2 : Motion trajectory generation experiment with two additional baseline decision criteria of always selecting the most or least restrictive warps. We observe that they perform strictly worse than the two methods proposed in the main paper.
Method
Decision Criteria
Precision
Recall
F1 Score
GPT-4.1
—
62.0
96.9
75.6
GPT-5
—
74.0
84.7
79.0
Ours
Majority-Consensus
85.8
66.1
74.6
Ours
Most-Restrictive
86.1
42.1
56.5
Ours
Hierarchical-Consistency
80.5
89.0
84.6
Ours
Least-Restrictive
65.0
91.8
76.1
Appendix
Table S3 : Motion trajectory verification experiment (Table 2 of the main paper) with additional decision criteria of always selecting the most or least restrictive warps.
Figure S2 : We demonstrate a limitation of our self-consistency approach in terms of the definition of a shape family. In text prompt 1, “heptagram” has two distinct prototypes: heptagram {7/2} and heptagram {7/3}. Under the warp Wsim-ref , cluster 1 contains correct {7/3} trajectories, while cluster 2 has {7/2}. While, for generation, our approach would produce a correct interpretation of heptagram, we would mark all heptagram {7/2} as false for verification. Similarly, for the letter Ms in text prompt 2, our approach would generate a correct form of the letter, but verify all other letter Ms that are not an affine warp of the shapes in cluster 1 as false.
Figure S3 : When a shape is aligned with the screen space’s coordinate frame, applying an anisotropic scale of 2 along the x-axis does not introduce skews to the shapes (top). However, once a shape is rotated out of alignment, non-uniform scales effectively skew the shapes out of their respective shape families (bottom).
Decision Criteria
More Restrictive
Match
Less Restrictive
Majority-Consensus
48.2%
49.6%
1.8%
Hierarchical-Consistency
4.0%
72.3%
22.3%
Appendix
Table S4 : Our decision criteria has different failure modes when not selecting the ground truth warp. Majority-Consensus selects more restrictive distance metrics for 108 prompts (95.6% of the time when it is not matching with the ground truth), while Hierarchical-Consistency is more conservative for 50 prompts (80.6% of the time). Note that both decision criteria choose distance metrics that are on the same level of the hierarchy but different from the ground truth for 1 and 3 prompts respectively.
Figure S4 : We demonstrate a case where Hierarchical-Consistency chooses the correct warp but Majority-Consensus fails by selecting a more restrictive warp. The prompt (top) asks for a rhombus-shaped path. We show clustering results of all warps in our hierarchy, with largest clusters under each warp highlighted in green. Each trajectory has a ground truth correctness label on the upper right.
Figure S5 : An example case where Majority-Consensus correctly chooses the ground truth warp but Hierarchical-Consistency fails by selecting a less restrictive warp. The prompt (top) asks for a path shaped like an equilateral triangle traversed in a counterclockwise manner. We show clustering results of all warps in our hierarchy.
Figure S6 : More results of applying our self-consistency approach with the oracle warps. The top half shows two success cases where the largest clusters do correspond to the correct shape families (figure 8 and 4-petal rose). The bottom half shows two failure cases where the LLM did not produce more members of the ground truth shape families than non-members (pentagram outline and reuleaux triangle).
τ=0.25
τ=0.5
τ=1.0
τ=2.0
τ=4.0
τ=8.0
F1 score
84.0
84.6
83.8
83.1
80.3
76.8
Clusters (avg ± std)
4.0 ± 4.9
3.7 ± 4.8
3.5 ± 4.5
3.1 ± 4.0
2.6 ± 3.1
2.1 ± 2.1
Appendix
Table S5 : F1 scores of verification by Hierarchical-Consistency and the average number of DBSCAN clusters across the distance threshold τ .
Figure S7 : We prompt a VLM with a scatter plot representation of a motion trajectory. Each dot is the center position of the animated element and the color indicates time (transition from blue to red).
Figure S8 : Motion trajectory benchmark: prompt 1–36 with ground truth shape families.
Figure S9 : Motion trajectory benchmark: prompt 37–72 with ground truth shape families.
Figure S10 : Motion trajectory benchmark: prompt 73–108 with ground truth shape families.
Figure S11 : Motion trajectory benchmark: prompt 109–144 with ground truth shape families.
Figure S12 : Motion trajectory benchmark: prompt 145–180 with ground truth shape families.
Figure S13 : Motion trajectory benchmark: prompt 181–216 with ground truth shape families.
Figure S14 : Motion trajectory benchmark: prompt 217–224 with ground truth shape families.
Video generation has achieved remarkable progress in visual fidelity and controllability, enabling conditioning on text, layout, or motion. Among these, motion control - specifying object dynamics and camera trajectories - is essential for composing complex, cinematic scenes, yet existing interfaces remain limited. We introduce LAMP that leverages large language models (LLMs) as motion planners to translate natural language descriptions into explicit 3D trajectories for dynamic objects and (relatively defined) cameras. LAMP defines a motion domain-specific language (DSL), inspired by cinematography conventions. By harnessing program synthesis capabilities of LLMs, LAMP generates structured motion programs from natural language, which are deterministically mapped to 3D trajectories. We construct a large-scale procedural dataset pairing natural text descriptions with corresponding motion programs and 3D trajectories. Experiments demonstrate LAMP's improved performance in motion controllability and alignment with user intent compared to state-of-the-art alternatives establishing the first framework for generating both object and camera motions directly from natural language specifications. Code, models and data are available on our project page.
Muhammed Burak Kizil, Enes Sanli, Niloy J. Mitra +3
Koç University · University College London · Adobe Research +1
Recent advances in motion-aware large language models have shown remarkable promise for jointly learning motion understanding and generation knowledge. However, these models typically treat understanding and generation separately, limiting the mutual benefits that could arise from interactive feedback between tasks. In this work, we reveal that motion assessment and refinement tasks can act as crucial bridges to enable knowledge flow from motion understanding to generation. Specifically, we propose Interleaved Reasoning for Motion Generation (IRMoGen), a novel paradigm that tightly couples motion generation with assessment and refinement through iterative text-motion dialogue. To realize this, we introduce IRG-MotionLLM, the first model that seamlessly interleaves motion generation, assessment, and refinement to improve the alignment between generated motion and goal text. IRG-MotionLLM is developed progressively with a novel three-stage training scheme, initializing and subsequently enhancing native IRMoGen capabilities. To facilitate this development, we construct an automated data engine to synthesize interleaved reasoning annotations from existing text-motion datasets. Extensive experiments demonstrate the properties brought by IRMoGen training, and the advanced cross-benchmark and cross-evaluator performance of IRG-MotionLLM. Code and models are available at https://github.com/HumanMLLM/IRG-MotionLLM.
Yuan-Ming Li, Qize Yang, Nan Lei +5
Sun Yat-sen University · Tongyi Lab, Alibaba Group · Key Laboratory of Machine Intelligence and Advanced Computing, Ministry of Education, China +1
Text-to-motion (T2M) generation aims to control the behavior of a target character via textual descriptions. Leveraging text-motion paired datasets, existing T2M models have achieved impressive performance in generating high-quality motions within the distribution of their training data. However, their performance deteriorates notably when the motion descriptions differ significantly from the training texts. To address this issue, we propose Re2MoGen, a Reasoning and Refinement open-vocabulary Motion Generation framework that leverages enhanced Large Language Model (LLM) reasoning to generate an initial motion planning and then refine its physical plausibility via reinforcement learning (RL) post-training. Specifically, Re2MoGen consists of three stages: We first employ Monte Carlo tree search to enhance the LLM's reasoning ability in generating reasonable keyframes of the motion based on text prompts, specifying only the root and several key joints' positions to ease the reasoning process. Then, we apply a human pose model as a prior to optimize the full-body poses based on the planned keyframes and use the resulting incomplete motion to supervise fine-tuning a pre-trained motion generator via a dynamic temporal matching objective, enabling spatiotemporal completion. Finally, we use post-training with physics-aware reward to refine motion quality to eliminate physical implausibility in LLM-planned motions. Extensive experiments demonstrate that our framework can generate semantically consistent and physically plausible motions and achieve state-of-the-art performance in open-vocabulary motion generation.
Jiakun Zheng, Ting Xiao, Shiqin Cao +3
1East China University of Science and Technology · 2Hong Kong University of Science and Technology · Institute of Artificial Intelligence (TeleAI), China Telecom