OmniLottie: Generating Vector Animations via Parameterized Lottie Tokens

Yiying Yang; Wei Cheng; Sijin Chen; Honghao Fu; Xianfang Zeng; Yujun Cai; Gang Yu; Xingjun Ma

cs.CVMar 2, 2026

OmniLottie: Generating Vector Animations via Parameterized Lottie Tokens

Authors: Yiying Yang, Wei Cheng, Sijin Chen, Honghao Fu, Xianfang Zeng, Yujun Cai, Gang Yu, Xingjun Ma

Organizations: Fudan University · StepFun · HKU MMLab · University of Queensland

Abstract

OmniLottie is a versatile framework that generates high quality vector animations from multi-modal instructions. For flexible motion and visual content control, we focus on Lottie, a light weight JSON formatting for both shapes and animation behaviors representation. However, the raw Lottie JSON files contain extensive invariant structural metadata and formatting tokens, posing significant challenges for learning vector animation generation. Therefore, we introduce a well designed Lottie tokenizer that transforms JSON files into structured sequences of commands and parameters representing shapes, animation functions and control parameters. Such tokenizer enables us to build OmniLottie upon pretrained vision language models to follow multi-modal interleaved instructions and generate high quality vector animations. To further advance research in vector animation generation, we curate MMLottie-2M, a large scale dataset of professionally designed vector animations paired with textual and visual annotations. With extensive experiments, we validate that OmniLottie can produce vivid and semantically aligned vector animations that adhere closely to multi modal human instructions.

Figures & tables

Figure 2: Overview of the Vector Animation Data Construction Pipeline. Data Collection & Conversion : We convert SVG assets into static Lottie files and apply randomized animation effects to generate effect-tagged animated Lotties. In parallel, we gather professionally created Lottie animations from five online platforms and perform thorough filtering and cleaning. Data Processing : Each animation undergoes spatio-temporal normalization, followed by video rendering and random keyframe extraction. Data Annotation : Finally, we provide multi-granularity annotations emphasizing geometric structure, color attributes, and motion characteristics.

Figure 3: Overview of OmniLottie . Lottie Structure : We reorganize the Lottie JSON representation, with a particular focus on the structure of its layers, including both common layer attributes and five special layer types. Lottie Parametrization : The hierarchical JSON format of Lottie is flattened into a sequence of function calls, which are further parameterized to define a dedicated vocabulary and token set for Lottie. OmniLottie : Built upon this parameterization, OmniLottie extends Qwen2.5-VL with a new tokenizer and vocabulary for Lottie, and is trained on our curated Lottie dataset.

Subset	Task	Methods	Time(s)	# Tokens	Success Rate	FVD $\downarrow$	CLIP $\uparrow$	Obj. $\uparrow$	Motion $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	DINO $\uparrow$
		DeepSeekV3	43.40	2.3k	9.3%	671.80	0.2677	1.51	2.09	-	-	-
		Qwen2.5-VL(3B)	27.97	0.5k	0.0%	-	-	-	-	-	-	-
		GPT-5	43.40	1.4k	12.7%	715.73	0.2600	0.73	0.71	-	-	-
		Recraft	-	54.1k	77.3%	300.70	0.2950	4.70	4.68	-	-	-
	Text-to-Lottie	Ours	33.71	21.2k	88.3%	202.14	0.2748	4.44	5.94	-	-	-
		Qwen2.5-VL(3B)	33.60	0.4k	0.0%	-	-	-	-	-	-	-

Table 1: Quantitative Evaluations. We provide a comprehensive quantitative comparison between Ours and baseline methods across both Real subset and Synthetic subset . The bold numbers and underlined numbers represents the best and second best performance repectively. Obj. and Motion stand for object alignment and motion alignment.

Figure 4: Qualitative comparison of Text-to-Lottie generation. Methods that failed to generate valid animations are omitted to ensure a clear comparison.

Figure 5: Qualitative comparison of Text-Image-to-Lottie generation. Methods that failed to generate valid animations are omitted to ensure a clear comparison.

Figure 6: Qualitative comparison of Video-to-Lottie generation. Methods that failed to generate valid animations are omitted to ensure a clear comparison.

Figure 7: Qualitative Ablation on Data Composition .

Training Data	Text-to-Lottie				Text-Image-to-Lottie
Training Data	FVD $\downarrow$	CLIP $\uparrow$	Obj.	Motion	FVD $\downarrow$	CLIP $\uparrow$	Obj.	Motion
1*	305.57	0.2573	1.85	4.92	405.39	0.2611	1.64	3.32
0.5 +0.5	285.22	0.2715	4.12	3.38	380.44	0.2642	3.91	3.25
1*	342.61	0.2308	4.02	2.35	441.83	0.2552	3.88	2.63
0.7* +0.3	269.50	0.2748	4.31	5.63	359.56	0.2666	4.10	3.44

Table 2: Ablation Study on SVG Data Mixing. We evaluate different combinations of Lottie and SVG training data. The optimal performance is achieved with 30% SVG data, which enhances geometric understanding while maintaining motion complexity. Obj. stands for object alignment.

Figure 8: Qualitative Ablation Study on Lottie Tokenizer .

Task	Setting	SR $\uparrow$	FVD $\downarrow$	CLIP $\uparrow$	Obj. $\uparrow$	Motion $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	DINO $\uparrow$
		0.00%	-	-	-	-	-	-	-
		13.4%	459.39	0.2600	1.33	1.82	-	-	-
T2Lottie		97.3%	269.50	0.2748	4.31	5.63	-	-	-
		0.00%	-	-	-	-	-	-	-
		15.9%	476.23	0.2400	1.27	0.98	-	-	-
TI2Lottie		92.0%	359.56	0.2666	4.10	3.44	-	-	-

Table 3: Ablation on Lottie Tokenizer . Comparing pretrained Qwen2.5-VL , fine-tuned on raw Lottie JSON (including SVG-Lottie data), and trained with our tokenizer . SR is the abbreviation of success rate, and Obj. stands for object alignment.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Additional Qualitative Results Across Three Generation Tasks . We present additional examples for Text-to-Lottie, Text-Image-to-Lottie, and Video-to-Lottie generation.

Failure Type	T2L	TI2L	V2L
Category I: Empty Layers	40.0%	30.0%	35.0%
Category II: Invalid Rendering
Missing Style	30.0%	35.0%	20.0%
Temporal Errors	10.0%	15.0%	25.0%
Opacity Issues	10.0%	10.0%	10.0%
Scale Collapse	5.0%	5.0%	5.0%

Appendix

Table 4: Failure Case Statistics. Distribution of failure types across different tasks.

Panel of Table 5 — Table 5: Participant Instructions and Evaluation Procedure for Lottie Animation Generation.

Metric vs. Human Rating	Pearson $r$ ( $p$ )	Spearman $\rho$ ( $p$ )	Kendall $\tau$ ( $p$ )
Object Align vs. Geometric Fidelity	0.4521 (3.21e-42)	0.4638 (8.45e-45)	0.3512 (2.18e-41)
Motion Align vs. Animation Quality	0.4823 (1.52e-48)	0.4917 (4.23e-51)	0.3724 (5.67e-47)
CLIP vs. Condition Adherence	0.3867 (6.78e-34)	0.3945 (2.14e-35)	0.2918 (1.43e-32)
FVD vs. Visual Quality	-0.3245 (4.89e-28)	-0.3312 (1.67e-29)	-0.2456 (3.21e-26)

Appendix

Table 6: Correlation Statistics Between Automated Metrics and Human Rankings. Reported values include Pearson’s

r

, Spearman’s

\rho

, and Kendall’s

\tau

with corresponding

p

-values. Higher absolute correlation values indicate stronger agreement between automated metrics and human judgments.

Figure 8: Parameter Distribution Analysis for Vocabulary Offset Design. We analyze the distribution of six key parameter types across our entire dataset to determine optimal vocabulary offset ranges. Each violin plot shows the probability density of parameter values, with solid horizontal lines indicating median values and dashed lines marking quartiles. The shaded regions highlight concentrated value ranges where the majority of parameters cluster, guiding our offset allocation strategy.

Figure 9: Token Efficiency Through Progressive Abstraction. Our Lottie tokenizer progressively reduces sequence length from raw JSON (2,562 tokens) through structured text sequences (884 tokens) to our final command-based representation (486 tokens), achieving 5.3× compression. The discretization operates purely at the representation level and does not compromise the vector nature of the output, as continuous parameter values are faithfully recovered during detokenization.

Panel of Table 7 — Table 7: Instructions for Different Tasks . Instructions including annotation, text-to-SVG, image-to-SVG and character-reference SVG generation.

Figure 10: Statistics of the MMLottie-2M Dataset . (a) Source distribution across five major online platforms. (b) Temporal statistics showing animation duration distribution and normalized frame counts. (c) Spatial statistics showing resolution distribution before normalization to $512\times 512$ .

Panel of Figure 11 — Figure 11: Vocabulary Analysis of MMLottie-2M Annotations. Word clouds visualizing the most frequent terms in our dataset captions. (a) Visual Elements : Dominant terms include geometric primitives (circle, rectangle, star), object types (icon, logo, character), and color descriptors (blue, red, white, green), confirming comprehensive coverage of visual attributes. (b) Motion Attributes : Prevalent motion verbs (rotate, scale, fade, move, slide, bounce) and directional modifiers (clockwise, left, right, up, down) demonstrate rich temporal semantics in our annotations.

Panel of Table 8 — Table 8: Object Alignment Evaluation Prompt for LLM-as-Judge.

Panel of Table 9 — Table 9: Motion Alignment Evaluation Prompt for LLM-as-Judge.

Attribute	Type	Title	Description
Base Layer Properties
nm	string	Name	Human-readable name
mn	string	Match Name	Used in expressions
ddd	0-1 int	3D	Whether layer is 3D
hd	boolean	Hidden	Whether layer is hidden
ty	integer	Type	Layer type (0:Precomp, 1:Solid, 2:Image, 3:Null, 4:Shape, 5:Text, 6:Audio, 13:Camera, 15:Data)

Appendix

Table 10: Properties of Lottie Layers .

Multimodal Generation

Multimodal generation creates coordinated text, images, audio, video, 3D, and other outputs from mixed inputs while preserving meaning and consistency.

See all 238 papers

LottieGPT: Tokenizing Vector Animation for Autoregressive Generation
Apr 13, 2026Junhao Chen, Kejun Gao, Yuehan Cui +8Vector Graphics Generation Visual Tokenization
Shenzhen International Graduate School, Tsinghua University · AIR, Tsinghua University · The Hong Kong Polytechnic University +3
VAnim: Rendering-Aware Sparse State Modeling for Structure-Preserving Vector Animation
May 2, 2026Guotao Liang, Zhangcheng Wang, Chuang Wang +6
School of Software, Beihang University, Beijing, China · College of Computer Science and Technology, Zhejiang University, Hangzhou, China · Paradigm 3 +1
LiveSVG: Zero-Shot SVG Animation via Video Generation
May 28, 2026Matan Levy, Ran Margolin, Bar Cavia +6Vector Graphics Generation
Google, Israel · The Hebrew University of Jerusalem, Israel · Bar-Ilan University, Israel +1

Apr 13, 2026J/K move · Enter open · S save

LottieGPT: Tokenizing Vector Animation for Autoregressive Generation

Junhao Chen, Kejun Gao, Yuehan Cui, Mingze Sun+7

Shenzhen International Graduate School, Tsinghua University · AIR, Tsinghua University · The Hong Kong Polytechnic University+3

Vector Graphics Generation Visual Tokenization Autoregressive Generation

Open paper AskPDF arXiv

Despite rapid progress in video generation, existing models are incapable of producing vector animation, a dominant and highly expressive form of multimedia on the Internet. Vector animations offer resolution-independence, compactness, semantic structure, and editable parametric motion representations, yet current generative models operate exclusively in raster space and thus cannot synthesize them. Meanwhile, recent advances in large multimodal models demonstrate strong capabilities in generating structured data such as slides, 3D meshes, LEGO sequences, and indoor layouts, suggesting that native vector animation generation may be achievable. In this work, we present the first framework for tokenizing and autoregressively generating vector animations. We adopt Lottie, a widely deployed JSON-based animation standard, and design a tailored Lottie Tokenizer that encodes layered geometric primitives, transforms, and keyframe-based motion into a compact and semantically aligned token sequence. To support large-scale training, we also construct LottieAnimation-660K, the largest and most diverse vector animation dataset to date, consisting of 660k real-world Lottie animation and 15M static Lottie image files curated from broad Internet sources. Building upon these components, we finetune Qwen-VL to create LottieGPT, a native multimodal model capable of generating coherent, editable vector animations directly from natural language or visual prompts. Experiments show that our tokenizer dramatically reduces sequence length while preserving structural fidelity, enabling effective autoregressive learning of dynamic vector content. LottieGPT exhibits strong generalization across diverse animation styles and outperforms previous state-of-the-art models on SVG generation (a special case of single-frame vector animation).

Abstract

Multimodal Generation

Explore similar work

LottieGPT: Tokenizing Vector Animation for Autoregressive Generation

VAnim: Rendering-Aware Sparse State Modeling for Structure-Preserving Vector Animation

LiveSVG: Zero-Shot SVG Animation via Video Generation