Large-scale Codec Avatars: The Unreasonable Effectiveness of Large-scale Avatar Pretraining

Junxuan Li; Rawal Khirodkar; Chengan He; Zhongshi Jiang; Giljoo Nam; Lingchen Yang; Jihyun Lee; Egor Zakharov

cs.CVApr 2, 2026

Large-scale Codec Avatars: The Unreasonable Effectiveness of Large-scale Avatar Pretraining

Authors: Junxuan Li, Rawal Khirodkar, Chengan He, Zhongshi Jiang, Giljoo Nam, Lingchen Yang, Jihyun Lee, Egor Zakharov, +32 more

Organizations: Codec Avatars Lab, Meta

Abstract

High-quality 3D avatar modeling faces a critical trade-off between fidelity and generalization. On the one hand, multi-view studio data enables high-fidelity modeling of humans with precise control over expressions and poses, but it struggles to generalize to real-world data due to limited scale and the domain gap between the studio environment and the real world. On the other hand, recent large-scale avatar models trained on millions of in-the-wild samples show promise for generalization across a wide range of identities, yet the resulting avatars are often of low-quality due to inherent 3D ambiguities. To address this, we present Large-Scale Codec Avatars (LCA), a high-fidelity, full-body 3D avatar model that generalizes to world-scale populations in a feedforward manner, enabling efficient inference. Inspired by the success of large language models and vision foundation models, we present, for the first time, a pre/post-training paradigm for 3D avatar modeling at scale: we pretrain on 1M in-the-wild videos to learn broad priors over appearance and geometry, then post-train on high-quality curated data to enhance expressivity and fidelity. LCA generalizes across hair styles, clothing, and demographics while providing precise, fine-grained facial expressions and finger-level articulation control, with strong identity preservation. Notably, we observe emergent generalization to relightability and loose garment support to unconstrained inputs, and zero-shot robustness to stylized imagery, despite the absence of direct supervision.

Figures & tables

Figure 2: ( Left ) Overview. Given multiple images of a subject, we extract image tokens from full-body images and face crops, and geometric tokens from a template mesh. The LCA encoder alternates image-only, geometry-only, and multimodal attention to fuse information across streams. Our decoders, canonical and pose-dependent, predict Gaussian attributes, which are skinned via linear blend skinning (LBS) and rendered to novel views. Training uses photometric reconstruction losses. ( Right ) Pretraining vs. Post-Training. LCA pretrains on large-scale, unconstrained monocular videos of single subjects with mixed (mid/low) quality, then post-trains on high-quality, multi-view studio captures. Pretraining drives broad generalization whereas post-training improves fidelity and 3D completeness.

Figure 3: Node-Based Deformation Model. We use a flexible two-level learnable deformation model to adapt skinning weights learning for post-training.

Figure 4: Pretraining vs. Post-Training. Qualitative comparison of models trained on multiple data sources and training strategies.

Train-Source	Capture-Studio			In-the-Wild
Train-Source	L1 $\downarrow$	LPIPS $\downarrow$	PSNR $\uparrow$	L1 $\downarrow$	LPIPS $\downarrow$	PSNR $\uparrow$
In-the-Wild	0.0117	0.0904	27.464	0.0099	0.0507	27.669
Capture-Studio	0.0091	0.0766	29.741	0.0128	0.0706	26.632
Mixed	0.0087	0.0724	30.025	0.0099	0.0519	27.998
Pre $\rightarrow$ Post (Ours)	0.0082	0.0688	30.514	0.0096	0.0491	28.175

Table 1: Effect of training schemes evaluated across domains.

(a) Multi-view
Method	Capture-Studio			In-the-Wild
Method	L1 $\downarrow$	LPIPS $\downarrow$	PSNR $\uparrow$	L1 $\downarrow$	LPIPS $\downarrow$	PSNR $\uparrow$
ExAvatar [ 44 ]	0.011	0.065	23.925	0.033	0.120	18.010
UP2You [ 9 ]	0.021	0.085	19.585	0.025	0.086	19.062
MV-LHM* [ 50 ]	0.008	0.043	26.606	0.009	0.040	25.796
LCA (Ours)	0.007	0.041	27.483	0.008	0.029	27.802

Table 2: Quantitative comparison with state-of-the-art 3D avatar methods. * denotes methods trained by us for multi-view inputs.

Figure 5: Qualitative Comparison with State-of-the-Art Methods. LCA outperforms in both multi-view and monocular settings.

Figure 6: Attention Map Visualization. Post-training yields cleaner semantic correspondences in last-layer attention maps between geometric and image tokens. The selected geometric tokens on the mesh are shown in red.

# Videos	Capture-Studio			In-the-Wild
# Videos	L1 $\downarrow$	LPIPS $\downarrow$	PSNR $\uparrow$	L1 $\downarrow$	LPIPS $\downarrow$	PSNR $\uparrow$
$10$ K	0.0083	0.0681	30.393	0.0109	0.0536	27.665
$100$ K	0.0083	0.0677	30.457	0.0108	0.0533	27.755
$1$ M	0.0082	0.0688	30.514	0.0096	0.0491	28.175

Table 3: Effect of scaling pretraining data on downstream performance across data distributions.

Figure 7: Loose Garment Support (LGS). ( Left ) Frontal view of the input condition. ( Middle ) Post-trained LCA avatar without loose garment support, while the general shape is recovered, skirts behave like pants when moving. ( Right ) LCA with loose garment support produces plausible animations without splitting garments.

Panel of Figure 8 — Figure 8: Relightable LCA. We demonstrate relighting under HDRI environment maps and point lights, alongside the recovered intrinsic properties of the avatar.

Figure S1 : PCA of Geometric Token Features. Visualization of the feature space distributions produced by models trained with different strategies. Green points denote studio-captured subjects, while red points denote in-the-wild subjects.

Decoder Architecture
Configuration	Capture-Studio			In-the-Wild
Configuration	L1 $\downarrow$	LPIPS $\downarrow$	PSNR $\uparrow$	L1 $\downarrow$	LPIPS $\downarrow$	PSNR $\uparrow$
Single-Branch	0.0088	0.0742	30.370	0.0103	0.0516	27.936
Dual-Branch (Ours)	0.0082	0.0688	30.514	0.0096	0.0491	28.175
Post-Training Learning Rate Decay ( $\gamma$ )
$\gamma=0.00$	0.0117	0.0904	27.464	0.0096	0.0507	27.669

Table S1 : Ablation study on decoder architecture and post-training learning rate decay. Our dual-branch residual design outperforms a single-branch variant. The learning rate decay is critical for preserving pretraining knowledge, with

\gamma{=}0.00

(no decay) severely degrading studio performance.

Train Data Size Num. Identities	Capture-Studio			In-the-Wild
Train Data Size Num. Identities	L1 $\downarrow$	LPIPS $\downarrow$	PSNR $\uparrow$	L1 $\downarrow$	LPIPS $\downarrow$	PSNR $\uparrow$
Pre-100K + Post-500	0.0088	0.0723	30.398	0.0130	0.0606	26.754
Pre-1M + Post-2.7K (Ours)	0.0082	0.0688	30.514	0.0096	0.0491	28.175

Table S2 : Effect of training data scale. Pre/post-training benefits persist even at

10\times

smaller scale (100K pretraining identities, 500 post-training identities), with consistent trends across both domains.

Method	L1 $\downarrow$	LPIPS $\downarrow$	PSNR $\uparrow$
No Deformer	0.0304	0.2452	24.538
LCA (Ours)	0.0238	0.2189	26.013

Table S3 : Loose garment deformer ablation. Quantitative evaluation on loose-garment sequences. The full model with the deformer improves all metrics and reduces splitting artifacts.

Figure S2 : Qualitative Comparison with Alternative Paradigms. Comparison with Wan-Animate [ 13 ] (2D video diffusion) and GUAVA [ 73 ] (upper-body 3D Gaussian avatar).

Explore similar work

GenLCA: 3D Diffusion for Full-Body Avatars from In-the-Wild Videos
Apr 8, 2026Yiqian Wu, Rawal Khirodkar, Egor Zakharov +63D Avatar Generation 3D Diffusion Models
State Key Laboratory of CAD&CG, Zhejiang University, China · Codec Avatars Lab, Meta, USA · State Key Lab of CAD&CG, Zhejiang University, China
FFAvatar: Few-Shot, Feed-Forward, and Generalizable Avatar Reconstruction
May 14, 2026Thuan Hoang Nguyen, Jiahao Luo, Yinyu Nie +33D Gaussian Splatting 3D Reconstruction
Snap Inc. · MBZUAI · University of California, Santa Cruz
HumanNOVA: Photorealistic, Universal and Rapid 3D Human Avatar Modeling from a Single Image
Jun 1, 2026Hezhen Hu, Wangbo Zhao, Lanqing Guo +63D Avatar Generation 3D Shape Generation
University of Texas at Austin · National University of Singapore · Texas A&M University

Apr 8, 2026J/K move · Enter open · S save

GenLCA: 3D Diffusion for Full-Body Avatars from In-the-Wild Videos

Yiqian Wu, Rawal Khirodkar, Egor Zakharov, Timur Bagautdinov+5

State Key Laboratory of CAD&CG, Zhejiang University, China · Codec Avatars Lab, Meta, USA · State Key Lab of CAD&CG, Zhejiang University, China

3D Avatar Generation 3D Diffusion Models Human Avatar Animation

Open paper AskPDF arXiv

We present GenLCA, a diffusion-based generative model for generating and editing photorealistic full-body avatars from text and image inputs. The generated avatars are faithful to the inputs, while supporting high-fidelity facial and full-body animations. The core idea is a novel paradigm that enables training a full-body 3D diffusion model from partially observable 2D data, allowing the training dataset to scale to millions of real-world videos. This scalability contributes to the superior photorealism and generalizability of GenLCA. Specifically, we scale up the dataset by repurposing a pretrained feed-forward avatar reconstruction model as an animatable 3D tokenizer, which encodes unstructured video frames into structured 3D tokens. However, most real-world videos only provide partial observations of body parts, resulting in excessive blurring or transparency artifacts in the 3D tokens. To address this, we propose a novel visibility-aware diffusion training strategy that replaces invalid regions with learnable tokens and computes losses only over valid regions. We then train a flow-based diffusion model on the token dataset, inherently maintaining the photorealism and animatability provided by the pretrained avatar reconstruction model. Our approach effectively enables the use of large-scale real-world video data to train a diffusion model natively in 3D. We demonstrate the efficacy of our method through diverse and high-fidelity generation and editing results, outperforming existing solutions by a large margin. The project page is available at https://onethousandwu.com/GenLCA-Page.

Abstract

Explore similar work

GenLCA: 3D Diffusion for Full-Body Avatars from In-the-Wild Videos

FFAvatar: Few-Shot, Feed-Forward, and Generalizable Avatar Reconstruction

HumanNOVA: Photorealistic, Universal and Rapid 3D Human Avatar Modeling from a Single Image