FlowAvatar: Real-Time Full-Body Avatars from Sparse Egocentric Inputs on Consumer XR Devices

Chenfeng Gao; Taeyoung Yeon; Sungheon Park; Vasco Xu; Anish Prabhu; Mar Gonzalez-Franco; Karan Ahuja

cs.CVOct 12, 2026

FlowAvatar: Real-Time Full-Body Avatars from Sparse Egocentric Inputs on Consumer XR Devices

Authors: Chenfeng Gao, Taeyoung Yeon, Sungheon Park, Vasco Xu, Anish Prabhu, Mar Gonzalez-Franco, Karan Ahuja

Organizations: Northwestern University · University of Chicago · Google

Abstract

Prior avatar embodiment research has studied tracking completeness (how faithfully the body, hands, and face are sensed and reproduced) and personalized appearance in isolation, each requiring specialized lab equipment. How these factors interact on consumer hardware remains an open question, because no existing system unifies body pose, face, hands, and personalized appearance on a standalone headset. We present FlowAvatar, a system enabling real-time personalized look-alike avatars with full-body pose, hand articulation, facial expression, and eye gaze tracking from sparse egocentric inputs on consumer XR devices (Meta Quest Pro). Our pose model achieves competitive spatial accuracy and state-of-the-art temporal consistency on the AMASS benchmark against prior works with comprehensive spatial and temporal losses validated through architecture and loss ablations, while running in real-time. Two user studies (N=12, N=10) provide converging evidence that tracking completeness and personalized appearance serve complementary roles: tracking completeness broadly improves perceived motion quality, while only their combination achieves positive total embodiment.

Figures & tables

Method	Body Pose	Face	Hands	Personalized	On-Device	MPJPE	MPJVE	Jitter
Meta Quest Full Body [ 43 ]	✓	$\times$	✓	$\times$	✓	–	–	–
Apple Vision Pro Persona [ 6 ]	$\times$	✓	✓	✓	✓	–	–	–
Pixel Codec Avatar [ 39 ]	$\times$	✓	$\times$	✓	✓	–	–	–
AvatarPoser [ 30 ]	✓	$\times$	$\times$	$\times$	$\times$	6.62	33.16	10.79
AGRoL [ 17 ]	✓	$\times$	$\times$	$\times$	$\times$	6.74	24.14	6.33
AvatarJLM [ 72 ]	✓	$\times$	$\times$	$\times$	$\times$	5.96	27.50	6.91

Table 1: Comparison with related work for user digitization on consumer XR devices without additional instrumented sensors.

Figure 1: Pose estimation architecture: three parallel BiGRU networks (upper body, lower body, root) process shared embedded features to generate SMPL-X parameters.

Figure 2: Examples of hand (A–C) and facial expression (D–F) capture on Meta Quest, remapped to SMPL-X mesh.

Figure 3: Texture generation pipeline: (A) preprocessing, (B) face alignment, (C) region-specific differentiable rendering, (D) compositing and upscaling.

Method	MPJRE $\downarrow$	MPJPE $\downarrow$	MPJVE $\downarrow$	Jit. $\downarrow$
AvatarPoser [ 30 ]	4.68	6.62	33.16	10.79
AGRoL [ 17 ]	4.38	6.74	24.14	6.33
AvatarJLM [ 72 ]	4.45	5.96	27.50	6.91
HMDPoser [ 14 ]	4.27	5.44	30.15	5.62
EgoPoser [ 29 ]	4.45	6.85	35.20	18.64
FlowAvatar (ours)	4.19	5.40	21.72	3.77

Table 2: FlowAvatar compared with prior works on AMASS.

Strategy	MPJRE $\downarrow$	MPJVE $\downarrow$	Jitter $\downarrow$	H-PE $\downarrow$
No temporal loss	4.55	25.58	5.17	2.50
No hand loss	4.41	23.26	3.91	2.98
No ground loss	4.31	22.55	4.05	2.25
All losses	4.19	21.72	3.77	2.31

Table 3: Ablation across different loss configurations.

Backbone	FLOPs	FPS	MPJRE $\downarrow$	MPJPE $\downarrow$	MPJVE $\downarrow$
Transformer	171M	308	4.83	6.94	29.55
FastViT [ 64 ]	1027M	39.1	5.49	7.65	29.01
LSTM	190M	542.3	4.43	5.75	22.95
GRU (ours)	154M	565.0	4.19	5.40	21.72

Table 4: Backbone architecture comparison (all 3-layer).

Condition	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
Baseline	25.289	0.9456	0.0644
+ Face align	25.289	0.9456	0.0640
+ Upscaling	25.291	0.9473	0.0639
+ Both	25.294	0.9478	0.0630

Table 5: Ablation study on face alignment and upscaling.

Figure 4: Texture ablation on (A) our capture data and (B) People Snapshot dataset. Face alignment and upscaling improve facial detail.

Figure 5: Example sequence (Anger) from the third-person motion quality study: ground truth, then representations (R1) Stick Figure, (R2) Minimal Body (no hands/face), (R3) Base Body (no texture), and (R4) Photo-real (personalized look-alike texture).

Motion Sequences
S1: Greeting	Waving, smiling, clapping — upper body gestures in social interaction
S2: Anger	Frustrated gestures, angry expressions, assertive locomotion — emotional expressivity
S3: Joy	Jumping, smiling, rhythmic movement — whole-body motion with subtle expressions
S4: Exercise	Squatting, jumping, boxing — dynamic full-body tracking under high intensity
S5: Workplace	Sitting/standing transitions, desktop tasks — fine hand movements and body transitions
Evaluation Dimensions

Table 6: Motion sequences and evaluation dimensions (the four avatar representations R1–R4 are shown in Figure 5 ).

	+Tracking (R2 $\to$ R3)		+Texture (R3 $\to$ R4)
	$p$ ( $p_{adj}$ )	$r$ [CI]	$p$ ( $p_{adj}$ )	$r$ [CI]
Q1	$\textless.001^{***}$ ( .004 )	+0.97 [+0.5, +1.1]	$.098^{\dagger}$ (.488)	+0.58 [ $-$ 0.0, +0.8]
Q2	$\textless.001^{***}$ ( .001 )	+1.00 [+0.6, +1.5]	$.512$ (.948)	+0.25 [ $-$ 0.2, +0.4]
Q3	$\textless.001^{***}$ ( .001 )	+1.00 [+0.8, +1.7]	$.316$ (.948)	+0.33 [ $-$ 0.1, +0.5]
Q4	$.014^{*}$ ( .014 )	+0.85 [+0.3, +1.1]	$.004^{**}$ ( .023 )	+0.96 [+0.3, +1.0]
Q5	$.002^{**}$ ( .004 )	+1.00 [+0.7, +1.8]	$.152$ (.609)	+0.56 [ $-$ 0.0, +0.3]

Table 7: Motion quality: paired Wilcoxon results for the two key contrasts across Q1–Q6.

r

is the rank-biserial effect size, CI the 95% bootstrap CI of the paired mean difference. Exact-

p

markers:

{}^{\dagger}p{<}.10

{}^{*}p{<}.05

{}^{**}p{<}.01

{}^{***}p{<}.001

; bold

p_{adj}

survives Holm correction.

Figure 6: Third-person motion quality study: Mean ratings with standard error for four avatar representations across five motion sequences and six evaluation dimensions.

Figure 7: Embodiment study: (A) participant’s pose; avatar representations: (B) FlowAvatar Base Body; (C) Meta Avatar; (D) FlowAvatar Photo-real (personalized look-alike texture). (E) Five participants’ avatars during study.

	Photo vs. Base		Photo vs. Meta
Subscale	$p$ ( $p_{adj}$ )	$r$ [CI]	$p$ ( $p_{adj}$ )	$r$ [CI]
Ownership	$.062^{\dagger}$ (.250)	+0.67 [+0.7, +9.2]	$.668$ (1.000)	+0.18 [ $-$ 4.6, +6.6]
Agency	$.125$ (.250)	+0.64 [+0.0, +3.1]	$.043^{*}$ (.258)	+0.73 [+0.8, +7.3]
Location	$.102$ (.250)	+0.67 [+0.1, +3.9]	$.109$ (.547)	+0.64 [+0.2, +6.4]
Appearance	$.031^{*}$ (.164)	+1.00 [+0.9, +5.0]	$1.000$ (1.000)	$-$ 0.04 [ $-$ 1.3, +2.4]
Response	$.078^{\dagger}$ (.250)	+0.82 [+0.3, +3.0]	$.500$ (1.000)	+0.67 [ $-$ 0.2, +2.8]

Table 8: Embodiment (N=10, exploratory): paired Wilcoxon results. Photo vs. Base isolates texture (tracking held constant). Photo vs. Meta additionally varies tracking and style.

r

is rank-biserial, CI the 95% bootstrap CI of the paired mean difference. Exact-

p

markers:

{}^{\dagger}p{<}.10

{}^{*}p{<}.05

; no comparison survives Holm correction (

p_{adj}

Figure 8: First-person embodiment study: CoolMoves [ 2 , 50 ] subscale scores (centered at 0) for three avatar conditions. Photo-real was the only condition with positive Total Embodiment.

Human Avatar Animation

Human avatar animation builds digital people that can be reconstructed, driven by motion or speech, and rendered consistently across poses and views.

See all 91 papers

FFAvatar: Few-Shot, Feed-Forward, and Generalizable Avatar Reconstruction
May 14, 2026Thuan Hoang Nguyen, Jiahao Luo, Yinyu Nie +33D Gaussian Splatting 3D Reconstruction
Snap Inc. · MBZUAI · University of California, Santa Cruz
SARAH: Spatially Aware Real-time Agentic Humans
Feb 20, 2026Evonne Ng, Siwei Zhang, Zhang Chen +2Human Motion Generation Co-Speech Gesture Generation
Meta Reality Labs, Redmond, WA, USA
PHAF-Personalized Hand Avatars in a Flash
Jun 2, 2026Meghana Shankar, Akanxit Upadhyay, Anmol Namdev +23D Reconstruction 3D Hand Reconstruction
Samsung R&D Institue Bangalore Bangalore, Karnataka, India

May 14, 2026J/K move · Enter open · S save

FFAvatar: Few-Shot, Feed-Forward, and Generalizable Avatar Reconstruction

Thuan Hoang Nguyen, Jiahao Luo, Yinyu Nie, Hao Li+2

Snap Inc. · MBZUAI · University of California, Santa Cruz

3D Gaussian Splatting 3D Reconstruction 3D Avatar Generation Sparse-View 3D Reconstruction Human Avatar Animation 3D Face Reconstruction+1

Open paper AskPDF arXiv

Avatar reconstruction has traditionally relied on per-subject optimization that requires hours of computation or on expensive preprocessing that limits scalability. We introduce FFAvatar, a generalizable feed-forward framework that reconstructs high-quality, animatable 3D Gaussian head avatars from few-shot unposed portrait images in seconds. FFAvatar fuses information from multiple source images into a unified canonical Gaussian representation through Multi-View Query-Former, which is animated via FLAME parameters predicted end-to-end directly from pixels, eliminating the overhead of offline FLAME extraction. We further propose a three-stage training curriculum that achieves both broad generalization and high-fidelity reconstruction: (i) scalable pretraining on extensive monocular video data with over 1M identities to learn strong generalizable priors; (ii) multi-view fine-tuning on a small but high-quality dataset of 360-degree captures to enhance geometric fidelity and extreme-view awareness; and (iii) optional personalization that adapts to specific identities for maximum fidelity within 500 optimization steps. Extensive experiments demonstrate that FFAvatar sets a new standard for identity preservation, geometric consistency, and animation fidelity. On the NeRSemble benchmark, it outperforms the state-of-the-art LAM by a substantial 5.5 PSNR gain. Furthermore, FFAvatar enables real-time deployment, reconstructing avatars in 2 seconds without personalization and 10 seconds with personalization, while supporting 49 FPS animation on a single NVIDIA A100 GPU.

Abstract

Human Avatar Animation

Explore similar work

FFAvatar: Few-Shot, Feed-Forward, and Generalizable Avatar Reconstruction

SARAH: Spatially Aware Real-time Agentic Humans

PHAF-Personalized Hand Avatars in a Flash