NBAvatar: Neural Billboards Avatars with Realistic Hand-Face Interaction

David Svitov; Mahtab Dahaghin; Pietro Morerio; Alessio Del Bue

cs.CVMar 12, 2026

NBAvatar: Neural Billboards Avatars with Realistic Hand-Face Interaction

Authors: David Svitov, Mahtab Dahaghin, Pietro Morerio, Alessio Del Bue

Organizations: Università degli Studi di Genova, Genoa, Italy · Istituto Italiano di Tecnologia (IIT), Genoa, Italy

Abstract

We present NBAvatar - a method for realistic rendering of head avatars handling non-rigid deformations caused by hand-face interaction. To this end, we introduce a novel hybrid implicit-explicit representation for animated avatars by combining the training of explicit oriented planar primitives with implicit neural rendering. Such a combination of representations in the end-to-end pipeline enables NBAvatar to handle temporally and pose-consistent geometry, along with fine-grained appearance details provided by the neural rendering technique. To enable joint optimization of different representations we propose a geometry-aware training scheme that allows our hybrid representation to surpass existing approaches in terms of novel-view and novel-pose rendering quality. Specifically, NBAvatar achieves up to 53% LPIPS reduction compared to Gaussian-based avatar methods, while also improving PSNR and SSIM, and achieves higher structural similarity compared to the state-of-the-art hand-face interaction method InteractAvatar.

Figures & tables

Figure 1: Novel view synthesis. We present the method for photo-realistic novel-view rendering of non-rigid hand-face interaction scenarios for human avatars. a) Our method achieves photo-realistic rendering quality for novel views and poses, surpassing state-of-the-art InteractAvatar [ 8 ] results. b) The method enables cross-actor enactment by transferring hand and face poses across different subjects.

Figure 2: Method description. a) For multi-view input video, we fit FLAME [ 29 ] and MANO [ 42 ] parametric models and reconstruct coarse face deformation with position-based dynamics (PBD) [ 34 ] . b) Then, for each polygon in the mesh, we anchor Neural Billboards by computing its orientation as offsets relative to the polygon. We rasterize Neural Billboards into a 6-channel image and a corresponding alpha map. c) Finally, we transform this rasterization to RGB with the U-Net renderer $R$ while regularizing the alpha map to fit the ground truth silhouette.

Figure 3: Neural billboards rasterization. a) We initialize the alpha texture $T_{i}^{\alpha}$ with a Gaussian distribution, and fill neural texture $T_{i}^{\textrm{NT}}$ with corresponding spectral coordinates on the mesh surface. During training, we optimize both textures with gradients from the rasterizer. b) We rasterize screen point $x$ by accumulating corresponding texture values along the ray. The resulting $I_{f}^{\textrm{NB}}$ transforms to RGB with a trainable renderer $R$ .

	Novel view			Self-enact
	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
SplattingAvatar [ 46 ]	24.67	0.9313	0.0942	24.83	0.9399	0.0847
GaussianAvatar [ 40 ]	24.95	0.9397	0.0756	25.14	0.9479	0.0663
Ours	26.08	0.9484	0.0439	25.25	0.9496	0.0479
Ours (MSE)	26.55	0.9517	0.0518	25.73	0.9523	0.0543

Table 1: Quantitative metrics. We report metrics for novel views and self-enactment (novel pose) rendering on the Decaf [ 47 ] dataset.

	Novel view			Self-enact
	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
InteractAvatar	29.85	0.9335	0.0338	28.17	0.9356	0.0396
Ours	26.08	0.9484	0.0439	25.25	0.9496	0.0479
Ours (MSE)	26.55	0.9517	0.0518	25.73	0.9523	0.0543

Table 2: Comparison with the metrics reported in the InteractAvatar [ 8 ] paper. We reproduced the evaluation pipeline for our method as closely as possible, and InteractAvatar results are taken directly from the original paper.

Figure 4: Qualitative comparison with InteractAvatar [ 8 ] . NBAvatar produces sharper details and more realistic hand-face deformations with fewer artifacts. InteractAvatar exhibits characteristic 3DGS artifacts including blurry facial textures and protruding Gaussians along the avatar boundary.

Figure 5: Qualitative comparison on novel views. We compare our method with SplattingAvatar [ 46 ] and GaussianAvatars [ 40 ] on the Decaf [ 47 ] dataset. NBAvatar produces sharp, high-fidelity reconstructions of non-rigid facial deformations and dynamic hand appearance, accurately capturing facial details.

Figure 6: Novel-pose synthesis (self-reenactment). We compare our method with SplattingAvatar [ 46 ] and GaussianAvatars [ 40 ] on held-out poses from the Decaf [ 47 ] dataset. NBAvatar generalizes to unseen hand-face interaction poses, faithfully reproducing contact-induced deformations and appearance changes.

Configuration	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
w/o $\mathcal{L}_{NB}$	25.41	0.9461	0.0453
w/o normals	25.86	0.9474	0.0451
w/o poses	25.12	0.9425	0.0509
w/o DNR	24.34	0.9333	0.0655
Full Method	26.08	0.9484	0.0439

Table 3: Ablation study on novel view synthesis for Decaf dataset.

Figure 7: Ablation study. We demonstrate renders and billboards’ masks for one of the Decaf subjects with ablated key components of the proposed algorithm.

Figure 8: Position-based dynamics ablation. We report examples of rendering all methods under novel poses with PBD and without.

	w/ PBD			w/o PBD
	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
SplattingAvatar	24.67	0.9313	0.0942	24.63	0.9312	0.0943
GaussianAvatar	24.95	0.9397	0.0756	24.86	0.9394	0.0755
Ours	26.08	0.9484	0.0439	25.84	0.9475	0.0455

Table 4: Position-based dynamics ablation for all methods under novel views.

Subject	Training Cameras	Evaluation Cameras
S1	110, 100, 108, 084, 111, 121, 122	102
S2	122, 110, 102, 100, 084, 111, 121	108
S3	121, 108, 122, 110, 102, 111, 084	100
S4	108, 122, 110, 102, 100, 084, 121	111

Table 5: Cameras setup. We used the following train-test camera split for the Decaf dataset for each subject.

Stage	Speed (ms) $\downarrow$	FPS $\uparrow$
Rasterization - RGB	1.1	880
Rasterization - NT	1.3	750
Neural Rendering - fp32	5.52	181
Neural Rendering - fp16	2.97	336
Total (NT; fp16)	4.13	242

Table 6: Inference speed evaluation. We report inference speed and corresponding FPS for different stages of the rendering pipeline. We additionally investigate the inference speed of rasterization with RGB and Neural Textures (NT).

	S1			S2			S3			S4
Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
Splatt. Av. [ 46 ]	25.17	0.9363	0.0892	25.83	0.9501	0.0689	24.45	0.9309	0.0938	23.24	0.9080	0.1251
Gauss. Av. [ 40 ]	25.31	0.9436	0.0691	26.55	0.9605	0.0533	24.55	0.9390	0.0763	23.40	0.9159	0.1040
Ours	26.04	0.9490	0.0401	28.36	0.9707	0.0247	25.37	0.9506	0.0457	24.55	0.9234	0.0651
Ours (MSE)	26.60	0.9525	0.0482	28.82	0.9721	0.0256	25.66	0.9538	0.0560	25.11	0.9286	0.0774

Table 7: Novel-view metrics per person. We report novel-view quantitative objective metrics separately for each person used in the evaluation on the Decaf dataset.

	S1			S2			S3			S4
Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
Splatt. Av. [ 46 ]	25.59	0.9443	0.0834	23.61	0.9313	0.0939	25.68	0.9490	0.0730	24.42	0.9353	0.0886
Gauss. Av. [ 40 ]	26.36	0.9556	0.0614	23.68	0.9357	0.0766	25.85	0.9556	0.0570	24.67	0.9447	0.0703
Ours	27.12	0.9611	0.0324	21.99	0.9287	0.0782	26.35	0.9591	0.0370	25.56	0.9498	0.0443
Ours (MSE)	27.87	0.9646	0.0416	22.10	0.9311	0.0751	26.81	0.9608	0.0454	26.13	0.9527	0.0552

Table 8: Self-enact metrics per person. We report novel-pose quantitative objective metrics separately for each person used in the evaluation on the Decaf dataset.

Figure 9: Qualitative comparison on novel views for additional subjects. We compare our method with SplattingAvatar [ 46 ] and GaussianAvatars [ 40 ] on the Decaf [ 47 ] dataset. Our NBAvatar produces sharp, high-fidelity reconstructions of non-rigid facial deformations and dynamic hand appearance, accurately capturing facial details.

Figure 10: Novel-pose synthesis for additional subjects. We compare our method with SplattingAvatar [ 46 ] and GaussianAvatars [ 40 ] on held-out poses from the Decaf [ 47 ] dataset. Our NBAvatar generalizes to unseen hand-face interaction poses, faithfully reproducing contact-induced deformations and appearance changes.

Figure 11: Cross-reenactment for different subjects. We report more examples of cross-reenactment when we drive one actor with the poses of another.

Channels	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	FPS $\uparrow$
3	27.54	0.9671	0.0286	153
6	28.12	0.9691	0.0259	149
12	27.89	0.9686	0.0276	144
24	28.22	0.9701	0.0270	27

Table 9: Number of channels. We analyzed the metrics and inference speed for Subject 2 from the Decaf dataset, and selected the optimal number of channels for Neural Textures.

Human Avatar Animation

Human avatar animation builds digital people that can be reconstructed, driven by motion or speech, and rendered consistently across poses and views.

See all 91 papers

FFAvatar: Few-Shot, Feed-Forward, and Generalizable Avatar Reconstruction
May 14, 2026Thuan Hoang Nguyen, Jiahao Luo, Yinyu Nie +33D Gaussian Splatting 3D Reconstruction
Snap Inc. · MBZUAI · University of California, Santa Cruz
PIAvatar: Physically Interactive Avatars via Deformation Gradient Decoupling
Jun 19, 2026Sang-Hun Han, Min-Gyu Park, Jisu Shin +3Material Point Method Human Avatar Animation
1GIST · 2KETI · 3Polygom +2
F3G-Avatar : Face Focused Full-body Gaussian Avatar
Apr 10, 2026Willem Menu, Erkut Akdag, Pedro Quesado +23D Avatar Generation 3D Human Reconstruction
AIMS Group, Department of Electrical Engineering, Eindhoven University of Technology

May 14, 2026J/K move · Enter open · S save

FFAvatar: Few-Shot, Feed-Forward, and Generalizable Avatar Reconstruction

Thuan Hoang Nguyen, Jiahao Luo, Yinyu Nie, Hao Li+2

Snap Inc. · MBZUAI · University of California, Santa Cruz

3D Gaussian Splatting 3D Reconstruction 3D Avatar Generation Sparse-View 3D Reconstruction Human Avatar Animation 3D Face Reconstruction+1

Open paper AskPDF arXiv

Avatar reconstruction has traditionally relied on per-subject optimization that requires hours of computation or on expensive preprocessing that limits scalability. We introduce FFAvatar, a generalizable feed-forward framework that reconstructs high-quality, animatable 3D Gaussian head avatars from few-shot unposed portrait images in seconds. FFAvatar fuses information from multiple source images into a unified canonical Gaussian representation through Multi-View Query-Former, which is animated via FLAME parameters predicted end-to-end directly from pixels, eliminating the overhead of offline FLAME extraction. We further propose a three-stage training curriculum that achieves both broad generalization and high-fidelity reconstruction: (i) scalable pretraining on extensive monocular video data with over 1M identities to learn strong generalizable priors; (ii) multi-view fine-tuning on a small but high-quality dataset of 360-degree captures to enhance geometric fidelity and extreme-view awareness; and (iii) optional personalization that adapts to specific identities for maximum fidelity within 500 optimization steps. Extensive experiments demonstrate that FFAvatar sets a new standard for identity preservation, geometric consistency, and animation fidelity. On the NeRSemble benchmark, it outperforms the state-of-the-art LAM by a substantial 5.5 PSNR gain. Furthermore, FFAvatar enables real-time deployment, reconstructing avatars in 2 seconds without personalization and 10 seconds with personalization, while supporting 49 FPS animation on a single NVIDIA A100 GPU.

Abstract

Human Avatar Animation

Explore similar work

FFAvatar: Few-Shot, Feed-Forward, and Generalizable Avatar Reconstruction

PIAvatar: Physically Interactive Avatars via Deformation Gradient Decoupling

F3G-Avatar : Face Focused Full-body Gaussian Avatar