Stable Signer V3: 4D Deformable Primitive Splatting Is All You Need for Sign Language Production

Sen Fang; Liwei Che; Dimitris N. Metaxas

cs.CVOct 18, 2026

Stable Signer V3: 4D Deformable Primitive Splatting Is All You Need for Sign Language Production

Authors: Sen Fang, Liwei Che, Dimitris N. Metaxas

Organizations: Rutgers University

Abstract

We present Stable Signer V3, an interactive high-resolution sign language avatar system built on 4D Deformable Primitive Splatting. It addresses three limitations of existing avatars: restricted facial expression, inaccurate identity-specific body deformation, and loss of fine appearance details. Shared Face Primitive Splatting adds transferable superquadric primitives for richer expression control; Test-Time Body Primitive Splatting learns an identity-specific virtual muscle layer for more faithful motion; and Test-Time Tunneling Super-Resolution uses TunnelingNet to recover identity-specific high-resolution details. Both test-time modules are optimized once per signer identity; inference then runs entirely from the stored identity parameters. Together, they reduce facial geometry error by 71.4%, body-boundary error by 35.1% across ten identities, and high-resolution RGB reconstruction error by 24.2% on held-out evaluations, while retaining interactive streaming.

Figures & tables

Figure 2: Overview of Stable Signer V3. Motion controls drive an avatar through deformable primitives. Gaussian feature splatting and a tunneling-enhanced decoder produce high-resolution signing videos.

Figure 3: Deformation and detail adaptation. (a) Shared facial primitives learn geometric residuals with FLAME supervision. (b) Identity-specific body primitives adapt the avatar carrier. (c) A late Softmax tunneling pathway learns high-resolution appearance. Dashed arrows denote training supervision; solid arrows denote inference.

Figure 4: Face primitive reconstruction on a held-out frame. Top: restricted FLAME. Bottom: SQ-corrected geometry, colored by primitive assignment. The second column amplifies surface deviations from the target by

5\times

for visibility. The third column shows the full-expression target; the last shows unamplified geometric error on a shared scale (darker indicates larger error).

Figure 5: Test-time body adaptation. Frozen GUAVA (top) and identity-specific Body SQ (bottom). Columns show the rendered avatar, reference frame with adapted SQ patches overlaid at 60% opacity, geometric carrier, target, and RGB error on a shared scale. The top-row reference has no SQ overlay. We show the held-out moment with the largest body-region RGB error reduction in this clip.

Body model	RGB L1 $\downarrow$	Boundary L1 $\downarrow$	SSIM $\uparrow$
Baseline	0.0649	0.0350	0.9038
$K=512$	0.0599	0.0282	0.9076
$K=1{,}024$	0.0575	0.0270	0.9096

Table 2: Body primitive count. Baseline and Body SQ use the same chronological 80%/20% train–evaluation split for one identity. Both SQ variants share the adaptation budget and loss weights. Metrics are computed in the body region; RGB L1 uses refined renderings.

Figure 6: Detail adaptation. Frozen baseline, Adapter, and our tunneling decoder at the same timestamp. Circular insets magnify identical facial regions. The baseline is enlarged from 512 resolution; both adapted outputs are generated at 1024. Parentheses report mean held-out RGB L1 reductions relative to the frozen baseline.

Stage	Output	Throughput $\uparrow$
Motion synthesis	SMPLest-X [ 74 ] motion	198.7 FPS
Primitive deformation	Face and body primitives	320.9 FPS
Avatar rendering	HD RGB	52.0 FPS
Detail refinement	4K RGB	47.6 FPS
Resident pipeline	Text $\rightarrow$ HD RGB	25.3 FPS

Table 3: Online efficiency. Component rows report warmed batch-one throughput on one Quadro RTX 8000. The resident row reports measured wall-clock throughput over 784 frames, including composition, motion synthesis, HD rendering, encoding, and concatenation.

Configuration	B-T BLEU-4 $\uparrow$	ROUGE-L $\uparrow$
Enhancement disabled	25.6450	59.4771
Enhancement enabled	26.8897	61.7298

Table 4: Semantic preservation under visual enhancement. Both configurations render identical motions. The enabled configuration activates shared face primitives, test-time body primitives, and detail tunneling. Signing poses are re-extracted from the final RGB videos and translated by the same frozen model.

Figure 7: Qualitative rendering comparison. Each column corresponds to one method: LHM++, SplattingAvatar, GaussianAvatar, ExAvatar, Ours, and Ground Truth. Each row shows the same held-out identity and signing pose across methods; the comparison focuses on facial detail, finger structure, and garment boundaries.

Four-identity shared held-out-pose evaluation
Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
LHM++ [ 45 ]	8.987	0.646	0.576
SplattingAvatar [ 55 ]	18.396	0.764	0.348
GaussianAvatar [ 14 ]	10.246	0.684	0.460
ExAvatar [ 36 ]	18.141	0.775	0.291
Stable Signer V3	19.415	0.787	0.276

Table 5: Rendering fidelity. Macro-averaged results under two matched protocols: shared held-out poses for four identities (top) and crops sampled directly from held-out 4K targets for ten identities (bottom). PSNR uses GT foreground; SSIM and LPIPS use the shared GT person crop.

Region	Restricted FLAME	Face primitives	Reduction
	(mm) $\downarrow$	(mm) $\downarrow$	(%) $\uparrow$
Full head	1.267	0.362	71.4
Face	1.492	0.518	65.3
Lips	1.739	0.990	43.1
Left eye	1.029	0.587	43.0
Right eye	1.483	0.626	57.8

Table 6: Region-wise face reconstruction error. Mean Euclidean point error on the held-out 20% of the tracked sequence. Both methods use identical articulation and coordinates.

Model	RGB L1 $\downarrow$	Boundary L1 $\downarrow$	SSIM $\uparrow$	Disp. (m) $\downarrow$
Frozen carrier	0.0649	0.0350	0.9038	–
$K{=}512$ , matched budget	0.0599	0.0282	0.9076	0.0111
$K{=}1{,}024$ , matched budget	0.0575	0.0270	0.9096	0.0111
$K{=}1{,}024$ , final checkpoint	0.0562	0.0230	0.9115	0.0125

Table 7: Body adaptation over the held-out segment. The first three rows reproduce the controlled capacity study; the final row reports the converged model on the complete held-out segment.

Method	RGB L1 $\downarrow$	Boundary L1 $\downarrow$	SSIM $\uparrow$
Frozen carrier	0.0661	0.0308	0.9086
Body primitives	0.0551	0.0200	0.9155

Table 8: Ten-identity body adaptation. Macro-average over temporally held-out segments. The same model configuration and evaluation code are used for all identities.

Method	RGB L1 $\downarrow$	PSNR $\uparrow$
Frozen decoder	0.0260	22.16
LoRA ( $r{=}32$ )	0.0234	22.62
Adapter ( $r{=}32$ )	0.0225	22.90
Tunneling ( $r{=}8$ , 64 slots)	0.0197	23.77
Tunneling ( $r{=}8$ , 128 slots)	0.0196	23.85

Table 9: Equal-budget detail adaptation. Every trainable method is evaluated after 900 update steps on the same identity, held-out frames, Gaussian features, and native-resolution targets.

Figure 8: Stable Signer V3 interface overview. The complete interface presents text or gloss input, signer identity, staged motion and avatar outputs, execution state, and reusable results within one workspace.

Panel of Figure 9 — Figure 9: Stable Signer V3 interactive generation workflow. The interface exposes input translation, identity selection, stage-level progress, motion inspection, and final avatar outputs in one view. The example loads the same completed generation across all output modes.

Figure 10: Instruction-conditioned generation. Prompt input, editable gloss, semantic fallback, signer identity, and the complete generation command remain visible within one uninterrupted workspace.

Panel of Figure 11 — Figure 11: Execution and inspection controls. Stable Signer V3 exposes reproducible runtime settings and keeps completed generations available for immediate comparison.

Panel of Figure 12 — Figure 12: Contextual text-to-sign access. Highlighting a passage invokes the floating signer in place and provides both its 3D motion and photorealistic avatar views without leaving the document.

Human Avatar Animation

Human avatar animation builds digital people that can be reconstructed, driven by motion or speech, and rendered consistently across poses and views.

See all 91 papers

PHOSA: Photorealistic 3D Sign Avatar Modeling and Benchmark
Sep 24, 2026Haodong Wang, Hezhen Hu, Wengang Zhou +13D Avatar Generation 3D Human Reconstruction
University of Science and Technology of China · University of Texas at Austin
SignSparK: Efficient Multilingual Sign Language Production via Sparse Keyframe Learning
Mar 11, 2026Jianhe Low, Alexandre Symeonidis-Herzig, Maksym Ivashechkin +2Human Motion Generation Sign Language Generation
CVSSP, University of Surrey, United Kingdom
Conditional Collapse in Sign Language Production: A Diagnostic and a Scaling Argument
Jun 1, 2026Rui Hong, Jana KošeckáSign Language Translation Human Motion Generation
George Mason University

Sep 24, 2026J/K move · Enter open · S save

PHOSA: Photorealistic 3D Sign Avatar Modeling and Benchmark

Haodong Wang, Hezhen Hu, Wengang Zhou, Houqiang Li

University of Science and Technology of China · University of Texas at Austin

3D Avatar Generation 3D Human Reconstruction Sign Language Generation Sign Language Datasets

Open paper AskPDF arXiv

In this work, we focus on photorealistic sign avatar modeling, which is crucial for effective communication with the Deaf community and is characterized by complex hand gestures and nuanced facial expressions. To this end, we introduce MVSign, the first multi-view Chinese sign language dataset co-designed with Deaf experts, featuring diverse gestures and rich annotations. For precise SMPL-X annotation, we develop a hybrid fitting pipeline that produces accurate body, hand, and facial parameters and can also be applied to the monocular setting. Building on MVSign, we propose a decoupled sign avatar representation that isolates body, head, and hand components to capture complex articulations, together with a motion-aware sampling strategy to handle motion blur and balance gesture diversity. Extensive experiments demonstrate that our method achieves high-fidelity visual results on MVSign, particularly in detailed hand and facial regions, and generalizes well to in-the-wild monocular sign language videos. Project page: https://naaapi.github.io/PHOSA.

Abstract

Human Avatar Animation

Explore similar work

PHOSA: Photorealistic 3D Sign Avatar Modeling and Benchmark

SignSparK: Efficient Multilingual Sign Language Production via Sparse Keyframe Learning

Conditional Collapse in Sign Language Production: A Diagnostic and a Scaling Argument