PoseCraft: Tokenized 3D Body Landmark and Camera Conditioning for Photorealistic Human Image Synthesis

Zhilin Guo; Jing Yang; Kyle Fogarty; Jingyi Wan; Boqiao Zhang; Tianhao Wu; Weihao Xia; Chenliang Zhou

cs.CVFeb 22, 2026

PoseCraft: Tokenized 3D Body Landmark and Camera Conditioning for Photorealistic Human Image Synthesis

Authors: Zhilin Guo, Jing Yang, Kyle Fogarty, Jingyi Wan, Boqiao Zhang, Tianhao Wu, Weihao Xia, Chenliang Zhou,

Organizations: University of Cambridge · Google

Abstract

Digitizing humans and synthesizing photorealistic avatars with explicit 3D pose and camera controls are central to VR, telepresence, and entertainment. Existing skinning-based workflows require laborious manual rigging or template-based fittings, while neural volumetric methods rely on canonical templates and re-optimization for each unseen pose. We present PoseCraft, a diffusion framework built around tokenized 3D interface: instead of relying only on rasterized geometry as 2D control images, we encode sparse 3D landmarks and camera extrinsics as discrete conditioning tokens and inject them into diffusion via cross-attention. Our approach preserves 3D semantics by avoiding 2D re-projection ambiguity under large pose and viewpoint changes, and produces photorealistic imagery that faithfully captures identity and appearance. To train and evaluate at scale, we also implement GenHumanRF, a data generation workflow that renders diverse supervision from volumetric reconstructions. Our experiments show that PoseCraft achieves significant perceptual quality improvement over diffusion-centric methods, and attains better or comparable metrics to latest volumetric rendering SOTA while better preserving fabric and hair details.

Figures & tables

Figure 1: PoseCraft uses tokenized 3D landmarks and camera parameters to synthesize photorealistic humans. This explicit 3D control delivers sharp silhouettes, preserves high-frequency details, and ensures structural coherence across novel viewpoints.

Figure 2: Pipeline overview of (A) RigCraft and (B, C) PoseCraft. (A) RigCraft performs multi-view fusion to generate temporally stable 3D landmarks from 2D poses. (B) During training, PoseCraft learns to denoise latent representations conditioned on tokenized 3D landmarks and camera parameters. (C) At inference, PoseCraft synthesizes a photorealistic image from a target 3D pose and camera view.

Figure 3: PoseCraft Architecture. We guide a latent diffusion UNet with two conditioning streams. First, a 3D Control Tokenizer converts camera extrinsics and 3D body landmarks into discrete tokens, which are injected via cross-attention for explicit 3D control. Second, a 2D skeleton projection is concatenated with the noisy latent input to provide direct spatial guidance.

Actor	Seq.	Gender	Clothing	Train	Test	Views
01	1	F	Loose	0–2000	2000–2200	100
01	2	F	Loose	0–2200	–	100
02	1	M	Tight	0–1900	1900–2100	100
02	2	M	Tight	0–2400	–	100
06	1	F	Tight	0–2100	2100–2300	100
06	2	F	Tight	0–2400	–	100

Table 1: Train/test split for three selected ActorsHQ [ 12 ] subjects. For each actor and sequence, we list gender, clothing style, training-frame and held-out test-frame ranges, and the number of virtual camera views per frame. We specifically choose Actor 01 (female, loose clothing), Actor 02 (male, tight clothing), and Actor 06 (female, tight clothing) to cover both genders and garment variety for comprehensive validation coverage.

Figure 4: Qualitative comparison on the GenHumanRF test split [ 12 ] . The proposed PoseCraft method largely outperforms the image-based CFLD [ 24 ] , CHAMP [ 53 ] , T2I-Adapter [ 30 ] and ControlNet [ 47 ] , and compare better or comparatively to the volumetric rendering SOTA Animatable Gaussian [ 19 ] .

Method	Actor01				Actor02				Actor06
Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	FID $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	FID $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	FID $\downarrow$
CFLD [ 24 ]	16.01	0.8035	0.2508	120.49	15.91	0.7843	0.2823	116.08	16.15	0.8149	0.2270	101.85
AnimateAnyone [ 11 ]	19.70	0.8438	0.1508	70.27	20.47	0.8398	0.1591	59.55	18.13	0.8299	0.1630	104.82
CHAMP [ 53 ]	20.75	0.8536	0.1270	53.67	21.12	0.8559	0.1352	42.00	19.04	0.8601	0.1418	66.94
T2I-Adapter [ 30 ]	23.01	0.9042	0.1053	41.89	25.16	0.9217	0.0823	38.32	23.99	0.9268	0.0798	40.34
ControlNet [ 47 ]	26.93	0.6737	0.0489	34.72	23.02	0.2080	0.0944	27.65	26.66	0.4622	0.0494	29.08

Table 2: Quantitative comparison of methods on our GenHumanRF train/test split. Our PoseCraft method largely outperforms diffusion-based CFLD [ 24 ] , AnimateAnyone [ 11 ] , CHAMP [ 53 ] , T2I-Adapter [ 30 ] , ControlNet [ 47 ] , and compares better or competitively to best volumetric rendering SOTA Animatable Gaussians [ 19 ] . (

\uparrow

higher is better,

\downarrow

lower is better, boldface indicates best result.)

Camera & Skeleton	Test	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	FID $\downarrow$
Plucker & OP-map	view	26.28	0.9505	0.0385	20.83
	frame	29.05	0.9620	0.0278	30.78
	all	26.23	0.9518	0.0398	22.60
Plucker & RC-map	view	27.16	0.9562	0.0359	22.32
	frame	30.32	0.9679	0.0247	31.87
	all	27.49	0.9580	0.0360	23.59

Table 3: Camera encoding ablation . Spherical Harmonics (SH) for camera rotation achieves better metrics than Plucker Ray with either RigCraft skeleton or OpenPose skeleton.

Token Embedding	Test	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	FID $\downarrow$
No MLP	view	27.50	0.9500	0.0386	21.94
	frame	29.53	0.9627	0.0255	31.08
	all	28.10	0.9532	0.0376	23.57
Share MLP	view	28.09	0.9534	0.0362	23.39
	frame	29.27	0.9616	0.0264	31.88
	all	28.37	0.9548	0.0360	24.33

Table 4: 3D Control Tokenizer ablation. Our Per-token MLP treats each 3D information token independently to elevate model’s 3D awareness, outperforming No MLP and Shared MLP ablations.

Control Method	Test	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	FID $\downarrow$
3D-CT only	view	24.59	0.9526	0.05883	28.25
	frame	28.26	0.9574	0.02858	32.19
	all	25.45	0.9471	0.0553	28.32
OP-Map only	view	26.73	0.9443	0.0427	24.04
	frame	28.56	0.9570	0.0302	34.53
	all	26.95	0.9456	0.0433	25.05

Table 5: Control method ablation. Our RigCraft (RC) skeleton together with 3D Control Tokenizer (3D-CT) architecture outperforms all other control combinations, including 3D-CT only ; OP(OpenPose)-map only ; OP-Map & 3D-CT ; and RC-Map only .

Figure 5: Qualitative visualization of the RigCraft 3D landmark estimation and refinement. From top to bottom: (1) noisy 3D landmarks from re-projected 2D OpenPose detections; (2) landmarks after multi-view triangulation; and (3) the final, temporally coherent RigCraft outputs after smoothing.

3D Avatar Generation

3D avatar generation creates realistic, controllable digital humans from images, video, text, or other inputs.

See all 90 papers

GenLCA: 3D Diffusion for Full-Body Avatars from In-the-Wild Videos
Apr 8, 2026Yiqian Wu, Rawal Khirodkar, Egor Zakharov +63D Avatar Generation 3D Diffusion Models
State Key Laboratory of CAD&CG, Zhejiang University, China · Codec Avatars Lab, Meta, USA · State Key Lab of CAD&CG, Zhejiang University, China
PoseDreamer: Scalable and Photorealistic Human Data Generation Pipeline with Diffusion Models
Mar 30, 2026Lorenza Prospero, Orest Kupyn, Ostap Viniavskyi +2Synthetic Data Generation Human Mesh Recovery
The Podium Institute for Sports Medicine and Technology University of Oxford Oxford, UK · Visual Geometry Group University of Oxford Oxford, UK · Ukrainian Catholic University Lviv, Ukraine
DiGS-Avatar: Single-Image Animatable 3D Human Reconstruction via UV-Space Diffusion
Aug 21, 2026Jiakun Li, Li Fang, Hao Zhu +43D Avatar Generation 3D Human Reconstruction
Key Laboratory of Media Audio and Video (Communication University of China), Ministry of Education, Beijing 100024, China · School of Intelligence Science and Technology, Nanjing University, Nanjing 210023, China

Apr 8, 2026J/K move · Enter open · S save

GenLCA: 3D Diffusion for Full-Body Avatars from In-the-Wild Videos

Yiqian Wu, Rawal Khirodkar, Egor Zakharov, Timur Bagautdinov+5

State Key Laboratory of CAD&CG, Zhejiang University, China · Codec Avatars Lab, Meta, USA · State Key Lab of CAD&CG, Zhejiang University, China

3D Avatar Generation 3D Diffusion Models Human Avatar Animation

Open paper AskPDF arXiv

We present GenLCA, a diffusion-based generative model for generating and editing photorealistic full-body avatars from text and image inputs. The generated avatars are faithful to the inputs, while supporting high-fidelity facial and full-body animations. The core idea is a novel paradigm that enables training a full-body 3D diffusion model from partially observable 2D data, allowing the training dataset to scale to millions of real-world videos. This scalability contributes to the superior photorealism and generalizability of GenLCA. Specifically, we scale up the dataset by repurposing a pretrained feed-forward avatar reconstruction model as an animatable 3D tokenizer, which encodes unstructured video frames into structured 3D tokens. However, most real-world videos only provide partial observations of body parts, resulting in excessive blurring or transparency artifacts in the 3D tokens. To address this, we propose a novel visibility-aware diffusion training strategy that replaces invalid regions with learnable tokens and computes losses only over valid regions. We then train a flow-based diffusion model on the token dataset, inherently maintaining the photorealism and animatability provided by the pretrained avatar reconstruction model. Our approach effectively enables the use of large-scale real-world video data to train a diffusion model natively in 3D. We demonstrate the efficacy of our method through diverse and high-fidelity generation and editing results, outperforming existing solutions by a large margin. The project page is available at https://onethousandwu.com/GenLCA-Page.

Abstract

3D Avatar Generation

Explore similar work

GenLCA: 3D Diffusion for Full-Body Avatars from In-the-Wild Videos

PoseDreamer: Scalable and Photorealistic Human Data Generation Pipeline with Diffusion Models

DiGS-Avatar: Single-Image Animatable 3D Human Reconstruction via UV-Space Diffusion