cs.CVOct 29, 2026
SavePicturePerfect: Diffusion-based Same-identity Facial Replacement
Organizations: Qualcomm AI Research⋆⋆
Abstract
With the growing success of diffusion models in computer vision, we explore their potential for same-identity facial replacement in photographs. Specifically, we propose a diffusion-based method, built on top of a pre-trained text-to-image model, that takes as input a portrait image of a person and a second reference image of the same individual, potentially captured under different conditions. The goal is to seamlessly replace the input face with the reference face, while keeping the background intact. Surprisingly, despite the clear real-world utility of this task, no recently published work has directly addressed face replacement in this specific setting. To support this goal, we construct a large dataset of image pairs depicting the same person under varying facial expressions and poses. Experimental results demonstrate that our approach produces more realistic and identity-consistent results than existing face reenactment models.
Figures & tables
Figure 1: We present PicturePerfect , a diffusion-based identity preserving facial replacement approach. Prior methods like FaceAdapter [ 20 ] tend to alter the facial identity and lose realism on both in-distribution (ID) ( left ) and out-of-distribution (OOD) ( right ) face replacement tasks.
Figure 2: PicturePerfect network overview: Given a source ( ) and target ( ) image pair, we extract the driving image ( ) by segmenting the face from the target image. To simulate real-world variations, we apply data augmentations such as scaling, rotation, translation, and relighting. The image triplet is then encoded using the SDXL [ 42 ] VAE to obtain latent representations and concatenated spatially.
Figure 3: PicturePerfect dataset samples. For each neutral face (top row), we generate 20 expressive variants using a face animation model [ 19 ] . For visualization purposes, we display three randomly sampled expressive variants per neutral face.
Figure 4: Face Replacement ( in-distribution ). Our method produces high-quality, realistic reconstructions with stronger identity preservation by leveraging both the source and masked driving image (red inset), outperforming other approaches for diverse identities.
Figure 5: Face Replacement ( out-of-distribution ). Our method demonstrates strong generalization for a wide range of facial poses, expressions, identities, and lighting conditions. Moreover, it effectively generates reenacted source images that preserve the subject’s identity. Rows 1-3 showcase results on authentic image-pairs from the PicturePerfect test-set, while row 4 contains results on synthetic images.
| Replacement out-of-distribution (OOD) | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| PicturePerfect (Real) | VoxCeleb2 (LivePortrait) | |||||||||
| Methods | CSIM | FID | AED | APD | AGD | CSIM | FID | AED | APD | AGD |
| DAM | 0.350 | 55.16 | 2.61 | 0.2222 | 15.29 | 0.246 | 38.51 | 2.91 | 0.2472 | 15.59 |
| FADM | 0.342 | 74.98 | 2.66 | 0.2252 | 16.51 | 0.219 | 61.76 | 3.01 | 0.2490 | 16.45 |
| IP-Adapter | 0.275 | 39.16 | 2.45 | 0.2666 | 16.59 | 0.214 | 23.68 | 3.14 | 0.2761 | 23.13 |
| FaceAdapter | 0.343 | — | 2.53 | 0.2076 | 11.31 | 0.207 | — | 3.06 | 0.3104 | 14.61 |
Table 1: Quantitative comparison with prior approaches using ArcFace CSIM [ 12 ] , FID, pose (APD) [ 31 ] , expression (AED) [ 31 ] , gaze (AGD) [ 5 ] errors, LPIPS [ 70 ] , and PSNR. Best scores are highlighted in blue , and second-best are underlined . FFHQ and VoxCeleb2 samples are generated with LivePortrait, while PicturePerfect uses curated real-world pairs. Baselines with unreliable FID scores are marked as (—).
Figure 6: User study: Replacement ID and OOD. Participants were asked to mark {source, output} pairs as “real” or “fake” based on perceptual quality. We report the proportion of images marked as “real” for each approach. The error bars represent the standard error of the mean (SEM).
Figure 7: Ablation study: Effects of Relighting. Without relighting, lighting artifacts from the driving image often leak into the output. Incorporating random relighting into the data augmentation pipeline produces significantly more realistic and consistent illumination.
Figure 8: Ablation study. We evaluate replacing spatial to channel-wise concatenation, swapping SDXL to SDv1.5, as well as disabling relighting, all data augmentations and inner face extraction.
| Replacement out-of-distribution (OOD) | ||||||||||
| PicturePerfect (Real) | VoxCeleb2 (LivePortrait) | |||||||||
| Methods | CSIM | FID | AED | APD | AGD | CSIM | FID | AED | APD | AGD |
| Ours | 0.631 | 28.08 | 2.50 | 0.2262 | 9.87 | 0.579 | 12.44 | 2.78 | 0.2366 | 12.78 |
| Channel-wise | 0.618 | 80.70 | 3.37 | 0.3113 | 8.47 | 0.548 | 62.26 | 3.69 | 0.3418 | 11.80 |
| SDv1.5 | 0.655 | 30.98 | 2.50 | 0.2164 | 9.71 | 0.576 | 12.96 | 2.84 | 0.2413 | 13.19 |
Table 2: Ablation study. We compare spatial versus channel-wise concatenation of the input images and assess the performance with SDv1.5 as the backbone UNet architecture [ 45 ] . We report the main ablation results here and defer the remaining variants to the supplementary material.
Figure 9: Real-world simulation of composite picture editing (“burst” mode reconstruction). Each one of the source faces is individually edited using the corresponding target face as the driving image. The model automatically adjusts for orientation, head pose, and blending issues to naturally fit the source composite photograph.
Figure 10: Comparing our facial replacement with instruction-based image editing approaches [ 8 , 7 ] . Note that, our approach uses a masked driving image (show in red inset), while the image editing approaches do not require a driving image. Our approach makes realistic edits following the driving image, while the image editing methods have no control over the facial edits, leading to unrealistic changes to personal identity.
Figure 11: Comparing our facial replacement with multimodal-based editing approach [ 48 ] . Note that, our approach uses a masked driving image (showed in red inset), while the SeeDream4.0 uses the full driving image. Our approach makes realistic edits following the driving image, while the image editing method has quality output but has tendency to change person’s identity.
Figure 12: Although our proposed approach achieves more realistic facial replacement compared to prior methods, it still exhibits limitations in certain areas, particularly around the eyes and hair.
| Replacement in-distribution (ID) | |||||||
|---|---|---|---|---|---|---|---|
| FFHQ | |||||||
| Methods | CSIM | FID | AED | APD | AGD | LPIPS | PSNR |
| DAM | 0.557 0.0034 | 25.28 | 2.91 0.0128 | 0.2796 0.0022 | 11.89 0.1247 | 0.0380 0.0003 | 21.37 0.0491 |
| FADM | 0.542 0.0034 | 43.24 | 2.97 0.0138 | 0.2849 0.0022 | 13.28 0.1278 | 0.0453 0.0004 | 19.76 0.0464 |
| IP-Adapter | 0.312 0.0018 | 17.22 | 3.01 0.0116 | 0.2910 0.0022 | 16.28 0.1839 | 0.0835 0.0004 | 14.78 0.0312 |
| FaceAdapter | 0.429 0.0064 | 122.47 | 2.97 0.0327 | 0.2439 0.0049 | 10.37 0.3383 | 0.1252 0.0009 | 10.32 0.0824 |
Table 3: We quantitatively compare our method with prior approaches ( Standard Error of Mean) on replacement-ID testsets FFHQ [ 26 ] and VoxCeleb [ 37 , 11 ] .
Figure 13: Metrics like PSNR and LPIPS often favor blurry, low-quality reconstructions due to sensitivity to slight misalignments, failing to align with human judgment.
Figure 14: Ablation: Robustness to data augmentations. Our solution demonstrates the ability to mitigate the effects of different data augmentations, resulting in more stable and visually appealing results.
Figure 15: Additional face replacement (in-distribution) results
Figure 16: Additional face replacement (out-of-distribution) results
| Replacement out-of-distribution (OOD) | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| PicturePerfect (Real) | VoxCeleb2 (LivePortrait) | |||||||||
| Methods | CSIM | FID | AED | APD | AGD | CSIM | FID | AED | APD | AGD |
| Ours | 0.631 | 28.08 | 2.50 | 0.2262 | 9.87 | 0.579 | 12.44 | 2.78 | 0.2366 | 12.78 |
| Channel-wise | 0.618 | 80.70 | 3.37 | 0.3113 | 8.47 | 0.548 | 62.26 | 3.69 | 0.3418 | 11.80 |
| SDv1.5 | 0.655 | 30.98 | 2.50 | 0.2164 | 9.71 | 0.576 | 12.96 | 2.84 | 0.2413 | 13.19 |
| No Relighting | 0.759 | 26.92 | 2.43 | 0.2116 | 8.67 | 0.685 | 11.91 | 2.83 | 0.2465 | 13.00 |
Table 4: Additional ablation study of our identity-preserving face replacement framework. We compare the final model against variants without relighting, without data augmentation, and using the driving image as-is. While some ablations achieve higher reconstruction-oriented metrics, these settings introduce shortcut cues that can artificially increase similarity to the target image and therefore do not reflect robust identity-preserving face replacement.
Figure 17: Experiment with cross-identity facial replacement where the identity along with the expression from the driving image gets transferred to source.
Figure 18: Experiment with cross-identity facial replacement where only the expression from the driving image gets transferred to source while keeping the identity unaltered.
Figure 19: PicturePerfect dataset samples. For each neutral face (top row), we generate 20 expressive variants using a face animation model [ 19 ] . For visualization purposes, we display four randomly sampled expressive variants per neutral face.
Figure 20: PicturePerfect Dataset generation process overview: Neutral looking faces are detected and extracted from source videos, which are guided via driving videos to generate expressive animated faces using LivePortrait [ 19 ] .
Figure 21: Overview of some example driving videos used to animate the neutral faces with LivePortrait [ 19 ] .
Figure 22: Random samples from the dataset. The PicturePerfect dataset features a wide diversity of backgrounds, lighting conditions, ethnicities, clothing styles, poses, and facial expressions.
More in this topic
Identity-Preserving Image Generation
Identity-preserving image generation creates or edits images while retaining a person's recognizable features across changes in pose, style, or scene.
See all 51 papers