High-fidelity removal of eyeglasses from video is a major challenge in facial attribute editing, as the underlying facial geometry is often obscured by complex refractive distortions and view-dependent specular reflections. While large-scale generative priors have shown promise in eye-glasses removal via static image inpainting, they often lack the structural constraints necessary to maintain identity, expression, and pose, leading to visible "identity drift" in both static images and dynamic sequences. In this paper, we propose a novel transfer framework that addresses the stochastic nature of generative priors. Our pipeline first extracts high-fidelity synthetic face images from a commercial-grade generative model (Nano Banana, Gemini 3 Pro Image), regularizes them via a three-stage structural filtering process to preserve identity, expression, and pose, and finally applies physically-based simulation of lens optics during training to provide diverse, paired data. This process transfers Nano Banana's photo-realistic, multi-view knowledge into a specialized restoration architecture, JFSnet (Joint Feature-Spatial network). JFSnet integrates DINOv2-based semantic features with a convolutional decoder for spatial reconstruction, leveraging translation equivariance constraints to improve temporal consistency and high-frequency detail preservation. Evaluations on the curated Flickr-Faces-HQ (FFHQ) subset (12,163 images) show that our approach achieves high fidelity and structural accuracy, while maintaining inference speed of 27.68 FPS. In perceptual studies on CelebV-Text video sequences, our results are consistently preferred over diffusion and GAN-based baselines for ocular consistency, temporal stability, and overall restoration quality.
Figures & tables
Figure 1: Joint Feature-Spatial network removes glasses in videos producing results that are both temporally consistent and structurally faithful. Given an input video frame (a), our approach (b) effectively erases the glasses while preserving facial identity. In comparison, the commercial video editing model Runway Gen-4.5 [ 30 ] alters the subject’s expression and introduces blurriness (c). Similarly, the image-based editor Nano Banana [ 10 ] fails to preserve the expression when applied to video frames (d). The video inpainting method ProPainter [ 47 ] faces different limitations, resulting in incomplete glasses removal (e). In contrast, our method (b) ensures coherent removal and maintains high-fidelity structural integrity throughout the sequence. Source: CelebV-Text [ 39 ] .
Figure 2: Dataset creation pipeline. Raw multi-pose face sets are generated from the Nano Banana prior and enriched with segmentation masks, monocular depth, and parametric face model fits. These sets undergo multi-stage filtering (joint background/ocular L1 Filtering , followed by U-Net reconstruction) and an As-Rigid-As-Possible (ARAP) warp to align clean and glasses-wearing pairs, giving the final dataset.
Figure 3: Refraction simulation. Physics-based lens distortion for −6,−4,−2 , and +1 diopters on a sample from our dataset. Our refraction model captures realistic magnification and minification while preserving facial identity and structure.
Figure 4: Qualitative comparison on high-refraction samples from FFHQ [ 17 ] . Close-up eye regions are shown for each subject. While Nano Banana [ 10 ] removes reflections, it alters structural details (e.g., gaze shift in row 1, eye closure in row 2). Diffusion-based LEDITS [ 37 ] exhibits identity drift or fails to remove glasses (row 1). Video inpainting methods [ 41 , 44 , 47 ] often fail to recover underlying facial structures. Our method effectively removes reflections and corrects refraction while preserving fine facial details and maintaining high structural integrity. See Sec. 4.5 for more details.
Figure 5: Qualitative comparison on CelebV-Text [ 39 ] . Runway Gen-4.5 removes reflections but changes the gaze and eye identity. V-LASIK causes blur and changes eye shape. TOE struggles with strong reflections. ProPainter, FGT, and STTN only perform inpainting, leaving behind shadows, reflections, or parts of the glasses. Our Joint Feature-Spatial network preserves gaze and fine facial details. See Sec. 4.6 .
Video Survey
Image Survey
Our method is preferred in
Our method is preferred in
Nano Banana [ 10 ]
97%
100%
100%
81%
42%
75%
55%
74%
LEDITS [ 37 ]
100%
100%
96%
90%
85%
98%
90%
96%
FGT [ 44 ]
83%
96%
100%
80%
92%
82%
98%
97%
TOE [ 23 ]
91%
97%
98%
85%
96%
89%
100%
99%
V-LASIK [ 32 ]
88%
88%
100%
N/A
N/A
N/A
N/A
N/A
Figure 6: Preference study on CelebV-Text [ 39 ] and FFHQ [ 17 ] . Our approach is consistently preferred across both video and image surveys. The heatmap shows the percentage of participants who preferred our result over the baseline for each evaluation criterion. See Sec. 4.7 for more details.
Method
FID ↓
Pupil (px) ↓
L2 (px) ↓
FPS ↑
Nano Banana [ 10 ]
0.389
2.854±
2.387
0.699±
0.462
1.29
Inpaint Anything [ 40 ]
0.391
2.259±
2.295
0.664±
0.902
1.84
InstructPix2Pix [ 5 ]
1.089
5.541±
6.522
8.038±
18.293
0.05
LEDITS [ 37 ]
1.953
3.905±
2.999
1.527±
2.437
0.08
Take-Off-Eyeglasses [ 23 ]
3.256
2.593±
1.696
0.877±
0.773
13.31
IP-FaceDiff [ 1 ]
3.832
17.607±
8.718
21.257±
27.171
0.04
Table 1: Quantitative evaluation on the Flickr-Faces-HQ (FFHQ) dataset [ 16 ] . We compare our proposed approach against several state-of-the-art image-based, video editing, and video inpainting methods using the Fréchet Inception Distance (FID) ↓ , Pupil Displacement ↓ , landmark L2 error (L2) ↓ , and inference speed (FPS) ↑ . Best results in bold . †
Figure S1 : Visualization of our generated dataset. Row 1: Clean images with warp grids (green points), dense facial landmarks (green: certain, purple: uncertain), and gaze vectors derived from rotation matrices. Row 2: Glasses compositions. Row 3: Clean images with precise lens and frame segmentations. Row 4: Estimated depth maps. Row 5: environment map (left) and the glasses asset (right) used for composition.
Fréchet Inception Distance (FID) ↓
Method
glasses
sunglasses
combined
FPS ↑
Inpaint Anything [ 40 ]
0.391
1.796
0.337
1.84
InstructPix2Pix [ 5 ]
1.089
1.608
0.927
0.05
LEDITS [ 37 ]
1.953
—
1.953
0.08
Take-Off-Eyeglasses [ 23 ]
3.256
2.741
2.839
13.31
RAVE [ 15 ]
11.559
16.741
12.065
0.16
Table S1 : Quantitative evaluation on the Flickr-Faces-HQ (FFHQ) dataset [ 16 ] . We compare our proposed approach against several state-of-the-art image-based, video editing, and video inpainting methods using the Fréchet Inception Distance (FID) ↓ and inference speed (FPS) ↑ . We report results for regular glasses , sunglasses , and the full combined test set. Best results are highlighted in bold .
Approach
RAFT-L1 ×103↓
tLPIPS ×103↓
Laplacian variance ↑
Nano Banana [ 10 ]
9.763
21.215
83.9
LEDITS [ 37 ]
14.455
27.904
73.9
FGT [ 44 ]
6.529
17.722
37.0
TOE [ 23 ]
6.517
16.569
19.9
V-LASIK [ 32 ]
7.760
16.803
54.9
STTN [ 41 ]
7.152
19.658
66.7
Table S2 : Temporal consistency metrics on CelebV-Text clips. RAFT-L1 is the flow-warping error (previous output warped into the current frame by RAFT-Sintel [ 35 ] flow estimated on the inputs); temporal LPIPS (tLPIPS) [ 45 ] is an adjacent-frame perceptual distance. Both measure stability, not restoration quality. Laplacian variance is a sharpness/detail proxy: lower-detail outputs are smoother and score better on stability metrics, so the comparison is conservative for our method. Best per column in bold .
Global
Lens region
Temporal
JFSnet λtemp
SSIM ↑
LPIPS ↓
L1 ×102↓
PSNR ↑
LPIPS ↓
RAFT-L1 ×103↓
tLPIPS ×103↓
✘
0.947
0.025
1.01
27.44
0.00307
8.023
19.520
✔
0.950
0.024
1.00
27.65
0.00300
7.682
18.528
Table S3 : Temporal-loss ablation. Effect of the translation-equivariance loss λtemp . Restoration metrics use a freshly generated, identity-disjoint paired split (300 identities, 3,900 paired samples); temporal metrics use the CelebV-Text video protocol of Table S2 . Despite no paired-video supervision, the temporal loss reduces temporal errors with no spatial-restoration trade-off. Best per column in bold .
Global
Lens region
Approach
SSIM ↑
LPIPS ↓
L1 ↓
PSNR ↑
LPIPS ↓
ProPainter [ 47 ]
0.907
0.054
0.021
15.28
0.037
ProPainter [ 47 ]
0.913
0.044
0.020
16.71
0.027
TOE [ 23 ]
0.916
0.050
0.022
18.98
0.012
TOE [ 23 ]
0.940
0.028
0.014
24.19
0.004
Ours (JFSnet)
0.950
0.024
0.010
27.65
0.003
Table S4 : Prior baselines retrained on our data. We retrain a video inpainter (ProPainter [ 47 ] , run on the input image with our lens masks) and a glasses-specific method (TOE [ 23 ] ) on JFSnet ’s training data, and evaluate on a 300-identity paired set (identity-disjoint, freshly generated, not used in any prior training). : as originally reported; : retrained on our data. Global metrics cover the whole image; lens-region metrics isolate the actual restoration task. Best in bold .
Table S5 : Establishing the “Expected Biometric Drop” for glass removal. To model the biometric penalty of removing glasses, we treat an image with synthetically heavy sunglasses (70% dark, 15% reflection) as the baseline identity. We then compute the identity shift when analytically “removing” these artifacts: restoring the true clear eyes (“Un-Lensing”) and inpainting the frames (“Un-Framing”) with OpenCV’s implementation of [ 36 ] . Combining both establishes a theoretical ceiling for identity preservation during complete glass removal.
Frame Only
Lenses Only
Glasses Area
Full Generated
Method
(Orig. Lenses)
(Orig. Frame)
(Frames + Lenses)
Output
Nano Banana
0.954 ±
0.023
0.962 ±
0.022
0.919 ±
0.031
0.883 ±
0.036
Inpaint Anything
0.961 ±
0.020
0.985 ±
0.024
0.950 ±
0.029
0.948 ±
0.030
InstructPix2Pix
0.961 ±
0.026
0.929 ±
0.055
0.893 ±
0.083
0.625 ±
0.304
LEDITS
0.965 ±
0.028
0.938 ±
0.045
0.900 ±
0.068
0.755 ±
0.126
TOE
0.954 ±
0.026
0.975 ±
0.014
0.934 ±
0.029
0.908 ±
0.030
Table S6 : Component Analysis of Generative Glass Removal across State-of-the-Art Methods. By selectively substituting generated content into specific geometric regions of the original image, we identify the primary sources of identity shift. The results demonstrate that models with high overall ArcFace scores frequently leave structural frame artifacts (as seen in higher Gen. Frame Only scores) and fail to deeply reconstruct the ocular region (as seen in near-perfect Gen. Lenses Only scores). Our pipeline provides the most structurally complete removal, incurring the expected biometric penalty.
Global
Encoder
Decoder
Skip
Encoder
SSIM ↑
LPIPS ↓
L1 ↓
FID ↓
ViT
ViT
—
—
0.941
0.070
0.024
1.131
ViT
ViT
—
✓
0.940
0.170
0.025
0.572
CNN
CNN
—
—
0.895
0.110
0.020
0.493
CNN
CNN
✓
—
0.911
0.113
0.017
0.439
ViT
ViT
✓
✓
0.943
0.171
0.026
0.392
Table S7 : Architectural Ablation. We evaluate the impact of different encoder-decoder backbones, global skip connections, and encoder fine-tuning. Our hybrid JFSnet (ViT encoder, CNN decoder, and global skip) achieves the best performance, effectively balancing high-level semantic consistency with low-level spatial detail. Beyond FID, we report paired restoration metrics (SSIM/LPIPS/L1) for each configuration on the 300-identity paired set.
Figure S2 : Overview of the proposed Joint Feature-Spatial network (JFSnet). Our architecture integrates a pre-trained DINOv2 encoder with a convolutional decoder to restore facial details behind eyeglasses. By leveraging multi-level feature integration and selective fine-tuning, the network bridges high-level semantic identity preservation with high-resolution spatial precision, ensuring consistent restoration across diverse facial poses and optical conditions.
Refr.
Refl.
λperc
λeye-pix
λeye-perc
λadv
λtemp
Enc.
dataset filtering
ARAP warping
FID ↓
—
✓
✓
✓
✓
✓
✓
✓
✓
✓
0.403
✓
✓
—
—
—
—
—
✓
✓
✓
0.402
✓
—
✓
✓
✓
✓
✓
✓
✓
✓
0.398
✓
✓
—
✓
✓
✓
✓
✓
✓
✓
0.398
✓
✓
✓
—
✓
✓
✓
✓
✓
✓
0.394
✓
✓
✓
✓
—
✓
✓
✓
✓
✓
0.394
Table S8 : Ablation Study. Impact of different augmentations, loss components, and encoder freezing strategies on FID scores. Best results are highlighted in bold .
Figure S3 : Additional qualitative comparison on FFHQ. We show results from image-to-image and video-to-video translation methods that showed lower quantitative performance. While these methods can remove glasses, they often fail to preserve the original facial features or introduce artifacts.
Figure S4 : Qualitative comparison on Flickr-Faces-HQ (FFHQ) [ 17 ] . Generative models like LEDITS and Nano Banana struggle to maintain gaze consistency and preserve fine facial details. Video completion methods like ProPainter and FGT, being designed for inpainting, often fail to recover underlying facial structures accurately. Our Joint Feature-Spatial network preserves gaze and fine facial details while maintaining high-fidelity structural integrity. See Sec. 4.5 for more details.
Figure S5 : Failure cases of our approach. We identify several failure modes, shown in row pairs (input and our result) from top to bottom: (i) incomplete removal of eyewear, (ii) incomplete removal of shades cast by glasses on person’s face, (iii) iris deformation, and (iv) refraction undercorrection.
Face Video Restoration (FVR) aims to recover high-fidelity facial videos from degraded input while preserving identity and semantic consistency across frames. Existing methods often struggle to simultaneously address three key challenges: identity shift, viewpoint-entangled guidance, and perceptual realism. To tackle these issues, we propose TIGER, a structured tri-prior fusion framework that Tames Identity, Geometry, and gEnerative pRiors for high-quality FVR. Specifically, an Identity Prior is first established by injecting subject-discriminative embeddings into the latent space, effectively anchoring the subject's identity against severe degradations. Then, to provide temporally consistent structural guidance for dynamic videos, TIGER constructs a Geometry Prior by lifting 2D reference cues into a disentangled 3D parameter space, creating a geometric anchor through cross-source parameter fusion. Moreover, to achieve maximum efficiency without compromising realism, we harness the video generation model's Generative Prior through a one-step rectified flow. We further design a progressive three-stage training optimization strategy that refines structural fidelity, textural reconstruction, and distribution-level realism to ensure robust optimization. We also construct a large-scale FVR dataset to facilitate robust training and standardized evaluation. Extensive experiments demonstrate that TIGER achieves state-of-the-art performance in both identity fidelity and temporal stability, delivering a high-quality, efficient and identity-consistent FVR. Project page: https://yzhoulv.github.io/Tiger/.
Yang Zhou, Wenxue Li, Peng Zhang +3
MiLM Plus, Xiaomi Inc. · The Hong Kong University of Science and Technology (Guangzhou)
Face video restoration from degraded observations is challenging, as it requires simultaneously recovering visual fidelity, temporal consistency, and subject identity. Existing approaches are often either reference-free, which can lead to identity loss when person-specific facial details are lost, or subject-specific, which limits generalization to unseen identities. We propose a subject-agnostic, reference-guided framework for identity-preserving face video restoration. Our method introduces bimodal perceptual-descriptive identity conditioning into a pretrained flow-based text-to-video generator and employs a two-stage training strategy to strengthen identity guidance during restoration. Experiments show that our approach improves restoration fidelity, temporal consistency, and identity preservation, achieving superior performance under challenging video degradations, including downsampling, blur, noise, and compression artifacts. The code is available under: https://github.com/batuhanntosun/RG-FVR.
Cem Eteke, Batuhan Tosun, Eckehard Steinbach
Chair of Media Technology Munich Institute of Robotics and Machine Intelligence School of Computation, Information, and Technology Technical University of Munich 80333 Munich, Germany
Video face restoration aims to enhance degraded face videos into high-quality results with realistic facial details, stable identity, and temporal coherence. Recent diffusion-based methods have brought strong generative priors to restoration and enabled more realistic detail synthesis. However, existing approaches for face videos still rely heavily on generic diffusion priors and multi-step sampling, which limit both facial adaptation and inference efficiency. These limitations motivate the use of one-step diffusion for video face restoration, yet achieving faithful facial recovery alongside temporally stable outputs remains challenging. In this paper, we propose, DVFace, a one-step diffusion framework for real-world video face restoration. Specifically, we introduce a spatio-temporal dual-codebook design to extract complementary spatial and temporal facial priors from degraded videos. We further propose an asymmetric spatio-temporal fusion module to inject these priors into the diffusion backbone according to their distinct roles. Evaluation on various benchmarks shows that DVFace delivers superior restoration quality, temporal consistency, and identity preservation compared to recent methods. Code: https://github.com/zhengchen1999/DVFace.