cs.LGOct 15, 2026
SaveTime to Fake It: Continual Machine Learning Intensifies Unfairness in Deepfake Detection
Organizations: Faculty of Mathematics and Computer Science, University of Bremen, Bremen, Germany
Abstract
Deepfake detectors safeguard against misinformation, rendering their failures as much a societal as a technical concern. As generators constantly evolve, detectors trained on static datasets miss newly arising threats, while progressive training interferes catastrophically. Yet, mitigating forgetfulness in terms of accuracy, the prevalent focus of continual learning (CL), is insufficient to reflect real-world impact. We therefore scrutinize the effects of CL methods with respect to evolving fairness. To this end, we analyze four dimensions of fairness across diverse demographics for detectors trained progressively on a chronological sequence of 11 real-world deepfake generation methods. Tracking fairness metrics over time, we expose non-trivial interplays between performance and fairness that depend on both the choice of CL algorithm and demographic intersections. We find accuracy degradation to typically exacerbate unfairness and, conversely, fairness improvements to remain scarce over time. Dissecting the CL method families (rehearsal, prompts, regularization), we observe that inequitable forgetting is not explained by underrepresentation alone, but likely entangled with representation learning. We thus issue a call to action for CL methods to take social outcomes into account when balancing the stability-plasticity tradeoff.
Figures & tables
Figure 1: Our chronological sequence of 11 Deepfake datasets, on which a single detector is trained continually. Each task shows its index, the generator, size ( = thousands), date and an example image. Beads mark publication dates; bead area is proportional to dataset size and color denotes the generator family. Fairness is evaluated on demographic test annotations after every training step, but training only ever uses real/fake labels.
Figure 2: Demographic distribution across the 11 test sets. Colors denote disjoint attributes, each summing to 100%; green shows binary gender intersected with three skin-tone groups (of ten). Some groups are underrepresented throughout (e.g. very light skin tones), while others fluctuate heavily over time.
Figure 5: Subgroup ( ) trajectories over tasks 1 11 (circle arrow) against cumulative share of training data (log scale). Top (GradMatch, memory-based): memory share , the ’s share of the replay buffer, i.e. how much memory is spent rehearsing it. Middle (SOYO, prompt-based): routing accuracy , the fraction of the ’s test samples from all seen tasks routed to their own task’s prompt. Lower means more often matched with another task’s prompt. Bottom (EWC, regularization-based): Fisher cosine similarity between the diagonal Fisher on the ’s samples and on each task’s full dataset at the final checkpoint. Lower means the ’s important parameters are less protected. Scores are averaged over five seeds. The shared log x-axis allows comparing each subgroup across method families.
Appendix figures & tables16 assets
Supplementary material from the paper’s appendix.
Appendix
| Demographic Attributes | Description |
| Gender | 1 - Male, 0 - Female |
| Age | 0 - Child, 1 - Youth, 3 - Adult, |
| 4 - Middle-aged, 5-Senior | |
| Skin Tone | According to Monk Skin Tone Scale |
| Intersection | 0-(Female,Light), 1-(Female,Medium), |
| 2-(Female,Dark), 3-(Male,Light), |
Table 1: Summary of demographic attributes provided by AI‑Face‑FairnessBench and used in our fairness analysis.
| Datasets | Train | Test | ||||
| Real | Fake | Total | Real | Fake | Total | |
| FaceForensics (FF++) | 17,218 | 84,346 | 101,564 | 4,375 | 21,233 | 25,608 |
| DFD | 37,231 | 55,099 | 92,330 | 9,461 | 13,823 | 23,284 |
| CelebDF | 18,883 | 124,647 | 143,530 | 4,762 | 30,842 | 35,604 |
| DFDC | 30,136 | 30,038 | 60,174 | 7,700 | 7,494 | 15,194 |
Table 2: FaceForensics, DFD, CelebDF and DFDC are the four datasets which have real data already included and provided as part of AI-Face-FairnessBench. Here we include a detailed overview of the number of samples provided for each of these datasets. of every respective training set is held out as validation set.
| SL. No. | Dataset Publication Date | Dataset or Generator Used | Type of Generation | Samples | Real Dataset Used in Training | |
| Train | Test | |||||
| 1 | 25.01.19 | FaceForensics (FF++) | Hybrid | 101,564 | 25,608 | NA |
| 2 | 24.09.19 | DFD | Hybrid | 92,330 | 23,284 | NA |
| 3 | 27.09.19 | CelebDF | Hybrid | 143,530 | 35,604 | NA |
| 4 | 11.12.19 | DFDC | Hybrid | 60,174 | 15,194 | NA |
| 5 | 17.12.20 | VQGAN | GAN | 39,943 | 10,057 | CelebA, MetFaces |
Table 3: Task definitions and chronological ordering used across all our experiments. of every respective training set is held out as validation set. This shows the corresponding date of publication of the dataset sources as provided in ( Lin et al., 2025 ) . Dates refer to the public release of the images, which for some generators (e.g., StyleGAN) may lag the generator’s first publication. The table also shows real data sources used for training and/or evaluation stage. A detailed overview of these real datasets are provided in table 4 .
| Dataset | Source | # Images | Generators | Training | Validation | Test |
| CelebA | https://mmlab.ie.cuhk.edu.hk/ projects/CelebA.html | 202,599 | VQGAN | 35,347 | 3,928 | NA |
| Palette | 3,722 | 414 | NA | |||
| MetFaces | https://github.com/NVlabs/metfaces-dataset | 1,336 | VQGAN | 601 | 67 | NA |
| Palette | 601 | 67 | NA | |||
| CASIA Webfaces | https://github.com/deepinsight/insightface | 490,623 | DCFace | 382,007 | 42,445 | NA |
| FFHQ | AI-Face-FairnessBench | StyleGAN | 3,244 | 360 | 908 |
Table 4: We fetched real data for generators if needed from external datasets or internal ones in AI-Face-FairnessBench. External data are not considered for testing as they do not have demographic annotations available. Here we include a detailed overview of the number of samples provided for each of these datasets. of every respective training set is held out as validation set unless otherwise mentioned.
| Train | Validation | Test | |||||
| # | Dataset | Real | Fake | Real | Fake | Real | Fake |
| 1 | FaceForensics++ | 15,496 | 75,911 | 1,722 | 8,435 | 4,375 | 21,233 |
| 2 | DFD | 33,508 | 49,589 | 3,723 | 5,510 | 9,461 | 13,823 |
| 3 | CelebDF | 16,995 | 112,182 | 1,888 | 12,465 | 4,762 | 30,842 |
| 4 | DFDC | 27,122 | 27,034 | 3,014 | 3,004 | 7,700 | 7,494 |
| 5 | VQGAN | 35,948 | 35,949 | 3,995 | 3,994 | 26,298 | 10,057 |
Table 5: Number of real and fake images per task and split. For VQGAN, Palette and DCFace, real training and validation images come from external sources (Table 4 ), and real test images are re-used from the test sets of earlier tasks.
| Methods | Epochs | Learning Rate | Buffer/ Prompt Size | Specific Parameter |
| Accumulated | 25 | NA | NA | |
| Naive | 25 | NA | NA | |
| EWC | 25 | NA | ||
| DER++ | 25 | Buffer Size = 3000 | ||
| GradMatch | 25 | Buffer Size = 3000 | ||
| SOYO | 25 | Prompt Length = 10 | NA |
Table 6: Overview of method specific and shared hyperparameter settings used in all our experiments.
Figure 6: Inter-task accuracy matrix (mean std over five seeds) for the accumulated upper-bound and naive sequential lower-bound by training step (y-axis) and evaluated task (x-axis). The matrix highlights that each task is solvable in itself (diagonal values), but accuracy on generators not yet seen in training (top-right off-diagonal) remains limited and varies strongly across generators.
Figure 7: Inter-task accuracy matrix (mean std over five seeds) for EWC, GradMatch, PINA, SOYO and S-Prompts by training step (y-axis) and evaluated task (x-axis). The matrix highlights that each task is solvable in itself (diagonal values), but accuracy on generators not yet seen in training (top-right off-diagonal) remains limited and varies strongly across generators. CL methods mitigate forgetting for most generators (bottom-left off-diagonal).
Table 7: Comparison of (Un)fairness at the end of continually learning across the 11 generator sequence with accumulated and naive sequential baselines along with EWC and GradMatch for test sets of all previously observed tasks, i.e., test set 1 after training on 11 tasks, test set 2 after 11 tasks, and so on. To focus on whether the CL algorithm has overall improved or deteriorated fairness, the shown values (mean std over five seeds) are differences of the final fairness with respect to the fairness obtained right after training on the respective generator, e.g. discrepancy in fairness of task 2 after training on task 2 vs. task 2 after training on all 11 tasks. As before, red (blue) marks the many deteriorating (few improving) fairness dimensions. Cells remain white if the absolute difference is less than 2.5% to ensure the figure remains legible.
Table 8: Comparison of (Un)fairness at the end of continually learning across the 11 generator sequence with PINA, SOYO, and S-Prompts for test sets of all previously observed tasks, i.e. test set 1 after training on 11 tasks, test set 2 after 11 tasks, and so on. To focus on whether the CL algorithm has overall improved or deteriorated fairness, the shown values (mean std over five seeds) are differences of the final fairness with respect to the fairness obtained right after training on the respective generator, e.g. discrepancy in fairness of task 2 after training on task 2 vs. task 2 after training on all 11 tasks. As before, red (blue) marks the many deteriorating (few improving) fairness dimensions. Cells remain white if the absolute difference is less than 2.5% to ensure the figure remains legible.
Figure 8: Evolution of fairness of EWC across demographics (x-axes) for (top left), (top right), (bottom left) and (bottom right). Values are absolute fairness on each earlier dataset at the end of training (step 11), averaged over five seeds. As in earlier tables, higher means less fair. Color shows improvement (blue) or deterioration (red) relative to the naive sequential baseline, our reference to emphasize that fairness often deteriorates overall, and in parts even more than under naive fine-tuning.
Figure 9: Evolution of fairness of DER++ across demographics (x-axes) for (top left), (top right), (bottom left) and (bottom right). Values are absolute fairness on each earlier dataset at the end of training (step 11), averaged over five seeds. As in earlier tables, higher means less fair. Color shows improvement (blue) or deterioration (red) relative to the naive sequential baseline, our reference to emphasize that fairness often deteriorates overall, and in parts even more than under naive fine-tuning.
Figure 10: Evolution of fairness of GradMatch across demographics (x-axes) for (top left), (top right), (bottom left) and (bottom right). Values are absolute fairness on each earlier dataset at the end of training (step 11), averaged over five seeds. As in earlier tables, higher means less fair. Color shows improvement (blue) or deterioration (red) relative to the naive sequential baseline, our reference to emphasize that fairness often deteriorates overall, and in parts even more than under naive fine-tuning.
Figure 11: Evolution of fairness of SOYO across demographics (x-axes) for (top left), (top right), (bottom left) and (bottom right). Values are absolute fairness on each earlier dataset at the end of training (step 11), averaged over five seeds. As in earlier tables, higher means less fair. Color shows improvement (blue) or deterioration (red) relative to the naive sequential baseline, our reference to emphasize that fairness often deteriorates overall, and in parts even more than under naive fine-tuning.
Figure 12: Evolution of fairness of PINA across demographics (x-axes) for (top left), (top right), (bottom left) and (bottom right). Values are absolute fairness on each earlier dataset at the end of training (step 11), averaged over five seeds. As in earlier tables, higher means less fair. Color shows improvement (blue) or deterioration (red) relative to the naive sequential baseline, our reference to emphasize that fairness often deteriorates overall, and in parts even more than under naive fine-tuning.
Figure 13: Evolution of fairness of S-Prompts across demographics (x-axes) for (top left), (top right), (bottom left) and (bottom right). Values are absolute fairness on each earlier dataset at the end of training (step 11), averaged over five seeds. As in earlier tables, higher means less fair. Color shows improvement (blue) or deterioration (red) relative to the naive sequential baseline, our reference to emphasize that fairness often deteriorates overall, and in parts even more than under naive fine-tuning.
More in this topic
Algorithmic Fairness
Algorithmic fairness studies how automated decisions distribute errors, opportunities, and harms across groups, and how to measure and reduce disparities.
See all 413 papers