Attributing authorship to paintings is a historically complex task, and one of its main challenges is the limited availability of real artworks for training computational models. This study investigates whether synthetic images, generated through DreamBooth fine-tuning of Stable Diffusion, can improve the performance of classification models in this context. We propose a hybrid approach that combines real and synthetic data to enhance model accuracy and generalization across similar artistic styles. Experimental results show that adding synthetic images leads to higher ROC-AUC and accuracy compared to using only real paintings. By integrating generative and discriminative methods, this work contributes to the development of computer vision techniques for artwork authentication in data-scarce scenarios.
Figures & tables
Fig. 1: Overview of the image generation pipeline. Starting from Gaussian noise and a prompt (e.g., ”a full owhx painting”), the CLIP encoder conditions the U-Net denoising process in latent space. After multiple refinement steps, the latent image is decoded by a VAE into a full-resolution painting. This process is repeated to generate a synthetic dataset per artist.
Fig. 2: Patch sampling strategies. (A) Original image. (B) M1: baseline sampling with moderate overlap. (C) M2: denser sampling obtained by doubling the number of patches along both axes. Axis color reflects the level of overlap between samples.
Fig. 3: Examples of three synthetic images generated for each artist. The images reflect stylistic variations captured by the model, showcasing differences in composition, subject matter, and brushwork. Despite efforts to mitigate generation biases, some partial figures and cropped elements remain, likely influenced by the characteristics of the training dataset.
Fig. 4: Heatmap of ROC-AUC values for different training settings. The figure illustrates the variation in the model’s ability to discriminate between classes depending on the dataset used for training and testing. Higher values indicate better discrimination between authors.
Fig. 5: Heatmap of Accuracy values across training settings. The figure highlights differences in overall classification performance, showing how accuracy varies depending on whether the model was trained with real, synthetic, or mixed data.
Experiment
Metric
Most Accurate Artist
Best
Least Accurate Artist
Worst
Real-Only
ROC-AUC
TB
0.9593
JH
0.5976
Precision
GD
0.9644
JH
0.8346
Recall
GD
0.9919
JN
0.7163
F1-Score
GD
0.9780
JH
0.8288
Accuracy
GD
0.9573
JH
0.7193
Synthetic-Only
ROC-AUC
GM
0.9991
JN
0.9833
TABLE I: Performance comparison per experiment: best and worst artist-specific results across evaluation metrics
Institute of Automation, Chinese Academy of Sciences, Beijing, China · Beijing Academy of Artificial Intelligence, Beijing, China · University of Chinese Academy of Sciences, Beijing, China +3