The RenAIssance: Caption-Guided Diffusion Models for Historical-Art Visual Media
Organizations: Sorbonne University, Paris, France
Abstract
Generative AI can expand cultural collections into prompt-responsive visual media, but historical artworks rarely provide the paired descriptions needed to adapt modern image generators. We present The RenAIssance, a scalable pipeline that turns a collection of historical European artworks into a reusable text-to-image media resource. The pipeline generates scene-level captions with BLIP and uses the resulting approximately 15,000 image--text pairs from DEArt to train a compact LoRA adapter for Stable Diffusion v1.5. The adapter shifts generated images toward the historical-art distribution while retaining the base model's ability to render novel concepts. On historical and contemporary prompts of different lengths, it improves FID to the DEArt reference distribution in every setting with only a modest decrease in CLIP text--image similarity. Qualitative results show that the same adapter can render both period-appropriate subjects and contemporary concepts in a coherent historical visual language. By decoupling a reusable style representation from the base generator, the approach offers a practical building block for controllable historical-art media creation, interactive authoring, and rich-media experiences.
Figures & tables
| CLIP Similarity | Base Model | LoRA (5000 Steps) |
|---|---|---|
| Short Historical Prompts | 31.407 | 30.311 |
| Short Contemporary Prompts | 31.080 | 29.654 |
| Long Historical Prompts | 34.610 | 34.430 |
| Long Contemporary Prompts | 35.820 | 34.740 |
| FID | Base Model | LoRA (5000 Steps) |
|---|---|---|
| Short Historical Prompts | 230.11 | 207.57 |
| Short Contemporary Prompts | 252.08 | 229.18 |
| Long Historical Prompts | 230.91 | 221.45 |
| Long Contemporary Prompts | 270.17 | 256.66 |
T2I Generation
Text-to-image generation creates images from natural-language prompts using generative models, with research focused on fidelity, control, efficiency, and safety.
See all 329 papers