cs.CVNov 25, 2024
SaveText-to-Image Synthesis: A Decade Survey
Organizations: School of Software & Microelectronics, Peking University, Beijing 102600, China. Beijing 100871, China. · School of Computer Science, Peking University, Beijing 100871, China.
Abstract
Text-to-image synthesis (T2I) focuses on generating high-quality images from textual descriptions In this survey, we review 490 papers on T2I spanning the past decade, with representative advances covered through 2026. We start by briefly introducing how GANs, autoregressive models, and diffusion models have been used for image generation, and we additionally examine the shared components whose evolution has driven recent progress. Building on this foundation, we discuss the development of these models for T2I and organize cutting-edge research by problem rather than by chronology alone, covering controllability, personalized and identity-preserving generation, instruction-based image editing, preference alignment and post-training with human and AI feedback, efficiency, safety, and consistency in content and spatial relationships. We further review unified multimodal understanding-and-generation models, summarize the datasets and evaluation metrics commonly used in T2I research, and provide a comparative discussion of the three paradigms together with their respective trade-offs. Finally, we discuss the potential applications of T2I within AIGC, along with the challenges and forward-looking research opportunities in this field.
Figures & tables
Figure 1: Supplementary survey timeline of representative T2I milestones from 2015 to 2026. The lanes indicate historical emphasis rather than mutually exclusive model families; hybrid systems can combine different representations, objectives, backbones, and tasks. This figure is a qualitative synthesis and is not a performance ranking.
Figure 2: The text-conditional convolutional GAN architecture Reed et al. (2016) . Text encoding is used by both generator and discriminator. It is projected to lower dimensions and depth concatenated with image feature maps for further stages of convolutional processing.
Figure 3: The training of DALL-E Ramesh et al. (2021) is divided into two stages. The first stage trains the codebook of VQ-VAE, while the second stage trains the Transformer, corresponding to Stage One and Stage Two indicated in the figure.
Figure 4: The framework of CogView Ding et al. (2021) . [ROI1], [BASE1], etc., are seperator tokens.
Figure 5: DDPM Ho et al. (2020) generates new images by adding noise and then denoising.
Figure 6: Selected samples from GLIDE Nichol et al. (2021) using classifier-free guidance. We observe that the model can produce photorealistic images with shadows and reflections, can compose multiple concepts in the correct way, and can produce artistic renderings of novel concepts.
Figure 7: The condition LDM Rombach et al. (2022) either via concatenation or by a more general cross-attention mechanism.
Figure 8: The Diffusion Transformer (DiT) architecture Peebles and Xie (2023) . The convolutional U-Net denoiser is replaced by a Transformer that operates on a sequence of noised latent patches, with conditioning such as timestep and class or text embeddings injected through adaptive layer normalization.
| Aspect | GAN | Autoregressive (AR) | Diffusion / Flow |
|---|---|---|---|
| Training objective | Adversarial min–max game Goodfellow et al. (2014) | Likelihood over image tokens Ramesh et al. (2021) | Denoising score matching Ho et al. (2020) or flow matching Lipman et al. (2022) |
| Image representation | Continuous pixels/latents | Discrete (or continuous) tokens | Continuous latents (or pixels) |
| Sampling | Single forward pass | Sequential token decoding | Iterative denoising / ODE integration |
| Main inference cost | Low (one step) | Grows with sequence length | Grows with number of steps |
| Typical risk | Training instability, mode collapse | Exposure bias, slow decoding | High sampling cost (mitigated by distillation / few-step flow) |
| Control & editing | Latent manipulation | Token in-filling / prompting | Cross-attention control, inversion, instruction editing |
Table 1: Qualitative comparison of the three T2I paradigms. Entries describe typical designs and tendencies, not absolute guarantees, and are not a performance ranking.
| Year | Source | Model and Description |
|---|---|---|
| 2016 | ICML | Text-conditional GAN Reed et al. (2016) : The first GAN used for T2I tasks. |
| 2017 | arXiv | TAC-GAN Dash et al. (2017) : Incorporated auxiliary classification tasks into the discriminator. |
| ICCV | StackGAN Zhang et al. (2017) : 1. Generate Stage-I low-resolution images; 2. Generate high-resolution images using text descriptions. | |
| TPAMI | StackGAN++ Zhang et al. (2018a) : Composed of multiple generators and discriminators in a tree-like structure. | |
| 2018 | ECCV | Fused GAN Bodla et al. (2018) : Two generators for conditional and unconditional image synthesis. |
| CVPR | HDGAN Zhang et al. (2018b) : Accompanying hierarchical-nested adversarial objectives inside the network hierarchies. |
Table 2: Overview of GAN models in T2I by year.
| Year | Source | Model and Description |
|---|---|---|
| 2021 | ICML | DALL-E Ramesh et al. (2021) : The first to use AR for generation in T2I. |
| NeurIPS | CogView Ding et al. (2021) : A 4-billion-parameter Transformer with a VQ-VAE tokenizer. | |
| arXiv | M6 Lin et al. (2021) : The largest dataset for Chinese multimodal pre-training. | |
| 2022 | TMLR | Parti Yu et al. () : Uses image token sequences as the target output instead of text tokens. |
| NeurIPS | CogView2 Ding et al. (2022) : A solution based on hierarchical transformers and local parallel autoregressive generation. | |
| ECCV | Make-A-Scene Gafni et al. (2022) : Control mechanism complementary to text in the form of a scene. |
Table 3: Overview of AR models in T2I by year.
| Year | Source | Model and Description |
|---|---|---|
| 2021 | arXiv | GLIDE Nichol et al. (2021) : The first diffusion work in T2I. |
| 2022 | CVPR | VQ-Diffusion Gu et al. (2022) : Based on a vector quantization diffusion model. |
| NeurIPS | Imagen Saharia et al. (2022) : Combines Transformer language models and diffusion processes. | |
| CVPR | Stable Diffusion Rombach et al. (2022) : Applies the diffusion model to the latent space of a powerful pretrained autoencoder. | |
| arXiv | DALLE-2 Ramesh et al. (2022) : A two-stage model generating CLIP image embeddings based on text. | |
| CVPR | Blended Diffusion Avrahami et al. (2022) : Using natural language guidance to produce realistic and diverse images. |
Table 4: Overview of DM models in T2I by year.
| Model | Visual representation / sharing | Image-generation objective | Main survey interpretation |
|---|---|---|---|
| Unified-IO Lu et al. (2022) | Unified task/token interface | Token generation | Connects image generation with a broad set of vision-language tasks. |
| CM3Leon Yu et al. (2023) | Token interface, LM recipe | Autoregressive tokens | Shows a text-LM training recipe transfers to multimodal generation. |
| Janus Wu et al. (2025b) | Decoupled visual encoders | Autoregressive generation | A unified framework need not share one visual encoder. |
| Show-o Xie et al. (2024) | Unified Transformer | Discrete diffusion | Image-side discrete diffusion differs from continuous latent diffusion. |
| Transfusion Zhou et al. (2024a) | Mixed-modal shared model | Continuous diffusion | Language and image prediction use different objectives in one model. |
| BAGEL Deng et al. (2025) | Mixture of Transformers | Flow-based generation | Joint task support must be separated from evidence of cross-task benefit. |
Table 5: Representative unified understanding-and-generation models. The table records architecture choices, not a cross-paper performance ranking obtained under different protocols.
| Research Area | Model |
|---|---|
| Sec. 4.1 : Alignment and Human Feedback | RAHF Liang et al. (2024) , DifffAgent Zhao et al. (2024a) , HPS Wu et al. (2023e) , ImageReward Xu et al. (2024a) , Multi-dimensional Human |
| Preference (MHP) Zhang et al. (2024e) , MPS Zhang et al. (2024e) | |
| Sec. 4.2 : Personalized Generation | Textual Inversion Gal et al. (2022) , PIA Zhang et al. (2024h) , Prompt-Free Diffusion Xu et al. (2024b) , ADI Huang et al. (2024c) , HyperDreamBooth Ruiz et al. (2024) , |
| DreamBooth Ruiz et al. (2023) , Personalized Residuals Ham et al. (2024) , PALP Arar et al. (2024) , PhotoMaker Li et al. (2024m) , DreamTurner Hua et al. (2023) , | |
| ELITE Wei et al. (2023) , Tailored Visions Chen et al. (2024f) , RealCustom Huang et al. (2024a) , SAG Chan et al. (2024) , CoRe Wu et al. (2024a) , Imagine Yourself He et al. (2024c) , | |
| FastComposer Xiao et al. (2024a) , InstantID Wang et al. (2024e) , Specialist Diffusion Lu et al. (2023) , ControlStyle Chen et al. (2023a) , UniPortrait He et al. (2024a) , |
Table 6: Overview of T2I research directions.
Figure 9: With just a few images (typically 3-5) of a subject (left), DreamBooth Ruiz et al. (2023) , the AI-powered photo booth, can generate a myriad of images of the subject in different contexts (right), using the guidance of a text prompt. The results exhibit natural interactions with the environment, as well as novel articulations and variation in lighting conditions, all while maintaining high fidelity to the key visual features of the subject.
Figure 10: Fine-tuning of DreamBooth Ruiz et al. (2023) . Given approximately 3-5 images of a subject, we fine-tune a text-to-image diffusion model with the input images paired with a text prompt containing a unique identifier and the name of the class the subject belongs to (e.g., “A [V] dog”). In parallel, we apply a class-specific prior preservation loss, which leverages the semantic prior that the model has on the class and encourages it to generate diverse instances belonging to the subject’s class using the class name in a text prompt (e.g., “A dog”).
Figure 11: Controlling Stable Diffusion Rombach et al. (2022) with Learned Conditions. ControlNet Zhang et al. (2023c) allows users to add conditions like Canny edges (top), human pose (bottom), etc., to control the image generation of large pretrained diffusion models. The default results use the prompt “a high-quality, detailed, and professional image”. Users can optionally give prompts like “chef in kitchen”.
Figure 12: Stable Diffusion Rombach et al. (2022) ’s U-net architecture is connected with a ControlNet Zhang et al. (2023c) on the encoder blocks and middle block. The locked, gray blocks show the structure of Stable Diffusion V1.5 (or V2.1, as they use the same U-net architecture). The trainable blue blocks and the white zero convolution layers are added to build a ControlNet.
Figure 13: Style transfer results using the proposed method. Given only a single input painting, InST Zhang et al. (2023d) can accurately transfer the style attributes such as semantics, material, object shape, brushstrokes and colors of the references to a natural image with a very simple learned textual description “[C]”.
Figure 14: Different target texts applied to the same image. Imagic Kawar et al. (2023) edits the same image differently depending on the input text.
Figure 15: SVDiff Han et al. (2023) proposes the Cut-Mix-Unmix data augmentation technique to enhance the quality of multi-subject image generation, which is a simple text-based image editing framework.
Figure 16: Images shown in each group share the same source image. AdvDm Liang et al. (2023) use textual inversion Gal et al. (2022) to extract the style of training samples from WikiArt, shown in a separate column. For each group, the top row shows the generated images based on the style extracted from the clean examples. The bottom row shows the generated images based on the style extracted from the adversarial examples. Strength is a hyper-parameter that indicates how much the style of the source image is covered by the target style. LDM fails to capture the style from adversarial examples, compared to clean images.
Figure 17: Given a pretrained text-to-image diffusion model (e.g., Stable Diffusion Rombach et al. (2022) ) method in GSN Chefer et al. (2023) , Attend-and-Excite, guides the generative model to modify the cross-attention values during the image synthesis process to generate images that more faithfully depict the input text prompt. Stable Diffusion alone (top row) struggles to generate multiple objects (e.g., a horse and a dog). However, by incorporating Attend-and-Excite (bottom row) to strengthen the subject tokens (marked in blue), we achieve images that are more semantically faithful with respect to the input text prompts.
Figure 18: The existing vision-language datasets do not capture spatial relationships well. To alleviate this shortcoming, we synthetically re-caption 6M images with a spatial focus, and create the SPRIGHT (SPatially RIGHT) Chatterjee et al. (2025) dataset. Shown above are samples from the COCO Validation Set, where text in denotes ground-truth captions and text in are corresponding captions from SPRIGHT.
Figure 19: Emotional Image Content Generation (EICG). Given an emotion category, The network EmoGen Yang et al. (2024b) produces images that exhibit unambiguous meanings (semantic-clear), reflect the intended emotion (emotion-faithful) and incorporate varied semantics (semantic-diverse).
Figure 20: Colors and contrast. The top row is SDXL Podell et al. (2023) , bottom row is Playground v2.5. The model Playground v2.5 Li et al. (2024a) can generate samples with more vibrant colors and contrast.
Figure 21: Overview of MLLM-Guided Image Editing (MGIE) Fu et al. (2023b) , which leverages MLLMs to enhance instruction-based image editing. MGIE learns to derive concise and expressive instructions while providing explicit visual-related guidance for the intended goal. The diffusion model jointly trains and achieves image editing with latent imagination through the edit head in an end-to-end manner, showing that the module is trainable or frozen, respectively.
| Dataset | Year | Size | Source |
|---|---|---|---|
| Oxford 102 Flower Dataset Nilsback and Zisserman (2008) | 2008 | 8,189 | University of Oxford |
| UIUC Pascal Sentence Rashtchian et al. (2010) | 2010 | 1,000 | VOC2008 |
| SBU Captions Ordonez et al. (2011) | 2011 | 1,000,000 | Flickr.com |
| Flickr8K Hodosh et al. (2013) | 2013 | 8,092 | Flickr.com |
| COCO Lin et al. (2014) | 2014 | 330,000+ | Microsoft |
| ABSTRACT-50S Vedantam et al. (2015) | 2015 | 500 | ASD |
Table 7: Datasets in the T2I task. The “Size” column follows the unit reported by each resource (images, image-text pairs, prompts, or QA items) and is not directly comparable across rows.
| Metric | Year | Human Evaluated | Positive Correlation |
|---|---|---|---|
| SSIM Wang et al. (2004) | 2004 | ||
| PSNR Hore and Ziou (2010) | 2010 | ||
| CIDEr Vedantam et al. (2015) | 2015 | ||
| IS Salimans et al. (2016) | 2016 | ||
| C2ST Lopez-Paz and Oquab (2016) | 2016 | ||
| FID Heusel et al. (2017) | 2017 |
Table 8: Evaluation metrics in the T2I task. “Human Evaluated” and “Positive Correlation” are coarse descriptors taken from each metric’s own reporting, not a unified re-evaluation.
| Benchmark | What it evaluates | Scoring | Main limitation |
|---|---|---|---|
| MME-Unify Xie et al. (2025) | Unified understanding, generation, and mixed-modality interaction | Multi-task, model-based | Mixed-task protocols vary; interaction gains hard to isolate |
| KontextBench Labs et al. (2025) | In-context generation and single-/multi-turn editing (1,026 image-prompt pairs) | Human and automatic | Introduced with a specific model family |
| LongT2IBench Yang et al. (2025) | Long-prompt generation with graph-structured annotations | Structured, automatic | Focused on long-prompt setting |
| WISE Niu et al. (2025) | World-knowledge-informed semantic correctness | Knowledge-grounded checks | Depends on knowledge coverage |
| GenExam Wang et al. (2025c) | Multidisciplinary “exam-style” prompts | Rubric/answer-based | Subject coverage is finite |
| Qwen-Image-Bench Li et al. (2026) | Creator-oriented real-world fidelity and creative generation | Human and model-based | Creator pool and judge may share source bias |
Table 9: Representative 2025–2026 T2I benchmarks, organized by evaluation target. Scores across these resources are not directly comparable because inputs, judges, and model pools differ.
Figure 22: Example storytelling images generated by the method AutoStory Wang et al. (2023b) . it can generate text-aligned, identity-consistent, and high-quality story images from user-input stories and characters (the dog and cat on the left, specified by about 5 images per character), without additional inputs like sketches.
Figure 23: Example images generated by Kolors Team (2024) , with the text description “refreshing summer juice drink”. Kolors can produce a variety of clear image results.
Figure 24: Given Kolors Team (2024) the content “explain the process of photosynthesis”, it can generate a series of images related to photosynthesis to enhance educational significance.
Figure 25: Given the text description “a futuristic city”, a series of images can be generated to capture users’ attention. This example was generated by Kolors Team (2024) .
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 26: The model, GigaGAN [ 157 ] , shows GAN frameworks can also be scaled up for general T2I synthesis and superresolution tasks, generating a 512px output at an interactive speed of 0.13s, and 4096px within 3.7s. Selected examples at 2K resolution.
Figure 27: Sampling is derived from Stable Diffusion 3 [ 82 ] , generating fantastical pictures.
Figure 28: Hunyuan-DiT [ 208 ] can generate images containing Chinese elements; the examples shown are generated directly from Chinese prompts.
Figure 29: RAHF [ 213 ] region inpainting with Muse [ 32 ] generative model. From left to right, original images with artifacts from Muse, implausibility heatmaps predicted by RAHF, masks by processing (thresholding, dilating) the heatmaps, and new images from Muse region inpainting with the mask, respectively.
More in this topic
T2I Generation
Text-to-image generation creates images from natural-language prompts using generative models, with research focused on fidelity, control, efficiency, and safety.
See all 290 papers