Limitations of Synthetic Data Generation in Specialized Data-Scarce Domains

Class imbalance is a persistent challenge in visual recognition, particularly in safety-critical domains where collecting positive examples is expensive and rare events are inherently underrepresented. We propose a lightweight synthetic data augmentation pipeline that fine-tunes a LoRA adapter on as few as 20-50 real images of a rare class and uses a pretrained diffusion model to generate synthetic samples for training. We systematically vary the synthetic-to-real ratio and evaluate the approach across two structurally different domains: chest X-ray pathology classification (NIH ChestX-ray14) and industrial surface crack detection (Magnetic Tile Defect dataset). All evaluations are performed on held-out sets of real images only. Across both domains, synthetic augmentation consistently improves rare-class recall and F1 compared to training with real data alone. Performance improves with moderate synthetic augmentation and shows diminishing returns as the synthetic ratio increases. These results suggest that LoRA-adapted diffusion models provide a simple and scalable mechanism for augmenting rare classes, enabling effective learning in data-scarce scenarios across heterogeneous visual domains.

Recent generative models can produce high-quality synthetic images, offering scalable training training data for data-hungry models. Existing approaches to exploiting this potential typically involve 1) training or fine-tuning generators, or 2) using lightweight post-hoc adaptation like prompt engineering or inference-time guidance, making them generator-specific and expertise-intensive. We study a complementary question: given a fixed pool of generated images, can downstream utility be improved purely by selecting an informative subset? The answer is yes. We show that effective selection must counter a structural bias of modern generators: they tend to over-produce canonical modes of each class while underrepresenting intra-class variation. Building on this insight, we split each real class into a canonical Homogeneous (HO) subset and a non-redundant Heterogeneous (HE) subset, then score synthetic images by a fidelity-diversity criterion that rewards semantic alignment while penalizing canonical redundancy. The method is generator-agnostic and requires no retraining. Across multiple benchmarks, it consistently outperforms state-of-the-art data selection baselines and matches the real-data performance with up to 40% fewer synthetic samples. The same criterion remains effective when applied on top of stronger task-tuned generators, with gains on both classification and segmentation tasks. Post-generation selection is therefore not a substitute for better generators, but a complementary mechanism for improving the utility of synthetic data.

Generative Adversarial Networks (GANs) can help overcome data scarcity in computer vision tasks by generating additional training samples. In this work, we explore generative data augmentation in two low-resource domains: Bangla handwritten character recognition and chest X-ray image analysis. We use DCGAN-based models trained on 64x64 images to generate synthetic samples and evaluate their quality using Inception Score (IS), Fréchet Inception Distance (FID), and visualization methods such as t-SNE and UMAP. To measure practical usefulness, we train image classifiers using real data and a combination of real and synthetic data. Experimental results show that synthetic augmentation improves data diversity and consistently increases classification performance in limited-data settings. We also investigate training stability techniques, including gradient penalty and spectral normalization, and perform ablation studies on synthetic-to-real data ratios and sample filtering strategies. In addition, we discuss challenges related to medical image evaluation, dataset licensing, and privacy concerns of synthetic data. Our approach is simple, reproducible, and provides a strong baseline for generative augmentation in resource-constrained imaging applications.