Machine learning for neglected tropical diseases is limited by data, not algorithms: public annotated image sets for leprosy (Hansen's disease) number in the hundreds, orders of magnitude below what generative models require. We ask whether a model trained on abundant chronic wound photography transfers to this low-data regime. We build a three-stage pipeline. First, a DeepLabV3-ResNet50 segmentation network (validation Dice 0.876, IoU 0.799) supplies lesion masks for two wound datasets that ship without them. Second, we assemble a mask-conditioned latent diffusion model from Stable Diffusion 1.5 components and train it on 3,280 region-of-interest wound crops, widening the UNet input convolution from 4 to 11 channels to admit three mask feature maps and a blurred low-frequency context latent. Third, we fine-tune this model on 708 leprosy image-mask pairs drawn from 764 images of approximately 150 patients. We evaluate with LPIPS perceptual distance, anchored by a real-versus-real baseline computed on the same 242 anchor images as the cross-set comparisons; without that reference the cross-set distances cannot be interpreted. The generated set shows no mode collapse: its internal perceptual diversity (0.662) is statistically indistinguishable from that of the real leprosy set (0.672, 95% CI [0.664, 0.680]). Generated images sit 0.044 LPIPS outside the real distribution - measurably apart, but under half of one standard deviation. Fine-tuning shifted the output distribution only marginally, which we trace to lesion geometry reaching the network through input concatenation alone. Chronic wound photography is therefore a viable donor domain for leprosy lesion synthesis: low-level appearance transfers well, and the remaining barrier is semantic control rather than image quality.
Figures & tables
Figure 1: Color calibration card removal (left: original, right: processed). The card is replaced by a flat rectangle filled with the median surrounding color. Two limitations are visible: the top strip of the card is not fully covered, and the replacement region is perfectly uniform. This procedure affected 41.0% of the leprosy corpus.
Corpus
Pairs
Empty
<24 px
Usable
Base training (wound)
Kaggle (ground truth)
2760
24
217
2519
AZH (predicted)
930
204
24
702
WoundsDB (predicted)
79
5
15
59
Total
3769
233
256
3280
Fine-tuning (leprosy)
Table 1: Dataset composition and attrition under the ROI filter. Mask source indicates whether lesion masks are distributed with the corpus or predicted by our Stage 1 network. Usable counts the pairs that actually contributed gradient during training.
Figure 2: Preprocessing artifacts in the training corpora. (a) Uniform letterbox padding is near-universal in the FUSeg and Medetec sub-corpora and near-absent in WSNET. (b) A majority of images on both sides of the transfer contain a large synthetic flat region, from padding in the wound corpus and from card redaction in the leprosy corpus.
Figure 3: Pairs discarded by the ROI filter. AZH loses 21.9% of its pairs to completely empty predicted masks, the clearest available signal of pseudo-label degradation under domain shift.
Figure 4: The three-stage pipeline. Stage 1 trains a segmentation network on the one wound corpus that ships with ground-truth masks and uses it to label the two that do not. Stage 2 pretrains a mask-conditioned latent diffusion model on all wound crops. Stage 3 transfers that model to leprosy. At sampling time the model receives a lesion mask and Gaussian noise only; no real image pixels enter the generation path.
Version
Framing
Mask ch.
UNet in
v1
full frame
1 (binary)
5
v2
ROI, 20% margin
3 (+ distance)
7
v3
ROI, 30% margin
3 (+ distance)
11
Table 2: Architecture progression. Only the final version was evaluated quantitatively; earlier comparisons were qualitative.
Figure 5: Samples from the preliminary DCGAN at 128×128 . Skin tone and coarse lesion structure are present; fine texture and spatial control are not. This limitation motivated the move to mask-conditioned latent diffusion.
Figure 6: Stage 1 segmentation on held-out validation images. Left to right: input, ground-truth mask, predicted mask.
Figure 7: Min-SNR-weighted training loss. Base training (blue) declines steadily over 25,000 micro-steps. Fine-tuning (orange) remains approximately flat over 5,000 , indicating a small parameter movement.
Figure 8: Reverse diffusion from Gaussian noise to a synthetic lesion, sampled every five DPM-Solver++ steps. Global structure resolves early; later steps refine texture.
Figure 9: Base and fine-tuned models on an identical mask and seed. Differences are subtle—slightly redder, more granular tissue after fine-tuning. Paired LPIPS for this example is 0.186 .
Figure 10: Four seeds, one mask, fine-tuned model. Lesion appearance varies widely, supporting the diversity result of Section 7.4 . Shape also varies more than tight mask adherence would allow, and flat rectangular regions of the kind audited in Section 3.4 are visible in two panels.
Comparison
Mean
SD
95% CI
real vs. real
0.672
0.100
[0.664, 0.680]
fine-tuned vs. fine-tuned
0.662
0.065
—
fine-tuned vs. base
0.663
0.065
—
real vs. fine-tuned
0.716
0.089
—
Table 3: LPIPS between-image distances (AlexNet, 2562 ). Each comparison aggregates 1,210 pairs from 242 anchors. Lower means more perceptually similar. The real-versus-real row is the baseline contributed by this work; its confidence interval is a cluster bootstrap over anchor images. Per-pair values for the other three comparisons were not recoverable, so their intervals are omitted.
Figure 11: Perceptual distance between image sets. The dashed line and shaded band give the real-set baseline and its 95% confidence interval. Generated-set diversity ( 0.662 ) essentially matches real-set diversity ( 0.672 ); generated images sit 0.044 outside the real distribution.
Despite recent advances in deep generative modeling, skin lesion classification systems remain constrained by the limited availability of large, diverse, and well-annotated clinical datasets, resulting in class imbalance between benign and malignant lesions and consequently reduced generalization performance. We introduce DermaFlux, a rectified flow-based text-to-image generative framework that synthesizes clinically grounded skin lesion images from natural language descriptions of dermatological attributes. Built upon Flux.1, DermaFlux is fine-tuned using parameter-efficient Low-Rank Adaptation (LoRA) on a large curated collection of publicly available clinical image datasets. We construct image-text pairs using synthetic textual captions generated by Llama 3.2, following established dermatological criteria including lesion asymmetry, border irregularity, and color variation. Extensive experiments demonstrate that DermaFlux generates diverse and clinically meaningful dermatology images that improve binary classification performance by up to 6% when augmenting small real-world datasets, and by up to 9% when classifiers are trained on DermaFlux-generated synthetic images rather than diffusion-based synthetic images. Our ImageNet-pretrained ViT fine-tuned with only 2,500 real images and 4,375 DermaFlux-generated samples achieves 78.04% binary classification accuracy and an AUC of 0.859, surpassing the next best dermatology model by 8%.
Data availability remains a critical bottleneck in many deep learning applications. Large-scale datasets are often expensive to collect, curate and annotate, which can limit the scalability and applicability of supervised learning methods. In this work, we evaluate the classification performance of models trained on synthetic image datasets produced by generative deep learning. In particular, we use latent diffusion models conditioned on learned representations from DINOv2, DINOv3, and CLIP. Our results demonstrates that this representation-conditioned formulation significantly outperforms class-conditioned generation by a large margin (+10.76 p.p. top-1 accuracy on ImageNet100), by improving sample quality and mode coverage. Furthermore, by scaling the size of the synthetic dataset, we are able to outperform a classifier trained on the real data (+2.0 p.p top-1 accuracy). We also demonstrate how generated images can be used for augmentation purposes, outperforming classical augmentation methods, and how the conditioning space can be used for sample filtering to further improve training value. Collectively, these findings highlight that representation-conditioned diffusion models provide a promising approach for augmenting, complementing, or potentially replacing real-world datasets in large-scale visual learning tasks.
Nithesh Chandher Karthikeyan, Jonas Unger, Gabriel Eilertsen
Class imbalance is a persistent challenge in visual recognition, particularly in safety-critical domains where collecting positive examples is expensive and rare events are inherently underrepresented. We propose a lightweight synthetic data augmentation pipeline that fine-tunes a LoRA adapter on as few as 20-50 real images of a rare class and uses a pretrained diffusion model to generate synthetic samples for training. We systematically vary the synthetic-to-real ratio and evaluate the approach across two structurally different domains: chest X-ray pathology classification (NIH ChestX-ray14) and industrial surface crack detection (Magnetic Tile Defect dataset). All evaluations are performed on held-out sets of real images only. Across both domains, synthetic augmentation consistently improves rare-class recall and F1 compared to training with real data alone. Performance improves with moderate synthetic augmentation and shows diminishing returns as the synthetic ratio increases. These results suggest that LoRA-adapted diffusion models provide a simple and scalable mechanism for augmenting rare classes, enabling effective learning in data-scarce scenarios across heterogeneous visual domains.
Daniil Dushenev, Nazariy Karpov, Daniil Zinovjev +2
National University of Science and Technology MISIS, Moscow, Russia