BlendFusion -- Scalable Synthetic Data Generation for Diffusion Model Training

Thejas Venkatesh; Suguna Varshini Velury

cs.CVApr 10, 2026

BlendFusion -- Scalable Synthetic Data Generation for Diffusion Model Training

Authors: Thejas Venkatesh, Suguna Varshini Velury

Organizations: Samaya AI, Inc. · Independent Researcher

Abstract

With the rapid adoption of diffusion models, synthetic data generation has emerged as a promising approach for addressing the growing demand for large-scale image datasets. However, images generated purely by diffusion models often exhibit visual inconsistencies, and training models on such data can create an autophagous feedback loop that leads to model collapse, commonly referred to as Model Autophagy Disorder (MAD). To address these challenges, we propose BlendFusion, a scalable framework for synthetic data generation from 3D scenes using path tracing. Our pipeline incorporates an object-centric camera placement strategy, robust filtering mechanisms, and automatic captioning to produce high-quality image-caption pairs. Using this pipeline, we curate FineBLEND, an image-caption dataset constructed from a diverse set of 3D scenes. We empirically analyze the quality of FineBLEND and compare it to several widely used image-caption datasets. We also demonstrate the effectiveness of our object-centric camera placement strategy relative to object-agnostic sampling approaches. Our open-source framework is designed for high configurability, enabling the community to create their own datasets from 3D scenes.

Figures & tables

Figure 1: Images generated by BlendFusion for different scenes

Scene	Total	Heuristic Passed	VLM Passed	CLIP Score	LAION Aesthetic Score
Bistro Exterior	10368	4666 (45.0%)	3308 (31.9%)	$24.80\pm 3.79$	$4.24\pm 0.74$
Bistro Interior	9488	3172 (33.4%)	1557 (16.4%)	$26.20\pm 3.43$	$4.27\pm 0.68$
Pavillion	840	329 (39.2%)	203 (24.2%)	$25.27\pm 3.36$	$3.54\pm 0.71$
Sun Temple	4136	984 (23.8%)	630 (15.2%)	$23.73\pm 2.86$	$3.80\pm 0.78$
Emerald Square	8240	5105 (62.0%)	4397 (53.4%)	$26.57\pm 3.65$	$5.04\pm 0.94$
Classroom	1456	225 (15.5%)	91 (6.2%)	$28.40\pm 2.89$	$3.22\pm 0.55$

Table 1: Scene-wise statistics for images generated by the BlendFusion pipeline before final dataset curation.

Figure 3: Scene composition for the BlendFusion and FineBLEND datasets. Bars show the number of images per scene, with percentages indicating the proportion of each scene within the respective dataset.

Dataset	N	CLIPScore	Aesthetic
COCO	10,000	25.43 $\pm$ 3.82	5.79 $\pm$ 0.90
CC	10,000	24.72 $\pm$ 4.84	5.22 $\pm$ 1.19
ReLAION	10,000	28.74 $\pm$ 4.99	4.75 $\pm$ 1.23
BlendFusion	11,224	25.76 $\pm$ 3.80	4.50 $\pm$ 0.94
FineBLEND	7,500	25.91 $\pm$ 3.37	4.52 $\pm$ 0.86

Table 2: CLIPScore and LAION aesthetic score comparison across datasets.

Panel of Figure 4 — Figure 4: Qualitative comparison between BlendFusion path-traced renders (top) and SDXL-generated images (bottom) conditioned on the same captions (shown below each pair). The diffusion-generated images exhibit geometric hallucinations (a, c), semantic drift (b), illegible text (d), and structural incoherence (e).

Scene	Method	Heuristic (%)	VLM (%)
Barbershop	Object-centric	16.1	5.2
Emerald Square	Object-centric	62.0	53.4
Sun Temple	Object-centric	23.8	15.2

Table 3: Ablation of camera placement strategies. Our object-centric sampling consistently produces more usable samples after filtering compared to object-agnostic camera sampling baselines.

Synthetic Data Generation

Synthetic data generation creates artificial examples, simulations, and datasets to train, test, or protect machine-learning systems.

See all 623 papers

AC3S: Adaptive Conditioning for 3D-Aware Synthetic Data Generation
Jun 30, 2026Eric Ji, Qiran Hu, Wufei Ma +4Synthetic Data Generation Diffusion-Based Image Generation
University of Illinois Urbana-Champaign · Johns Hopkins University · College of Engineering and Computer Science, VinUniversity
PoseDreamer: Scalable and Photorealistic Human Data Generation Pipeline with Diffusion Models
Mar 30, 2026Lorenza Prospero, Orest Kupyn, Ostap Viniavskyi +2Synthetic Data Generation Human Mesh Recovery
The Podium Institute for Sports Medicine and Technology University of Oxford Oxford, UK · Visual Geometry Group University of Oxford Oxford, UK · Ukrainian Catholic University Lviv, Ukraine
SynCity 3000: Bootstrapping Scene-Scale 3D Diffusion
Jul 6, 2026Paul Engstler, Iro Laina, Christian Rupprecht +13D Scene Generation 3D Scene Layout Generation
Visual Geometry Group, University of Oxford

Jun 30, 2026J/K move · Enter open · S save

AC3S: Adaptive Conditioning for 3D-Aware Synthetic Data Generation

Eric Ji, Qiran Hu, Wufei Ma, Sarthak Jain+3

University of Illinois Urbana-Champaign · Johns Hopkins University · College of Engineering and Computer Science, VinUniversity

Synthetic Data Generation Diffusion-Based Image Generation Conditional Diffusion Models Diffusion Models

Open paper AskPDF arXiv

Synthetic data generation has emerged as a powerful tool for improving data scalability in computer vision. Recent diffusion-based pipelines have demonstrated strong photorealism. However, how to enforce precise 3D structure and pose consistency in generated images remains challenging. Existing methods leverage visual prompts such as edge maps to guide diffusion models, but often suffer from over-conditioning artifacts that degrade image realism and limit dataset quality. In this paper, we present a diffusion-based image generation framework that enforces 3D structural alignment while preserving photorealism through adaptive conditioning. Our framework, Adaptive Conditioning for 3D-Aware Synthetic Data Generation (AC3S), introduces a self-supervised visual prompt modulator that dynamically adjusts the strength of ControlNet conditioning, preventing over-conditioning and enabling the diffusion model to retain its generative expressiveness. To further enhance diversity and semantic consistency, we develop a multi-agent vision language model framework that composes detailed and 3D-aware prompts aligned with the underlying geometric structure. Together, these components enable the scalable generation of high-quality synthetic datasets with accurate 2D and 3D annotations. Extensive experiments demonstrate that our method significantly improves image quality and downstream utility.

Abstract

Synthetic Data Generation

Explore similar work

AC3S: Adaptive Conditioning for 3D-Aware Synthetic Data Generation

PoseDreamer: Scalable and Photorealistic Human Data Generation Pipeline with Diffusion Models

SynCity 3000: Bootstrapping Scene-Scale 3D Diffusion