Metric-Guided Synthetic Image Data Rendering for Deep Learning compatible with Agentic AI

Synthetic 3D scene generation is increasingly used as a data source for computer vision and embodied AI, but existing generators often optimize perceptual realism without reliably satisfying task-critical functional constraints. This mismatch limits the usefulness of synthetic data for downstream training, where accessibility, traversability, and spatial rule compliance are often essential. We present iARCS, an iterative agentic reinforcement learning framework that adapts a pretrained scene generator to natural-language task requirements. iARCS uses a two-stage strategy: universal-reward pretraining to improve physical plausibility and layout quality, followed by task-specific fine-tuning with LLM-generated reward programs that are iteratively refined from training feedback. Experiments show improved constraint fidelity on walkability, reachability, and clearance-focused tasks, effective task-specific constraint optimization, and competitive scene diversity. We further show that data generated by iARCS improves a base generator, supporting its value as a practical synthetic data generation tool rather than only a controllable scene editing method.

Acquiring labeled datasets for 3D human mesh estimation is challenging due to depth ambiguities and the inherent difficulty of annotating 3D geometry from monocular images. Existing datasets are either real, with manually annotated 3D geometry and limited scale, or synthetic, rendered from 3D engines that provide precise labels but suffer from limited photorealism, low diversity, and high production costs. In this work, we explore a third path: generated data. We introduce PoseDreamer, a novel pipeline that leverages diffusion models to generate large-scale synthetic datasets with 3D mesh annotations. Our approach combines controllable image generation with Direct Preference Optimization for control alignment, curriculum-based hard sample mining, and multi-stage quality filtering. Together, these components naturally maintain correspondence between 3D labels and generated images, while prioritizing challenging samples to maximize dataset utility. Using PoseDreamer, we generate more than 500,000 high-quality synthetic samples, achieving a 76% improvement in image-quality metrics compared to rendering-based datasets. Models trained on PoseDreamer achieve performance comparable to or superior to those trained on real-world and traditional synthetic datasets. In addition, combining PoseDreamer with synthetic datasets results in better performance than combining real-world and synthetic datasets, demonstrating the complementary nature of our dataset. We will release the full dataset and generation code.

Synthetic images are increasingly used to augment scarce real data for object detection. However, not all synthetic sets help equally, and the only way to know a set's value is to train a detector on it, which is slow and demands dense annotation. We ask whether a training-free metric can instead rank candidate synthetic training sets by their downstream utility. Existing image-set metrics such as FID, KID, and MMD compare two feature distributions with a single global statistic, which we show is mis-specified for detection-data selection in two ways: it is blind to per-image composition (object count, box scale, class mix), and even at fixed composition its global averaging washes out the appearance differences that separate high-mAP pools from low-mAP ones. We propose Conditional-Composition Domain Match (CCDM), which converts any feature-space distance into a composition-stratified comparison, matching candidate and target within metadata-defined strata without training a detector. On COCO and VisDrone-DET, the best CCDM variant ranks 19 candidate training sets in strong agreement with YOLOv8 mAP (Spearman \r{ho} = 0.97 and 0.96), outperforming FID, KID, and MMD. Furthermore, CCDM holds when reference metadata comes from detector pseudo-labels rather than ground-truth boxes.