cs.CVNov 7, 2026

Semantic Synthesis over Brute-Force Search: LLM-Guided Transformation Design for Efficient Learning

Authors: Usha Shrestha, Radu Timofte, Dmitry Ignatov

Organizations: Computer Vision Lab, CAIDAS & IFI, University of Würzburg, Germany

Abstract

Data transformation pipelines shape the training distribution in computer vision, but designing them is a computationally expensive problem. We investigate whether large language models can enable efficient discovery by synthesizing executable transformation programs outside predefined augmentation spaces. We compare nine strategies under a unified training protocol, including standard augmentation policies, brute-force search, direct code generation with two LLMs, and performance-guided generation using LLMs fine-tuned on accuracy-ordered transformation pairs. When the best candidates are retrained for 20 epochs, LLM-generated transformation programs lead on every dataset, reaching 0.8877 accuracy on CIFAR-10, 0.5825 on CIFAR-100, and 0.9588 on SVHN, surpassing brute-force search over 6,000 candidates. The best transform on CIFAR-10 and SVHN comes from a fine-tuned generator using 270 candidates, while the best on CIFAR-100 comes from an off-the-shelf generator using only 10. This needs 22x fewer candidate evaluations in the generation step and 600x fewer for off-the-shelf generations, relative to brute-force search. Beyond transformation discovery, we analyze the reliability of the reduced-budget evaluation commonly used for augmentation search. Correlating one-epoch selection scores with 20-epoch outcomes reveals strong, weak, and absent predictive relationships depending on the dataset and architecture, with correlations ranging from 0.756 to near zero. A positive control indicates that range restriction alone is unlikely to explain the low correlations. Our results indicate that semantic synthesis with code LLMs provides an efficient alternative to search-based transformation discovery and highlight the importance of validating evaluation fidelity when optimizing data transformations.

Figures & tables

Explore similar work

CardsList
  1. Delta-Based Neural Architecture Search: LLM Fine-Tuning via Code Diffs

    May 6, 2026Santosh Premi Adhikari, Radu Timofte, Dmitry IgnatovLarge Language Model-Guided NASLLM Fine-Tuning

  2. VisionFoundry: Teaching VLMs Visual Perception with Synthetic Images

    Apr 10, 2026Guanyu Zhou, Yida Yin, Wenhao Chai +3Visual Question AnsweringSynthetic Data Augmentation