cs.CVApr 3, 2026

Learning from Synthetic Data via Provenance-Based Input Gradient Guidance

Authors: Koshiro Nagano, Ryo Fujii, Ryo Hachiuma, Fumiaki Sato, Taiki Sekii, Hideo Saito

Organizations: Keio University · Independent Researcher · CyberAgent

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Learning methods using synthetic data have attracted attention as an effective approach for increasing the diversity of training data while reducing collection costs, thereby improving the robustness of model discrimination. However, many existing methods improve robustness only indirectly through the diversification of training samples and do not explicitly teach the model which regions in the input space truly contribute to discrimination; consequently, the model may learn spurious correlations caused by synthesis biases and artifacts. Motivated by this limitation, this paper proposes a learning framework that uses provenance information obtained during the training data synthesis process, indicating whether each region in the input space originates from the target object, as an auxiliary supervisory signal to promote the acquisition of representations focused on target regions. Specifically, input gradients are decomposed based on information about target and non-target regions during synthesis, and input gradient guidance is introduced to suppress gradients over non-target regions. This suppresses the model's reliance on non-target regions and directly promotes the learning of discriminative representations for target regions. Experiments demonstrate the effectiveness and generality of the proposed method across multiple tasks and modalities, including weakly supervised object localization, spatio-temporal action localization, and image classification.

Figures & tables

Figure 2: Examples of provenance information I\bm{I} obtained during synthesis. I\bm{I} corresponds to each supervisory label.
Figure 2
BackboneMethodMaxBoxAccV2 (%)
δ\delta = 0.30.50.7Mean
VGG16 [ 35 ]CAM +CutMix [ 45 ]91.167.328.662.3
+Ours96.874.623.165.1
CAM +ResizeMix [ 23 ]92.462.418.157.6
+Ours95.970.620.162.2
CAM +PuzzleMix [ 19 ]94.661.314.456.8
Table 1: Comparison of weakly supervised object localization accuracy between baseline methods and the proposed method on the CUB dataset. Underlined values are reference values whose trends are inconsistent across IoU thresholds.
Table 4
Table 5
Table 6
Figure 6: Visualization of image editing synthesis and the corresponding provenance masks.

Explore similar work

CardsList
May 11, 2026

LiBaGS: Lightweight Boundary Gap Synthesis for Targeted Synthetic Data Selection

Synthetic data is useful only when the added samples fill missing parts of the training distribution that matter for the downstream task. We introduce LiBaGS, a lightweight, generator-agnostic method for targeted synthetic training data selection. LiBaGS scores candidate synthetic samples by combining decision-boundary proximity, predictive uncertainty, real-data density, and support validity, so that selected samples are both informative and likely to remain on the real data manifold. We then use a boundary-gap allocation rule that targets sparse but realistic decision-boundary neighborhoods, rather than simply adding more data or selecting only the most uncertain candidates. LiBaGS also learns when enough synthetic samples have been added through a marginal-value stopping rule, assigns softer labels near ambiguous boundaries, and uses a diversity objective to avoid redundant near-duplicate selections. Experiments show that LiBaGS improves accuracy over classical oversampling, hard augmentation, uncertainty and density ablations, and targeted-generation selection criteria.

Abhishek Moturu, Anna Goldenberg, Babak Taati

Department of Computer Science University of Toronto The Hospital for Sick Children UHN KITE Research Institute T-CAIREM Vector Institute · Department of Computer Science Department of Laboratory Medicine and Pathobiology University of Toronto The Hospital for Sick Children T-CAIREM Vector Institute · Department of Computer Science Institute of Biomedical Engineering University of Toronto Rehabilitation Sciences Institute UHN KITE Research Institute Vector Institute

Synthetic Data AugmentationTraining Data SelectionSynthetic Data Selection
AskPDF
May 11, 2026

An Information-Theoretic Criterion for Efficient Data Synthesis

Synthetic data becomes crucial for large language model training, but its effectiveness is highly inconsistent. We provide an information-theoretic account of this inconsistency: synthetic data improves a model only when the generation-training loop is information-open, i.e., shaped by external signals (verifiers, environments, or rubrics) that inject task-relevant information beyond the model's current distribution. When the loop is information-closed (relying on the model's own outputs without such signals), the data processing inequality ensures that task-relevant information can only decrease, making collapse a predicted outcome. Among information-open pipelines, both efficiency and generalization hinge on the meta-level of supervision: a coarser signal such as binary correctness treats all acceptable outputs as equivalent, so the behavior it teaches is not tied to any particular domain or surface form and generalizes naturally across tasks and domains. These observations lead to a guiding thesis: learning preferentially converges to the most information-efficient signal component available, which accelerates learning when that component is the intended one, but causes reward hacking when a spurious pattern happens to be simpler.

Hanyu Li, Zhengqi Sun, Xiaotie Deng

CFCS, School of Computer Science, Peking University, Beijing, China · Department of Information Management, Peking University, Beijing, China

Synthetic Data PretrainingSynthetic Data GenerationSynthetic Data Generation for Language Models
AskPDF
Oct 7, 2026

Concentration, Not Uncertainty: Why Targeted Synthetic Data Doesn't Help Camouflaged Object Detection

Camouflaged object detection requires pixel-accurate masks, but obtaining such annotations is slow and costly, making synthetic training images an attractive alternative. Under a fixed generation budget, however, it remains unclear which real-image regions to target for synthetic data generation. We study an uncertainty-guided generation strategy that clusters the unlabelled real images, identifies clusters on which the model is least certain, allocates synthetic generation toward those clusters, and iteratively retrains the model. Across 103 training runs, uncertainty-based targeting does not outperform random allocation. Five independent controls further show that this null result is not an artifact: targeted training sets are measurably different from random sets, but the difference is explained by concentrating the generation budget rather than by where uncertainty is concentrated, as every concentration rule we test reproduces the effect and, on boundary accuracy, so does aiming at the clusters the model was most certain about. Separately, we find substantial data contamination in CHAMELEON, with 50 of its 76 images duplicated from training data despite the standard overlap check reporting zero overlap. Together, these results show that, under a fixed synthetic-data budget, budget concentration, not uncertainty-based targeting, accounts for the observed training-set effects.

Akshat Dobhal, Sanjay Singh

Manipal Institute of Technology, Manipal Academy of Higher Education, Manipal, India

Synthetic Data SelectionSynthetic Data GenerationCamouflaged Object Detection
AskPDF