cs.CVAug 28, 2026

ActiveAugment: Online Active Learning for Augmentation Selection in Deep Learning

Authors: Noah Videcrantz, Mostafa Mehdipour Ghazi

Organizations: Pioneer Centre for AI University of Copenhagen

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Data augmentation is a cornerstone of deep learning pipelines, yet existing strategies treat it as a static, model-agnostic preprocessing step, either relying on expensive dataset-specific policy search or applying transformations uniformly at random, regardless of what the model has already learned. We introduce ActiveAugment, a unified framework that treats augmentation selection as an online active learning problem. For each training minibatch, ActiveAugment generates a pool of candidate augmented views and scores each candidate using a combination of the model's predictive uncertainty and the feature discrepancy induced by the augmentation. The augmentation under which the current model is most fragile is selected per sample, and the model is then trained with a joint supervised classification and supervised contrastive objective that enforces intra-class invariance to the selected augmentations while maintaining inter-class separation. We evaluate ActiveAugment on eight benchmark datasets spanning natural and medical imaging, using CNN and transformer architectures across three training regimes (training from scratch, full fine-tuning, and linear probing), and comparing eight active selection strategies for augmentation scoring. ActiveAugment outperforms AutoAugment, RandAugment, and TrivialAugment under controlled augmentation shifts across all domains and budgets, with the most pronounced gains at low labelling budgets. On medical imaging datasets, where data is scarce and domain shift relative to natural-image pretrained models is large, ActiveAugment achieves higher test F1 than all baselines, demonstrating strong cross-domain adaptability. Our analysis reveals that the augmentation selection policy evolves meaningfully during training and that strategy choice has a direct impact on generalisation. Code is available at: https://github.com/noahvide/ActiveAugment.

Figures & tables

Figure 1: Overview of ActiveAugment. For each minibatch, every sample is augmented with all available augmentations at a randomly sampled intensity level. An active selection strategy scores each candidate and selects the most informative augmentation per sample. The model is then trained on both the original and selected augmented samples using the combined supervised classification and contrastive loss.
Figure 2: Test results on natural image datasets (ResNet-18 backbone, full fine-tuning, BADGE strategy). Shaded bands indicate ±1\pm 1 standard deviation. Overall : test set including augmented views. Clean : original test images only.
Figure 3: Per-augmentation test F1 on MNIST (ResNet-18 backbone, full fine-tuning). BADGE, Margin, Entropy, and Coreset substantially outperform BAL, Least Confident, and TypiClust, with the largest gap observed under the Flip augmentation.
Figure 4: Left: Augmentation selection frequency (%) on natural image datasets (ResNet-18 backbone, full fine-tuning), aggregated over all datasets and budgets. Each column corresponds to one active selection strategy; the rightmost column is the random baseline. Random frequencies are non-uniform because the final score is weighted by feature discrepancy. Strategies that favour Gaussian Blur consistently underperform (see Section 4.3 ). Right: Evolution of augmentation selection frequencies (top) and selection confidence (bottom) over normalised training epochs, aggregated over all natural image datasets and budgets (ResNet-18 backbone, full fine-tuning, BADGE strategy). Confidence is measured as the coefficient of variation of the spread (Std/Mean) and of the margin (Margin/Mean) across augmentation scores. Both confidence metrics rise monotonically, indicating that the selection policy becomes increasingly decisive as the encoder representations stabilise.
Figure 5: Test results on medical image datasets (ResNet-18 backbone, full fine-tuning, BADGE strategy). Shaded bands indicate ±1\pm 1 standard deviation. Overall : test set including augmented views. Clean : original test images only. ActiveAugment is consistently among the top-performing methods on the clean test set across all four datasets and all budgets.
Figure 6: Per-augmentation test F1 on FETAL-PLANES (ResNet-18 backbone, full fine-tuning). The performance gap between top and bottom strategies is largest under Gaussian Noise, suggesting that noise robustness is discriminative for medical image recognition.
Figure 7: Left: Augmentation selection frequency (%) on medical image datasets (ResNet-18 backbone, full fine-tuning), aggregated over all datasets and budgets. Compared to natural images, Gaussian Noise is selected more frequently (on par with Flip for several strategies). Right: Policy evolution on medical image datasets (ResNet-18 backbone, full fine-tuning, BADGE strategy). Selection confidence rises monotonically as on natural image data, confirming the domain-agnostic behaviour of the selection mechanism.
More in this topic

Data Augmentation

Data augmentation expands or transforms training examples to improve learning, data efficiency, and robustness to distribution shifts.

See all 242 papers

Explore similar work

CardsList
Mar 23, 2026

Ctrl-A: Control-Driven Online Data Augmentation

We introduce ControlAugment (Ctrl-A), an automated data augmentation algorithm for image-vision tasks, which incorporates principles from control theory for online adjustment of augmentation strength distributions during model training. Ctrl-A eliminates the need for initialization of individual augmentation strengths. Instead, augmentation strength distributions are dynamically, and individually, adapted during training based on a control-loop architecture and what we define as relative operation response curves. Using an operation-dependent update procedure provides Ctrl-A with the potential to suppress augmentation styles that negatively impact model performance, alleviating the need for manually engineering augmentation policies for new image-vision tasks. Experiments on the CIFAR-10, CIFAR-100, and SVHN-core benchmark datasets using the common WideResNet-28-10 architecture demonstrate that Ctrl-A is highly competitive with existing state-of-the-art data augmentation strategies.

Jesper B. Christensen, Ciaran Bench, Spencer A. Thomas +4

Danish Fundamental Metrology (DFM), Kogle Allé 5, DK-2970 Hørsholm, Denmark · National Physical Laboratory (NPL), Teddington, United Kingdom · TÜV SÜD, Warrington, United Kingdom +2

Data AugmentationImage Classification
AskPDF
Aug 19, 2026

Learning-State-Aware Dynamic Generative Data Augmentation on Small-Scale Datasets

Small-scale image classification is often limited by the scarcity of training data. Generative data augmentation (GDA) based on pretrained generative models has emerged as an effective solution. However, existing methods rely on task-agnostic augmentation strategies that overlook downstream model needs. Although recent dynamic GDA methods incorporate model feedback to guide augmentation, they still struggle to reliably determine sample-specific augmentation strengths and adapt augmentation strategies to different image regions while balancing image diversity and class semantics. To address these issues, we propose learning-state-aware dynamic generative data augmentation (LSADA). Specifically, LSADA constructs a learning state for each sample based on its current loss and loss-decrease rate, which is then mapped to a sample-specific augmentation strength. Furthermore, LSADA introduces a decoupled data augmentation and diffusion fusion strategy that applies strength-controlled transformations to class-relevant regions and generates diverse class-irrelevant regions, progressively fusing them to improve image diversity while preserving class semantics. Experiments on nine public datasets show that LSADA outperforms the existing SOTA dynamic GDA method by an average of 4.5% on six natural image datasets and 2.5% on three medical image datasets.

Ting Xiang, Chenxi Deng, Jinhui Zhao +4

Hunan University

Data AugmentationSynthetic Data AugmentationFew-Shot Image Classification
AskPDF
Jul 21, 2026

InstructMixup: Instruction-Guided Salient Patch Editing for Robust Data Augmentation

In image and video technologies, data augmentation is widely used to improve the generalization of deep visual models, and mixup-based strategies that interpolate between samples have become the dominant approach. However, computing informative mixing regions adds substantial overhead, and blending content across different images frequently disrupts the semantic integrity of the resulting sample. We propose \our{}, a data augmentation method that constructs challenging yet label-consistent training samples entirely within a single visual sample. \our{} first extracts multi-scale salient patches from the sample using a lightweight saliency detector, refines each patch with an instruction-guided generative model, and blends the edited patch back into the non-salient regions of the same sample; because the generative edits are computed once and cached offline, this step adds negligible training cost. To further diversify the learned representation, \our{} injects self-similar fractal structure into the same salient regions at an adaptive ratio, so each training sample carries both fractal and non-fractal structure. We derive a second-order approximation of the resulting vicinal risk, showing that the method simultaneously enforces invariance to the generative edit and suppresses curvature along the perturbed salient directions, and we verify both predictions empirically. We evaluate on small to large backbones for instance Convolutional Neural Networks (CNNs), Vision Transformers (ViTs) and Vision-Language Foundational Models (VLMs) across seven benchmarks covering coarse- and fine-grained classification, robustness to corruption and occlusion, calibration, and transfer and self-supervised learning, InstructMixup outperforms nine competing augmentation methods, surpassing the strongest baseline across all benchmarks.

Khawar Islam, Arif Mahmood, Xin Jin +1

The University of Melbourne, Melbourne, Australia · The Information Technology University, Lahore, Pakistan · Westlake University, China

Synthetic Data AugmentationData Augmentation
AskPDF