Ctrl-A: Control-Driven Online Data Augmentation

Jesper B. Christensen; Ciaran Bench; Spencer A. Thomas; Hüsnü Aslan; David Balslev-Harder; Nadia A. S. Smith; Alessandra Manzin

cs.CVMar 23, 2026

Ctrl-A: Control-Driven Online Data Augmentation

Authors: Jesper B. Christensen, Ciaran Bench, Spencer A. Thomas, Hüsnü Aslan, David Balslev-Harder, Nadia A. S. Smith, Alessandra Manzin

Organizations: Danish Fundamental Metrology (DFM), Kogle Allé 5, DK-2970 Hørsholm, Denmark · National Physical Laboratory (NPL), Teddington, United Kingdom · TÜV SÜD, Warrington, United Kingdom · Royal Surrey NHS Foundation Trust, Guildford, United Kingdom ·

Abstract

We introduce ControlAugment (Ctrl-A), an automated data augmentation algorithm for image-vision tasks, which incorporates principles from control theory for online adjustment of augmentation strength distributions during model training. Ctrl-A eliminates the need for initialization of individual augmentation strengths. Instead, augmentation strength distributions are dynamically, and individually, adapted during training based on a control-loop architecture and what we define as relative operation response curves. Using an operation-dependent update procedure provides Ctrl-A with the potential to suppress augmentation styles that negatively impact model performance, alleviating the need for manually engineering augmentation policies for new image-vision tasks. Experiments on the CIFAR-10, CIFAR-100, and SVHN-core benchmark datasets using the common WideResNet-28-10 architecture demonstrate that Ctrl-A is highly competitive with existing state-of-the-art data augmentation strategies.

Figures & tables

Figure 1: Illustration of the ControlAugment framework. Model training proceeds in phases of $n_{p}$ epochs, separated by the ControlAugment algorithm, which regulates the informed augmentation pool by adjusting the augmentation strength parameters $\boldsymbol{\Gamma}$ and $\boldsymbol{\alpha}$ for the subsequent training phase. The adjustment is executed by the ControlAugment block which quantifies the response of each operation using augmented validation data and through an internal control parameter ( $\xi$ ) which is updated by comparing the setpoint value ( $\kappa_{sp}$ ) with a relative training/validation performance metric ( $\kappa$ ).

Figure 2: Examples of augmentation strength distributions with increasing distribution means from left to right.

Figure 3: Illustration of the Ctrl-A update procedure (Eqs. 3 and 4 ), which relies on relative operation response curves to determine new values for the ASD parameters $\Gamma$ and $\alpha$ based on the value of the control parameter, $\xi$ . Implicitly, $\Gamma_{3}=1$ and $\alpha_{1}=\alpha_{2}=0$ .

Figure 4: CIFAR-10 test accuracy (model: airbench-94) as a function of the control setpoint

\kappa_{sp}

for (a) CtrlA(

1

), (b) CtrlA(

2

), and (c) CtrlA(

3

). Search-optimized RA is represented by the shaded region (

95\,\%

confidence interval).

	Standard setups						Modified setups
	Base	AA	RA s	DeepAA s	TA w	CtrlA c (2)	TA w	CtrlA c (1)	CtrlA c (2)	CtrlA c (3)
CIFAR-10	96.1	97.4	97.3	97.56(14)	97.46(06)	97.54(08)	97.96(08)	98.14(11)	98.19(06)	98.10(05)
CIFAR-100	81.2	82.9	83.3	84.02(18)	84.33(17)	84.29(17)	84.43(24)	84.02(24)	84.80(13)	84.44(29)
SVHN-c	96.9	98.1	98.3	-	98.11(03)	98.06(07)	98.14(07)	98.18(03)	98.25(03)	98.27(03)

Table 1: Top-1 accuracy (model: WRN-28-10) on CIFAR-10, CIFAR-100, and SVHN-core. The first five columns are literature values [ 38 , 20 , 28 , 24 , 21 ] , and the last five columns are from this work. Parenthesis represent shorthand notation for 95

\%

uncertainty coverage, i.e.

97.54(08)

97.54\pm 0.08

. Lowercase font represents the used DA pool, s: Standard pool [ 28 ] , w: Wide pool [ 21 ] , and c: Control pool from this work (Appendix A). Dash indicates no available result.

Dataset	Validation	Test	Test TTA
CIFAR-10	98.06(23)	98.20(12)	98.35(11)
CIFAR-100	84.60(40)	84.68(15)	85.15(26)
SVHN-c	97.74(59)	98.27(06)	98.27(06)

Table 2: Top-1 accuracy (

\%

) (model: WRN-28-10) on CIFAR-10, CIFAR-100, and SVHN-core using CtrlA(2) with

\kappa_{sp}=1.5

. The validation dataset, containing 1000 image-label pairs, is a subset of the test dataset, i.e.

\mathcal{D}_{\mathrm{Val}}\subset\mathcal{D}_{\mathrm{Test}}

. Notation xx.xx(yy) is shorthand for xx.xx

\pm

0.yy, representing a 95

\%

uncertainty coverage.

Dataset	Validation	Test	Test TTA
CIFAR-10	98.14(36)	98.11(14)	98.25(11)
CIFAR-100	86.10(95)	84.57(36)	85.03(32)
SVHN-c	96.52(18)	98.28(07)	98.31(07)

Table 3: Top-1 accuracy (

\%

) (model: WRN-28-10) on CIFAR-10, CIFAR-100, and SVHN-core using CtrlA(2) with

\kappa_{sp}=1.5

. The validation dataset, containing 1,000 image-label pairs, is obtained from a random training-validation split, i.e.

\mathcal{D}_{\mathrm{Val}}\subset\mathcal{D}_{\mathrm{Train}}^{(full)},~\mathcal{D}_{\mathrm{Train}}=\mathcal{D}_{\mathrm{Train}}^{(full)}\setminus\mathcal{D}_{\mathrm{Val}}

. Notation xx.xx(yy) is shorthand for xx.xx

\pm

0.yy, representing a 95

\%

uncertainty coverage.

Figure 5: Convergence results for CIFAR-10 performance.

Aug. pool	TA	CtrlA(2)
Standard [ 28 ]	97.60(11)	97.76(09)
Wide [ 21 ]	97.96(08)	97.93(10)
Ours (App. A)	98.03(06)	98.19(06)

Table 4: Top-1 test accuracy (

\%

) (model: WRN-28-10) on CIFAR-10 for different augmentation pools. Notation xx.xx(yy) is shorthand for xx.xx

\pm

0.yy, representing a 95

\%

uncertainty coverage.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Op. $\#$	Op. Name	Parametrization
	TranslateX (s)	$\gamma_{1}/2\times$ Img. width
	TranslateY (s)	$\gamma_{2}/2\times$ Img. height
	ShearX (s)	$45^{\circ}\gamma_{3}$
	ShearY (s)	$45^{\circ}\gamma_{4}$
	Scale (s)	$1+\gamma_{5}/2$
	Rotation (s)	$60^{\circ}\gamma_{6}$

Appendix

Table 5: List of the “control” augmentation pool and respective parametrizations that use normalized augmentation strength

\gamma_{i}\in[0,1]

. All operations converge to the identity operator for

\gamma\rightarrow 0

, and the identity operator is therefore explicitly excluded from the augmentation pool. Operations appended by (s) indicates that the sign of

\gamma_{i}

is flipped, i.e.

\gamma_{i}<0

, with a

50\,\%

probability.

Figure 6: Visualization of uni-transformed images for each of the operations in the full augmentation pool with augmentation strength $\gamma=+1$ . The original image is shown in the lower right.

Figure 7: Examples of relative operation response data (crosses) and fitting curves (dashed lines) used to determine

\Gamma_{i}

and

\alpha_{i}

. CIFAR-10 was used as example dataset.

Figure 8: Comparison study exploiting an 8000/2000 validation-test split to investigate whether the Ctrl-A algorithm leads to overfitting to the validation data used to update augmentation strengths. The results are obtained by training a (custom) LeNet model on CIFAR-10 employing CtrlA(2) augmentation.

Data Augmentation

Data augmentation expands or transforms training examples to improve learning, data efficiency, and robustness to distribution shifts.

See all 242 papers

ActiveAugment: Online Active Learning for Augmentation Selection in Deep Learning
Aug 28, 2026Noah Videcrantz, Mostafa Mehdipour GhaziData Augmentation Active Learning
Pioneer Centre for AI University of Copenhagen
InstructMixup: Instruction-Guided Salient Patch Editing for Robust Data Augmentation
Jul 21, 2026Khawar Islam, Arif Mahmood, Xin Jin +1Synthetic Data Augmentation Data Augmentation
The University of Melbourne, Melbourne, Australia · The Information Technology University, Lahore, Pakistan · Westlake University, China
How (and How Not) to Use Data Augmentation in VLA Post-Training
Oct 5, 2026Bram Grooten, Joaquin VanschorenRL Post-Training Robustness of VLA Models
TU Eindhoven

Aug 28, 2026J/K move · Enter open · S save

ActiveAugment: Online Active Learning for Augmentation Selection in Deep Learning

Noah Videcrantz, Mostafa Mehdipour Ghazi

Pioneer Centre for AI University of Copenhagen

Data Augmentation Active Learning Image Classification

Open paper AskPDF arXiv

Data augmentation is a cornerstone of deep learning pipelines, yet existing strategies treat it as a static, model-agnostic preprocessing step, either relying on expensive dataset-specific policy search or applying transformations uniformly at random, regardless of what the model has already learned. We introduce ActiveAugment, a unified framework that treats augmentation selection as an online active learning problem. For each training minibatch, ActiveAugment generates a pool of candidate augmented views and scores each candidate using a combination of the model's predictive uncertainty and the feature discrepancy induced by the augmentation. The augmentation under which the current model is most fragile is selected per sample, and the model is then trained with a joint supervised classification and supervised contrastive objective that enforces intra-class invariance to the selected augmentations while maintaining inter-class separation. We evaluate ActiveAugment on eight benchmark datasets spanning natural and medical imaging, using CNN and transformer architectures across three training regimes (training from scratch, full fine-tuning, and linear probing), and comparing eight active selection strategies for augmentation scoring. ActiveAugment outperforms AutoAugment, RandAugment, and TrivialAugment under controlled augmentation shifts across all domains and budgets, with the most pronounced gains at low labelling budgets. On medical imaging datasets, where data is scarce and domain shift relative to natural-image pretrained models is large, ActiveAugment achieves higher test F1 than all baselines, demonstrating strong cross-domain adaptability. Our analysis reveals that the augmentation selection policy evolves meaningfully during training and that strategy choice has a direct impact on generalisation. Code is available at: https://github.com/noahvide/ActiveAugment.

Abstract

Data Augmentation

Explore similar work

ActiveAugment: Online Active Learning for Augmentation Selection in Deep Learning

InstructMixup: Instruction-Guided Salient Patch Editing for Robust Data Augmentation

How (and How Not) to Use Data Augmentation in VLA Post-Training