Pairwise Learning Increases Data Efficiency in Deep Neural Networks
Organizations: Department of Neurobiology & Behavior, University of California, Irvine, Irvine, CA 92697, USA · Department of Cognitive Sciences, University of California, Irvine, Irvine, CA 92697, USA
Abstract
Deep neural networks often fail at small-sample regression, collapsing to low-variance predictions that miss individual differences. Here, we introduce a pairwise comparison framework that reframes regression as aggregated relative rankings. This method improves data efficiency across model architectures and dataset sizes. On the UTKFace dataset (23k images), pairwise learning generates over one billion training comparisons without additional supervision. The paired ranking allows a lightweight ResNet10 trained from scratch on home hardware to achieve a 3.64-year MAE-surpassing previous models, including those with large-scale pretraining. For classification into categories, pairwise supervision consistently outperforms an otherwise identical 10-way crossentropy model on STL10 with 500-2000 labeled examples, improving accuracy by up to 5.2%. Overall, pairwise supervision enhances data efficiency and enables strong regression and classification performance when labeled data are limited, particularly in clinical settings.
Data-Efficient Learning
Data-efficient learning develops methods that achieve accurate, robust models with fewer labeled examples, training samples, or interactions.
See all 239 papers