Data-Efficient Learning

Ask about this topic

Overview

Data-efficient learning studies how to build useful models when labels, examples, simulations, or interaction data are costly or limited. Common approaches include active learning and coreset selection, semi-supervised and self-supervised pretraining, transfer and meta-learning, and curricula that prioritize examples suited to a model’s current abilities. In scientific and clinical settings, physical laws, domain knowledge, and synthetic data can provide useful structure when observations are scarce. A central challenge is that the best strategy depends on the task, data distribution, and compute budget, so gains in accuracy must be weighed against training cost, robustness, and reliable evaluation.

Latest papers

CardsList

245 papers28 papers in the last four weeks, up 56% on the four weeks before. 0.2% of all new papers.

  1. Efficient Learning of Mechanism Designs and Compositionality

    Oct 29, 2026João Pedro Gandarela, Thiago Rios, Stefan Menzel +1Compositional Representation LearningData-Efficient Learning

    Idiap Research Institute, Switzerland · École Polytechnique Fédérale de Lausanne (EPFL), Switzerland · Honda Research Institute Europe, Germany +2

  2. Sometimes less is more: Investigating inductive biases of MLPs and Transformers on noisy k-parity features

    Oct 27, 2026Alasdair ParenTransformerMultilayer Perceptrons

    Chalmers University of Technology

  3. TRIDENT-Seq: Profile-Matched Prefix-Future Contrast for Label-Efficient Payment Risk Modeling

    Oct 27, 2026Weiyu Han, Haoyu Jiang, Jie Sheng +7Financial Fraud DetectionContrastive Learning

    Ant International · East China Normal University · Tongji University +1

  4. Vision Transformers for Estimating Irradiance Using Data Scarce Sky Images

    Oct 22, 2026David Hamlyn, Sunny Chaudhary, Tasmiat RahmanVision TransformerData-Efficient Learning

    School of Electronics and Computer Science, University of Southampton, University Road, Southampton, SO17 1BJ, United Kingdom

  5. Learning Computational Structure for Sample-Efficient Prediction

    Oct 20, 2026Hangyu Zhou, Constantine Dovrolis, Stephen MussmannData-Efficient LearningCompositional Representation Learning

    Georgia Institute of Technology · University of Cyprus

  6. Pairwise Learning Increases Data Efficiency in Deep Neural Networks

    Oct 20, 2026Robert Bain, Marjan Rashidi, Elizabeth ChrastilData-Efficient LearningSmall-Data Learning

    Department of Neurobiology & Behavior, University of California, Irvine, Irvine, CA 92697, USA · Department of Cognitive Sciences, University of California, Irvine, Irvine, CA 92697, USA

  7. Sample-efficient learning with Dense Associative Memories

    Oct 19, 2026Andrew Lucas, Dmitry KrotovAssociative MemoryData-Efficient Learning

    Department of Physics and Center for Theory of Quantum Matter, University of Colorado, Boulder, CO 80309, USA · Dynamical Mind, Cambridge, MA 02141, USA

  8. Preserving the Manifold: Frozen Expert Alignment for Label-Efficient SAR Scene Understanding

    Oct 18, 2026Eli A. Underwood, I. Taylor McKechnie, Leslie M. CollinsRemote Sensing VLMsCross-Modal Alignment

    Duke University

  9. Shallow Probes Are Sufficient for Near-Oracle Personalization Depth Allocation in Federated Learning

    Oct 18, 2026Beyazit Bestami Yuksel, Emrah Dikbiyik, Ayse Yilmazer MetinFederated LearningData-Efficient Learning

    Department of Computer Engineering, Istanbul Technical University, Istanbul, Turkey · Department of Computer Engineering, Marmara University, Istanbul, Turkey

  10. Diversity Without Labels: Task-Agnostic Coreset Selection Across Perception and Action

    Oct 16, 2026Julia Hindel, Benjamin Ramtoula, Kyuhwan Yeon +4Data SelectionCoreset Selection

    University of Freiburg · University of Oxford

  11. Data-Efficient Classifier Pretraining Through Diffusion Models

    Oct 16, 2026Qinyu Li, Yee Whye Teh, Chris Holmes +1Data-Efficient LearningSelf-Supervised Pre-Training

    Department of Statistics, University of Oxford · Faculty of Medicine and Health Technology, Tampere University · Tampere Institute for Advanced Study, Tampere University

  12. Efficient Pairwise Difference Learning: A Comparative Study of Encoding Strategies and Training Pair Selection

    Oct 15, 2026Bernard Georges, Mohamed Karim Belaid, Manuel RodriguesData-Efficient LearningTraining Data Selection

    University of Minho, ALGORITMI CENTER LASI, Braga, Portugal · IDIADA Fahrzeugtechnik GmbH, Munich, Germany

  13. L2GSSL: Label-Efficient Local-to-Global Self-Supervised Learning for Volumetric Healthcare Data

    Oct 15, 2026Po-Cheng Cheng, Hung-Hsuan ChenLung Nodule DetectionMedical Image Representation Learning

    Computer Science and Information Engineering, National Central University

  14. Mind the Tail: Data Selection Through Sparse Concepts

    Oct 14, 2026Ziheng Zhang, Neha Hulkund, Colin Lee +7Training Data SelectionData Selection

    University of Texas at Austin · The Ohio State University · Massachusetts Institute of Technology +2

  15. Diminishing Returns of Data in Sparse Autoencoders: Functional Behavior does not Guarantee Feature Reproducibility

    Oct 14, 2026Sambit Ghosh, Madelyn Mathai, R. Venkatesh Babu +1Sparse AutoencodersData-Efficient Learning

    IBM · Indian Institute of Science · University of Virginia

  16. pi-Net: Empirical Latent Representations for Differentiable Physics Heads in Data-Scarce Systems

    Oct 12, 2026Akshay Dhalpe, Arnab Chattopadhyay, Shahriar Bakrani Balani +2Physics-Informed MLSurrogate Modeling

    Tampere University · University of Helsinki

  17. HAND: A Biologically-Inspired Activation Function that Improves Generalisation and Sample Efficiency in Image Classification

    Oct 8, 2026Michael W. Spratling, Heiko H. SchüttNeural Network Activation FunctionsNeural Network Generalization

    University of Luxembourg, Department of Behavioural and Cognitive Sciences, L-4366 Esch-sur-Alzette, Luxembourg.

  18. SPD-MetaFormer is what you need for small-data brain decoding

    Oct 7, 2026Zhida Wang, Wei Lyu, Guo Yu +1Small-Data LearningEEG Decoding

  19. Stream-Based Active Learning with Cooperative Neural Networks for Data-Efficient Partial Inverse Design: An Automotive Glass Run Channel Case Study

    Oct 7, 2026Agung Nugraha, Hyerin Kwon, Heungjun Im +2Inverse DesignData-Efficient Learning

    Department of Industrial and Data Engineering, Major in Industrial Data Science and Engineering, Pukyong National University, Busan, 48513, Republic of Korea · DRB Co., Ltd., Busan, 46329, Republic of Korea

  20. Label-Free Coreset Selection with Foundation Models for Efficient Annotation in Computational Pathology

    Oct 5, 2026Tuo Yin, Jennifer DhontComputational PathologyCoreset Selection

    Department of Medical Physics, Institut Jules Bordet, Hˆpital Universitaire de Bruxelles (HUB), Belgium · Radiophysics and MRI Physics Laboratory, Universit´e Libre de Bruxelles (ULB), Belgium · Laboratory of Image Synthesis and Analysis (LISA), Universit´e Libre de Bruxelles (ULB), Belgium

  21. On architectural choices for interpretability and thermodynamic consistency in Physically Recurrent Neural Networks in the low-data regime

    Oct 2, 2026M. A. Maia, K. A. Meyer, A. M. C. M. van Gils +2Recurrent Neural NetworksNeural Network Interpretability

    Delft University of Technology, Department of Civil Engineering and Geosciences, P.O.Box 5048, 2600GA, Delft, The Netherlands · Chalmers University of Technology, Department of Mechanical Engineering, Gothenburg, Sweden

  22. Geometry Meets Physics: Data-Efficient Pre-Training for Unstructured Neural PDE Solvers

    Oct 2, 2026Luis Medrano-Navarro, Giacomo Baldan, Qiang Liu +4Synthetic Data PretrainingNeural PDE Solvers

    Technical University of Munich · University of Stuttgart

  23. Label-Efficient Time Series Classification at Scale: A Dual-Stream OSSE-LSTM with Counterfactual Attribution

    Oct 2, 2026Nguyen Ho, Bach Tung Tran, Trung Ky Nguyen +3Time Series ClassificationData-Efficient Learning

    Loyola University Maryland, USA · School of Computer Science and Engineering, International University, Vietnam · School of Compute Science and Artificial Intelligence, Wuhan University of Technology, China

  24. DeepStratNet: A Context-Aware Coordinate Regression Framework for Seismic Horizon Tracking under Sparse Labels

    Oct 1, 2026Aniq Ahmad, Musham Ahmad Malik, Ahmad Mustafa +1Data-Efficient Learning

    Attribute Assisted Seismic Processing and Interpretation (AASPI), University of Oklahoma, USA · CIVIL Lab, Information Technology University, Lahore, Pakistan · SDAIA Joint Research Center for AI (JRCAI), King Fahd University of Petroleum and Minerals, Dhahran, Saudi Arabia

  25. Sharpen Before You Adapt: Data-Free Entry-State Sharpening for Test-Time Reinforcement Learning

    Oct 1, 2026Zhanming Zhang, Vinoth SelvendranRL for Language Model ReasoningTest-Time RL

    Independent Researcher New York, NY · Independent Researcher Palo Alto, CA

  26. Less is more: error-distance scaling relation for data-efficient kilometer-scale downscaling of extreme heat

    Sep 30, 2026Ahmed Marey, Henry Lu, Abhishek Gaur +6Spatial GeneralizationScaling Laws

    Centre for Zero Energy Building Studies, Department of Building, Civil and Environmental Engineering, Concordia University, Montreal, H3G 1M8 Canada · Building and Climate Interface, Construction Research Centre, National Research Council Canada, Ottawa, ON, K1A 0R6, Canada · Department of Architecture, School of Sciences and Engineering, The American University in Cairo, New Cairo 11835, Egypt +3

  27. Can Domain Generalization be Guaranteed in Small-Sample Learning?

    Sep 30, 2026Hong ZhengDomain GeneralizationSmall-Data Learning

    This work was completed during my Ph.D. period and was the result of a flash of inspiration. I decided to submit it to arXiv and leave its value to the readers.

  28. Asymptotic Properties of Support Vector Machines in High-Dimension, Low-Sample-Size Settings under a Spiked Model

    Sep 30, 2026Yugo NakayamaSupport Vector MachinesClassification

    Independent researcher

  29. Scale-Split Neural Operator for Memory- and Data-Efficient 3D Turbulence Prediction

    Sep 30, 2026Shaoxiang Qin, Yucheng Zhao, Zongyi Li +2PDE Surrogate ModelingNeural Operators

    University of Tennessee, Knoxville · McGill University · New York University +1

  30. Statistical Benefits of Fine-Tuning from Pretrained Initialization in Diagonal Linear Networks

    Sep 28, 2026Alexandre Declèves, Etienne Boursier, Nicolas FlammarionFine-TuningDeep Linear Networks

    TML, EPFL, Lausanne, Switzerland · INRIA, LMO, Université Paris-Saclay, Orsay, France