Data-Efficient Learning

Follow this topic Ask about this topic

Momentum

14 papers in the last four weeks, up 75% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 158

All topics

HAND: A Biologically-Inspired Activation Function that Improves Generalisation and Sample Efficiency in Image Classification
Oct 8, 2026Michael W. Spratling, Heiko H. SchüttNeural Network Activation Functions Neural Network Generalization
University of Luxembourg, Department of Behavioural and Cognitive Sciences, L-4366 Esch-sur-Alzette, Luxembourg.
SPD-MetaFormer is what you need for small-data brain decoding
Oct 7, 2026Zhida Wang, Wei Lyu, Guo Yu +1EEG Decoding Attention Mechanisms
Stream-Based Active Learning with Cooperative Neural Networks for Data-Efficient Partial Inverse Design: An Automotive Glass Run Channel Case Study
Oct 7, 2026Agung Nugraha, Hyerin Kwon, Heungjun Im +2Inverse Design Data-Efficient Learning
Department of Industrial and Data Engineering, Major in Industrial Data Science and Engineering, Pukyong National University, Busan, 48513, Republic of Korea · DRB Co., Ltd., Busan, 46329, Republic of Korea
Label-Free Coreset Selection with Foundation Models for Efficient Annotation in Computational Pathology
Oct 5, 2026Tuo Yin, Jennifer DhontComputational Pathology Coreset Selection
Department of Medical Physics, Institut Jules Bordet, Hˆpital Universitaire de Bruxelles (HUB), Belgium · Radiophysics and MRI Physics Laboratory, Universit´e Libre de Bruxelles (ULB), Belgium · Laboratory of Image Synthesis and Analysis (LISA), Universit´e Libre de Bruxelles (ULB), Belgium
Geometry Meets Physics: Data-Efficient Pre-Training for Unstructured Neural PDE Solvers
Oct 2, 2026Luis Medrano-Navarro, Giacomo Baldan, Qiang Liu +4Synthetic Data Pretraining Neural PDE Solvers
Technical University of Munich · University of Stuttgart
Sharpen Before You Adapt: Data-Free Entry-State Sharpening for Test-Time Reinforcement Learning
Oct 1, 2026Zhanming Zhang, Vinoth SelvendranRL for Language Model Reasoning Test-Time RL
Independent Researcher New York, NY · Independent Researcher Palo Alto, CA
Less is more: error-distance scaling relation for data-efficient kilometer-scale downscaling of extreme heat
Sep 30, 2026Ahmed Marey, Henry Lu, Abhishek Gaur +6Spatial Generalization Scaling Laws
Centre for Zero Energy Building Studies, Department of Building, Civil and Environmental Engineering, Concordia University, Montreal, H3G 1M8 Canada · Building and Climate Interface, Construction Research Centre, National Research Council Canada, Ottawa, ON, K1A 0R6, Canada · Department of Architecture, School of Sciences and Engineering, The American University in Cairo, New Cairo 11835, Egypt +3
Can Domain Generalization be Guaranteed in Small-Sample Learning?
Sep 30, 2026Hong ZhengDomain Generalization Statistical Learning Theory
This work was completed during my Ph.D. period and was the result of a flash of inspiration. I decided to submit it to arXiv and leave its value to the readers.
Asymptotic Properties of Support Vector Machines in High-Dimension, Low-Sample-Size Settings under a Spiked Model
Sep 30, 2026Yugo NakayamaSupport Vector Machines Classification
Independent researcher
Scale-Split Neural Operator for Memory- and Data-Efficient 3D Turbulence Prediction
Sep 30, 2026Shaoxiang Qin, Yucheng Zhao, Zongyi Li +2PDE Surrogate Modeling Neural Operators
University of Tennessee, Knoxville · McGill University · New York University +1
Statistical Benefits of Fine-Tuning from Pretrained Initialization in Diagonal Linear Networks
Sep 28, 2026Alexandre Declèves, Etienne Boursier, Nicolas FlammarionFine-Tuning Deep Linear Networks
TML, EPFL, Lausanne, Switzerland · INRIA, LMO, Université Paris-Saclay, Orsay, France
Two-Timescale Fine-tuning Provably Learns New Features for Two-Layer ReLU Networks
Sep 28, 2026Etienne Boursier, Nicolas FlammarionRepresentation Learning Fine-Tuning
INRIA, LMO, Universit´e Paris-Saclay, Orsay, France · EPFL, Lausanne, Switzerland
From Grey-Box to Green-Box: When can Physics-Informed Machine Learning Reduce Carbon Footprints in Structural Health Monitoring?
Sep 27, 2026Daisy R. Bradley, Nathan A. Hinchliffe, Daniel J. Pitchforth +2Energy-Efficient ML Physics-Informed ML
Dynamics Research Group, University of Sheffield, Sheffield, UK
The Statistical Benefits of Multiple Responses for Learning from Demonstrations
Sep 27, 2026Chandramauli Chakraborty, Cong MaLearning from Demonstration Statistical Learning Theory
Department of Statistics, University of Chicago
When Labels Are Scarce: An Oscillatory State Space Model for Vibration Diagnosis
Sep 23, 2026Mainak Mallick, Seung-Kyum ChoiPredictive Maintenance Tiny ML
Georgia Institute of Technology
Label-Efficient Learning for Ground-Based Sky-Image Classification: A Benchmark of Transfer Learning, Active Learning, and Pseudo-Labeling on GCD
Sep 22, 2026Esther Bou Dagher, Viktoriya Bu-Dager, Boguslaw ZegarlinskiActive Learning Transfer Learning
CEREMADE, Universit´e Paris Dauphine-PSL, Paris, France. · School of Computing and Information Science, Anglia Ruskin University, Cambridge, United Kingdom. · Institute of Mathematics, Polish Academy of Sciences, Warsaw, Poland.
QUALS: Corpus Equilibrium for Universal Forecasting via Pattern Quantization and Learnability Synchronization
Sep 17, 2026Yujie Li, Zezhi Shao, Chengqing Yu +7Zero-Shot Time Series Forecasting Data Selection
State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China · University of Chinese Academy of Sciences, Beijing, China · School of Data Science and Engineering, East China Normal University, Shanghai, China
A Kernel-Based Modular Discriminant Analysis Framework for Small-Sample Learning
Sep 9, 2026Lingxiao Qu, Yan PeiKernel Methods Data-Efficient Learning
Graduate School of Computer Science and Engineering, University of Aizu, 90 Kamiiawase, Tsuruga, Ikki-machi, Aizuwakamatsu, 965-8580, Fukushima, Japan · Computer Science Division, University of Aizu, 90 Kamiiawase, Tsuruga, Ikki-machi, Aizuwakamatsu, 965-8580, Fukushima, Japan
DE-Venus: A Data-Efficient RLVR Framework for Large Language Models
Sep 3, 2026Shenzhi Yang, Guangcheng Zhu, Kai Tang +11RL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards
1Zhejiang University · 2Ant Group · 3Hong Kong Baptist University +2
Task-Level Natural Language Priors as Learning Signals for Low-Resource LLM Training
Sep 2, 2026Jian Gao, Xiao Zhang, Xun Zhu +2LLM Fine-Tuning Data-Efficient Learning
College of AI, Tsinghua University · Department of Electronic Engineering, Tsinghua University
Data-Efficient Networks for Multi-Contrast MRI Reconstruction based on a Generalized Content/Style Prior
Sep 2, 2026Chinmay Rao, Efe Ilıcak, Matthias J. P. van Osch +5Accelerated MRI Reconstruction Magnetic Resonance Imaging
Department of Radiology, Leiden University Medical Center, Leiden, The Netherlands · Philips Innovative Technologies, Hamburg, Germany · Philips Cardiologs, Paris, France +1
Leveraging Speech Acts for Low-Data and Cross-Domain Conversation Derailment Forecasting
Aug 26, 2026Angela Yifei Yuan, Christine De Kock, Christopher LeckieCross-Domain Generalization Data-Efficient Learning
The University of Melbourne
Foundation Model-Enabled Efficient Data Sampling (FEEDS): A label-efficient training strategy for pan-cancer, multi-tracer PET/CT datasets
Aug 11, 2026Biratal Raj Wagle, Bashirul Azam Biswas, Grant Chau +5Data Selection Medical Image Segmentation
Department of Biomedical Data Science Geisel School of Medicine at Dartmouth Hanover, NH 03755, USA · Radiology Dartmouth Hitchcock Medical Center Lebanon, NH 03766, USA · Department of Urology Yale University New Haven, CT, USA +1
DEFT: Data-Efficient Frequency-domain Top-k Sampling via Inverse Discrete Fourier Transform for Spatiotemporal Dynamical Systems Modeling
Aug 11, 2026Hengbo Xiao, Jiale Liu, Jiahao Song +1PDE Operator Learning Data-Efficient Learning
Peking University
Grid-Preserving Knowledge Distillation: Transferring Convolutional Inductive Bias to Vision Transformers under Data Scarcity
Aug 11, 2026Junyong Choi, Cheolhyeon Park, Jaehoon ChoVision Transformer Cross-Architecture Knowledge Distillation
Hyundai Motor Company · School of Electronics and Avionics Engineering, Korea Aerospace University
Rethinking Data Efficiency in Industrial Dense Prediction: Pretraining Coherence, Not Inductive Bias, Determines ViTs Low-Data Advantage
Aug 11, 2026Haoran Sui, Yaoyuan JiaVision Transformer Object Detection
ZTE Corporation, Shenzhen, Guangdong Province, P.R. China · The University of Hong Kong, Hong Kong SAR, P.R. China
GLOBE: Trajectory-Aligned Gradient Matching with Structured SparseOptimization for Coreset Selection
Aug 3, 2026Hetian Liu, Jin Cui, Mengcheng Shi +4Training Data Selection Coreset Selection
State Key Laboratory of Human-Machine Hybrid Augmented Intelligence, and Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University · School of Software Engineering, Xi’an Jiaotong University · School of Computer Science and Technology, Xi’an Jiaotong University
Bootstrapping Self-Supervised Learning of Binary Classification Using Error Bounds: A Case Study on a Robotic Insertion Task
Jul 31, 2026Zebin Duan, Norbert Krüger, Juan Heredia +2Self-Supervised Learning Robotic Data Collection
Mærsk Mc-Kinney Møller Institute, University of Southern Denmark, 5230 Odense M, Denmark · Danish Institute for Advanced Study (DIAS), University of Southern Denmark, 5230 Odense M, Denmark
Curriculum Matters: Data-Efficient Relational PFN Pretraining with Synthetic Data
Jul 31, 2026Mohammad Sadeq Abolhasani, Viswanath GanapathySynthetic Data Pretraining Synthetic Data Generation
SAP Labs, LLC. Palo Alto, CA, United States
SDO: Structure-Aware Data Organization for Efficient LLM Post-Training
Jul 29, 2026Jinliang Gao, Ning Yang, Hai Wang +2Training Data Curation Efficient Language Model Training
Institute of Automation, Chinese Academy of Sciences · National University of Defense Technology

Oct 8, 2026, cs.CVJ/K move · Enter open · S save

HAND: A Biologically-Inspired Activation Function that Improves Generalisation and Sample Efficiency in Image Classification

Michael W. Spratling, Heiko H. Schütt

University of Luxembourg, Department of Behavioural and Cognitive Sciences, L-4366 Esch-sur-Alzette, Luxembourg.

Neural Network Activation Functions Neural Network Generalization Data-Efficient Learning Image Classification Convolutional Neural Networks

Open paper AskPDF arXiv

DNNs exhibit robustness and generalisation issues not seen in humans. They are also far less data-efficient learners, requiring considerably more training samples to accurately classify novel exemplars. Inductive bias could help with these issues by providing in-built mechanisms to improve generalisation, and hence, reduce reliance on learning from data. We incorporate a biologically-inspired inductive bias into a new activation function, HAND (Homeostasis, Accelerating Nonlinearity, and Divisive-nomalisation), and show its effectiveness with CNNs trained on image classification. Using HAND a ConvNeXt-tiny required 25 training epochs to reach the same accuracy on ImageNet1k as the unmodified model achieved after 200 epochs. Consistent with the effects of an inductive bias, the performance gap reduced with training time and increased data augmentation. When the volume of training data was reduced and unevenly distributed between classes (Long-tailed ImageNet) the improvements in accuracy were even larger and did not reduce with increased training time. Generalisation performance with the common-corruptions data, and the ability to reject samples from unknown classes, were unaffected or improved by HAND. Results generalised across CNN architectures and training data-sets. HAND can, therefore, reduce the required training time and/or the required volume and variety of training data, helping to improve sample efficiency.

Momentum

Related topics

Latest papers 158