Data-Efficient Learning

Follow this topic Ask about this topic

Momentum

14 papers in the last four weeks, up 75% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 158

All topics

Accurate and Resource-Efficient Federated Continual Learning
Jun 9, 2026Jebacyril Arockiaraj, Dhruv Parikh, Jayashree Adivarahan +2Communication-Efficient Distributed Training Continual Learning
University of Southern California · Los Angeles, USA · DEVCOM Army Research Office +1
Unifying Data, Memory, and Compute Efficiency in LLM training: A Survey
Jun 9, 2026Vanessa Schmidt, Huy Hoang Nguyen, Cédric Jung +2Efficient Language Model Training LLM Training
Chair of Information Theory and Data Analytics (INDA), RWTH Aachen University, 52074 Aachen, Germany · AIT Austrian Institute of Technology GmbH, 1210 Vienna, Austria · Automation and Control Institute, Technische Universität Wien (TUW), 1040 Vienna, Austria
Deep Active Re-Labeling: Toward Noise-Resilient Annotation Efficiency
Jun 7, 2026Md Abdullah Al Forhad, Weishi ShiHuman-in-the-Loop Annotation Active Learning
Active Machine Learning Lab · Learning · Language · Vision (LLaVi) Lab · Department of Computer Science and Engineering +1
OrderDP: A Theoretically Guaranteed Lossless Dynamic Data Pruning Framework
Jun 7, 2026Chenhan Jin, Shengze Xu, Qingsong Wang +3Dataset Pruning Data-Efficient Learning
The Chinese University of Hong Kong · Xiangtan University · University of Utah +3
Making the Most of Limited Data: Score-Aware Training for Text-to-Music Generation
Jun 5, 2026Yun-Chen Cheng, Tzu-Hung Huang, Chih-Pin TanText-to-Music Generation Audio Diffusion Models
Graduate Institute of Communication Engineering, National Taiwan University, Taipei, Taiwan
CLaaS: Continual learning as a service for sample efficient online learning
Jun 4, 2026Kion Fallah, Silen Naihin, Barak Widawsky +1Continual Learning for LLM Agents Continual Learning
1Resolute Labs · 2South Park Commons · 3Incept Labs
GOTabPFN: From Feature Ordering to Compact Tokenization for Tabular Foundation Models on High-Dimensional Data
Jun 3, 2026Al Zadid Sultan Bin Habib, Md Younus Ahamed, Prashnna Kumar Gyawali +2Tabular Foundation Models Tabular Prediction
Lane Department of Computer Science and Electrical Engineering, West Virginia University, Morgantown, WV 26506, USA · Scientific Computing and Imaging Institute & Department of Biomedical Informatics, The University of Utah, Salt Lake City, UT 84112, USA
GeoMin: Data-Efficient Semi-Supervised RLVR via Geometric Distribution Modeling
Jun 3, 2026Guangcheng Zhu, Shenzhi Yang, Haobo Wang +9RL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards
1Zhejiang University · 2Ant Group
Smart Picks in the Dark: Towards Efficient RLVR for Reasoning via Tracing Metacognitive Pivots
Jun 3, 2026Guangcheng Zhu, Shenzhi Yang, Haobo Wang +7Uncertainty Quantification RL for Language Model Reasoning
1Zhejiang University · 2Ant Group
q0: Primitives for Hyper-Epoch Pretraining
Jun 2, 2026Bishwas Mandal, Shmuel Berman, Akshay Vegesna +1Language Model Pretraining Efficient Language Model Training
Q Labs · Princeton University
Hybrid Adaptive Kalman Filtering for Data-Efficient Joint Tracking and Classification
Jun 1, 2026Jiho Lee, Nisar R. Ahmed, Rebecca RussellKalman Filtering Self-Supervised Learning
Charles Stark Draper Laboratory, Inc., Cambridge, MA · Ann and H. J. Smead Department of Aerospace Engineering Sciences at the University of Colorado, Boulder, Boulder, CO · Draper
Hybrid Neural Ordinary Differential Equations for Data-Efficient Polymerization Modeling with Incomplete Kinetics
Jun 1, 2026Marah Almanasreh, Alexander Mitsos, Eike CramerNeural Surrogate Modeling Materials Science
RWTH Aachen University, Process Systems Engineering (AVT.SVT), 52074 Aachen, Germany · JARA-CSD, 52425 Jülich, Germany · Energy Systems Engineering (ICE-1), Forschungszentrum Jülich, 52425 Jülich, Germany +1
Provable Data Scaling Law for Meta Learning via Complexity Minimization
Jun 1, 2026Kazuto Fukuchi, Ryuichiro Hataya, Kota MatsuiMeta-Learning Few-Shot Learning
University of Tsukuba, Japan · RIKEN AIP, Japan · SB Intuitions Corp., Japan +3
Improvise, Adapt, Overcome: An On-The-Fly Multifidelity Algorithm for Efficient Machine Learning
Jun 1, 2026Vivin Vinod, Peter ZaspelQuantum Chemistry Multifidelity Modeling
School of Mathematics and Natural Sciences, University of Wuppertal, Germany.
KDH-CAD: Knowledge-data hybrid CAD learning under data scarcity
Jun 1, 2026Ziqin Gao, Zhijie Yang, Qiang ZouData-Efficient Learning
State Key Laboratory of CAD&CG, Zhejiang University, Hangzhou, 310027, China
When Data Is Scarce: Scaling Sparse Language Models with Repeated Training
May 31, 2026Boqian Wu, Qiao Xiao, Patrik Okanovic +6Language Model Scaling Laws Efficient Language Model Training
University of Luxembourg · University of Twente · Eindhoven University of Technology +1
LARK: Learnability-Grounded Trajectory Selection for Efficient Reasoning Distillation
May 28, 2026Tianrun Yu, Kaixiang Zhao, Chih-Chun Chen +5CoT Distillation Language Model Distillation
Brigham Young University · The Pennsylvania State University · University of North Carolina at Chapel Hill
MEMENTO: Leveraging Web as a Learning Signal for Low-Data Domains
May 28, 2026Ashutosh Ojha, Vinay Aggarwal, Ashutosh Srivastava +3Lifelong Learning Agents Agent Skill Learning
Adobe, Media & Data Science Research Lab
Learning Robust and Task-Invariant Functional Representation from fMRI through Siamese Self-Supervised Learning
May 27, 2026Jiyao Wang, Peiyu Duan, Nicha C. Dvornek +4Self-Supervised Learning Siamese Neural Networks
Department of Biomedical Engineering, Yale University, New Haven, CT, USA · Radiology & Biomedical Imaging, Yale School of Medicine, New Haven, CT, USA · Electrical Engineering, Yale University, New Haven, CT, USA +1
Is Backpropagation Optimal? When Synthetic Gradients Improve Sample Efficiency
May 27, 2026Yibo Jacky Zhang, Zeyu Tang, Sanmi KoyejoDeep Learning Optimization Backpropagation
Stanford University
Which Anatomy Matters Under Limited Labels? A Data-Efficient Anatomy-Aware Benchmark for Cardiac Pathology Prediction
May 25, 2026Himanshu SinghHealthcare Medical Image Analysis
Computing & Artificial Intelligence Division, Los Alamos National Laboratory, Los Alamos, NM 87545, USA.
FDDet: Achieving Data-Efficient Food Defect Detection Under Real-World Scenarios
May 23, 2026Ruihao Xu, Yong Liu, Yansong TangObject Detection Semi-Supervised Learning
Tsinghua Shenzhen International Graduate School, Tsinghua University
ChainzRule: Sample-Efficient, Robust Deep Learning Across Tabular, NLP, and Vision Tasks
May 23, 2026Rowan MartnishnPolynomial Neural Networks Neural Network Robustness
Sentivity AI
Knowledge Graph Modulated Deep Learning for Limited-Sample Clinical Data Analysis
May 22, 2026Yuwei Xue, Sakib Mostafa, James Zou +5Graph Representation Learning Medical Diagnosis
Department of Radiation Oncology, Stanford University, Stanford, California, USA · Department of Electrical Engineering, Stanford University, Stanford, California, USA · Department of Biomedical Data Science, Stanford University School of Medicine, Stanford, California, USA +6
Multitask learning with semiempirical orbital charges enables sample-efficient MLIPs
May 22, 2026Ihor Neporozhnii, Sjoerd Hoogland, Oleksandr VoznyyMachine Learning Interatomic Potentials Multi-Task Learning
Department of Physical and Environmental Sciences, University of Toronto, 1065 Military Trail, Toronto, M1C 1A4, Ontario, Canada. · The Alliance for AI-Accelerated Materials Discovery, 10 King’s College Road, Toronto, M5S 3G4, Ontario, Canada. · Department of Electrical and Computer Engineering, University of Toronto, 10 King’s College Road, Toronto, M5S 3G4, Ontario, Canada.
Label-Efficient Dataset Pruning via Semi-Supervised Pseudo-Labeling
May 22, 2026Yeseul Cho, Baekrok Shin, Changmin Kang +1Dataset Pruning Coreset Selection
Kim Jaechul Graduate School of AI, KAIST
Balancing Uncertainty and Diversity of Samples: Leveraging Diversity of Least, High Confidence Samples for Effective Active Learning
May 21, 2026Vipul Arya, S. H. Shabbeer Basha, Srikrishna U N +2Active Learning Uncertainty Sampling
School of Computer Science and Engineering, RV University, Bengaluru, Karnataka-560059, India · School of Engineering & Technology, Vidyashilp University, Bangalore, India · Shiv Nadar Institution of Eminence, Delhi NCR
Algebraic Machine Learning for Small-to-Medium Datasets Is Competitive against Strong Standard Baselines
May 21, 2026David Mendez, Fernando Martin-Maroto, Gonzalo G. de PolaviejaClassification Image Classification
Mathematics of Behavior and Intelligence Lab, Champalimaud Foundation, Lisbon
EasyVFX: Frequency-Driven Decoupling for Resource-Efficient VFX Generation
May 21, 2026Yue Ma, Xu Ye, Qinghe Wang +9Frequency-Domain Feature Learning Data-Efficient Learning
1HKUST · 2DUT · 3THU +1
Data-Efficient Neural Operator Training via Physics-Based Active Learning
May 20, 2026Alicja Polanska, Lorenzo Zanisi, Vignesh Gopakumar +1Active Learning Physics-Informed ML
University College London · United Kingdom Atomic Energy Authority

Jun 9, 2026, cs.LGJ/K move · Enter open · S save

Accurate and Resource-Efficient Federated Continual Learning

Jebacyril Arockiaraj, Dhruv Parikh, Jayashree Adivarahan, Rajgopal Kannan+1

University of Southern California · Los Angeles, USA · DEVCOM Army Research Office+1

Communication-Efficient Distributed Training Continual Learning Federated Continual Learning Random Feature Methods Federated Learning Data-Efficient Learning

Open paper AskPDF arXiv

Federated continual learning (FCL) must learn from distributed task streams under limited resources, such as communication, computation, memory, and label availability. Existing FCL methods often rely on repeated local optimization, replay, and full supervision. Analytic alternatives avoid iterative training and replay, but using high-dimensional random features to improve accuracy requires a second-order feature statistic, the Gram matrix, which has a quadratic communication cost in the random feature size

M

. We propose FedRAN, a resource-aware analytic FCL framework that replaces gradient-based updates with compact random feature statistics. Each client transmits a truncated-SVD summary of its Gram matrix, reducing the dominant second-order upload from quadratic to linear in

M

for fixed rank. The server performs a two-level QR-SVD subspace merge, spatially across clients and temporally across tasks, and solves a ridge classifier in closed form. FedRAN further supports label scarcity through prototype-based pseudo-labeling. Across CIFAR-100, ImageNet-R, and VTAB datasets, FedRAN improves average accuracy by up to 4.8 percentage points over the strongest baseline, uses 30.6-121.8

\times

less per-client communication than optimization-based FCL, and is 190.3

\times

faster on average than gradient-based baselines; with only 20% labels, pseudo-labeling improves average accuracy by up to 6.61 points. These results show that FedRAN enables accurate and resource-efficient FCL under communication, computation, and label constraints. The source code is available at https://github.com/JebacyrilArockiaraj/Fed-RAN-SSL.

Momentum

Related topics

Latest papers 158