Data-Efficient Learning

Follow this topic Ask about this topic

Momentum

14 papers in the last four weeks, up 75% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 158

All topics

From Parameters to Data: A Task-Parameter-Guided Fine-Tuning Pipeline for Efficient LLM Alignment
May 20, 2026Hao Chen, Qi Zhang, Liyao Li +7LLM Alignment LLM Fine-Tuning
Zhejiang University · Ant Group · Eastern Institute of Technology
Sample Complexity of Transfer Learning: An Optimal Transport Approach
May 19, 2026Haoyang Cao, Xin Guo, Wenpin Tang +1Sample Complexity Transfer Learning
Department of Applied Mathematics and Statistics, John Hopkins University, Baltimore, MD, USA · Department of Industrial Engineering and Operations Research, University of California, Berkeley, CA, USA · Department of Industrial Engineering and Operations Research, Columbia University, New York, NY, USA +1
Less Data, Faster Training: repeating smaller datasets speeds up learning via sampling biases
May 19, 2026Jingwen Liu, Ezra Edelman, Surbhi Goel +1Deep Learning Optimization Statistical Learning Theory
Columbia University · University of Pennsylvania · Kempner Institute, Harvard University
Efficient coding along the visual hierarchy
May 18, 2026Ananya Passi, Brian S. Robinson, Michael F. BonnerUnsupervised Learning Neural Processes
Department of Cognitive Science, Johns Hopkins University, Baltimore, MD, USA · Applied Physics Laboratory, Johns Hopkins University, Baltimore, MD, USA
Network Knowledge Prior Guided Learning for Data-Efficient Surface Defect Detection
May 18, 2026Hang-Cheng Dong, Guodong Liu, Dong Ye +1Interpretable ML Industrial Inspection
School of Instrumentation Science and Engineering, Harbin Institute of Technology, Harbin, China · Harbin Institute of Technology Suzhou Research Institute, Suzhou, China
Brain-inspired spike-timing plasticity for reliable label-efficient event-camera vision
May 17, 2026Mohamad Yazan Sadoun, Sarah Sharif, Yaser Mike BanadEvent-Based Vision Spiking Neural Networks
School of Electrical and Computer Engineering, University of Oklahoma, Norman, OK, USA
D $^2$ Evo: Dual Difficulty-Aware Self-Evolution for Data-Efficient Reinforcement Learning
May 16, 2026Ru Zhang, Renda Li, Ziyu Ma +4RL for Language Model Reasoning Curriculum RL
Alibaba Group · AMAP, Alibaba Group · BUPT
Learning-Zone Energy: Online Data Selection for Efficient RL Post-Training
May 16, 2026Peng Cui, Boyao Yang, Jun ZhuRL for Language Model Reasoning RL Post-Training
Dept. of Comp. Sci. & Tech., Institute for AI, BNRist Center, Tsinghua-Bosch Joint ML Center, THBI Lab, Tsinghua University, Beijing 100084, China · Dept. of Automation, Tsinghua University, Beijing 100084, China
Distributionally Robust Multi-Task Reinforcement Learning via Adaptive Task Sampling
May 14, 2026Nicholas E. Corrado, Wenyuan Huang, Josiah P. HannaDistributionally Robust RL Multi-Task RL
Computer Sciences Department · University of Wisconsin – Madison
EMA: Efficient Model Adaptation for Learning-based Systems
May 13, 2026Daiyang Yu, Xinyu Chen, Yihan Zhang +3Adaptive Learning Data-Efficient Learning
University of Illinois Urbana-Champaign USA · The Hong Kong University of Science and Technology Hong Kong, China
Genetic algorithm vs. gradient descent for training a neural network architecture dedicated to low data regimes in small medical datasets
May 13, 2026Amine Boukhari, Boglarka Ecsedi, Laszlo Papp +1Gradient Descent Neural Network Optimization
Laboratory of medical information processing (LaTIM), INSERM, UMR 1101,university of western brittany · Center for Medical Physics and Biomedical Engineering, Medical University of Vienna · Georgia Institute of Technology
ECG-NAT: A Self-supervised Neighborhood Attention Transformer for Multi-lead Electrocardiogram Classification
May 13, 2026Mahsa Gazeran, Sayvan Soleymanbaigi, Fatemeh Daneshfar +2Self-Supervised Time Series Representation Learning Transformer Attention
Department of Computer Engineering, University of Kurdistan, Sanandaj, Iran · Department of Mathematics and Operational Research, University of Mons, Mons, Belgium
SMA: Submodular Modality Aligner For Data Efficient Multimodal Learning
May 13, 2026Truong Pham, Anay Majee, Rishabh IyerCross-Modal Learning Cross-Modal Alignment
The University of Texas at Dallas
A Data Efficiency Study of Synthetic Fog for Object Detection Using the Clear2Fog Pipeline
May 12, 2026Mohamed Ahmed Mohamed, Xiaowei HuangSynthetic Data Pretraining Synthetic-to-Real Domain Adaptation
School of Computer Science and Informatics, University of Liverpool, Liverpool, L69 3DR, UK
On What We Can Learn from Low-Resolution Data
May 12, 2026Theresa Dahl Frehr, Niels Henrik Pontoppidan, Hiba Nassar +1Data-Efficient Learning
1Technical University of Denmark · 2Eriksholm Research Center
Spectral Vision Transformer for Efficient Tokenization with Limited Data
May 12, 2026Alexandra G. Roberts, Maneesh John, Jinwei Zhang +10Medical Imaging Image Tokenization
Understanding Sample Efficiency in Predictive Coding
May 12, 2026Gaspard Oliviers, Elene Lominadze, Rafal BogaczBackpropagation Predictive Coding
Nuffield Department of Clinical Neurosciences, University of Oxford, United Kingdom · MRC Centre of Research Excellence in Restorative Neural Dynamics, United Kingdom
Constraint-Data-Value-Maximization: Utilizing Data Attribution for Effective Data Pruning in Low-Data Environments
May 11, 2026Danilo Brajovic, David A. Kreplin, Marco F. HuberDataset Pruning Training Data Selection
Fraunhofer IPA, Stuttgart, Germany · Institute of Industrial Manufacturing and Engineering IFF, University of Stuttgart, Germany · Hochschule Heilbronn, Germany
Semi-Supervised Neural Super-Resolution for Mesh-Based Simulations
May 10, 2026Jiyeon Kim, Youngjoon Hong, Won-Yong ShinSemi-Supervised Learning Data-Efficient Learning
School of Mathematics and Computing (Computational Science and Engineering), Yonsei University, Seoul, Republic of Korea · Department of Mathematical Sciences, Seoul National University, Seoul, Republic of Korea
How Much is Brain Data Worth for Machine Learning?
May 10, 2026Lane Lewis, Zhixin Wang, David Schwab +1Neural Scaling Laws Data-Efficient Learning
Neuroscience Institute, Carnegie Mellon University, Pittsburgh, PA, USA · Department of Machine Learning, Carnegie Mellon University, Pittsburgh, PA, USA · NSF AI Institute for Artificial and Natural Intelligence (ARNI) +2
Quantum Transfer Learning Shows Improved Robustness in Low-Data Regimes
May 9, 2026Li-An Lo, Li-Yi Hsu, Hsien-Yi HsiehQuantum Machine Learning Transfer Learning
Department of Engineering Science, National Cheng Kung University, Tainan 701401, Taiwan · Department of Physics, Chung Yuan Christian University, Chungli 32081, Taiwan · Physics Division, National Center for Theoretical Sciences, Taipei 106319, Taiwan +1
Anchoring the Eigengap: Cross-Modal Spectral Stabilization for Sample-Efficient Representation Learning
May 9, 2026Nikhil J. Dhinagar, Vidhi Chhatbar, Chirag Jagad +6Representation Learning Spectral Methods
Imaging Genetics Center, Mark & Mary Stevens Neuroimaging & Informatics Institute, Keck School of Medicine, University of Southern California, Los Angeles, CA, USA · Neuroscience Graduate Program, Mark & Mary Stevens Neuroimaging & Informatics Institute, Chan Division of Occupational Science & Occupational Therapy, Biomedical Engineering, University of Southern California, Los Angeles, CA, USA
Probing the Impact of Scale on Data-Efficient, Generalist Transformer World Models for Atari
May 9, 2026Jooyeon KimWorld Model Learning Transformer
Graduate School of Artificial Intelligence UNIST
The Geometric Structure of Models Learning Sparse Data
May 8, 2026Thomas Walker, T. Mitchell Roddenberry, Ahmed Imtiaz Humayun +2Neural Representation Geometry Statistical Learning Theory
Rice University · Rice University† · Now at Google Research +1
A meshfree exterior calculus for generalizable and data-efficient learning of physics from point clouds
May 8, 2026Benjamin D. Shaffer, Brooks Kinch, M. Ani Hsieh +1Point Cloud Learning PDE Surrogate Modeling
University of Pennsylvania Philadelphia, PA 19104 · *Mechanical Engineering and Applied Mechanics
Toward Privileged Foundation Models:LUPI for Accelerated and Improved Learning
May 8, 2026Xueying Ding, Leman AkogluTabular Foundation Models Learning Using Privileged Information
Carnegie Mellon University
Learning Large-Scale Modular Addition with an Auxiliary Modulus
May 8, 2026Hanato Kikuchi, Ryosuke Masuya, Kazuhiko Kawamoto +1Boolean Function Learning Distribution Shift
Chiba University · National Institute of Informatics
Transfer Learning Across Fast- and Full-Simulation Domains in High-Energy Physics
May 8, 2026Matthias Schott, Lucie FlekDomain Adaptation High-Energy Physics
Institute of Physics, University of Bonn, Germany · Bonn-Aachen International Center for Information Technology (b-it), University of Bonn, Germany
Accelerated and data-efficient flow prediction in stirred tanks via physics-informed learning
May 8, 2026Mahdi Naderibeni, Liang Wu, David M. J. TaxPDE Surrogate Modeling Physics-Informed ML
aPattern Recognition and Bio-informatics Group, Delft University of Technology, Delft, the Netherlands · bScience & Research and Innovation, dsm-firmenich, Delft, the Netherlands
ProtoSSL: Self-Supervised Pretraining and Downstream Transfer for Projection-Based Prototype Models
May 7, 2026Steven Song, Sahil Sethi, Brett Beaulieu-Jones +1Self-Supervised Learning Self-Supervised Pre-Training
Department of Computer Science, University of Chicago, Chicago, IL 60637, USA · Center for Translational Data Science, University of Chicago, Chicago, IL 60637, USA · Medical Scientist Training Program, University of Chicago, Chicago, IL 60637, USA +3

May 20, 2026, cs.LGJ/K move · Enter open · S save

From Parameters to Data: A Task-Parameter-Guided Fine-Tuning Pipeline for Efficient LLM Alignment

Hao Chen, Qi Zhang, Liyao Li, Zhanming Shen+6

Zhejiang University · Ant Group · Eastern Institute of Technology

LLM Alignment LLM Fine-Tuning Training Data Selection Attention Head Pruning Attention Head Specialization Data-Efficient Learning

Open paper AskPDF arXiv

Adapting Large Language Models (LLMs) to specialized domains typically incurs high data and computational overhead. While prior efficiency efforts have largely treated data selection and parameter-efficient fine-tuning as isolated processes, our empirical analysis suggests they may be intrinsically coupled. We posit the Strong Map Hypothesis: a sparse subset of attention heads plays a dominant role in task-specific adaptation, acting as keys that unlock specific data patterns. Building on this observation, we propose From Parameters to Data (P2D), a unified framework that leverages these task-sensitive attention heads as a dual compass for both sample mining and structural pruning. To rigorously quantify the total pipeline cost, we introduce the Alignment Efficiency Ratio (AER) metric for both selection latency and training time. Mechanistically, P2D identifies critical heads via a lightweight proxy and uses them as a functional filter to curate high-affinity data, establishing a synergistic pipeline. Empirically, by updating merely 10% of attention heads on 10% of the data, P2D achieves an 8.3 pp performance gain over strong baselines and delivers a 7.0x end-to-end time speedup. These results validate that precise parameter-data synchronization eliminates redundancy, offering a new paradigm for efficient alignment.

Momentum

Related topics

Latest papers 158