Transfer Learning

Follow this topic Ask about this topic

Transfer learning asks how a model trained on one task or domain can help with another, reducing the need for target-specific data and computation. Common approaches include pretraining followed by fine-tuning or parameter-efficient adapters, feature alignment for domain shifts, and knowledge transfer through distillation, model merging, or shared representations. Evaluations span zero-shot and few-shot transfer, cross-domain benchmarks in vision and language, and adaptation in settings such as clinical data, robotics, and scientific modeling. A central tension is that transferred knowledge can improve accuracy and sample efficiency or cause negative transfer; predicting when it will help, and separating reusable knowledge from dataset or architecture effects, remains difficult.

Latest papers

422 papers62 papers in the last four weeks, up 29% on the four weeks before. 0.4% of all new papers.

Cross-simulator transfer with foundation model summaries: Towards robust SKA-era reionization inference
Aug 26, 2026Yannic Pietschke, Caroline Heneka, Ayodele Ore +1Simulation-Based Inference Transfer Learning
Institut für Theoretische Physik, Universität Heidelberg, Philosophenweg 16, 69120 Heidelberg, Germany · Max Planck Institute for Radioastronomy, Auf dem Hügel 69, Bonn, 53121, Germany
A General-Purpose Molecular Foundation Model Transfers Across Diverse Olfactory Tasks
Aug 26, 2026Yikun Han, Yi Wang, Neil Mankodi +2Molecular Foundation Models Transfer Learning
Department of Statistics, University of Michigan, Ann Arbor, Michigan, USA · Computer Science and Engineering Division, University of Michigan, Ann Arbor, Michigan, USA
DeMMO: Longitudinal and Cross-Disease Modelling of Digital Mobility Outcomes via Multi-Task Learning
Aug 25, 2026Menghui Zhou, Zhipeng Yuan, Vitaveska Lanfranchi +1Clinical Outcome Prediction Multi-Task Learning
School of Computer Science, University of Sheffield, Sheffield, United Kingdom
UHI-Bench: Benchmarking Dual-Source Urban Heat Island Modeling Across Cities in Diverse Climate Regimes
Aug 24, 2026Wanyun Ling, Chenxi Liu, Yi Xie +3Benchmark Design Climate Science
Department of Operations and Technology Technical University of Munich Heilbronn, Germany · Hong Kong Institute of Science & Innovation, Chinese Academy of Sciences Hong Kong SAR, China · Faculty of Computer Science RWTH Aachen University Aachen, Germany +2
From Symmetry to Invariance: Learning Galois Equivalent Representations in Finite Fields
Aug 23, 2026Zheng Zhang, Na ZhangInvariant Representation Learning Equivariant Representation Learning
Department of Mathematics, Towson University, 7800 York Rd, Towson, MD 21204, USA
Tracing the Unlabeled Storm: Cross-Variable Transfer in a Lagrangian Atmospheric JEPA Framework
Aug 23, 2026K M Anirudh, S Sandeep, Hariprasad KodamanaPrecipitation Forecasting Spatiotemporal Forecasting
Centre for Atmospheric Sciences, IIT Delhi, New Delhi, India · Yardi School of Artificial Intelligence, IIT Delhi, New Delhi, India · Department of Chemical Engineering, IIT Delhi, New Delhi, India
TransHands: Repurposing Human Pose Encoders as Hand Pose Encoders
Aug 23, 2026Milo Piccioli, Gianluca Amprimo, Claudia Ferraris +13D Hand Pose Estimation Transfer Learning
Politecnico di Torino, Torino, Italy · Consiglio Nazionale delle Ricerche, Torino, Italy
How Architecture and Training Affect TPC Representations Across Experiments
Aug 22, 2026Tyler Wheeler, Michelle P. Kuchera, Raghuram Ramanujan +12Representation Learning Transfer Learning
Department of Computational Math, Science, and Engineering, Michigan State University, East Lansing, MI, USA · Department of Physics, Grand Valley State University, Allendale, Michigan, USA · Department of Physics, Davidson College, Davidson, NC, USA +5
When does fusing hand-crafted knowledge with learned representations pay? A cost-normalized benchmark of stacking, substitution, and interference
Aug 21, 2026Ahmad AlMughrabi, Albert Clop, Benjamin Busam +2Representation Learning Image Classification
Universitat de Barcelona, Barcelona, Spain · Technical University of Munich, Munich, Germany · Universitat Pompeu Fabra, Barcelona, Spain
Lift, Associate, and Fuse: A Decision-Centric Framework for 2D-to-3D Foundation Model Transfer
Aug 21, 2026Wentao Sun, Yiping Chen, John S. Zelek +1Transfer Learning 3D Semantic Segmentation
Department of Systems Design Engineering, University of Waterloo, N2L 3G1 Waterloo, Canada · School of Geospatial Engineering and Science, Sun Yat-sen University, 519082 Zhuhai, China
Break It Down, Pass It On: Cross-Task Skill Transfer in LLM Agents
Aug 20, 2026Yiyang Feng, Biddut Sarker Bijoy, Niranjan Balasubramanian +1LLM Agent Skill Learning LLM Agent Skill Evaluation
Computer Science Department, Stony Brook University, Stony Brook, USA
Transfer Learning in Nonparametric Regression with Deep ReLU Networks
Aug 20, 2026Junpeng Ren, Carlos Misael Madrid Padilla, Yanzhen Chen +1Transfer Learning Nonparametric Regression
Department of Statistics and Data Science, University of California, Los Angeles, California, USA · Department of Statistics and Data Science, Washington University in St. Louis, Missouri, USA · Department of Information Systems, Business Statistics and Operations Management, Hong Kong University of Science and Technology, Hong Kong, China.
SceneGTMM: A Conformal Mapping-based Scene-Aware Transferable GNN-Transformer Dual-Graph Interaction Framework for Map Matching
Aug 19, 2026Yongliang Zhang, Feng Song, Ji Chen +6Intelligent Transportation Systems Spatiotemporal GNNs
Amap, Alibaba Group,beijing,china
Beyond Forgetting: Diagnosing and Harnessing Shared Reasoning in Continual RLVR
Aug 19, 2026Lirui Luo, Guoxi Zhang, Hongming Xu +3RL for Language Model Reasoning Replay-Based Continual Learning
State Key Lab of General AI, School of Intelligence Science and Technology, Peking University · State Key Laboratory of General Artificial Intelligence, BIGAI · Beijing Institute of Technology +1
What Does Attention Transfer Transfer? Attention Structure and Robustness in Vision Transformers
Aug 19, 2026Jesse PonnockVision Transformer Transformer Attention
Johns Hopkins University
Where a New Concept Must Enter: Entry Point Gates Cross-Task Usability in Unified Multimodal Models
Aug 18, 2026Zongyang Qiu, Yihan Wu, Kaixuan Fan +2Unified Multimodal Models Multimodal Generation
The Hong Kong University of Science and Technology (Guangzhou) · Columbia University · MMLab, CUHK +1
Understanding Curriculum Learning in Large Language Models via Cross-Difficulty Optimization Dynamics
Aug 18, 2026Zhikai Ding, Ziyi YeCurriculum Learning LLM Training
Fudan University
CM-MAE: A Physics-Guided Cross-Modal Self-Supervised Learning Framework for Vision-Wireless Applications
Aug 16, 2026Yubo Zhang, Yiyao LiuCross-Modal Representation Learning Self-Supervised Pre-Training
TransfHAR: Self-Supervised Wrist Representations for On-Demand Activity Recognition
Aug 16, 2026Aidan Bradshaw, Riku Arakawa, Xin Liu +1Human Activity Recognition Wearable Sensing
Northwestern University Evanston, Illinois, USA · Carnegie Mellon University Pittsburgh, Pennsylvania, USA · Google Seattle, Washington, USA
Small Models Scout Bottleneck Order for Large-Model Data Control
Aug 14, 2026Seungmin Choi, Jiwon Sung, Muhammad Umer +4LLM Training Efficient Language Model Training
Stanford University · Seoul National University
CytoBERT: A Foundation Model for Cytometry Data
Aug 14, 2026Syed Abdul Haseeb Qadri, Bjarne C. Hiller, Felix Blanke +7Representation Learning Transfer Learning
University of Rostock, Rostock, Germany · Marburg University, Marburg, Germany · Hessian Center for Artificial Intelligence, Darmstadt, Germany
Attributing Preprocessing Invariance in Spectral Foundation Models
Aug 14, 2026Dongjun Wei, Hongyi WuTransfer Learning Raman Spectroscopy
ESCP Business School, Paris · OpluxCare Tech Group Ltd. · The University of Hong Kong, Hong Kong
From Fixed Grids to Moving Particles:A Transferable Latent Operator for Fluid Dynamics
Aug 14, 2026Meng Li, Chuqi Chen, Zhengqing Gao +4Neural Operators Computational Fluid Dynamics
Shanghai Artificial Intelligence Laboratory · The Hong Kong University of Science and Technology · University of Michigan +2
Emergence of Transfer Learning towards Specific Identification of Alzheimer's Disease A Prospective Approach
Aug 13, 2026Soumik Podder, Chandramouli HaldarAlzheimer's Disease Classification Transfer Learning
Department of Basic Science and Humanities Institute of Engineering and Management (IEM), University of Engineering and Management Kolkata Kolkata, India · Department of Computer Science and Engineering Guru Nanak Institute of Technology Kolkata, India
$\varepsilon$ -MemEvo: Adaptive Cross-Task Memory Transfer for LLM Program Evolution
Aug 12, 2026Aofan Liu, Shiyuan Song, Yiyan QiEvolutionary Optimization Large Language Model-Guided Optimization
XBridge: Entity-Grounded Latent Bridge for Heterogeneous LLM Communication
Aug 12, 2026Wooseong Yang, Wei-Chieh Huang, Weizhi Zhang +3LLM Grounding Transfer Learning
University of Illinois Chicago · Capital One AI Foundations · Hankuk University of Foreign Studies +1
Market-Information-Aware Gated-LoRA of Foundation Models for Transferable Day-Ahead Electricity Price Forecasting
Aug 11, 2026Hang Fan, Wei Wei, Shengwei MeiElectricity Price Forecasting Time Series Forecasting
Black-Box Knowledge Transfer across Distinct Feature Sets
Aug 10, 2026Oh-Ran Kwon, Daeyoung HamTransfer Learning Knowledge Distillation
Department of Statistics, The Ohio State University · Department of Statistics and Data Science, University of Texas at San Antonio
VERDI:Verified Contradictions for Continual World-Model Optimization
Aug 10, 2026Junyu Wu, Shiqin Nie, Youyi Kou +10World Model Learning World Models
* Equal contribution
TLDChoiceNet: Quantitatively Choosing a Transfer Learning Dataset
Aug 10, 2026Jing Ning, James D. BrazaFine-Tuning Transfer Learning
Department of Computer Science, Stanford University

Aug 26, 2026J/K move · Enter open · S save

Cross-simulator transfer with foundation model summaries: Towards robust SKA-era reionization inference

Yannic Pietschke, Caroline Heneka, Ayodele Ore, Romain Meriot

Institut für Theoretische Physik, Universität Heidelberg, Philosophenweg 16, 69120 Heidelberg, Germany · Max Planck Institute for Radioastronomy, Auf dem Hügel 69, Bonn, 53121, Germany

Simulation-Based Inference Transfer Learning Self-Supervised Pre-Training Vision Transformer

Open paper AskPDF arXiv

Simulation-based inference (SBI) for parameter estimation is vulnerable to model misspecification: neural summaries and density estimators trained on a specific forward model typically fail when applied to data drawn from another model, or from real observations, and no training simulator can capture the full observational pipeline of a real measurement exactly. We show that a self-supervised Vision Transformer (ViT), pretrained label-free on a fast approximate simulator, produces transferable data summaries that generalize across simulators. Without retraining, it can be reused as a frozen encoder to infer astrophysical parameters from a completely different simulator that resolves the radiative transfer explicitly, on which it has never seen either data or parameters. As a concrete use case in 21cm cosmology, SKATR, a ViT pretrained with a Joint Embedding Predictive Architecture (JEPA), serves as a foundation model for reionization inference from upcoming SKA measurements: SKATR is pretrained once on 67k low-cost, noiseless semi-numerical 21cmFAST lightcones, then frozen and applied to hydrodynamical Loreli II lightcones, where a lightweight conditional flow matching head infers five astrophysical parameters; the encoder is never shown Loreli data, its parameters, or any noise. In our comparison, SKATR yields the most precise and best-calibrated posteriors across all five parameters, matching the accuracy of the fully-supervised in-domain baseline while requiring 2.6x fewer radiative-transfer simulations. Under realistic SKA AA* noise, only SKATR remains simultaneously accurate, informative, and calibrated, outperforming even a supervised baseline retrained from scratch on noisy data. Self-supervised pretraining on computationally efficient semi-numerical simulations is therefore a viable route to calibrated, simulator- and noise-agnostic reionization inference for the SKA-era.