Backward Transfer

Backward transfer in machine learning refers to the improvement of a model's performance on previously learned tasks after training on new, related tasks. Current research focuses on mitigating negative backward transfer (where performance on older tasks degrades) and leveraging positive backward transfer to enhance overall learning efficiency. This is explored through various approaches, including prompt tuning, continual learning algorithms that manage forgetting, and the development of similarity metrics to guide the selection of beneficial transfer sources. Understanding and harnessing backward transfer is crucial for developing more efficient and robust AI systems capable of lifelong learning and adaptation in dynamic environments.

Papers

July 19, 2024

Quantifying the value of positive transfer: An experimental case study
Aidan J. Hughes, Giulia Delo, Jack Poole, Nikolaos Dervilis, Keith Worden
Case Study Net Present Value Structural Health Monitoring Information Transfer Population Based Structural Health Monitoring Backward Transfer Maintenance Decision

June 18, 2024

Mitigate Negative Transfer with Similarity Heuristic Lifelong Prompt Tuning
Chenyuan Wu, Gangwei Jiang, Defu Lian
Task Similarity Negative Transfer Sequence Task Backward Transfer

June 13, 2024

Data-dependent and Oracle Bounds on Forgetting in Continual Learning
Lior Friedman, Ron Meir
Continual LEArning Anti Forgetting Dependent Data Backward Transfer Oracle Inequality

February 3, 2024

Predicting positive transfer for improved low-resource speech recognition using acoustic pseudo-tokens
Nay San, Georgios Paraskevopoulos, Aryaman Arora, Xiluo He, Prabhjot Kaur, Oliver Adams, Dan Jurafsky
Automatic Speech Recognition Automatic Speech Recognition Performance Low Resource Speech Recognition Multilingual Speech Model Backward Transfer Acoustic Token

January 17, 2024

Beyond Anti-Forgetting: Multimodal Continual Instruction Tuning with Positive Forward Transfer
Junhao Zheng, Qianli Ma, Zhen Liu, Binquan Wu, Huawen Feng
Multimodal Large Language Model Catastrophic Forgetting Prompt Tuning Anti Forgetting Input Embeddings Backward Transfer Continual Instruction Tuning

November 2, 2023

Continual Learning Under Language Shift
Evangelia Gogoulou, Timothée Lesort, Magnus Boman, Joakim Nivre
Language Model Continual LEArning Pre Trained Language Model Monolingual Language Model Language Change Backward Transfer

March 14, 2023

Is forgetting less a good inductive bias for forward transfer?
Jiefeng Chen, Timothy Nguyen, Dilan Gorur, Arslan Chaudhry
Continual LEArning Catastrophic Forgetting Continual Learning Inductive Bias Continual Learning Algorithm Continual Learner Backward Transfer

November 30, 2022

General policy mapping: online continual reinforcement learning inspired on the insect brain
Angel Yanguas-Gil, Sandeep Madireddy
Reinforcement Learning Algorithm Efficient Reinforcement Learning Continual Reinforcement Learning Lifelong Reinforcement Learning Policy Generalization Insect Machine Hybrid System Backward Transfer

August 14, 2022

A Theory for Knowledge Transfer in Continual Learning
Diana Benavides-Prado, Patricia Riddle
Continual LEArning Theoretical Understanding Knowledge Transfer Continual Learner Backward Transfer Supervised Continual Learning

March 26, 2022

Quantum continual learning of quantum data realizing knowledge backward transfer
Haozhen Situ, Tianxiang Lu, Minghua Pan, Lvzhou Li
Continual LEArning Continual Learning Quantum Machine Learning Knowledge Based Quantum Physic Variational Quantum Classifier Backward Transfer

January 25, 2022

Dense Pixel-Labeling for Reverse-Transfer and Diagnostic Learning on Lung Ultrasound for COVID-19 and Pneumonia Detection
Gautam Rajendrakumar Gare, Andrew Schoenling, Vipin Philip, Hai V Tran, Bennett P deBoisblanc, Ricardo Luis Rodriguez, John Michael Galeotti
Segmentation Model Sparse Label Pre Trained Segmentation Model Pneumonia Detection Lung Ultrasound Dense Label Backward Transfer