Multi Task Learning

Multi-task learning (MTL) aims to improve model efficiency and generalization by training a single model to perform multiple related tasks simultaneously. Current research focuses on addressing challenges like task interference and optimization difficulties, exploring architectures such as Mixture-of-Experts (MoE), low-rank adaptors, and hierarchical models to enhance performance and efficiency across diverse tasks. MTL's significance lies in its potential to improve resource utilization and create more robust and adaptable AI systems, with applications spanning various fields including natural language processing, computer vision, and scientific modeling.

Papers

November 3, 2023

Multi-task Learning for Optical Coherence Tomography Angiography (OCTA) Vessel Segmentation
Can Koz, Onat Dalmaz, Mertay Dayanc
Multi Task Learning Multi Task Segmentation Task Optical Coherence Tomography Vessel Segmentation

November 1, 2023

Learning to optimize by multi-gradient for multi-objective optimization
Linxi Yang, Xinmin Yang, Liping Tang
LeArning Abstract Gradient Descent Multi Task Learning Multi Objective Optimization Superior Optimizer

October 31, 2023

Dynamically Updating Event Representations for Temporal Relation Classification with Multi-category Learning
Fei Cheng, Masayuki Asahara, Ichiro Kobayashi, Sadao Kurohashi
Multi Task Learning Temporal Relation Event Representation Multi Category Temporal Layer

October 28, 2023

Episodic Multi-Task Learning with Heterogeneous Neural Processes
Jiayi Shen, Xiantong Zhen, Qi, Wang, Marcel Worring
Multi Task Learning Multi Task Learning Framework Episodic Training Neuronal Heterogeneity Free Task

October 27, 2023

Towards a Unified Conversational Recommendation System: Multi-task Learning via Contextualized Knowledge Distillation
Yeongseo Jung, Eunseo Jung, Lei Chen
Multi Task Learning Conversational Recommender System Conversational Capability Natural Language Conversation

October 25, 2023

Subspace Chronicles: How Linguistic Information Emerges, Shifts and Interacts during Language Model Training
Max Müller-Eberstein, Rob van der Goot, Barbara Plank, Ivan Titov
Language Model Multi Task Learning Representation Space Interaction Generation Linguistic Information Two Shift Cross Task Subspace Prototype Syntactic Knowledge

October 24, 2023

Task Grouping for Automated Multi-Task Machine Learning via Task Affinity Prediction
Afiya Ayman, Ayan Mukhopadhyay, Aron Laszka
Multi Task Learning Task Specific Feature Single Task Learning Task Affinity Task Grouping

October 21, 2023

When MOE Meets LLMs: Parameter Efficient Fine-tuning for Multi-task Medical Applications
Qidong Liu, Xian Wu, Xiangyu Zhao, Yuanshao Zhu, Derong Xu, Feng Tian, Yefeng Zheng
Large Language Model Multi Task Learning Mixture of Expert Efficient Fine Tuning Multi Task Medical

October 20, 2023

October 19, 2023

LeTFuser: Light-weight End-to-end Transformer-Based Sensor Fusion for Autonomous Driving with Multi-Task Learning
Pedram Agand, Mohammad Mahdavian, Manolis Savva, Mo Chen
Vision Transformer Autonomous Driving Multi Task Learning Imitation Learning Perception Module RGB D Image Transformer Based Sensor Fusion

October 16, 2023

Interpreting and Exploiting Functional Specialization in Multi-Head Attention under Multi-task Learning
Chong Li, Shaonan Wang, Yunhao Zhang, Jiajun Zhang, Chengqing Zong
Multi Task Learning Human Attention Multi Head Attention Machine Translated Multi Task Training Task Specialization

October 13, 2023

October 11, 2023

Denoising Task Routing for Diffusion Models
Byeongjun Park, Sangmin Woo, Hyojun Go, Jin-Young Kim, Changick Kim
Diffusion Model Multi Task Learning Diffusion Training Denoising Network Task Specific Channel

October 8, 2023

Boosting Facial Action Unit Detection Through Jointly Learning Facial Landmark Detection and Domain Separation and Reconstruction
Ziqiao Shang, Li Yu
Multi Task Learning Full State Reconstruction Domain Discriminator Facial Landmark Detection Facial Action Unit Detection Unlabeled Face Facial Feature

October 4, 2023

AdaMerging: Adaptive Model Merging for Multi-Task Learning
Enneng Yang, Zhenyi Wang, Li Shen, Shiwei Liu, Guibing Guo, Xingwei Wang, Dacheng Tao
Multi Task Learning Model Merging

October 3, 2023

Inductive biases of multi-task learning and finetuning: multiple regimes of feature reuse
Samuel Lippl, Jack W. Lindsey
Deep Neural Network Fine Tuning Multi Task Learning Inductive Bias Implicit Regularization Chaotic Regime Feature Reuse

October 2, 2023

Multi Task Learning

Papers

Multi-task Learning for Optical Coherence Tomography Angiography (OCTA) Vessel Segmentation

Learning to optimize by multi-gradient for multi-objective optimization

Dynamically Updating Event Representations for Temporal Relation Classification with Multi-category Learning

Episodic Multi-Task Learning with Heterogeneous Neural Processes

Towards a Unified Conversational Recommendation System: Multi-task Learning via Contextualized Knowledge Distillation

Subspace Chronicles: How Linguistic Information Emerges, Shifts and Interacts during Language Model Training

Task Grouping for Automated Multi-Task Machine Learning via Task Affinity Prediction

When MOE Meets LLMs: Parameter Efficient Fine-tuning for Multi-task Medical Applications

FairBranch: Mitigating Bias Transfer in Fair Multi-task Learning

Provable Benefits of Multi-task RL under Non-Markovian Decision Making Processes

LeTFuser: Light-weight End-to-end Transformer-Based Sensor Fusion for Autonomous Driving with Multi-Task Learning

Interpreting and Exploiting Functional Specialization in Multi-Head Attention under Multi-task Learning

Disentangled Latent Spaces Facilitate Data-Driven Auxiliary Learning

Scalarization for Multi-Task and Multi-Domain Learning at Scale

Denoising Task Routing for Diffusion Models

Boosting Facial Action Unit Detection Through Jointly Learning Facial Landmark Detection and Domain Separation and Reconstruction

AdaMerging: Adaptive Model Merging for Multi-Task Learning

Inductive biases of multi-task learning and finetuning: multiple regimes of feature reuse

ScaLearn: Simple and Highly Parameter-Efficient Task Transfer by Learning to Scale

Multi-task Learning with 3D-Aware Regularization