Multi-Task Learning

Follow this topic Ask about this topic

Multi-task learning studies how one model can learn several related predictions at once, using shared information to improve accuracy, data efficiency, or deployment cost. Common designs combine a shared encoder with task-specific heads, adapters, or routed experts; other methods balance losses, adjust conflicting gradients, or group tasks by affinity. Applications range from dense vision benchmarks such as NYUv2 and Cityscapes to language, medical prediction, recommendation, and scientific modeling. The central tension is positive versus negative transfer: sharing can help when tasks have useful common structure, but differences in data, objectives, or capacity can cause gains for some tasks and losses for others.

Latest papers

389 papers50 papers in the last four weeks, up 16% on the four weeks before. 0.3% of all new papers.

Scalable Multi-Task Low-Rank Model Adaptation
Mar 2, 2026Zichen Tian, Antoine Ledent, Qianru SunLow-Rank Adaptation Multi-Task Learning
Singapore Management University
Adaptive Bidirectional Task Interaction for Joint Segmentation and Classification of Breast Ultrasound
Mar 1, 2026Abdullah Al Shafi, Md Kawsar Mahmud Khan Zunayed, Safin Ahmmed +2Ultrasound Image Segmentation Image Segmentation
Khulna University of Engineering and Technology, Bangladesh · University Clermont Auvergne, France
Machine Learning Grade Prediction Using Students' Grades and Demographics
Feb 28, 2026Mwayi Sonkhanani, Symon Chibaya, Clement N. NyirendaEducational Data Mining Educational Assessment
Malawi University of Science and Technology, P.O.Box 5196, Limbe, Malawi
TAP-SLF: Parameter-Efficient Adaptation of Vision Foundation Models for Multi-Task Ultrasound Image Analysis
Feb 28, 2026Hui Wan, Libin LanVision Foundation Model Adaptation Medical Image Analysis
College of Computer Science and Engineering, Chongqing University of Technology, Chongqing, China
Multidimensional Task Learning: A Unified Tensor Framework for Computer Vision Tasks
Feb 26, 2026Alaa El Ichi, Khalide JbilouMulti-Task Learning
Université du Littoral Cote d’Opale, LMPA, 50 rue F. Buisson, 62228 Calais-Cedex, France.
UniWhisper: Efficient Continual Multi-task Training for Robust Universal Audio Representation
Feb 25, 2026Yuxuan Chen, Peize He, Haoyuan Yu +1Audio Representation Learning Audio Understanding
Jilin University · University of Electronic Science and Technology of China · Hunan University +1
Wireless Federated Multi-Task LLM Fine-Tuning via Sparse-and-Orthogonal LoRA
Feb 24, 2026Nuocheng Yang, Sihua Wang, Ouwen Huan +3Federated Fine-Tuning Federated Learning
Beijing Laboratory of Advanced Information Network, and the Beijing Key Laboratory of Network System Architecture and Convergence, Beijing University of Posts and Telecommunications, Beijing 100876, China · Department of Electrical and Computer Engineering and Institute for Data Science and Computing, University of Miami, Coral Gables, FL, 33146 USA · Information Systems Technology and Design Pillar, Singapore University of Technology and Design, 487372, Singapore
Uncertainty-Aware Delivery Delay Duration Prediction via Multi-Task Deep Learning
Feb 23, 2026Stefan Faulkner, Reza Zandehshahvar, Vahid Eghbal Akhlaghi +3Supply Chain Management Multi-Task Learning
NSF Artificial Intelligence Institute for Advances in Optimization, H. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, GA, USA · Kinaxis
Ensemble Prediction of Task Affinity for Efficient Multi-Task Learning
Feb 20, 2026Afiya Ayman, Ayan Mukhopadhyay, Aron LaszkaMulti-Task Learning Task Transferability Estimation
Pennsylvania State University · College of William & Mary
The Geometry of Multi-Task Grokking: Transverse Instability, Superposition, and Weight Decay Phase Structure
Feb 19, 2026Yongzhong XuGrokking Neural Network Training Dynamics
On the Power of Source Screening for Learning Shared Feature Extractors
Feb 18, 2026Leo Muxing Wang, Connor Mclaughlin, Lili SuShared Representation Learning Representation Learning
Northeastern University
UrbanVerse: Learning Urban Region Representation Across Cities and Tasks
Feb 17, 2026Fengze Sun, Egemen Tanin, Shanika Karunasekera +3Urban Representation Learning Geospatial Representation Learning
School of Computing and Information System, University of Melbourne, Melbourne, Australia · School of Computer Science and Engineering, University of New South Wales, Sydney, Australia
DeepMTL2R: A Library for Deep Multi-task Learning to Rank
Feb 16, 2026Chaosheng Dong, Peiyao Xiao, Yijia Wang +1Learning to Rank Multi-Task Learning
Amazon USA
MultiGraspNet: A Multitask 3D Vision Model for Multi-gripper Robotic Grasping
Feb 6, 2026Stephany Ortuno-Chanelo, Paolo Rabino, Enrico Civitelli +2Robotic Grasping Multi-Task Learning
VANDAL Laboratory, Department of Control and Computer Engineering, Politecnico di Torino, Turin, Italy. · Comau S.p.A., Advanced Automation Solutions, Grugliasco, Italy. · Istituto Italiano di Tecnologia, Genoa, Italy.
Multi-Task Learning for Metal Alloy Property Prediction: An Empirical Study of Negative Transfer and Mitigation Strategies
Dec 28, 2025Sungwoo KangMulti-Task Learning Materials Property Prediction
Department of Electrical and Computer Engineering, Korea University, Seoul 02841, Republic of Korea
Efficiently Learning Branching Networks for Multitask Algorithmic Reasoning
Nov 30, 2025Dongyue Li, Zhenshuo Zhang, Minxuan Duan +2Neural Algorithmic Reasoning Multi-Task Learning
†Northeastern University, Boston, Massachusetts · ‡University of Pennsylvania, Philadelphia, Pennsylvania
HENet++: Hybrid Encoding and Multi-task Learning for 3D Perception and End-to-end Autonomous Driving
Nov 10, 2025Zhongyu Xia, Zhiwei Lin, Yongtao Wang +1Autonomous Driving Perception End-to-End Autonomous Driving
Wangxuan Institute of Computer Technology, Peking University, Beijing, China. · University of California, Merced, USA.
Multi-task Neural Diffusion Processes
Oct 3, 2025Joseph Rawson, Domniki Ladopoulou, Petros DellaportasDiffusion Models Multi-Task Learning
Department of Statistical Science, University College London, Gower Street, London, WC1E 6BT, United Kingdom. · Department of Statistics, Athens University of Economics and Business, Patission 76, Athens, 104 34, Greece.
Graph Coloring for Multi-Task Learning
Sep 21, 2025Santosh Patapati, Ian NoronhaGradient Interference Multi-Task Learning
Stony Brook University AI Innovation Institute, Stony Brook, NY 11794, USA · Purdue University, West Lafayette, IN 47907, USA
Energy Injection Identification enabled Disaggregation with Deep Multi-Task Learning
Aug 20, 2025Xudong Wang, Guoming Tang, Junyu Xue +3Non-Intrusive Load Monitoring Multi-Task Learning
The Chinese University of Hong Kong, Shenzhen · The Hong Kong University of Science and Technology (Guangzhou) · Southern University of Science and Technology +1
From Isolation to Alignment: Unified LoRA for Efficient Multi-Task Learning
Aug 7, 2025Jinda Liu, Yi Chang, Yuan WuMulti-Task Learning Low-Rank Adaptation
School of Artificial Intelligence, Jilin University · 2Southern University of Science and Technology · 3International Center of Future Science, Jilin University +1
Calibrating Biophysical Models for Grape Phenology Prediction via Multi-Task Learning
Aug 5, 2025William Solow, Sandhya SaisubramanianMulti-Task Learning Model Calibration
Oregon State University
Learning Task Mixtures from Task Affinities: A Probabilistic Graphical Model for Supervised Fine-Tuning
Jul 16, 2025Prateek Chanda, Saral Sureka, Parth Pratim Chatterjee +3Supervised Fine-Tuning Multi-Task Learning
IIT Bombay · IBM Research · Red Hat AI Innovation +1
Macro Graph of Experts for Billion-Scale Multi-Task Recommendation
Jun 12, 2025Hongyu Yao, Zijin Hong, Hao Chen +6Graph Representation Learning Multi-Task Learning
The Hong Kong Polytechnic University Hong Kong SAR, China · National University of Defense Technology Changsha, China · Beihang University Beijing, China
Q-SiT: Teaching LMMs for Image Quality Scoring and Interpreting
Mar 12, 2025Zicheng Zhang, Haoning Wu, Ziheng Jia +2Unified Multimodal Models Multi-Task Learning
Institute of Image Communication and Network Engineering, Shanghai Jiao Tong University, China · S-Lab, Nanyang Technological University, Singapore · School of Computer Science and Engineering, Nanyang Technological University, Singapore
Hierarchical Sparse Bayesian Multitask Learning for Disease Prediction in Pooled Microbiome Studies
Feb 4, 2025Haonan Zhu, Andre R. Goncalves, Camilo Valdes +10Clinical Outcome Prediction Multi-Task Learning
Lawrence Livermore National Laboratory · Electrical and Computer Engineering, University of Michigan
Variational Model Merging for Pareto Front Estimation in Multitask Finetuning
Dec 11, 2024Hugo Monzón Maldonado, Nico Daheim, Thomas Möllenhoff +2Bayesian Inference Multi-Task Learning
RIKEN Center for AI Project, Japan · Ubiquitous Knowledge Processing Lab (UKP Lab), Department of Computer Science, Technical University of Darmstadt and National Research Center for Applied Cybersecurity ATHENE, Germany · National Research Center for Applied Cybersecurity ATHENE, Germany
Scale-specific auxiliary multi-task contrastive learning for deep liver vessel segmentation
Sep 18, 2024Amine Sadikine, Bogdan Badic, Jean-Pierre Tasu +4Medical Image Segmentation 3D Medical Image Segmentation
LaTIM UMR 1101, Inserm, Brest, France · University of Western Brittany, Brest, France · University Hospital of Brest, Brest, France +3
Taskonomy: Disentangling Task Transfer Learning
Apr 23, 2018Amir Zamir, Alexander Sax, William Shen +3Transfer Learning Task Transferability Estimation
Stanford University · University of California, Berkeley

Mar 2, 2026J/K move · Enter open · S save

Scalable Multi-Task Low-Rank Model Adaptation

Zichen Tian, Antoine Ledent, Qianru Sun

Singapore Management University

Low-Rank Adaptation Multi-Task Learning Parameter-Efficient Fine-Tuning

Open paper AskPDF arXiv

Scaling multi-task low-rank adaptation (LoRA) to a large number of tasks induces catastrophic performance degradation, such as an accuracy drop from 88.2% to 2.0% on DOTA when scaling from 5 to 15 tasks. This failure is due to parameter and representation misalignment. We find that existing solutions, like regularization and dynamic routing, fail at scale because they are constrained by a fundamental trade-off: strengthening regularization to reduce inter-task conflict inadvertently suppresses the essential feature discrimination required for effective routing. In this work, we identify two root causes for this trade-off. First, uniform regularization disrupts inter-task knowledge sharing: shared underlying knowledge concentrates in high-SV components (89% alignment on Flanv2->BBH). Uniform regularization forces high-SV components to update in orthogonal directions, directly disrupting the shared knowledge. Second, Conflict Amplification: Applying LoRA at the component-level (e.g., W_q, W_v) amplifies gradient conflicts; we show block-level adaptation reduces this conflict by 76% with only 50% parameters. Based on these insights, we propose mtLoRA, a scalable solution with three novel designs: 1) Spectral-Aware Regularization to selectively orthogonalize low-SV components while preserving high-SV shared knowledge, 2) Block-Level Adaptation to mitigate conflict amplification and largely improve parameter efficiency, and 3) Fine-Grained Routing using dimension-specific weights for superior expressive power. On four large-scale (15-25 tasks) vision (DOTA and iNat2018) and NLP (Dolly-15k and BBH) benchmarks, mtLoRA achieves 91.7%, 81.5%, 44.5% and 38.5% accuracy on DOTA, iNat2018, Dolly-15k and BBH respectively, outperforming the state-of-the-art by 2.3% on average while using 47% fewer parameters and 24% less training time.