Multi-Task Optimization

Follow this topic Ask about this topic

Multi-task optimization studies how to improve several related tasks or objectives together, from training one model across datasets to searching for solutions across related optimization problems. Common approaches share representations or task information, balance losses, and use gradient surgery to reduce interference; multi-objective methods also use scalarization and Pareto-front search to expose trade-offs. Bayesian and evolutionary methods transfer information between tasks, sometimes using task graphs or learned task relationships. A central tension is that sharing can save data and computation but cause negative transfer when tasks differ, while conflicting objectives and mismatched loss scales make reliable balancing difficult.

Latest papers

49 papers7 papers in the last four weeks, level with the four weeks before. 0.0% of all new papers.

CRLoMTL: Conflict-Resolution through Low-rank matrices for Multi-Task Learning
Oct 26, 2026Ahmed Agiza, Tasneem Shaffee, Rahil Badkul +1Multi-Task Learning Multi-Task Optimization
School of Engineering, Brown University, Providence, RI, USA
Optimal Multitask Representations Under a Rate Constraint: An Information Bottleneck Analysis
Oct 21, 2026Nicholas Stiles, Gregory W. WornellRepresentation Learning Information Bottleneck
Massachusetts Institute of Technology
Hierarchical Interpretable Multitask Audio Representation Learning: An Analysis of Event-Scene Relationships through the Information Bottleneck
Oct 15, 2026Claudia Montero-Ramírez, Alba Martínez-Serrano, Carmen Peláez-MorenoAudio Classification Multi-Task Learning
First Mathematical Runtime Analyses of Evolutionary Multitask Multi-Objective Optimization
Oct 15, 2026Danyang Zhang, Weijie ZhengEvolutionary Multitask Optimization Multi-Objective Evolutionary Optimization
School of Computer Science and Technology, State Key Laboratory of Smart Farm Technologies and Systems, International Research Institute for Artificial Intelligence, Harbin Institute of Technology, Shenzhen, China · Pengcheng Laboratory, Shenzhen, China
Geometry-Aware Multi-Objective Optimization: Norm-Constrained Oracles with Entropic Weight Updates
Oct 9, 2026El Mahdi Chayti, Alireza Mirrokni, Martin JaggiLMO Methods Multi-Objective Optimization
Machine Learning and Optimization Laboratory (MLO), EPFL
CRT-HMAR: Causal Requirement Tracing-Guided Hierarchical Multi-Agent Regulation for Open-Task-Aware Infrared-Visible Image Fusion
Oct 7, 2026Zengyi Yang, Shuai Yuan, Zhong-Cheng Wu +3Infrared-Visible Image Fusion Multi-Task Optimization
School of Instrument Science and Opto-electronics Engineering, Hefei University of Technology, Hefei 230009, China · Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming 650500, China
A Broader Look at Model Merging: Rethinking Implicit Regularization Induced by Task Arithmetic
Oct 6, 2026Sin-Han Yang, Shih-Cheng Huang, Chieh-Yen Lin +3Task Vector Merging Multi-Task Optimization
Appier AI Research · National Taiwan University
Multi-Task Evolution for Zero-Shot Cross-Problem Generalization using LLMs
Oct 2, 2026Zhuoliang Xie, Changliang Zhou, Genghui Li +1Automated Heuristic Design Large Language Model-Guided Optimization
School of Automation and Intelligent Manufacturing, Southern University of Science and Technology, Shenzhen, China · Guangdong Provincial Key Laboratory of Fully Actuated System Control Theory and Technology, Southern University of Science and Technology, Shenzhen, China · College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, China
Gradient Surgery for Physics-Informed Neural Networks
Sep 25, 2026Thomas Borsani, Giuseppe Di FattaPhysics-Informed ML Gradient Interference
Faculty of Engineering Free University of Bozen - Bolzano Bozen - Bolzano, Italy
In-Context Guidance: Learning Inter-Task Synergies via Numerical Foundational Models for Few-Shot Multitask Optimization
Sep 22, 2026Tingyang Wei, Haofeng Wu, Jiao Liu +3Evolutionary Multitask Optimization Bayesian Optimization
College of Computing and Data Science, Nanyang Technological University, Singapore · Institute of Advanced Intelligence and Computing, Agency for Science, Technology and Research, Singapore · Singapore Institute of Manufacturing Technology (SIMTech), Agency for Science, Technology and Research, Singapore +1
On the Potential of Multi-Task Learning in Predictive Process Monitoring
Sep 11, 2026Lukas Kirchdorfer, Keyvan Amiri Elyasi, Heiner StuckenschmidtPredictive Process Monitoring Multi-Task Learning
University of Mannheim
SIMS: Scale-Invariant Merit-Function-Based Scalarization for Multi-Task Learning
Sep 11, 2026Zebin Chen, Fei Xing, Yang Chen +4Multi-Task Learning Multi-Task Optimization
Southern University of Science and Technology Shenzhen, China · City University of Hong Kong Hong Kong SAR, China · Shanxi University Taiyuan, China
Solving Few-Shot Multiobjective Multitask Optimization via Iterative Sequential Transfer
Sep 10, 2026Tingyang Wei, Haofeng Wu, Ananda Phan Iman +3Multi-Task Optimization Multi-Objective Optimization
College of Computing and Data Science Nanyang Technological University Singapore · Department of AI Convergence Gwangju Inst. of Sci. & Tech. (GIST) South Korea · Centre for Frontier AI Research (CFAR) A*STAR Singapore
Towards Efficient Evaluation of Evolutionary Transfer Optimization: Case Studies on Task-Parameterized Applications
Sep 4, 2026Yanchen Li, Xiaoming Xue, Kay Chen TanEvolutionary Optimization Multi-Task Optimization
Department of Data Science and Artificial Intelligence, The Hong Kong Polytechnic University, Hong Kong SAR, China · School of Petroleum Engineering, China University of Petroleum (East China), Qingdao, China
An Exploratory Study of Frequency-Aware Task Weighting for YOLOv8-Based Unified Driving Perception
Aug 31, 2026Zhiyuan Nie, Zixi Zhou, Xianbin GuAdaptive Loss Weighting Multi-Task Optimization
NYU Shanghai
Decentralized Multitask Learning over Learned Task Graphs
Aug 27, 2026Zirui Wan, Stefan VlaskiGraph Structure Learning Multi-Task Learning
Department of Electrical and Electronic Engineering, Imperial College London, UK
Bounded Precision-Geometry Scaling for Robust Multi-Task Learning under Loss Scale Mismatch
Aug 21, 2026Krishna SubediMulti-Task Learning Adaptive Loss Weighting
Neryva
Open-MOPD: Diagnosing and Fixing Capability Imbalance in Multi-Teacher On-Policy Distillation
Aug 19, 2026Huan-ang Gao, Haohan Chi, Yong Yan +7Multi-Teacher Knowledge Distillation On-Policy Distillation
SIA-Lab of Tsinghua AIR and ByteDance Seed · Institute for AI Industry Research (AIR), Tsinghua University · Department of Computer Science and Technology, Tsinghua University
MOON: Multi-Objective OrthoNormalized Updates for Multitask Learning
Aug 12, 2026Shiji Zhou, Kunlin Lyu, Lei Zhang +2Neural Network Optimization Multi-Task Learning
Institute of Artificial Intelligence, Beihang University · Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing, Beihang University · Beijing Academy of Artificial Intelligence (BAAI) +1
Beyond Full-Model Rollback: AuroSFT for Adapter-State Multi-Task Fine-Tuning
Aug 5, 2026Yue Han, Ziniu LiuSupervised Fine-Tuning Multi-Task Learning
College of Systems Engineering, National University of Defense Technology · College of Computer Science and Technology, National University of Defense Technology
Hierarchical Wireless Foundation Model for Multi-Task Optimization
Jul 18, 2026Yangjing Wang, Ouya Wang, Shenglong Zhou +1Wireless Foundation Models Hierarchical Representation Learning
Department of Electrical and Electronic Engineering, Faculty of Engineering, Imperial College London, SW7 2BX London, U.K. · School of Mathematics and Statistics, Beijing Jiaotong University, China
Capacity and Redundancy Trade-offs in Multi-Task Learning
Jul 17, 2026Asif KhanMulti-Task Learning Multi-Task Optimization
1Harvard Medical School, Boston, USA
Pitfalls and Remedies for Multi-Task Bayesian Optimization
Jul 10, 2026Carl Hvarfner, Sam Daulton, Max Balandat +1Bayesian Optimization Hyperparameter Transfer
Meta
The Double Dilemma in Multi-Task Radiology Report Generation: A Gradient Dynamics Analysis and Solution
May 21, 2026Erjian Zhang, Yatong Hao, Liejun Wang +1Radiology Report Generation Gradient Interference
School of Computer Science and Technology, Xinjiang University, Urumqi, China · Xinjiang Multimodal Intelligent Processing and Information Security Engineering Technology Research Center, Urumqi, China
optimize_anything: A Universal API for Optimizing any Text Parameter
May 19, 2026Lakshya A Agrawal, Donghyun Lee, Shangyin Tan +11Black-Box Optimization Large Language Model-Guided Optimization
UC Berkeley USA · MIT USA
DiffusionOPD: A Unified Perspective of On-Policy Distillation in Diffusion Models
May 14, 2026Quanhao Li, Junqiu Yu, Kaixun Jiang +7Diffusion Model Distillation Multi-Task RL
Fudan University, China · Alibaba Group, China
PEML: Parameter-efficient Multi-Task Learning with Optimized Continuous Prompts
May 13, 2026Anjir Ahmed Chowdhury, Syed Zawad, Xiaolong Ma +2LLM Fine-Tuning Multi-Task Learning
Department of Computer Science University of Houston · IBM Research · Argonne National Laboratory
TOPPO: Rethinking PPO for Multi-Task Reinforcement Learning with Critic Balancing
May 12, 2026Yuanpeng Li, Gefei Lin, Annie Qu +1Multi-Task RL Proximal Policy Optimization
UC Irvine · George Washington University · UC Santa Barbara +1
A Tale of Two Problems: Multi-Task Bilevel Learning Meets Equality Constrained Multi-Objective Optimization
May 9, 2026Zhiyao Zhang, Myeung Suk Oh, Zhen Qin +3Bilevel Optimization Multi-Task Optimization
Department of Electrical and Computer Engineering, The Ohio State University, Columbus, OH, USA · Meta Platforms, Inc., Menlo Park, CA, USA.
CoAction: Cross-task Correlation-aware Pareto Set Learning
May 3, 2026Xinyue Chen, Yingxuan Liang, Yiqin Huang +3Multi-Task Learning Multi-Task Optimization
Guangdong University of Technology, Guangzhou, China · Xiamen University, Xiamen, China

Oct 26, 2026J/K move · Enter open · S save

CRLoMTL: Conflict-Resolution through Low-rank matrices for Multi-Task Learning

Ahmed Agiza, Tasneem Shaffee, Rahil Badkul, Sherief Reda

School of Engineering, Brown University, Providence, RI, USA

Multi-Task Learning Multi-Task Optimization Task Interference Low-Rank Adaptation

Open paper AskPDF arXiv

Multi-task learning (MTL) improves model efficiency by sharing representations across related vision tasks, but it often suffers from negative transfer when task updates interfere with one another. Existing conflict-mitigation methods typically operate in gradient space, requiring explicit task-gradient manipulation and additional optimization steps. We propose Conflict-Resolution through Low-Rank Matrices for Multi-Task Learning (CRLoMTL), a parameter-efficient framework that mitigates task interference in the low-rank parameter-update space. CRLoMTL learns task-specific LoRA updates through task-specific passes, where each task updates its own low-rank matrices, and integrates these updates through global shared-subspace fusion. Specifically, we introduce a shared-subspace resolver based on singular value decomposition (SVD), referred to as the SVD resolver, which extracts a common low-dimensional orthonormal basis across task adapters and projects task updates into this shared space before merging them into a single deployment-time update. The resulting update can be folded into the backbone, preserving the inference structure of the original model without additional runtime branches. Experiments with Swin Transformer backbones on PASCAL-Context, NYUD-v2, and CelebA show that CRLoMTL achieves a favorable accuracy-parameter trade-off, improving over single-task baselines by up to 2.77% on PASCAL-Context, 5.03% on NYUD-v2, and 3.50% on CelebA, while training only a small fraction of the parameters used by single-task models.