Multi Task Fine Tuning

Multi-task fine-tuning (MFT) enhances the performance of pre-trained models, like large language models (LLMs) and reinforcement learning agents, by training them on multiple related tasks simultaneously. Current research focuses on optimizing MFT strategies, exploring efficient algorithms like LoRA and MAML, and investigating the impact of task selection and data composition on model generalization. This approach offers significant advantages, including improved sample efficiency, enhanced performance on downstream tasks, and the ability to leverage existing knowledge for faster adaptation to new domains, impacting fields from robotics to natural language processing.

Papers

October 7, 2024

Active Fine-Tuning of Generalist Policies
Marco Bagatella, Jonas Hübotter, Georg Martius, Andreas Krause
Imitation Learning Fast Adaptation Policy Generalization Multi Task Policy Active Finetuning Multi Task Fine Tuning Policy Fine Tuning

October 1, 2024

Mixing It Up: The Cocktail Effect of Multi-Task Fine-Tuning on LLM Performance -- A Case Study in Finance
Meni Brief, Oded Ovadia, Gil Shenderovitz, Noga Ben Yoash, Rachel Lemberg, Eitam Sheetrit
Large Language Model Case Study LLM Performance Domain Specific Large Language Model Multi Task Fine Tuning

August 22, 2024

Improving Factuality in Large Language Models via Decoding-Time Hallucinatory and Truthful Comparators
Dingkang Yang, Dongling Xiao, Jinjie Wei, Mingcheng Li, Zhaoyu Chen, Ke Li, Lihua Zhang
Semantic Representation Factual Claim Multi Task Fine Tuning Decoding Time

May 19, 2024

MAML-en-LLM: Model Agnostic Meta-Training of LLMs for Improved In-Context Learning
Sanchit Sinha, Yuguang Yue, Victor Soto, Mayank Kulkarni, Jianhua Lu, Aidong Zhang
Large Language Model Context Learning Meta Training LLM Robustness Multi Task Fine Tuning

January 22, 2024

A Framework to Implement 1+N Multi-task Fine-tuning Pattern in LLMs Using the CGC-LORA Algorithm
Chao Song, Zhihao Ye, Qiqiang Lin, Qiuying Peng, Jun Wang
Large Language Model Natural Language Processing New Framework Multi Task Fine Tuning

November 4, 2023

MFTCoder: Boosting Code LLMs with Multitask Fine-Tuning
Bingchang Liu, Chaoyu Chen, Cong Liao, Zi Gong, Huan Wang, Zhichao Lei, Ming Liang, Dajun Chen, Min Shen, Hailian Zhou, Hang Yu, Jianguo Li
Fine Tuning Multi Task Learning Code LLM Multitask Finetuning Multi Task Fine Tuning

October 15, 2023

MERTech: Instrument Playing Technique Detection Using Self-Supervised Pretrained Model With Multi-Task Finetuning
Dichucheng Li, Yinghao Ma, Weixing Wei, Qiuqiang Kong, Yulun Wu, Mingjin Che, Fan Xia, Emmanouil Benetos, Wei Li
Fine Tuning Multi Task Audio Data Self Supervised Pre Trained Model Multi Task Fine Tuning Onset Detection Playing Technique

October 4, 2023

Human-oriented Representation Learning for Robotic Manipulation
Mingxiao Huo, Mingyu Ding, Chenfeng Xu, Thomas Tian, Xinghao Zhu, Yao Mu, Lingfeng Sun, Masayoshi Tomizuka, Wei Zhan
Pre Trained Robotic Manipulation Interpretable Representation Generalizable Representation Task Specific Decoder Multi Task Fine Tuning

November 22, 2022

HyperTuning: Toward Adapting Large Language Models without Back-propagation
Jason Phang, Yi Mao, Pengcheng He, Weizhu Chen
Large Language Model Back Propagation Parameter Efficient Fine Tuning Language Task Hyper Tune Multi Task Fine Tuning

October 19, 2022

On the Feasibility of Cross-Task Transfer with Model-Based Reinforcement Learning
Yifan Xu, Nicklas Hansen, Zirui Wang, Yung-Chieh Chan, Hao Su, Zhuowen Tu
Reinforcement Learning Model Based Reinforcement Learning Task Feasibility Task Transfer Multi Task Fine Tuning

Multi Task Fine Tuning

Papers

Active Fine-Tuning of Generalist Policies

Mixing It Up: The Cocktail Effect of Multi-Task Fine-Tuning on LLM Performance -- A Case Study in Finance

Improving Factuality in Large Language Models via Decoding-Time Hallucinatory and Truthful Comparators

MAML-en-LLM: Model Agnostic Meta-Training of LLMs for Improved In-Context Learning

A Framework to Implement 1+N Multi-task Fine-tuning Pattern in LLMs Using the CGC-LORA Algorithm

MFTCoder: Boosting Code LLMs with Multitask Fine-Tuning

MERTech: Instrument Playing Technique Detection Using Self-Supervised Pretrained Model With Multi-Task Finetuning

Human-oriented Representation Learning for Robotic Manipulation

HyperTuning: Toward Adapting Large Language Models without Back-propagation

On the Feasibility of Cross-Task Transfer with Model-Based Reinforcement Learning