cs.CLNov 7, 2026

Adapting Large Language Models via Multi-Task Learning for Mental Health Problem, Emotion, and Sentiment Detection

Authors: Amir Rafiq Bhat, Rahim Dehkharghani, Zeki Erdem

Organizations: Computer Engineering Department, Faculty of Engineering and Natural Sciences, Kadir Has University, Istanbul, Turkey · Computer Science Department, College of Innovation and Technology, University of Michigan-Flint, Flint, MI, USA

Abstract

This study investigates sequential multi-task learning using Parameter-Efficient Fine-Tuning (PEFT), LoRA, and 4-bit quantization on Llama 3.1 8B Instruct for mental health problem detection, emotion detection, and sentiment analysis. Using 4,243 English social media posts, it compares two task sequences and demonstrates that task ordering significantly affects knowledge transfer and task interference. The MHP-first sequence achieves the best mental health problem detection (Micro-F1: 0.7624), while the Emotion-first sequence performs better in emotion (F1: 0.6052) and sentiment analysis (F1: 0.9459). A Joint-MTL baseline further highlights the challenges of simultaneous learning, including task interference and generative format collapse. Overall, the findings demonstrate the effectiveness of sequential PEFT for mental health NLP while showing that task order is critical to balancing performance across related tasks. The integration of LoRA and 4-bit quantization also reduces computational and memory requirements, supporting the adaptation of large language models in resource-constrained environments.

Explore similar work

CardsList
  1. From Isolation to Alignment: Unified LoRA for Efficient Multi-Task Learning

    Aug 7, 2025Jinda Liu, Yi Chang, Yuan WuMulti-Task LearningLow-Rank Adaptation

  2. PEML: Parameter-efficient Multi-Task Learning with Optimized Continuous Prompts

    May 13, 2026Anjir Ahmed Chowdhury, Syed Zawad, Xiaolong Ma +2LLM Fine-TuningMulti-Task Learning

  3. The Parts Are Greater Than the Sum: Automated Task Sequencing for Efficient Training of Multi-Policy LLMs

    Jul 31, 2026Jiajia Tang, Sizhe Yuen, Francisco Gomez Medina +2Continual Learning for LLMsLow-Rank Adaptation