From Baselines to Preferences: A Comparative Study of LoRA/QLoRA and Preference Optimization for Mental Health Text Classification

Mihael Arcan

cs.CLApr 1, 2026

From Baselines to Preferences: A Comparative Study of LoRA/QLoRA and Preference Optimization for Mental Health Text Classification

Authors: Mihael Arcan

Organizations: Home Lab, Galway, Ireland

Abstract

Mental health text classification has rapidly adopted modern adaptation methods, yet practical guidance on which optimization strategy to use, when, and why remains limited. This paper presents a systematic comparative study of optimization pathways for a joint mental-health classification task, moving from strong vanilla baselines to progressively more specialized techniques. We first establish classical and encoder references, then examine parameter-efficient supervised fine-tuning with LoRA/QLoRA under multiple objective and optimization settings, and finally evaluate preference-based optimization with DPO, ORPO, and KTO, including class-rebalanced training. Rather than emphasizing a single headline score, we focus on methodological insight: how performance changes with objective formulation, adapter choice, optimizer behavior, context windowing, and class-balance intervention. The results show that optimization effects are highly method-dependent: some approaches deliver stable, transferable gains, while others are sensitive to configuration and data balance. Preference optimization, in particular, exhibits large variation across objectives, indicating that method selection is more consequential than simply adding a preference-training stage. The central contribution is a clear optimization narrative for mental health NLP: start from transparent baselines, apply controlled tuning, and use preference optimization selectively where its gains are demonstrable. This provides a reproducible and practically grounded framework for choosing effective training strategies beyond architecture choice alone.

Figures & tables

Split	Text Length		Label Positives (count, %)
	Mean	Median	GAD1 $>0$	GAD2 $>0$	PHQ1 $>0$	PHQ2 $>0$	Anxiety-any / Depression-any
Train	1034.32	895	96 (67.61%)	47 (33.10%)	75 (52.82%)	83 (58.45%)	97 (68.31%) / 89 (62.68%)
Test	348.69	314	96 (67.61%)	47 (33.10%)	75 (52.82%)	83 (58.45%)	97 (68.31%) / 89 (62.68%)

Table 1: Combined train/test dataset statistics. Anxiety-any is defined as

(\mathrm{GAD\_1}>0)\vee(\mathrm{GAD\_2}>0)

and Depression-any as

(\mathrm{PHQ\_1}>0)\vee(\mathrm{PHQ\_2}>0)

Profile	Model Name	Window	F1 m	F1 w
Vanilla	xgboost	94	0.3106	0.5903
Vanilla	xgboost	128	0.3028	0.5875
Vanilla	xgboost	192	0.3097	0.5953
Vanilla	xgboost	256	0.3288	0.6312
Vanilla	xgboost	384	0.3110	0.6111
Vanilla	xgboost	512	0.3375	0.6480

Table 2: Macro- and weighted-F1 comparison of XGBoost and BERT-family models across context windows under vanilla and optimized (temperature-scaled) settings.

Profile	Model Name	Best F1	Mean F1
Vanilla	bert-base-uncased	0.3444	0.3367
Vanilla	distilbert-base-uncased	0.3333	0.2860
Vanilla	roberta-base	0.3406	0.3041
Optimized (Temp)	bert-base-uncased	0.3518	0.3386
Optimized (Temp)	distilbert-base-uncased	0.3311	0.2883
Optimized (Temp)	roberta-base	0.3286	0.2775

Table 3: BERT-family summary across all tested windows: per model/profile, Best F1 is the maximum macro-F1 over windows and Mean F1 is the average macro-F1 across windows; optimized denotes temperature-scaled evaluation.

Profile	Objective	Mode	Optimizer	N	Best F1	Mean F1
Vanilla	discriminative	lora	adamw	3	0.3031	0.2773
Vanilla	discriminative	lora	adafactor	3	0.2956	0.2729
Vanilla	discriminative	lora	adam8bit	3	0.2888	0.2738
Vanilla	discriminative	qlora	adamw	3	0.3022	0.2810
Vanilla	discriminative	qlora	adafactor	3	0.2908	0.2665
Vanilla	discriminative	qlora	adam8bit	3	0.2937	0.2752

Table 4: LoRA/QLoRA optimizer comparison across objectives and profiles. MeanF1 is averaged over runs with fixed (profile, objective, mode, optimizer) across included windows (and schemas).

Profile	Objective	N	F1	Mean F1 m	Mean F1 w
Vanilla	generative	45	0.3399	0.2877	0.5726
Vanilla	discriminative	18	0.3031	0.2744	0.5348
Optimized	generative	45	0.3339	0.2885	0.5734
Optimized	discriminative	18	0.3072	0.2741	0.5345

Table 5: Profile-level LoRA/QLoRA SFT summary by objective:

N

is the number of runs,

F1

is the best macro-F1 observed in each group, and Mean

F1_{m}

/Mean

F1_{w}

are macro- and weighted-F1 averages across runs.

Profile	Objective	Mode	Output Schema	Window	overlap	N	Mean F1	Best F1
Vanilla	discriminative	lora	label only	384	96	3	0.2884	0.2942
Vanilla	discriminative	lora	label only	512	128	3	0.2948	0.3031
Vanilla	discriminative	qlora	label only	384	96	3	0.2956	0.3022
Vanilla	discriminative	qlora	label only	512	128	3	0.2840	0.2939
Vanilla	generative	lora	label confidence	384	96	3	0.2255	0.3004
Vanilla	generative	lora	label confidence	512	128	3	0.2748	0.3200

Table 6: Cell-level SFT sensitivity analysis across profile, objective, adapter mode, output schema, and chunking setup (window/overlap):

N

denotes completed runs per cell, Mean F1 is the average macro-F1 across those runs, and Best F1 is the within-cell maximum.

Profile	Method	Mode	N	Best F1	Mean F1
Vanilla	dpo	lora	4	0.2474	0.1956
Vanilla	dpo	qlora	4	0.2174	0.1904
Vanilla	kto	lora	4	0.1200	0.1187
Vanilla	kto	qlora	4	0.1200	0.1187
Vanilla	orpo	lora	4	0.3181	0.2707
Vanilla	orpo	qlora	4	0.3173	0.2815

Table 7: Preference-optimization comparison (DPO, ORPO, KTO) by training profile and adapter mode;

N

is the number of runs per setting, with Best F1 and Mean F1 reported as maximum and average macro-F1, respectively.

Family	Profile	F1 m	F1 w	Prec m	Rec m
XGBoost	Vanilla	0.3375	0.6480	0.3816	0.3446
BERT	Vanilla	0.3444	0.6566	0.3446	0.3546
BERT	Optimized (Temp)	0.3518	0.6638	0.3393	0.3666
LoRA SFT	Vanilla	0.3323	0.6257	0.3155	0.3536
QLoRA SFT	Vanilla	0.3399	0.6454	0.3275	0.3547
LoRA SFT	Optimized	0.3339	0.6300	0.3172	0.3538

Table 8: Main cross-family comparison on the shared window set: each row reports the best available run within a method/profile group, with macro/weighted F1 and macro precision/recall shown for direct comparability across XGBoost, encoder baselines, SFT adapters, and preference-optimization methods.

Family	Profile	Window	F1 m	F1 w	P m	P w	R m	R w
XGBoost	Vanilla	512	0.7013	0.7519	0.8233	0.8030	0.6852	0.7803
BERT	Vanilla	384	0.7078	0.7533	0.7380	0.7558	0.6957	0.7647
BERT	Optimized (Temp)	384	0.7199	0.7572	0.7253	0.7556	0.7157	0.7598
LoRA SFT	Vanilla	512	0.6952	0.7219	0.6913	0.7328	0.7058	0.7168
QLoRA SFT	Vanilla	512	0.7016	0.7368	0.7072	0.7350	0.6975	0.7399
LoRA SFT	Optimized	512	0.6983	0.7265	0.6944	0.7335	0.7058	0.7225

Table 9: Secondary anxiety-task results derived from joint predictions, reporting macro/weighted F1, precision, and recall for the best selected run per family/profile (with its corresponding window).

Family	Profile	Window	F1 m	F1 w	P m	P w	R m	R w
XGBoost	Vanilla	512	0.6446	0.6841	0.7734	0.7562	0.6480	0.7225
BERT	Vanilla	384	0.6828	0.7129	0.7300	0.7302	0.6761	0.7304
BERT	Optimized (Temp)	384	0.7043	0.7278	0.7219	0.7301	0.6979	0.7353
LoRA SFT	Vanilla	512	0.6660	0.6834	0.6649	0.6850	0.6675	0.6821
QLoRA SFT	Vanilla	512	0.6862	0.7094	0.7010	0.7107	0.6810	0.7168
LoRA SFT	Optimized	512	0.6693	0.6879	0.6693	0.6879	0.6693	0.6879

Table 10: Secondary depression-task results derived from joint predictions, showing macro/weighted F1, precision, and recall for the best selected run per family/profile (with the corresponding window).

Explore similar work

Jun 11, 2026

Direct Preference Optimization for Chatbot Fine-Tuning: An Empirical Study

We present an approach to fine-tuning large language models using Direct Preference Optimization (DPO), a reinforcement learning technique. Our experimental results demonstrate that DPO simplifies the training pipeline, improves computational efficiency, and achieves competitive performance. The evaluation using BLEU, ROUGE, and cosine similarity metrics indicates effective learning and convergence, though further investigation is needed to address observed training instability.

Dezhi Yu, Yvonne Qiu, ShuoJia Fu

LLM Alignment Direct Preference Optimization LLM Fine-Tuning LLM Post-Training

Ask PDF

Feb 17, 2026

Multi-Objective Alignment of Language Models for Personalized Psychotherapy

Mental health disorders affect over 1 billion people worldwide, yet access to care remains limited by workforce shortages and cost constraints. While AI systems show therapeutic promise, current alignment approaches optimize objectives independently, failing to balance patient preferences with clinical safety. We survey 335 individuals with lived mental health experience to collect preference rankings across therapeutic dimensions, then develop a multi-objective alignment framework using direct preference optimization. We train reward models for six criteria -- empathy, safety, active listening, self-motivated change, trust/rapport, and patient autonomy -- and systematically compare multi-objective approaches against single-objective optimization, supervised fine-tuning, and parameter merging. Multi-objective DPO (MODPO) achieves superior balance (77.6% empathy, 62.6% safety) compared to single-objective optimization (93.6% empathy, 47.8% safety), and therapeutic criteria outperform general communication principles by 17.2%. Blinded clinician evaluation confirms MODPO is consistently preferred, with LLM-evaluator agreement comparable to inter-clinician reliability.

Mehrab Beikzadeh, Yasaman Asadollah Salmanpour, Ashima Suvarna +4

Department of Computer Science, University of California, Los Angeles, CA, USA · Department of Psychology, The University of Texas at Austin, Austin, TX, USA · Department of Psychiatry, NYU Grossman School of Medicine, New York, NY, USA

Personalized Language Model Alignment Multi-Objective Language Model Alignment Clinical Language Model Evaluation Direct Preference Optimization

Ask PDF

Apr 27, 2026

Intrinsic Mutual Information as a Modulator for Preference Optimization

Offline preference optimization methods, such as Direct Preference Optimization (DPO), offer significant advantages in aligning Large Language Models (LLMs) with human values. However, achieving optimal performance with these methods typically involves additional hyperparameter tuning, resulting in substantial time overhead. Although prior work has proposed a range of improvements, these methods remain limited in effectiveness and have not fully eliminated reliance on hyperparameter tuning. In this work, we propose RMiPO, a lightweight and efficient framework for offline preference optimization. RMiPO leverages intrinsic Response-level Mutual information for Preference Optimization with hyperparameter modulation, dynamically decoupling preference contributions at negligible additional computational cost. Extensive experimental results demonstrate that RMiPO achieves consistently superior performance over existing methods while reducing training overhead by more than 15%. Our code is available at https://github.com/liavonpenn/rmipo.

Peng Liao, Peijia Zheng, Lingbo Li +2

1Sun Yat-sen University · University of Warwick · 3Macao Polytechnic University

LLM Alignment Offline Preference Optimization LLM Training

Ask PDF