Multi-Task LLM with LoRA Fine-Tuning for Automated Cancer Staging and Biomarker Extraction

Jiahao Shao; Anam Nawaz Khan; Christopher Brett; Tom Berg; Xueping Li; Bing Yao

cs.LGApr 14, 2026

Multi-Task LLM with LoRA Fine-Tuning for Automated Cancer Staging and Biomarker Extraction

Authors: Jiahao Shao, Anam Nawaz Khan, Christopher Brett, Tom Berg, Xueping Li, Bing Yao

Organizations: Department of Industrial & Systems Engineering, The University of Tennessee, Knoxville, TN 37996, USA · The University of Tennessee Medical Center, Knoxville, TN 37920, USA

Abstract

Pathology reports serve as the definitive record for breast cancer staging, yet their unstructured format impedes large-scale data curation. While Large Language Models (LLMs) offer semantic reasoning, their deployment is often limited by high computational costs and hallucination risks. This study introduces a parameter-efficient, multi-task framework for automating the extraction of Tumor-Node-Metastasis (TNM) staging, histologic grade, and biomarkers. We fine-tune a Llama-3-8B-Instruct encoder using Low-Rank Adaptation (LoRA) on a curated, expert-verified dataset of 10,677 reports. Unlike generative approaches, our architecture utilizes parallel classification heads to enforce consistent schema adherence. Experimental results demonstrate that the model achieves a Macro F1 score of 0.976, successfully resolving complex contextual ambiguities and heterogeneous reporting formats that challenge traditional extraction methods including rule-based natural language processing (NLP) pipelines, zero-shot LLMs, and single-task LLM baselines. The proposed adapter-efficient, multi-task architecture enables reliable, scalable pathology-derived cancer staging and biomarker profiling, with the potential to enhance clinical decision support and accelerate data-driven oncology research.

Figures & tables

Fig. 1 : Detailed architecture of the proposed Multi-Task LLM framework. (a) Data Preprocessing: Raw pathology reports undergo native digital text extraction, de-identification, and human-verified ground truth generation before tokenization. (b) Data Splitting: The dataset is partitioned into training (80%), validation (10%), and testing (10%) sets to support model development and unbiased evaluation. (c) Model Architecture: A shared Llama-3 encoder (fine-tuned via LoRA) extracts high-dimensional semantic features from the terminal hidden state, which are simultaneously projected into parallel task-specific classification heads (T, N, M, Grade, ER, PR, HER2). (d) Optimization Loop: The training process minimizes a joint cross-entropy loss, updating only the low-rank adapters and head weights while keeping the heavy LLM backbone frozen. (e) Evaluation: The system produces a structured output vector containing the predicted class for each clinical variable, providing a comprehensive TNM and biomarker profile.

Fig. 2 : Schematic of the LoRA mechanism.

Fig. 3 : Fishbone diagram of the experimental design and ablation framework.

Clinical Task	Naive Baseline			Multi-Task LLM
Clinical Task	Accuracy	F1	AUC	Accuracy	F1	AUC
T Stage	0.627	0.447	0.747	0.994	0.993	1.000
N Stage	0.911	0.739	0.933	1.000	1.000	1.000
Histologic Grade	0.543	0.498	0.752	0.969	0.969	0.995
ER Status	0.919	0.618	0.950	0.977	0.961	0.986
PR Status	0.937	0.635	0.964	0.962	0.957	0.990

TABLE I : Performance comparison on the held-out test set (

S_{test}=1,068

Method	Trainable Params	Time (s)	Memory (MB)	Accuracy	Macro F1	AUROC
Frozen (Head)	81,940	5,539	5,433	0.664	0.490	0.762
(IA) 3	606,228	9,198	12,553	0.673	0.512	0.756
Prefix Tuning	213,012	8,517	10,716	0.978	0.972	0.994
LoRA (Ours)	13,713,428	3,434	14,211	0.981	0.976	0.996

TABLE II : Comparison of classification performance and computational resource utilization across PEFT strategies.

	Resource Efficiency			Aggregate Performance			Task-specific Macro F1
Architecture	Models	Params	Time	Accuracy	Macro F1	AUROC	T	N	Grade	ER	PR	HER2
STL Baseline	6	82.2 M	3,473 s	0.928	0.914	0.946	0.991	1.000	0.970	0.919	0.900	0.309
MTL (Ours)	1	13.7 M	3,434 s	0.981	0.976	0.996	0.993	1.000	0.969	0.961	0.957	0.967

TABLE III : Comparative analysis of efficiency, aggregate performance, and task-specific Macro F1 scores

Configuration	Accuracy	Macro F1	AUROC
Full Text (No Slicing)	0.982	0.977	0.995
Random Window Baseline	0.854	0.790	0.921
Window Slicing ( $\pm 5$ lines)	0.981	0.976	0.996

TABLE IV : Ablation study on Input Robustness.

Fig. S1 : Detailed workflow of the ground truth generation process. (a) Data Preprocessing: The pipeline begins with native PDF text extraction and de-identification to produce machine-readable text. (b) Human-Verified Bootstrapping: The core iterative loop initialized with a seed sample. The process splits into two feedback cycles: a Coverage Loop (red arrows) for unparsed reports, and an Accuracy Loop (blue arrows) where provisional regex outputs are rigorously validated via manual expert audit against the source documents. The cycle repeats until the logic stabilizes, yielding no further rule refinements on the manually verified cohort.

Fig. S2 : (a) Training Efficiency and (b) HER2 Classification Performance.

Fig. S3 : Class Distributions across Clinical Tasks. The histograms illustrate the frequency of ground truth labels for (a) T Stage, (b) N Stage, (c) Histologic Grade, (d) ER, (e) PR, and (f) HER2.

Variable	Class Label	Clinical Definition
T Stage	pTis	Ductal carcinoma in situ (early stage, non-invasive).
	T0	No evidence of primary tumor.
	T1*	Small tumor ( $\leq$ 20mm). Includes microinvasive cases ( $\leq$ 1mm).
	T2	Medium tumor ( $>$ 20mm but $\leq$ 50mm).
	T3	Large tumor ( $>$ 50mm).
	T4	Tumor of any size that has spread directly to the chest wall or skin.

TABLE S1 : Clinical Definitions of Extracted Variables. Definitions derived from AJCC TNM Staging System (8th Ed.) and NCCN Guidelines (v6.2024). Asterisks (*, **, ***) denote specific prognostic grouping conventions observed in the dataset labels.

Clinical Information Extraction

Clinical Information Extraction turns clinical notes, reports, and patient narratives into structured, searchable facts and relations.

See all 106 papers

Prompt, Plan, Extract: Zero-Shot Agentic LLMs Workflows for Lung Pathology Extraction from Clinical Narratives
Jun 18, 2026Aman Pathak, Cheng Peng, Mengxian Lyu +8Computational Pathology Zero-Shot Learning
Department of Health Outcomes and Biomedical Informatics, College of Medicine, University of Florida, Gainesville, FL, USA · Division of Pulmonary, Critical Care and Sleep Medicine, Department of Medicine, College of Medicine, University of Florida, Gainesville, FL, USA · College of Nursing, Florida State University, Tallahassee, FL, USA
Tracking Cancer Through Text: Longitudinal Extraction From Radiology Reports Using Open-Source Large Language Models
Mar 10, 2026Luc Builtjes, Alessa HeringClinical Information Extraction LLM Information Extraction
Department of Medical Imaging, Radboudumc, Nijmegen, The Netherlands
SIFTING: A Novel LLM-Based Framework for Structured and Transparent Information Extraction from Clinical Free-Text Reports, with Application to Tumor Staging in Lung Cancer
Sep 7, 2026Mirco Hess, Gerben van Veenendaal, Joris Wakkie +5Clinical Information Extraction LLM Information Extraction
Axana B.V., Plantage Kerklaan 18-0, Amsterdam, 1018 TB, North Holland, Netherlands · University College London Hospitals NHS Foundation Trust, London, United Kingdom · Mid & South Essex NHS Foundation Trust, Essex, United Kingdom +2

Jun 18, 2026J/K move · Enter open · S save

Prompt, Plan, Extract: Zero-Shot Agentic LLMs Workflows for Lung Pathology Extraction from Clinical Narratives

Aman Pathak, Cheng Peng, Mengxian Lyu, Ziyi Chen+7

Department of Health Outcomes and Biomedical Informatics, College of Medicine, University of Florida, Gainesville, FL, USA · Division of Pulmonary, Critical Care and Sleep Medicine, Department of Medicine, College of Medicine, University of Florida, Gainesville, FL, USA · College of Nursing, Florida State University, Tallahassee, FL, USA

Computational Pathology Zero-Shot Learning Clinical Information Extraction LLM Information Extraction Oncology

Open paper AskPDF arXiv

Information extraction from pathology reports is essential for cancer staging, tumor registry population. Yet key data remains embedded in narrative reports, making manual extraction labor-intensive and error-prone. Traditional supervised Natural Language Processing pipelines address this through fully supervised Named Entity Recognition and Relation Extraction, but require expensive manual annotation and suffer cascading failures when upstream entities are missed. In this study, we developed a zero-shot, agentic workflow, and evaluated five open-source generative Large Language Models (LLMs) to populate 13 College of American Pathologists synoptic fields from lung resection pathology reports. We compared them against a state-of-the-art supervised GatorTron NER-RE baseline using a novel, registry-aligned evaluation framework. The baseline achieved Micro-F1of 0.960, while the best zero-shot model (GPT-OSS-20B) achieved Micro-F1 of 0.893 (recall: 0.949), accurately extracting complex relations like Pathologic Stage without task-specific training. These results suggest that open-source, zero-shot agentic LLMs show great potential as a low-cost solution for extracting lung pathology information.

Abstract

Clinical Information Extraction

Explore similar work

Prompt, Plan, Extract: Zero-Shot Agentic LLMs Workflows for Lung Pathology Extraction from Clinical Narratives

Tracking Cancer Through Text: Longitudinal Extraction From Radiology Reports Using Open-Source Large Language Models

SIFTING: A Novel LLM-Based Framework for Structured and Transparent Information Extraction from Clinical Free-Text Reports, with Application to Tumor Staging in Lung Cancer