QU-NLP at ArchEHR-QA 2026: Two-Stage QLoRA Fine-Tuning of Qwen3-4B for Patient-Oriented Clinical Question Answering and Evidence Sentence Alignment

Mohammad AL-Smadi

cs.CLMar 26, 2026

QU-NLP at ArchEHR-QA 2026: Two-Stage QLoRA Fine-Tuning of Qwen3-4B for Patient-Oriented Clinical Question Answering and Evidence Sentence Alignment

Authors: Mohammad AL-Smadi

Organizations: Qatar University Doha, Qatar

Abstract

We present a unified system addressing both Subtask 3 (answer generation) and Subtask 4 (evidence sentence alignment) of the ArchEHR-QA Shared Task. For Subtask 3, we apply two-stage Quantised Low-Rank Adaptation (QLoRA) to Qwen3-4B loaded in 4-bit NF4 quantisation: first on 30,000 samples from the emrQA-MedSQuAD corpus to establish clinical domain competence, then on the 20 annotated development cases to learn the task-specific output style. Our system achieves an overall score of 32.87 on the official test-2026 split (BLEU = 9.42, ROUGE-L = 27.04, SARI = 55.42, BERTScore = 43.00, AlignScore = 25.28, MEDCON = 37.04). For Subtask 4, we develop a weighted ensemble of three retrieval methods - BM25 with relative thresholding, TF-IDF cosine similarity, and a fine-tuned cross-encoder - to identify note sentences supporting a given gold answer, achieving a micro-F1 of 67.16 on the 100-case test set. Experiments reveal that both subtasks expose the same fundamental challenge: 20 annotated training cases are insufficient to distinguish relevant from irrelevant clinical sentences, pointing to data augmentation as the highest-leverage future direction.

Figures & tables

Split	Cases	IDs
dev	20	1–20
test	100	21–120
test-2026	47	121–167
Total	167

Table 1: ArchEHR-QA dataset statistics.

Field	Case 4 — Cardiology
Patient question	“My doctor performed a cardiac catherization. Was this invasive, risky procedure necessary.”
Clinician question	“Why was cardiac catheterization recommended to the patient?”
Clinical note excerpt	1: History of Present Illness: 2: On the cardiology service his abdominal pain, nausea, vomitting was felt to be secondary to congestive hepatopathy, his cough due to CHF vs asthma, and his syncope was felt to be secondary to a coughing spell… 3: His ICD interrogation was negative for any events. 4: He was aggressively diuresed with a net 10 liters negative since admission. [E] 5: He underwent RHC for milrinone trial, which proved to be successful. 6: His mean PCW went from 30 to 22, and his Fick C.I. went from 1.72 to 2.79. [S] 7: Brief Hospital Course: 48M with idiopathic dilated cardiomyopathy (EF 25%, on coumadin & s/p ICD placement), HTN, CKD (baseline Cr 1.3–1.6) & asthma who presents with syncope, found to be in acute-on-chronic heart failure. 8: #) ACUTE-ON-CHRONIC SYSTOLIC HEART FAILURE [S] 9: Underlying exacerbation of chronic heart failure known idiopathic cardiomyopathy likely explanation for numerous presenting symptoms including syncope, abdominal pain, and dyspnea. [E] 10: Last echo showed LVEF = 25%. [E] 11: He was admitted in the setting of low output heart failure, leading to increased intra-abdominal pressures causing congestive hepatopathy/abdominal pain/RUQ tenderness. 12: Abdominal distension also thought to contribute to syncope. [E] 13: Cardiac output and wedge pressure significantly improved after milrinone infusion. 14: In the ICU he was maintained on milrinone at a rate of 0.5 mcg/hr and transferred to the floor on this stable dose. 15: Torsemide 80mg daily was restarted; he was discharged on milrinone and torsemide. 16: Heart failure specialists had honest discussions with the patient about his long-term prognosis… 17: Discharge Instructions: [E] 18: You were admitted to the hospital with worsening heart failure. [E] 19: You had a cardiac catheterization that showed you would benefit from milrinone. [E] 20: You were started on a milrinone drip, with improvement in your heart’s pump function. 21: You were also diuresed for 6 liters of fluid.
Gold answer	“The patient was recommended a cardiac catheterization for worsening heart failure confirmed by left ventricle ejection fraction of 25% on his echocardiogram [ 5,10,18 ]. He had low output heart failure which caused increasing intra-abdominal pressure resulting in congestive hepatopathy, abdominal pain, and right upper quadrant abdominal tenderness [ 11 ]. The cardiac catheterization showed the patient needed milrinone for treatment [ 19 ]. Milrinone infusion improved the patient’s heart pump function by significantly improving cardiac output and wedge pressure [ 13,20 ].”

Table 2: Subtask 3 example from the dev split (Case 4, Cardiology, MIMIC-III document 180932_37135). All 21 note sentences are shown verbatim from the XML excerpt ( note_excerpt_sentences ); sentences truncated with … exceed 120 characters in the source. Bold sentence numbers mark gold-cited sentences (Subtask 4 labels): E = essential, S = supplementary. MIMIC-III de-identified spans appear as […] .

Field	Case 20 — Neurology
Patient question	“So what is the dizziness from I never heard of a migraine that can cause your head to spin like that.”
Clinician question	“How did they diagnose her with migraine for spinning sensation?”
Clinical note excerpt	NR 1: Discharge Instructions: You were evaluated in the Neurology Wards of the ___ ___ for headaches that were of an unusual quality. E 2: We conducted a series of tests to investigate the cause of your headaches, including a CT scan and an MRI, a series of laboratory tests as well as a lumbar puncture. E 3: Through these tests, we were able to rule out serious intracranial causes that can cause a headache, including pseudotumor cerebri, which is a condition where elevated intracranial pressure can cause headaches and subsequent visual damage. E 4: We also checked an MRV (MR venogram) which ruled out cerebral venous thrombosis (a blood clot in the veins of your brain). E 5: — We started you on a new medication called verapamil. E 6: Take this once daily even if you do not have headaches (everyday, by mouth). NR 7: — Inform your PCP of this change, and please follow up with your primary care appointment and neurology appointment. NR 8: — You can request your PCP to set up a referral to see a neurologist in the community if you would prefer to do so. E 9: — To achieve optimal control of your migraines, avoid caffeinated products and keep a relatively regular sleep wake cycle (go to bed early, wake up early).
Gold answer	A1 Neurology doctors completed a series of tests and procedures to determine the cause of headaches. cites note [2] A2 They ruled out elevated intracranial pressure and blood clots. cites note [3, 4] A3 Verapamil was started once daily to prevent headaches. cites note [5, 6] A4 Based on clinical knowledge, spinning sensation with a migraine is vertigo caused by the migraine. cites note [ ] A5 A migraine with aura can also cause dizziness. cites note [ ] A6 To achieve optimal control of migraines, patient was recommended to avoid caffeine and keep a regular sleep wake cycle. cites note [9]

Table 3: Subtask 4 example from the dev split (Case 20, Neurology, MIMIC-IV document 22494097).The gold answer is split into its constituent sentences; each is annotated with the note sentences it cites. Sentence-level relevance labels ( answers field) are shown in the rightmost column: E = essential, NR = not-relevant. Answer sentences 4 and 5 carry no citations ( [ ] ) because they draw on general clinical knowledge not present in the note.

Hyperparameter	Stage 1	Stage 2
Base model	Qwen3-4B
Quantisation	4-bit NF4
LoRA rank $r$	32
LoRA $\alpha$	64
LoRA dropout	0.05
Max sequence length	2,048

Table 4: Training hyperparameters for Stage 1 (domain grounding) and Stage 2 (task fine-tuning). LoRA configuration and optimiser are shared across both stages.

Metric	Ours	#1	$\boldsymbol{\Delta}$
BLEU	9.42	9.92	$-$ 0.50
ROUGE-L	27.04	27.85	$-$ 0.81
SARI	55.42	58.64	$-$ 3.22
BERTScore	43.00	46.83	$-$ 3.83
AlignScore	25.28	31.74	$-$ 6.46
MEDCON	37.04	43.10	$-$ 6.06

Table 5: Official Subtask 3 test-2026 results (Cases 121–167,

n

= 47). Overall is the arithmetic mean of all six metrics. #1 is the top-ranked system;

\Delta

shows our gap.

Metric	CE alone	Ours (ensemble)	#1	$\boldsymbol{\Delta}$
Micro-precision	85.46	67.35	88.0	$-$ 20.65
Micro-recall	47.21	66.97	75.9	$-$ 8.93
Micro-F1	60.82	67.16	81.5	$-$ 14.34
Macro-precision	85.09	69.33	—	—
Macro-recall	51.00	69.71	—	—
Macro-F1	60.99	67.52	—	—

Table 6: Official Subtask 4 leaderboard results (Cases 21–120, 147 submissions). CE alone : cross-encoder without ensemble. Ours : weighted BM25 + TF-IDF + CE ensemble with

\tau_{\text{ens}}=0.85

. #1 is the top-ranked system;

\Delta

shows the gap between our ensemble and #1.

Clinical QA

Clinical question answering uses AI to answer medical and patient-specific questions from clinical knowledge, records, guidelines, and evidence.

See all 58 papers

Yale-DM-Lab at ArchEHR-QA 2026: Deterministic Grounding and Multi-Pass Evidence Alignment for EHR Question Answering
Apr 8, 2026Elyas Irankhah, Samah FodehEHR QA Clinical QA
Yale University, Department of Emergency Medicine, New Haven, CT, USA
Neural at ArchEHR-QA 2026: One Method Fits All: Unified Prompt Optimization for Clinical QA over EHRs
May 11, 2026Abrar Majeedi, Viswanatha Reddy Gajjala, Sai Prasanna Teja Reddy Bogireddy +1Healthcare Clinical QA
University of Wisconsin–Madison · University of Chicago · Independent Researcher
HealthNLP_Retrievers at ArchEHR-QA 2026: Cascaded LLM Pipeline for Grounded Clinical Question Answering
Apr 29, 2026Md Biplob Hosen, Md Alomgeer Hussein, Md Akmol Masud +3LLM Grounding Healthcare
University of Maryland Baltimore County, Baltimore, MD, USA · 2Jahangirnagar University, Savar, Dhaka, Bangladesh

Apr 8, 2026J/K move · Enter open · S save

Yale-DM-Lab at ArchEHR-QA 2026: Deterministic Grounding and Multi-Pass Evidence Alignment for EHR Question Answering

Elyas Irankhah, Samah Fodeh

Yale University, Department of Emergency Medicine, New Haven, CT, USA

EHR QA Clinical QA

Open paper AskPDF arXiv

We describe the Yale-DM-Lab system for the ArchEHR-QA 2026 shared task. The task studies patient-authored questions about hospitalization records and contains four subtasks (ST): clinician-interpreted question reformulation, evidence sentence identification, answer generation, and evidence-answer alignment. ST1 uses a dual-model pipeline with Claude Sonnet 4 and GPT-4o to reformulate patient questions into clinician-interpreted questions. ST2-ST4 rely on Azure-hosted model ensembles (o3, GPT-5.2, GPT-5.1, and DeepSeek-R1) combined with few-shot prompting and voting strategies. Our experiments show three main findings. First, model diversity and ensemble voting consistently improve performance compared to single-model baselines. Second, the full clinician answer paragraph is provided as additional prompt context for evidence alignment. Third, results on the development set show that alignment accuracy is mainly limited by reasoning. The best scores on the development set reach 88.81 micro F1 on ST4, 65.72 macro F1 on ST2, 34.01 on ST3, and 33.05 on ST1.

Abstract

Clinical QA

Explore similar work

Yale-DM-Lab at ArchEHR-QA 2026: Deterministic Grounding and Multi-Pass Evidence Alignment for EHR Question Answering

Neural at ArchEHR-QA 2026: One Method Fits All: Unified Prompt Optimization for Clinical QA over EHRs

HealthNLP_Retrievers at ArchEHR-QA 2026: Cascaded LLM Pipeline for Grounded Clinical Question Answering