cs.CLMar 26, 2026

QU-NLP at ArchEHR-QA 2026: Two-Stage QLoRA Fine-Tuning of Qwen3-4B for Patient-Oriented Clinical Question Answering and Evidence Sentence Alignment

Authors: Mohammad AL-Smadi

Organizations: Qatar University Doha, Qatar

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

We present a unified system addressing both Subtask 3 (answer generation) and Subtask 4 (evidence sentence alignment) of the ArchEHR-QA Shared Task. For Subtask 3, we apply two-stage Quantised Low-Rank Adaptation (QLoRA) to Qwen3-4B loaded in 4-bit NF4 quantisation: first on 30,000 samples from the emrQA-MedSQuAD corpus to establish clinical domain competence, then on the 20 annotated development cases to learn the task-specific output style. Our system achieves an overall score of 32.87 on the official test-2026 split (BLEU = 9.42, ROUGE-L = 27.04, SARI = 55.42, BERTScore = 43.00, AlignScore = 25.28, MEDCON = 37.04). For Subtask 4, we develop a weighted ensemble of three retrieval methods - BM25 with relative thresholding, TF-IDF cosine similarity, and a fine-tuned cross-encoder - to identify note sentences supporting a given gold answer, achieving a micro-F1 of 67.16 on the 100-case test set. Experiments reveal that both subtasks expose the same fundamental challenge: 20 annotated training cases are insufficient to distinguish relevant from irrelevant clinical sentences, pointing to data augmentation as the highest-leverage future direction.

Figures & tables

SplitCasesIDs  
dev201–20  
test10021–120  
test-202647121–167  
Total167   
Table 1: ArchEHR-QA dataset statistics.
FieldCase 4 — Cardiology
Patient question“My doctor performed a cardiac catherization. Was this invasive, risky procedure necessary.”
Clinician question“Why was cardiac catheterization recommended to the patient?”
Clinical note excerpt1: History of Present Illness: 2: On the cardiology service his abdominal pain, nausea, vomitting was felt to be secondary to congestive hepatopathy, his cough due to CHF vs asthma, and his syncope was felt to be secondary to a coughing spell… 3: His ICD interrogation was negative for any events. 4: He was aggressively diuresed with a net 10 liters negative since admission. [E] 5: He underwent RHC for milrinone trial, which proved to be successful. 6: His mean PCW went from 30 to 22, and his Fick C.I. went from 1.72 to 2.79. [S] 7: Brief Hospital Course: 48M with idiopathic dilated cardiomyopathy (EF 25%, on coumadin & s/p ICD placement), HTN, CKD (baseline Cr 1.3–1.6) & asthma who presents with syncope, found to be in acute-on-chronic heart failure. 8: #) ACUTE-ON-CHRONIC SYSTOLIC HEART FAILURE [S] 9: Underlying exacerbation of chronic heart failure known idiopathic cardiomyopathy likely explanation for numerous presenting symptoms including syncope, abdominal pain, and dyspnea. [E] 10: Last echo showed LVEF = 25%. [E] 11: He was admitted in the setting of low output heart failure, leading to increased intra-abdominal pressures causing congestive hepatopathy/abdominal pain/RUQ tenderness. 12: Abdominal distension also thought to contribute to syncope. [E] 13: Cardiac output and wedge pressure significantly improved after milrinone infusion. 14: In the ICU he was maintained on milrinone at a rate of 0.5 mcg/hr and transferred to the floor on this stable dose. 15: Torsemide 80mg daily was restarted; he was discharged on milrinone and torsemide. 16: Heart failure specialists had honest discussions with the patient about his long-term prognosis… 17: Discharge Instructions: [E] 18: You were admitted to the hospital with worsening heart failure. [E] 19: You had a cardiac catheterization that showed you would benefit from milrinone. [E] 20: You were started on a milrinone drip, with improvement in your heart’s pump function. 21: You were also diuresed for 6 liters of fluid.
Gold answer“The patient was recommended a cardiac catheterization for worsening heart failure confirmed by left ventricle ejection fraction of 25% on his echocardiogram [ 5,10,18 ]. He had low output heart failure which caused increasing intra-abdominal pressure resulting in congestive hepatopathy, abdominal pain, and right upper quadrant abdominal tenderness [ 11 ]. The cardiac catheterization showed the patient needed milrinone for treatment [ 19 ]. Milrinone infusion improved the patient’s heart pump function by significantly improving cardiac output and wedge pressure [ 13,20 ].”
Table 2: Subtask 3 example from the dev split (Case 4, Cardiology, MIMIC-III document 180932_37135). All 21 note sentences are shown verbatim from the XML excerpt ( note_excerpt_sentences ); sentences truncated with … exceed 120 characters in the source. Bold sentence numbers mark gold-cited sentences (Subtask 4 labels): E = essential, S = supplementary. MIMIC-III de-identified spans appear as […] .
FieldCase 20 — Neurology
Patient question“So what is the dizziness from I never heard of a migraine that can cause your head to spin like that.”
Clinician question“How did they diagnose her with migraine for spinning sensation?”
Clinical note excerptNR 1: Discharge Instructions: You were evaluated in the Neurology Wards of the ___ ___ for headaches that were of an unusual quality. E 2: We conducted a series of tests to investigate the cause of your headaches, including a CT scan and an MRI, a series of laboratory tests as well as a lumbar puncture. E 3: Through these tests, we were able to rule out serious intracranial causes that can cause a headache, including pseudotumor cerebri, which is a condition where elevated intracranial pressure can cause headaches and subsequent visual damage. E 4: We also checked an MRV (MR venogram) which ruled out cerebral venous thrombosis (a blood clot in the veins of your brain). E 5: — We started you on a new medication called verapamil. E 6: Take this once daily even if you do not have headaches (everyday, by mouth). NR 7: — Inform your PCP of this change, and please follow up with your primary care appointment and neurology appointment. NR 8: — You can request your PCP to set up a referral to see a neurologist in the community if you would prefer to do so. E 9: — To achieve optimal control of your migraines, avoid caffeinated products and keep a relatively regular sleep wake cycle (go to bed early, wake up early).
Gold answerA1 Neurology doctors completed a series of tests and procedures to determine the cause of headaches. cites note [2] A2 They ruled out elevated intracranial pressure and blood clots. cites note [3, 4] A3 Verapamil was started once daily to prevent headaches. cites note [5, 6] A4 Based on clinical knowledge, spinning sensation with a migraine is vertigo caused by the migraine. cites note [ ] A5 A migraine with aura can also cause dizziness. cites note [ ] A6 To achieve optimal control of migraines, patient was recommended to avoid caffeine and keep a regular sleep wake cycle. cites note [9]
Table 3: Subtask 4 example from the dev split (Case 20, Neurology, MIMIC-IV document 22494097).The gold answer is split into its constituent sentences; each is annotated with the note sentences it cites. Sentence-level relevance labels ( answers field) are shown in the rightmost column: E = essential, NR = not-relevant. Answer sentences 4 and 5 carry no citations ( [ ] ) because they draw on general clinical knowledge not present in the note.
HyperparameterStage 1Stage 2
Base modelQwen3-4B
Quantisation4-bit NF4
LoRA rank rr32
LoRA α\alpha64
LoRA dropout0.05
Max sequence length2,048
Table 4: Training hyperparameters for Stage 1 (domain grounding) and Stage 2 (task fine-tuning). LoRA configuration and optimiser are shared across both stages.
MetricOurs#1Δ\boldsymbol{\Delta}
BLEU9.429.92−- 0.50
ROUGE-L27.0427.85−- 0.81
SARI55.4258.64−- 3.22
BERTScore43.0046.83−- 3.83
AlignScore25.2831.74−- 6.46
MEDCON37.0443.10−- 6.06
Table 5: Official Subtask 3 test-2026 results (Cases 121–167, nn = 47). Overall is the arithmetic mean of all six metrics. #1 is the top-ranked system; Δ\Delta shows our gap.
MetricCE aloneOurs (ensemble)#1Δ\boldsymbol{\Delta}
Micro-precision85.4667.3588.0−- 20.65
Micro-recall47.2166.9775.9−- 8.93
Micro-F160.8267.1681.5−- 14.34
Macro-precision85.0969.33——
Macro-recall51.0069.71——
Macro-F160.9967.52——
Table 6: Official Subtask 4 leaderboard results (Cases 21–120, 147 submissions). CE alone : cross-encoder without ensemble. Ours : weighted BM25 + TF-IDF + CE ensemble with τens=0.85\tau_{\text{ens}}=0.85 . #1 is the top-ranked system; Δ\Delta shows the gap between our ensemble and #1.
More in this topic

Clinical QA

Clinical question answering uses AI to answer medical and patient-specific questions from clinical knowledge, records, guidelines, and evidence.

See all 58 papers

Explore similar work

CardsList
Apr 8, 2026

Yale-DM-Lab at ArchEHR-QA 2026: Deterministic Grounding and Multi-Pass Evidence Alignment for EHR Question Answering

We describe the Yale-DM-Lab system for the ArchEHR-QA 2026 shared task. The task studies patient-authored questions about hospitalization records and contains four subtasks (ST): clinician-interpreted question reformulation, evidence sentence identification, answer generation, and evidence-answer alignment. ST1 uses a dual-model pipeline with Claude Sonnet 4 and GPT-4o to reformulate patient questions into clinician-interpreted questions. ST2-ST4 rely on Azure-hosted model ensembles (o3, GPT-5.2, GPT-5.1, and DeepSeek-R1) combined with few-shot prompting and voting strategies. Our experiments show three main findings. First, model diversity and ensemble voting consistently improve performance compared to single-model baselines. Second, the full clinician answer paragraph is provided as additional prompt context for evidence alignment. Third, results on the development set show that alignment accuracy is mainly limited by reasoning. The best scores on the development set reach 88.81 micro F1 on ST4, 65.72 macro F1 on ST2, 34.01 on ST3, and 33.05 on ST1.

Elyas Irankhah, Samah Fodeh

Yale University, Department of Emergency Medicine, New Haven, CT, USA

EHR QAClinical QA
AskPDF
May 11, 2026

Neural at ArchEHR-QA 2026: One Method Fits All: Unified Prompt Optimization for Clinical QA over EHRs

Automated question answering (QA) over electronic health records (EHRs) demands precise evidence retrieval, faithful answer generation, and explicit grounding of answers in clinical notes. In this work, we present Neural1.5, our method for the ArchEHR-QA 2026 shared task at CL4Health@LREC 2026, which comprises four subtasks: question interpretation, evidence identification, answer generation, and evidence alignment. Our approach decouples the task into independent, modular stages and employs DSPy"s MIPROv2 optimizer to automatically discover high-performing prompts, jointly tuning instructions and few-shot demonstrations for each stage. Within every stage, self-consistency voting over multiple stochastic inference runs suppresses spurious errors and improves reliability, while stage-specific verification mechanisms (e.g., self-reflection and chain-of-verification for alignment) further refine output quality. Among all teams that participated in all four subtasks, our method ranks second overall (mean rank 4.00), placing 4th, 1st, 4th, and 7th on Subtasks 1-4, respectively. These results demonstrate that systematic, per-stage prompt optimization combined with self-consistency mechanisms is a cost-effective alternative to model fine-tuning for multifaceted clinical QA.

Abrar Majeedi, Viswanatha Reddy Gajjala, Sai Prasanna Teja Reddy Bogireddy +1

University of Wisconsin–Madison · University of Chicago · Independent Researcher

HealthcareClinical QAEvidence-Grounded GenerationElectronic Health RecordsEHR QA
AskPDF
Apr 29, 2026

HealthNLP_Retrievers at ArchEHR-QA 2026: Cascaded LLM Pipeline for Grounded Clinical Question Answering

Patient portals now give individuals direct access to their electronic health records (EHRs), yet access alone does not ensure patients understand or act on the complex clinical information contained in these records. The ArchEHR-QA 2026 shared task addresses this challenge by focusing on grounded question answering over EHRs, and this paper presents the system developed by the HealthNLP_Retrievers team for this task. The proposed approach uses a multi-stage cascaded pipeline powered by the Gemini 2.5 Pro large language model to interpret patient-authored questions and retrieve relevant evidence from lengthy clinical notes. Our architecture comprises four integrated modules: (1) a few-shot query reformulation unit which summarizes verbose patient queries; (2) a heuristic-based evidence scorer which ranks clinical sentences to prioritize recall; (3) a grounded response generator which synthesizes professional-caliber answers restricted strictly to identified evidence; and (4) a high-precision many-to-many alignment framework which links generated answers to supporting clinical sentences. This cascaded approach achieved competitive results. Across the individual tracks, the system ranked 1st in question interpretation, 5th in answer generation, 7th in evidence identification, and 9th in answer-evidence alignment. These results show that integrating large language models within a structured multi-stage pipeline improves grounding, precision, and the professional quality of patient-oriented health communication. To support reproducibility, our source code is publicly available in our GitHub repository

Md Biplob Hosen, Md Alomgeer Hussein, Md Akmol Masud +3

University of Maryland Baltimore County, Baltimore, MD, USA · 2Jahangirnagar University, Savar, Dhaka, Bangladesh

LLM GroundingHealthcareClinical QAEvidence-Grounded GenerationEHR QAEvidence Retrieval
AskPDF