cs.CLApr 8, 2026

Yale-DM-Lab at ArchEHR-QA 2026: Deterministic Grounding and Multi-Pass Evidence Alignment for EHR Question Answering

Authors: Elyas Irankhah, Samah Fodeh

Organizations: Yale University, Department of Emergency Medicine, New Haven, CT, USA

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

We describe the Yale-DM-Lab system for the ArchEHR-QA 2026 shared task. The task studies patient-authored questions about hospitalization records and contains four subtasks (ST): clinician-interpreted question reformulation, evidence sentence identification, answer generation, and evidence-answer alignment. ST1 uses a dual-model pipeline with Claude Sonnet 4 and GPT-4o to reformulate patient questions into clinician-interpreted questions. ST2-ST4 rely on Azure-hosted model ensembles (o3, GPT-5.2, GPT-5.1, and DeepSeek-R1) combined with few-shot prompting and voting strategies. Our experiments show three main findings. First, model diversity and ensemble voting consistently improve performance compared to single-model baselines. Second, the full clinician answer paragraph is provided as additional prompt context for evidence alignment. Third, results on the development set show that alignment accuracy is mainly limited by reasoning. The best scores on the development set reach 88.81 micro F1 on ST4, 65.72 macro F1 on ST2, 34.01 on ST3, and 33.05 on ST1.

Figures & tables

Figure 1: Dual-LLM pipeline for Subtask 1 clinician-interpreted question reformulation. The system extracts clinical context, retrieves few-shot examples, generates candidates using parallel LLMs, and selects the final question via scoring-based constraint enforcement. Clinical notes are optional and used only during development; test inference uses patient questions only.
Figure 2: Pipeline for Subtask 4 evidence alignment. The system combines ensemble inference, voting aggregation, and embedding-based recall to improve evidence matching.
Development Phase
Method / ConfigMetrics
Subtask 1 — Question Reformulation
 ScoreR1R2RLsumBERTAlign
Single (GPT-4o) + 5-shot ‡25.4934.6715.2831.7644.71–
Dual-model ensemble ‡26.1436.2816.5233.0645.34–
Dual + CoT + note grounding †23.9832.3312.4829.1942.74–
Table 1: Results across subtasks with Development Phase experiments shown first and Test Phase results summarized at the end. † indicates configurations using explicit reasoning or chain-of-thought(CoT) prompting. ‡ indicates baseline configurations without additional grounding or pattern-scoring components.
More in this topic

EHR QA

See all 14 papers

Explore similar work

CardsList

  1. Neural at ArchEHR-QA 2026: One Method Fits All: Unified Prompt Optimization for Clinical QA over EHRs

    May 11, 2026Abrar Majeedi, Viswanatha Reddy Gajjala, Sai Prasanna Teja Reddy Bogireddy +1HealthcareClinical QA

    University of Wisconsin–Madison · University of Chicago · Independent Researcher

  2. QU-NLP at ArchEHR-QA 2026: Two-Stage QLoRA Fine-Tuning of Qwen3-4B for Patient-Oriented Clinical Question Answering and Evidence Sentence Alignment

    Mar 26, 2026Mohammad AL-SmadiClinical QAEHR QA

    Qatar University Doha, Qatar

  3. HealthNLP_Retrievers at ArchEHR-QA 2026: Cascaded LLM Pipeline for Grounded Clinical Question Answering

    Apr 29, 2026Md Biplob Hosen, Md Alomgeer Hussein, Md Akmol Masud +3LLM GroundingHealthcare

    University of Maryland Baltimore County, Baltimore, MD, USA · 2Jahangirnagar University, Savar, Dhaka, Bangladesh