Objective: Case investigation and contact tracing are labor-intensive. Web forms struggle when respondents need clarification before disclosing contacts, and open-ended chatbots may deviate from protocols. We developed and evaluated Epira, a protocol-controlled large language model (LLM) interviewer for multi-disease case investigation and contact tracing. Materials and Methods: Epira pairs LLM dialogue with a rule-based controller enforcing required questions, safety handoffs, and a structured case record, and adapts to new diseases by questionnaire upload. Local (Qwen2.5) and cloud (GPT-5.4) backends were tested. Study 1 used 108 simulated cases with ground-truth fields and contacts for measles and gonorrhea under Epira and open-ended conditions (864 sessions). Study 2 enrolled 72 volunteers in a 2×2 disease-by-model interview assessing completion, usability, acceptability, and trust. Results: Across Study 1, Epira with a local LLM captured 88.9% of required fields and 60.1% of ground-truth contacts. Open-ended interviewers recovered 31.1-41.9 percentage points fewer required fields than the same models in Epira. In Study 2, completion was 100%; mean adapted System Usability Scale (SUS) was 82.95 ± 16.17 on a 0-100 scale and mean trust was 3.85 ± 0.77 on a 1-5 scale. Discussion: Protocol control drove completeness more than model choice. Local and cloud deployments produced similar records, supporting privacy-preserving local use with human review of routed interviews and missed contacts. Conclusion: Epira recovers required fields and contacts under diverse disclosure conditions, works with local and cloud LLMs, and received favorable usability and trust ratings.
Figures & tables
Figure 1: Epira’s System. (A) Epira’s user interface. The page shows the dialogue on the left and a structured case form on the right. The form fills in as the interview proceeds, from the answers Epira has accepted, and the respondent can edit any field at any time to correct it. A completion bar at the top of the page shows which stage the interview is in and how many stages remain. A Talk instead button at the top right switches the same interview from text to a voice call. (B) Epira’s backend system architecture . One controller runs every disease. The active disease is a specification (stages, required fields, and safety rules), so adding a disease does not change the model or the controller. On each turn the caller’s message, typed in the web chat or spoken through the voice service, is passed to the controller with the current stage prompt, the structured case record, and recent dialogue. A language model drafts the next reply and proposes structured updates and safety flags. Protocol rules then accept, correct, or reject that proposal. Only the accepted reply is returned to the caller, and only accepted fields are written to the case record. The voice path is part of the system shown here. Study 1 and Study 2 evaluated the text interface only.
Study 1 on measles: Routine bank (36 cases)
Metric
Qwen2.5 (local)
GPT-5.4 (cloud)
Difference (95% CI)
Field completeness (%)
89.8%
97.1%
− 7.3 pp ( − 10.9 to − 2.5)
Contact completeness (%)
65.3%
61.1%
+ 4.2 pp ( − 11.5 to + 20.3)
Interview efficiency
0.605 ± 0.126
0.716 ± 0.150
− 0.112 ( − 0.152 to − 0.072)
Average Number of Interview Turns
39.9 ± 3.8
35.2 ± 4.8
+ 4.7 ( + 3.2 to + 6.4)
Cost ($ per completed interview)
0.00
$0.39
−0.39(-0.41 to − $0.38)
Table 1: Primary outcomes of the simulated evaluation of Epira for measles and gonorrhea (Study 1) with two LLMs as the interview engines. Each block reports one disease and one case bank (routine, 36 cases; challenging, 72 cases). Columns are the local open-source backend (Qwen2.5), the cloud backend (GPT-5.4), and the paired difference (local minus cloud) with a 95% confidence interval from 10,000 case-level resamples. Field completeness is the percentage of required case fields captured; contact completeness is the pooled percentage of ground-truth contacts matched. Interview efficiency is information units per turn; average number of interview turns is the mean conversation length; cost is LLM API spend per completed interview (0forthelocalmodel).Performancewassimilaracrossmodels,supportinglocaldeploymentforprivacyorclouddeploymentwithoutmateriallossoffieldorcontactrecovery.Forlocal–localcostcomparisons,noexternalAPIcostexistedineitherconditionandthedifferencewasstructurallyzero.Dataaremean\pm$ SD, %, or difference (95% CI). CI=confidence interval, LLM=large language model, API=application programming interface, and pp=percentage points.
Figure 2: Statistics for Studies 1 and 2. (A) Study 1: Field and contact recovery in simulated interviews . Each group is one disease and one case bank (routine, 36 cases; challenging, 72 cases). The left panel shows field completeness, the percentage of that disease’s required fields that were filled. The right panel shows contact completeness, the percentage of known contacts recovered. The measles banks contained 190 known contacts across 36 routine cases and 339 across 72 challenging cases; the corresponding gonorrhea banks contained 65 and 128 known contacts. (B) Study 2: Human volunteer ratings of Epira usability and preference . Mean ± SD scores by disease and interview engine (local Qwen2.5 vs cloud GPT-5.4); n = 18 per arm. The left panel shows the Adapted System Usability Scale (SUS; 0–100). The dashed line marks the conventional above-average SUS benchmark of 68. The right panel shows preference items (1–5): agreement with preferring the tool over a long online form, and with preferring a human interviewer over the tool. The dotted line marks the scale midpoint (3 = Neutral).
Measles
Metric
Measles: Local Qwen2.5
Measles: Cloud GPT-5.4
Interview completion rate
100%
100%
Usability (0-100)
Adapted System Usability Scale (SUS)
80.6 ± 16.9
78.9 ± 19.9
Trust & comfort (1-5)
Primary trust index (mean)
3.91 ± 0.72
4.00 ± 0.84
Table 3: Human volunteer pilot of Epira for measles and gonorrhea (Study 2) by disease and LLM engine. Each experiment was done with 18 volunteers (72 total). User experience, acceptability and usability were assessed with a post-study questionnaire. For each metric with (1-5) scale, higher value = greater agreement. Data are mean ± SD and %. SUS=System Usability Scale.
There are now multiple proposals for systems based on Large Language Models (LLMs) to conduct automated qualitative interviews, but most of the current solutions rely on proprietary LLMs, which compromises reproducibility and data security. They also rely on LLMs for all interview tasks, which limits standardisation of question wording as well as control over question order. To address these issues, we introduce the AInterviewer platform, an opensource solution based on a multi-agent pipeline that combines controlled question administration of survey software with the flexibility of LLMs. AInterviewer is an interdisciplinary effort designed to implement best practices of qualitative interviewing in social science, and it can run with locally hosted models to ensure security, transparency, and reproducibility. Our platform provides a web-based GUI supporting each phase of data collection: from interview guide design and pilot testing to interview distribution and data collection monitoring.
Tobias Priesholm Gardhus, Nikolas Vitsakis, Fie Lejre Frederiksen +2
University of Copenhagen · IT University of Copenhagen
We describe DS@GT's submission to the eRisk 2026 Task 1 challenge on conversational depression screening, in which systems interview LLM personas that simulate individuals with varying depression profiles and produce a Beck Depression Inventory II (BDI-II) score plus four key symptoms per persona, without directly asking sensitive mental health questions. Our pipeline evolved through three stages: a monolithic single-model prototype to start off, a baseline multi-agent architecture that separates conversational interviewing from BDI-II scoring under a coordinating orchestration layer, and a final hybrid configuration that replaces the paid GPT-5-nano interviewer with the open-source Gemma 27B. To offset the model's weaker reasoning and instruction-following, the hybrid adds three algorithmic components: a precomputed dialogue tree that standardizes interview openers and follow-ups, a reliability-weighted consensus aggregation inspired by the Weaver framework, and a cluster-based imputation step for unprobed symptoms. We submitted three fully automated runs across all 20 personas, with Run 1 from the paid baseline and Runs 2 and 3 from the hybrid. Hybrid Run 3 achieved an ADODL of 0.9063, ranking 3rd among all complete-submission runs and placing DS@GT 2nd among the 21 teams overall, while outperforming our paid baseline Run 1 (0.8841) at roughly one-quarter of the per-persona API cost. These results support our central hypothesis that with sufficient algorithmic supervision, a weaker open-source model can compete with a stronger proprietary model in the conversational interviewer role. Our source code is available at https://github.com/dsgt-arc/erisk-task1-2026.
Victor Gong, David Guecha
Georgia Institute of Technology, North Ave NW, Atlanta, GA 30332
Reliable epidemiological reasoning requires synthesizing study evidence to infer disease burden, transmission dynamics, and intervention effects at the population level. Existing medical question answering benchmarks primarily emphasize clinical knowledge or patient-level reasoning, yet few systematically evaluate evidence-grounded epidemiological inference. We present EpiQAL, to our knowledge the first diagnostic benchmark for epidemiological question answering over research literature, comprising three subsets built from open-access articles across diverse diseases. The three subsets progressively test factual recall, multi-step inference, and conclusion reconstruction under incomplete information, and are constructed through a quality-controlled pipeline combining taxonomy guidance, multi-model verification, and difficulty screening. Experiments on fifteen models spanning open-source and proprietary systems reveal that current LLMs show limited performance on epidemiological reasoning, with multi-step inference posing the greatest challenge. Model rankings shift across subsets, and scale alone does not predict success. Chain-of-Thought prompting benefits multi-step inference but yields mixed results elsewhere. EpiQAL provides fine-grained diagnostic signals for evidence-grounding, inferential reasoning, and conclusion reconstruction.
Mingyang Wei, Dehai Min, Zewen Liu +8
1Emory University · University of Illinois Chicago · 3Microsoft