Design and Evaluation of Epira: An Automated Interview System for Case Investigation and Contact Tracing
Organizations: University of Illinois Urbana-Champaign, Urbana, IL, USA · Stanford University, Stanford, CA, USA
Abstract
Objective: Case investigation and contact tracing are labor-intensive. Web forms struggle when respondents need clarification before disclosing contacts, and open-ended chatbots may deviate from protocols. We developed and evaluated Epira, a protocol-controlled large language model (LLM) interviewer for multi-disease case investigation and contact tracing. Materials and Methods: Epira pairs LLM dialogue with a rule-based controller enforcing required questions, safety handoffs, and a structured case record, and adapts to new diseases by questionnaire upload. Local (Qwen2.5) and cloud (GPT-5.4) backends were tested. Study 1 used 108 simulated cases with ground-truth fields and contacts for measles and gonorrhea under Epira and open-ended conditions (864 sessions). Study 2 enrolled 72 volunteers in a 22 disease-by-model interview assessing completion, usability, acceptability, and trust. Results: Across Study 1, Epira with a local LLM captured 88.9% of required fields and 60.1% of ground-truth contacts. Open-ended interviewers recovered 31.1-41.9 percentage points fewer required fields than the same models in Epira. In Study 2, completion was 100%; mean adapted System Usability Scale (SUS) was 82.95 16.17 on a 0-100 scale and mean trust was 3.85 0.77 on a 1-5 scale. Discussion: Protocol control drove completeness more than model choice. Local and cloud deployments produced similar records, supporting privacy-preserving local use with human review of routed interviews and missed contacts. Conclusion: Epira recovers required fields and contacts under diverse disclosure conditions, works with local and cloud LLMs, and received favorable usability and trust ratings.
Figures & tables
| Study 1 on measles: Routine bank (36 cases) | |||
|---|---|---|---|
| Metric | Qwen2.5 (local) | GPT-5.4 (cloud) | Difference (95% CI) |
| Field completeness (%) | 89.8% | 97.1% | 7.3 pp ( 10.9 to 2.5) |
| Contact completeness (%) | 65.3% | 61.1% | 4.2 pp ( 11.5 to 20.3) |
| Interview efficiency | 0.605 0.126 | 0.716 0.150 | 0.112 ( 0.152 to 0.072) |
| Average Number of Interview Turns | 39.9 3.8 | 35.2 4.8 | 4.7 ( 3.2 to 6.4) |
| Cost ($ per completed interview) | 0.00 | $0.39 | -0.41 to $0.38) |
| Measles | ||
|---|---|---|
| Metric | Measles: Local Qwen2.5 | Measles: Cloud GPT-5.4 |
| Interview completion rate | 100% | 100% |
| Usability (0-100) | ||
| Adapted System Usability Scale (SUS) | 80.6 16.9 | 78.9 19.9 |
| Trust & comfort (1-5) | ||
| Primary trust index (mean) | 3.91 0.72 | 4.00 0.84 |
LLM Agents
Large language model agents use language models to plan, use tools, and act across multi-step tasks and environments.
See all 774 papers