cs.CLNov 2, 2026

Rethinking Next-Visit Diagnosis Prediction Beyond Aggregate Metrics: Retrieval--Memory Multi-Agent Adjudication

Authors: Heejoon Koo

Organizations: University of Illinois Urbana-Champaign

Abstract

Next-visit diagnosis prediction forecasts the diagnoses documented at a patient's next encounter, turning retrospective electronic health records into prospective clinical prediction. However, conventional aggregate evaluation obscures its multifaceted nature: distributionally, models must handle variable-size diagnosis sets and long-tailed labels; temporally, they must distinguish persistent, newly recorded, and recurrent diagnoses; and both vary with observed history depth. We introduce an evaluation framework along these dimensions and apply it to rule-based heuristics, LLM-based prompting, retrieval, memory, and multi-agent approaches. This reveals that repeating the last visit outperforms most LLM-based approaches overall despite predicting no new or recurrent diagnoses, and that retrieval and memory exhibit distinct profiles whose relative strengths vary across temporal transitions and backbones. Motivated by this, we propose LONGINUS, a retrieval-memory multi-agent adjudication framework combining evidence retrieved from patient records with structured memory of the patient trajectory. Using cross-view agreement as a reliability heuristic, it preserves shared predictions and adjudicates disagreements without revealing their source. Across MIMIC-III and MIMIC-IV with four LLM backbones, LONGINUS achieves the best transition-specific performance among LLM-based methods in nearly all settings while remaining competitive with them on aggregate metrics.

Figures & tables

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. One Panel Does Not Fit All: Case-Adaptive Multi-Agent Deliberation for Clinical Prediction

    Mar 31, 2026Yuxing Lu, Yushuhong Lin, Jason ZhangMulti-Agent CollaborationClinical Prediction

  2. Coverage-Aware Reasoning with Medical Tokens for Diagnosis Prediction

    Oct 7, 2026Kaisong Zhang, Haotian Fang, Junmeng Zhou +3Medical LLMsClinical Prediction

  3. MeDxAgent: Multi-Agent Consultation for Interactive Medical Diagnosis

    Jun 2, 2026Akshat Sanghvi, Naren Akash, Raza Imam +2Multi-Agent LLM Systemsinteractive medical diagnosis