Future Querying: Can LLMs Serve as Implicit Medical World Models?

Siri Willems; James Butterworth; Lore Goetschalckx; Peter Vrancx; Philippe Modard; Elke Giets; Ludovic Denoyer

cs.CLAug 24, 2026

Future Querying: Can LLMs Serve as Implicit Medical World Models?

Authors: Siri Willems, James Butterworth, Lore Goetschalckx, Peter Vrancx, Philippe Modard, Elke Giets, Ludovic Denoyer

Organizations: imec, AI-labs⋆⋆, Paris, France

Abstract

Traditional clinical prediction models rely on task-specific pipelines and curated, structured data, which scale poorly and underutilize unstructured text. To address this, we introduce future querying, a paradigm that probes whether large language models (LLMs) can function as implicit medical world models by evaluating their ability to answer time-indexed clinical queries about a patient's future. Our framework operates on unstructured clinical documentation using endpoint-agnostic training, enabling a single model to answer diverse clinical queries over patient trajectories without manual feature engineering or task-specific retraining. We show that small, locally fine-tuned open-weight models can match or approach larger proprietary systems, making the framework suitable for privacy-preserving, on-premise deployment. Evaluated on a new synthetic medical reports dataset and real ICU notes from the MIMIC-IV dataset, our results provide encouraging evidence that LLMs can capture aspects of clinical dynamics.

Figures & tables

Figure 1: The future querying paradigm. A patient timeline $p$ is partitioned by a virtual present $\hat{T}$ into an observed history and a hidden future. Given the historical context and a flexible, time-indexed clinical query $q_{k}$ , we investigate whether an LLM can serve as an implicit medical world model to predict the correct answer $a_{k}$ derived from the patient’s actual future timeline.

Figure 2: (Left) Characteristics of the two future querying datasets; values are reported as median (min–max). (Right) Distribution of query types computed on the test fold. Queries span six semantic categories and are binned by the time gap between the simulated present and the target clinical event.

Figure 3: Baseline performance of off-the-shelf LLMs on the Synthetic Medical Reports and MIMIC ICU Notes benchmarks. The top legend row shows models with reasoning capabilities, evaluated using OpenRouter’s default settings. For gemini-2.5-flash and claude-sonnet-5 , these defaults did not enable reasoning. Due to computational costs, three models were evaluated on a 4% subset of the MIMIC ICU Notes test set and are shown with reduced opacity.

Figure 4: Effect of fine-tuning on future querying accuracy with respect to the accuracy of proprietary models.

Explore similar work

Can we generate portable representations for clinical time series data using LLMs?
Mar 25, 2026Zongliang Ji, Yifei Sun, Andre Amaral +2Domain Generalization Time Series Representation Learning
University of Toronto, Canada · Vector Institute, Canada · Sunnybrook Health Sciences Centre, Canada
Assessing Large Language Models for Medical QA: Zero-Shot and LLM-as-a-Judge Evaluation
Feb 16, 2026Shefayat E Shams Adib, Ahmed Alfey Sani, Ekramul Alam Esham +2Medical QA Medical LLMs
Department of Computer Science and Engineering, Islamic University of Technology, Gazipur, Bangladesh
ReMedi: Reasoner for Medical Clinical Prediction
May 2, 2026Yushi Cao, Yiming Chen, Hongchao Jiang +2Fine-Tuning Clinical Outcome Prediction
ASUS Intelligent Cloud Services (AICS), Singapore · NTU Artificial Intelligence Center of Research Excellence (NTU AI-CoRE), Taiwan · National University of Singapore, Singapore

Mar 25, 2026J/K move · Enter open · S save

Can we generate portable representations for clinical time series data using LLMs?

Zongliang Ji, Yifei Sun, Andre Amaral, Anna Goldenberg+1

University of Toronto, Canada · Vector Institute, Canada · Sunnybrook Health Sciences Centre, Canada

Domain Generalization Time Series Representation Learning Longitudinal EHR Modeling Clinical Outcome Prediction

Open paper AskPDF arXiv

Deploying clinical ML is slow and brittle: models that work at one hospital often degrade under distribution shifts at the next. In this work, we study a simple question -- can large language models (LLMs) create portable patient embeddings i.e. representations of patients enable a downstream predictor built on one hospital to be used elsewhere with minimal-to-no retraining and fine-tuning. To do so, we map from irregular ICU time series onto concise natural language summaries using a frozen LLM, then embed each summary with a frozen text embedding model to obtain a fixed length vector capable of serving as input to a variety of downstream predictors. Across three cohorts (MIMIC-IV, HIRID, PPICU), on multiple clinically grounded forecasting and classification tasks, we find that our approach is simple, easy to use and competitive with in-distribution with grid imputation, self-supervised representation learning, and time series foundation models, while exhibiting smaller relative performance drops when transferring to new hospitals. We study the variation in performance across prompt design, with structured prompts being crucial to reducing the variance of the predictive models without altering mean accuracy. We find that using these portable representations improves few-shot learning and does not increase demographic recoverability of age or sex relative to baselines, suggesting little additional privacy risk. Our work points to the potential that LLMs hold as tools to enable the scalable deployment of production grade predictive models by reducing the engineering overhead.

Abstract

Explore similar work

Can we generate portable representations for clinical time series data using LLMs?

Assessing Large Language Models for Medical QA: Zero-Shot and LLM-as-a-Judge Evaluation

ReMedi: Reasoner for Medical Clinical Prediction