CBR-to-SQL: Rethinking Retrieval-based Text-to-SQL using Case-based Reasoning in the Healthcare Domain

Hung Nguyen; Hans Moen; Pekka Marttinen

cs.IRMar 5, 2026

CBR-to-SQL: Rethinking Retrieval-based Text-to-SQL using Case-based Reasoning in the Healthcare Domain

Authors: Hung Nguyen, Hans Moen, Pekka Marttinen

Organizations: Department of Computer Science Aalto University · Department of Clinical Medicine Aalborg University

Abstract

Extracting insights from Electronic Health Record (EHR) databases often requires SQL expertise, creating a barrier for clinical decision-making and research. A promising approach is to use Large Language Models (LLMs) to translate natural language questions into SQL through Retrieval-Augmented Generation (RAG), where relevant question-SQL examples are retrieved to generate new queries via few-shot learning. However, adapting this method to the medical domain is non-trivial, as effective retrieval requires examples that align with both the logical structure of the question and its referenced entities (e.g., drug names, procedure titles). Standard single-step RAG struggles to optimize both aspects simultaneously and often relies on near-exact matches to generalize effectively. This issue is especially severe in healthcare, as questions often contain noisy and inconsistent medical jargon. To address this, we present CBR-to-SQL, a framework inspired by Case-based Reasoning theory that decomposes RAG's single-step retrieval into two explicit stages: one that focuses on retrieving structurally relevant examples, and one that aligns entities with the target database schema. Evaluated on two clinical benchmarks, CBR-to-SQL achieves competitive accuracies compared to fine-tuned methods. More importantly, it demonstrates considerably higher sample efficiency and robustness than the standard RAG approach, particularly under data scarcity and retrieval perturbations.

Figures & tables

Figure 1: Overview of CBR-to-SQL’s architecture.

Method	$\mathbf{Acc_{EX}}(\uparrow)$	$\mathbf{Acc_{LF}}(\uparrow)$
SQLNet	0.260	0.142
PtrGen	0.292	0.180
Coarse2Fine	0.378	0.496
TREQS	0.654	0.556
MedTS	0.899	0.784
Chen et al. (2026)	0.942	–

Table 1: Comparison of methods on

Acc_{EX}

and

Acc_{LF}

metrics in CDB. Higher metric values indicate stronger performance. Cells marked “–” indicate that the authors do not report the metric.

Method	MIMICSQL		EHRSQL
Method	$\Delta_{brittle^{EX}}(\downarrow)$	$\Delta_{brittle_{LF}}(\downarrow)$	$\Delta_{brittle_{EX}}(\downarrow)$	$\Delta_{brittle_{LF}}(\downarrow)$
RAG-to-SQL	0.074	0.052	0.138	0.149
CBR-to-SQL	0.028	0.033	0.043	0.128

Table 2: Comparison of methods on brittleness metric in CDB. Lower

\Delta

values indicate less brittle behavior.

Method	MIMICSQL		EHRSQL
Method	$Acc_{EX}(\uparrow)$	$Acc_{LF}(\uparrow)$	$Acc_{EX}(\uparrow)$	$Acc_{LF}(\uparrow)$
RAG-to-SQL	0.781	0.779	0.718	0.542
CBR-to-SQL	0.871	0.836	0.837	0.644

Table 3: Comparison of methods on

Acc_{EX}

and

Acc_{LF}

metrics in IDB.

Method	MIMICSQL		EHRSQL
Method	$\Delta_{brittle_{EX}}(\downarrow)$	$\Delta_{brittle_{LF}}(\downarrow)$	$\Delta_{brittle_{EX}}(\downarrow)$	$\Delta_{brittle_{LF}}(\downarrow)$
RAG-to-SQL	0.082	0.079	0.171	0.184
CBR-to-SQL	0.065	0.061	0.086	0.182

Table 4: Comparison of methods on brittleness metric in IDB.

Method	$Acc_{EX}(\uparrow)$	$Acc_{LF}(\uparrow)$
CBR-to-SQL	0.894	0.862
Replace Proposal stage	0.889	0.851
No Revision stage	0.786	0.776

Table 5:

Acc_{EX}

and

Acc_{LF}

when each online component in CBR-to-SQL is ablated in CDB for MIMICSQL, compared to its original performance.

Figure 2: $Acc_{EX}$ and $Acc_{LF}$ of different methods on different numbers of $k$ -shot examples ( $k\in\{1,3,5\}$ ) in CDB for MIMICSQL.

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 3: CBR cycle adapted from Aamodt and Plaza (1994) .

Metric	MIMICSQL	EHRSQL
Tables Processed	5	17
Columns Analyzed	70	226
Text Columns	30	97
Passed Simple Filter	22	75
Passed LLM Filter	19	26
Entities Indexed	82,685	200,146

Appendix

Table 6: Lookup table construction statistics.

Category	MIMICSQL	EHRSQL
Accepted Columns
Demographics	NAME, MARITAL_STATUS,	admission_type,
	RELIGION, ETHNICITY	admission_location,
		discharge_location,
		insurance
Diagnoses	SHORT_TITLE, LONG_TITLE	long_title

Appendix

Table 7: Accepted and rejected column examples by category.

Statistic	Value
Number of patients	46,520
Number of tables	5
Number of columns in tables 2 2 2 Columns are listed in order: Demographics, Diagnoses, Procedures, Prescriptions, Laboratory Tests.	23 / 5 / 5 / 7 / 9
Number of Question-SQL pairs	10,000
Average template question length (words)	18.39
Average natural language question length (words)	16.45

Appendix

Table 8: Statistics of the MIMICSQL dataset ( Wang et al., 2020 ) .

Table	Description
Demographics	Contains patient demographic information such as age, gender, ethnicity, admission details, and other identifiers.
Diagnoses	Records all diagnosis codes (ICD-9) assigned during hospital stays, capturing the medical conditions and comorbidities of patients.
Procedures	Contains procedural codes (ICD-9) for interventions or surgeries performed on patients during their hospital stay.
Prescriptions	Details of medication orders including drug name, dosage, route, frequency, and timing, prescribed to patients.
Laboratory Tests	Laboratory measurements and test results such as blood tests, biochemistry, and hematology with timestamps.

Appendix

Table 9: Overview of tables in MIMICSQL ( Wang et al., 2020 ) .

	Train	Valid	Test
Answerable question template	100	134	134
Answerable samples	4674	931	934
Unanswerable samples	450	232	233
Total samples	5124	1163	1167

Appendix

Table 10: Data statistics for the EHRSQL shared task ( Lee et al., 2024 ) .

Figure 4: UMAP projection of the HBSCAN clustering results over masked SQL queries in the IDB setup. Each circle represents a cluster, and the circle size reflects the cluster size.

Dataset	Cluster Size	Representative Questions
MIMICSQL	124	How many patients were diagnosed with anxiety state NOS and were given drugs orally? What is the number of patients whose diagnoses title is acute nephritis nec and drug route is po? Provide the number of patients who are diagnosed with Wegener’s granulomatosis and have sc route of drug administration.
	102	Count the number of patients whose primary disease is left internal jugular vein thrombosis; left arm edema and who died in or before the year 2115. How many patients died of colangitis in or before the year 2180? What is the number of patients who had rash as primary disease and died in 2155 or before that?
EHRSQL	75	Tell me the price of trimethoprim. Prazosin – How much does it cost? What are the prices of the drug albumin 25% (12.5g / 50ml)?
	72	How much does the pH change in patient 10021666 last measured on the last hospital visit compared to the first value measured on the last hospital visit? What are patient 10027602’s changes in glucose, CSF second measured on the first hospital visit compared to the first value measured on the first hospital visit? How much difference is there in phosphate levels among patient 10015931 second measured on the first hospital visit compared to the value first measured on the first hospital visit?

Appendix

Table 11: Representative questions from the largest HDBSCAN clusters in the Incomplete Database (IDB) setup across MIMICSQL and EHRSQL datasets. The clustering effectively groups structurally similar SQL queries.

Figure 5: Overview of RAG-to-SQL’s architecture.

Method	MIMICSQL		EHRSQL
Method	Latency ( $\downarrow$ )	Token ( $\downarrow$ )	Latency ( $\downarrow$ )	Token ( $\downarrow$ )
RAG-to-SQL	1.67	2251.68	3.02	6819.99
CBR-to-SQL	2.60	5399.09	4.43	9512.79

Appendix

Table 12: Average latency (in seconds) and token usage per query across two datasets in CDB.

Metric	MIMICSQL		EHRSQL
Metric	Per 1% $Acc_{EX}$ Gain	Per 1% $Acc_{LF}$ Gain	Per 1% $Acc_{EX}$ Gain	Per 1% $Acc_{LF}$ Gain
Latency Overhead ( $\downarrow$ )	0.103	0.163	0.118	0.138
Token Overhead ( $\downarrow$ )	349.71	552.18	226.29	264.00

Appendix

Table 13: Marginal cost of performance gain for CBR-to-SQL over RAG-to-SQL across two datasets in IDB.

Method	$\mathbf{Acc_{EX}}(\uparrow)$	$\mathbf{Acc_{LF}}(\uparrow)$
RAG-to-SQL	0.818	0.677
RAG-to-SQL + Neural Re-ranking	0.828	0.725
RAG-to-SQL + Hybrid Retrieval	0.842	0.733
CBR-to-SQL	0.895	0.729
CBR-to-SQL + Neural Re-ranking	0.893	0.739
CBR-to-SQL + Hybrid Retrieval	0.909	0.775

Appendix

Table 14: Ablation study of RAG-based enhancements on EHRSQL in CDB, applied to both RAG-to-SQL and CBR-to-SQL.

Method	$Agg_{op}(\uparrow)$	$Agg_{col}(\uparrow)$	$Table(\uparrow)$	$Con_{col+op}(\uparrow)$	$Con_{val}(\uparrow)$
Coarse2Fine	0.321	0.313	0.321	0.260	0.214
Seq2Seq	0.978	0.872	0.926	0.471	0.174
SQLNet	0.994	0.939	0.933	0.722	0.080
PtrGen	0.987	0.917	0.944	0.795	0.236
TREQS	0.990	0.912	0.942	0.834	0.694
MedTS	0.994	0.988	0.971	0.893	0.785

Appendix

Table 15:

Acc_{LF}

of clause-specific breakdown matching in CDB.

Method	$Agg_{op}(\uparrow)$	$Agg_{col}(\uparrow)$	$Table(\uparrow)$	$Con_{col+op}(\uparrow)$	$Con_{val}(\uparrow)$
RAG-to-SQL	0.993	0.943	0.948	0.875	0.830
CBR-to-SQL	0.992	0.944	0.980	0.910	0.883

Appendix

Table 16:

Acc_{LF}

of clause-specific breakdown matching in IDB.

Method	$Acc_{EX}(\uparrow)$	$Acc_{LF}(\uparrow)$
RAG-to-SQL	0.841	0.820
CBR-to-SQL	0.862	0.835

Appendix

Table 17: Comparison of methods on

Acc_{EX}

and

Acc_{LF}

metrics in CDB for MIMICSQL using GPT-4.1-mini as the underlying LLM.

Method	$Agg_{op}(\uparrow)$	$Agg_{col(\uparrow)}$	$Table(\uparrow)$	$Con_{col+op}(\uparrow)$	$Con_{val}(\uparrow)$
CBR-to-SQL	0.994	0.957	0.985	0.927	0.900
Replace Proposal	0.993	0.958	0.971	0.909	0.889
No Revision	0.994	0.952	0.947	0.867	0.816

Appendix

Table 18: Logical form accuracy of break-down matching in CDB setup for MIMICSQL, in ablation settings.

Error	Representative Example
Canonical Form Mismatch	Question: How many patients born before the year 2109 were primarily diagnosed for guillain barre syndrome ? CBR-to-SQL (Fail): SELECT COUNT ( DISTINCT DEMOGRAPHIC."SUBJECT_ID" ) FROM DEMOGRAPHIC WHERE DEMOGRAPHIC."DIAGNOSIS" = "GUILLAIN-BARRE SYNDROME" AND DEMOGRAPHIC."DOB_YEAR" < "2109" RAG-to-SQL (Success): SELECT COUNT ( DISTINCT DEMOGRAPHIC."SUBJECT_ID" ) FROM DEMOGRAPHIC WHERE DEMOGRAPHIC."DIAGNOSIS" = "GUILLAIN BARRE SYNDROME" AND DEMOGRAPHIC."DOB_YEAR" < "2109"
Incorrect Source Retrieval	Question: Count the number of patients for whom csf lab test was ordered. CBR-to-SQL (Fail): SELECT COUNT ( DISTINCT DEMOGRAPHIC."SUBJECT_ID" ) FROM DEMOGRAPHIC INNER JOIN LAB on DEMOGRAPHIC.HADM_ID = LAB.HADM_ID WHERE LAB."LABEL" = "Glucose, CSF" RAG-to-SQL (Success): SELECT COUNT ( DISTINCT DEMOGRAPHIC."SUBJECT_ID" ) FROM DEMOGRAPHIC INNER JOIN LAB on DEMOGRAPHIC.HADM_ID = LAB.HADM_ID WHERE LAB."FLUID" = "Cerebrospinal Fluid (CSF)"
Unrecognized Entity	Question: What is the number of patients with replaced route of drug administration? CBR-to-SQL (Fail): SELECT COUNT ( DISTINCT DEMOGRAPHIC."SUBJECT_ID" ) FROM DEMOGRAPHIC INNER JOIN PRESCRIPTIONS ON DEMOGRAPHIC.HADM_ID = PRESCRIPTIONS.HADM_ID WHERE PRESCRIPTIONS."ROUTE" = "REPLACED" RAG-to-SQL (Success): SELECT COUNT ( DISTINCT DEMOGRAPHIC."SUBJECT_ID" ) FROM DEMOGRAPHIC INNER JOIN PRESCRIPTIONS ON DEMOGRAPHIC.HADM_ID = PRESCRIPTIONS.HADM_ID WHERE PRESCRIPTIONS."ROUTE" = "REPLACE"
Incorrect Entity Tagging	Question: What is the maximum number of days of hospital stay for patients with hypertension but not coronary artery disease/cardiac cath ? CBR-to-SQL (Fail): SELECT MAX ( DEMOGRAPHIC."DAYS_STAY" ) FROM DEMOGRAPHIC WHERE DEMOGRAPHIC."DIAGNOSIS" = "HYPERTENSION" AND DEMOGRAPHIC."DIAGNOSIS" <> "CORONARY ARTERY DISEASE\CARDIAC CATH" RAG-to-SQL (Success): SELECT MAX ( DEMOGRAPHIC."DAYS_STAY" ) FROM DEMOGRAPHIC WHERE DEMOGRAPHIC."DIAGNOSIS" = "HYPERTENSION;RULE OUT CORONARY ARTERY DISEASE\CARDIAC CATH"

Appendix

Table 19: Error categories where CBR-to-SQL consistently fails but RAG-to-SQL succeeds in MIMICSQL’s CDB. Each category is illustrated by a representative example. The underlined text highlights the specific part of the problem that CBR-to-SQL fails to address.

Case-Based Reasoning

See all 12 papers

EHR-Complex: Benchmarking Medical Agents for Complex Clinical Reasoning
Jun 22, 2026Yitong Qiao, Lei Liu, Yue Shen +4AI Agent Benchmarks Clinical Reasoning
1Zhejiang University · †Work done during an internship at Ant Group. · 2Ant Group
DecoSearch: Complexity-Aware Routing and Plan-Level Repair for Text-to-SQL
Jun 16, 2026Esteban Schafir, Xu Zheng, Hojat Allah Salehi +4Schema Linking Question Decomposition
Florida International University · NEC-Labs · Singapore Management University
IESR:Efficient MCTS-Based Modular Reasoning for Text-to-SQL with Large Language Models
Feb 5, 2026Tao Liu, Jiafan Lu, Bohan Yu +11Small Language Models Text-to-SQL
Zhengzhou University · Tianjin University · Zhongshan University

Jun 22, 2026J/K move · Enter open · S save

EHR-Complex: Benchmarking Medical Agents for Complex Clinical Reasoning

Yitong Qiao, Lei Liu, Yue Shen, Jian Wang+3

1Zhejiang University · †Work done during an internship at Ant Group. · 2Ant Group

AI Agent Benchmarks Clinical Reasoning Electronic Health Records EHR QA

Open paper AskPDF arXiv

Clinical agents promise to democratize access to electronic health records (EHRs), yet existing benchmarks fail to reflect the complexity of practical EHR analysis, e.g., often operating on idealized, clean EHRs via static SQL generation rather than interactive execution. In this work, we introduce EHR-Complex, a large-scale benchmark designed for interactive clinical database reasoning. Built on the large MIMIC-IV substrate (365K patients, 31 tables, 500M+ records), EHR-Complex comprises about 52K tasks spanning six clinical intents, supporting both patient-level and population-level queries, where each task requires an agent to interact with a sandboxed environment by executing SQL queries or Python code. Notably, EHR-Complex considers the real-world SQL task complexity for longitudinal multi-table aggregation and compositional reasoning, resulting in 31.93 SQL structural components per query on average. Evaluation results on EHR-Complex reveal the clinical difficulty of these EHR reasoning scenarios, with the top-performing model achieving only 62.3% exact-match accuracy. Pass^k consistency drops below 50% for nearly all evaluated models at k=4, exposing broad stochastic fragility. A fine-grained analysis of more than 3,800 failed trajectories for representative LLMs reveals three dominant failure modes: SQL logic errors, medical-code lookup failures, and semantic misunderstandings. EHR-Complex provides a rigorous testbed for clinical agents and highlights remaining gaps in robust reasoning for large-scale EHR analysis.

Abstract

Case-Based Reasoning

Explore similar work

EHR-Complex: Benchmarking Medical Agents for Complex Clinical Reasoning

DecoSearch: Complexity-Aware Routing and Plan-Level Repair for Text-to-SQL

IESR:Efficient MCTS-Based Modular Reasoning for Text-to-SQL with Large Language Models