Case2Flow: Bridging Patient Cases and Guideline Flowcharts through Multimodal Retrieval

Jiale Wei; Yufan Chen; Alexander Jaus; Zdravko Marinov; Julian Friedrich; Simon Reiß; Jens Kleesiek; Rainer Stiefelhagen

cs.CLAug 26, 2026

Case2Flow: Bridging Patient Cases and Guideline Flowcharts through Multimodal Retrieval

Authors: Jiale Wei, Yufan Chen, Alexander Jaus, Zdravko Marinov, Julian Friedrich, Simon Reiß, Jens Kleesiek, Rainer Stiefelhagen

Organizations: Karlsruhe Institute of Technology, Karlsruhe, Germany · Helmholtz Information and Data Science School for Health (HIDSS4Health), Karlsruhe/Heidelberg, Germany · University Hospital Essen, Essen, Germany

Abstract

Medical guidelines encode rich, evidence-based decision logic, yet the specific decision artifact a clinician needs is hard to locate within a guideline, let alone across guidelines covering plausible diseases and treatments. While guideline passages have supported end-to-end question answering, flowcharts remain largely underused in decision support despite their ability to encode actionable clinical pathways. We therefore introduce Case2Flow, a task designed to retrieve the most relevant guideline flowchart for a given patient case from a collection of guideline documents. To support it, we construct FlowAtlas, a curated corpus of 202 flowcharts extracted from 2,080 medical guidelines, together with a pipeline that synthesises 1,911 aligned case-flowchart pairs. Our evaluation of multimodal retrieval methods reveals systematic failure modes, including overreliance on keywords and spurious token-patch matches induced by uninformative background regions in flowcharts. Motivated by this, we propose CRISP, a training-free scoring method that sharpens late-interaction retrieval by suppressing uninformative patches, discounting ambiguous token matches, and incorporating bidirectional query-image alignment. CRISP improves Recall@1 by up to 18.71 percentage points, while a blinded physician assessment on published case narratives provides preliminary feasibility evidence beyond synthetic queries.

Figures & tables

Figure 1: The task Case2Flow : given a textual patient case (history, symptoms, key findings), retrieve the most relevant guideline flowchart from a large cross-provider candidate pool. We build FlowAtlas from $2{,}080$ guidelines by extracting and digitising $202$ flowcharts and synthesising $1{,}911$ aligned case-flowchart pairs through DFS path enumeration, case generation, and clinical rewriting.

Figure 2: CRISP sharpens late-interaction retrieval by reducing the influence of uninformative background patches, subtracting a soft uniform-match baseline from each query token to demote generic matches, and adding a reverse image-to-query coverage term. All three components are training-free and operate only at scoring time.

Method	Size	Type	CDC	WHO	ESMO	Onko.	NCCN	Mixed
RANDOM	–	–	7.69	1.92	1.12	2.08	1.82	0.39
Textual Pipeline-Based Flowchart Retrieval
Tesseract OCR + BM25	–	Original	83.82	79.55	77.65	88.70	50.59	62.55
Tesseract OCR + BM25	–	Rewritten	75.00	72.16	68.94	64.81	N/A	58.55
VLM Captioning + BM25	2B	Original	89.71	84.66	80.56	86.12	71.28	71.83
VLM Captioning + BM25	2B	Rewritten	83.82	76.42	75.00	67.38	N/A	68.62

Table 1: Recall@1 (%) across guideline sources and query types. Per-source : candidate pool from one provider. NCCN : external MedGUIDE Li et al. (2025b) flowcharts. Mixed : all

257

flowcharts pooled. MedGUIDE supplies its own case-chart pairs and has no rewriting pass, so the Rewritten row is not applicable (N/A) in the NCCN column. Best results per column in bold .

	Mixed		NCCN
Method	orig	rew	orig
OCR + BGE-M3	56.74	54.23	57.45
OCR hybrid	70.25	66.90	68.41
Graph-text + BGE-M3	55.50	53.46	59.06
Graph-text hybrid	67.60	65.82	70.12
OCR hybrid $\to$ BGE-reranker	51.34	49.53	54.30

Table 2: Recall@1 (%) of stronger text, reranking, and fusion baselines on the full query sets. Hybrid : equal-weight fusion of BM25 and BGE-M3 scores after per-query

z

-normalisation; graph-text : the digitised flowchart graph serialised instead of OCR text.

	ColPali		ColQwen3
Scoring variant	R@1	MRR	R@1	MRR
MaxSim (baseline)	54.67	66.91	79.91	87.55
+ PW	62.31	72.86	80.77	88.25
+ Adj	57.93	68.70	76.30	84.49
+ Bi	64.85	75.65	82.59	89.74
+ PW + Adj	59.27	69.69	75.24	84.12

Table 3: All combinations of the CRISP components on MedGUIDE (

7{,}747

cases,

55

flowcharts; Recall@1 and MRR in %). PW : patch weighting; Adj : adjusted forward scoring; Bi : bidirectional coverage.

	ColPali		ColQwen3
	orig	rew	orig	rew
Per-query rank change vs. MaxSim (%)
Improved	37.4	39.9	14.5	15.2
Unchanged	57.1	53.8	81.4	80.2
Hurt	5.4	6.3	4.0	4.6
Recovered to rank $1$ †	40.9	39.6	35.8	33.5

Table 4: Behavioural analysis of CRISP on the Mixed pool. Top : per-query rank change of the ground-truth flowchart relative to MaxSim; † Recovery is a fraction of the queries MaxSim ranks below top-

1

, not of all queries. Bottom : the top-

1

errors remaining under CRISP, classified by the deepest level at which the prediction matches the ground truth.

Top- $1$ system	Accept. ( $\geq 2$ )	Applic. ( $=3$ )
ColPali (MaxSim)	9/50 ( $18\%$ )	6/50 ( $12\%$ )
+ CRISP	13/50 ( $26\%$ )	6/50 ( $12\%$ )
ColQwen3 (MaxSim)	22/50 ( $44\%$ )	14/50 ( $28\%$ )
+ CRISP	24/50 ( $48\%$ )	14/50 ( $28\%$ )

Table 5: Blinded physician rating of the top-

1

chart on

50

published PMC-CaseReport narratives Wu et al. (2025) , scored

0

(irrelevant) to

3

(directly applicable).

Figure 3: Per-patch similarity heatmaps on a CDC flowchart (ColPali). Top: standard MaxSim assigns high scores almost uniformly. Bottom: CRISP suppresses background and highlights decision nodes.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Source	Guide.	Flow.	Cases	Path	Words	Jacc.
				(avg)	(o/r)
CDC	776	13	68	5.3	113/102	0.45
WHO	1,159	52	352	6.3	114/104	0.44
ESMO	91	89	792	5.8	103/103	0.44
Onkopedia	54	48	699	7.7	106/105	0.45
FlowAtlas	2,080	202	1,911	6.4	109/104	0.44

Appendix

Table 6: FlowAtlas dataset statistics per guideline source. Guide. : number of crawled guideline PDFs; Flow. : retained flowcharts; Cases : synthesised case-flowchart pairs; Path : mean number of decision nodes per case; Words : mean word count of (case + question) in original / rewritten mode; Jacc. : mean token-level Jaccard similarity between the two modes. MedGUIDE is included for the Mixed and external evaluations but is not part of FlowAtlas.

Figure 4: One representative flowchart from each of the five candidate pools (CDC, ESMO, Onkopedia, WHO and MedGUIDE).

	CDC					WHO					ESMO
Method	R@1	R@3	R@5	R@10	MRR	R@1	R@3	R@5	R@10	MRR	R@1	R@3	R@5	R@10	MRR
Original queries
OCR + BM25	83.82	95.59	100.00	100.00	90.00	79.55	91.48	93.18	93.18	85.52	77.65	87.75	89.14	91.16	83.12
VLM Cap. + BM25	89.71	94.12	100.00	100.00	92.89	84.66	95.45	96.59	96.59	90.10	80.56	90.03	90.28	91.54	85.30
CLIP	45.59	83.82	92.65	98.53	65.68	50.57	76.42	88.92	95.74	65.94	17.68	26.89	32.20	43.06	26.08
Qwen3-VL-Emb	80.88	98.53	100.00	100.00	89.58	85.80	95.17	96.59	96.59	90.47	82.83	91.54	92.42	93.43	87.38

Appendix

Table 7: Per-source Recall@

\{1,3,5,10\}

and MRR (%) on CDC, WHO and ESMO. Best score per column among the direct-image rows in bold . Continued in Table 8 .

	Onkopedia					MedGUIDE
Method	R@1	R@3	R@5	R@10	MRR	R@1	R@3	R@5	R@10	MRR
Original queries
OCR + BM25	88.70	93.85	96.42	98.14	91.92	50.59	68.40	76.20	88.00	62.49
VLM Cap. + BM25	86.12	92.27	93.56	95.28	89.69	71.28	89.14	92.89	96.48	81.02
CLIP	13.16	21.89	28.18	37.63	22.29	18.86	35.52	46.44	62.68	32.46
Qwen3-VL-Emb	92.70	96.57	98.86	100.00	95.16	79.24	94.45	98.36	99.56	87.10

Appendix

Table 8: Per-source Recall@

\{1,3,5,10\}

and MRR (%) on Onkopedia and the MedGUIDE pool. MedGUIDE does not provide separate rewritten cases, so its rewritten rows are marked N/A. Continued from Table 7 .

	Original queries					Rewritten queries
Method	R@1	R@3	R@5	R@10	MRR	R@1	R@3	R@5	R@10	MRR
OCR + BM25	62.55	77.74	83.23	88.75	71.71	58.55	74.64	80.90	86.85	68.40
VLM Cap. + BM25	71.83	87.71	91.20	93.86	80.50	68.62	85.69	89.86	92.97	78.07
CLIP	15.17	28.32	35.78	46.02	25.48	14.73	27.57	34.96	45.05	24.85
Qwen3-VL-Emb	74.39	90.33	93.95	96.75	82.90	73.05	89.98	93.69	96.57	82.03
ColPali	55.48	73.75	80.61	86.43	66.22	52.09	71.00	78.44	84.71	63.36

Appendix

Table 9: Recall@

\{1,3,5,10\}

and MRR (%) on the Mixed pool (

257

candidates,

9{,}658

queries per mode).

Figure 5: Per-patch decomposition of CRISP; each panel isolates one ingredient from the ablation table ( Table 3 ). (a) MaxSim : $\max_{t}\mathbf{q}_{t}^{\top}\mathbf{p}_{s}$ saturates over whitespace. (b) Patch weights : $w_{s}$ from Eq. ( 1 ); query-independent, concentrated on decision and outcome boxes. (c) Patch weights + Adjusted : per-token contrast $a_{t}$ from Eq. ( 2 ) credited to its $\arg\max_{s}$ patch; sparse by construction. (d) Complete CRISP : image-side proxy $w_{s}\cdot\max_{t}\mathbf{q}_{t}^{\top}\mathbf{p}_{s}$ for Eq. ( 3 ); the additive reverse term contributes a scalar per candidate with no separate per-patch signature. Each panel is normalised to its minimum and maximum values independently.

Biomedical IR

Biomedical information retrieval finds and ranks relevant evidence in scientific literature, clinical records, databases, and biomedical knowledge resources.

See all 43 papers

Guideline2Graph: Profile-Aware Multimodal Parsing for Executable Clinical Decision Graphs
Apr 2, 2026Onur Selim Kilic, Yeti Z. Gurbuz, Cem O. Yaldiz +4Clinical Decision Support Clinical Information Extraction
Georgia Institute of Technology · MetaDialog · Infuse Inc.
MED-COPILOT: A Medical Assistant Powered by GraphRAG and Similar Patient Case Retrieval
Feb 28, 2026Shuheng Chen, Namratha Patil, Haonan Pan +4Case-Based Reasoning Clinical Decision Support
University of Southern California
From PDF to Evidence: Structure-Aware Retrieval for Clinical Practice Guidelines
Sep 27, 2026Xingyu Lin, Dehui DuHealthcare Information Retrieval
Software Engineering Institute, East China Normal University Shanghai, China

Apr 2, 2026J/K move · Enter open · S save

Guideline2Graph: Profile-Aware Multimodal Parsing for Executable Clinical Decision Graphs

Onur Selim Kilic, Yeti Z. Gurbuz, Cem O. Yaldiz, Afra Nawar+3

Georgia Institute of Technology · MetaDialog · Infuse Inc.

Clinical Decision Support Clinical Information Extraction

Open paper AskPDF arXiv

Clinical practice guidelines are long, multimodal documents whose branching recommendations are difficult to convert into executable clinical decision support (CDS), and one-shot parsing often breaks cross-page continuity. Recent LLM/VLM extractors are mostly local or text-centric, under-specifying section interfaces and failing to consolidate cross-page control flow across full documents into one coherent decision graph. We present a decomposition-first pipeline that converts full-guideline evidence into an executable clinical decision graph through topology-aware chunking, interface-constrained chunk graph generation, and provenance-preserving global aggregation. Rather than relying on single-pass generation, the pipeline uses explicit entry/terminal interfaces and semantic deduplication to preserve cross-page continuity while keeping the induced control flow auditable and structurally consistent. We evaluate on an adjudicated prostate-guideline benchmark with matched inputs and the same underlying VLM backbone across compared methods. On the complete merged graph, our approach improves edge and triplet precision/recall from

19.6\%/16.1\%

in existing models to

69.0\%/87.5\%

, while node recall rises from

78.1\%

93.8\%

. These results support decomposition-first, auditable guideline-to-CDS conversion on this benchmark, while current evidence remains limited to one adjudicated prostate guideline and motivates broader multi-guideline validation.

Abstract

Biomedical IR

Explore similar work

Guideline2Graph: Profile-Aware Multimodal Parsing for Executable Clinical Decision Graphs

MED-COPILOT: A Medical Assistant Powered by GraphRAG and Similar Patient Case Retrieval

From PDF to Evidence: Structure-Aware Retrieval for Clinical Practice Guidelines