Clinical Knowledge Graphs for Chest X-Ray Device Reasoning

Harshil Lodhiya

cs.AIAug 25, 2026

Clinical Knowledge Graphs for Chest X-Ray Device Reasoning

Authors: Harshil Lodhiya

Organizations: Sliced Health

Abstract

Chest radiographs are routinely used to verify the position of catheters, tubes, and other support devices. Existing image models often return labels or segmentations, while report-processing systems structure text without access to image geometry. We present an uncertainty-aware clinical knowledge graph that represents device instances, tip estimates, placement assessments, provenance, report events, and temporal links as separate but connected evidence. We evaluate the implemented visual graph layer using saved predictions from the complete RANZCR CLiP test archive, comprising 30,083 studies from 3,255 patients across five non-overlapping outer folds. The graph builder materializes 914,632 B7 evidence nodes and 884,549 typed relationships. All 118,647 B7 predicted-device nodes retain tip covariance, placement probabilities, fragment provenance, and fragment counts, whereas the direct B2 baseline retains none of these fields. We further define typed data contracts, uncertainty representations, abstention rules, report-image grounding, and longitudinal query mechanisms for extending the graph to report-bearing cohorts. The reported graph-materialization analysis is post-hoc descriptive and does not establish report grounding, longitudinal performance, or clinical utility. It demonstrates a reproducible foundation for evidence-preserving AI reasoning over chest X-ray device assessments.

Figures & tables

Plane	Responsibility	Representative outputs
Visual extraction	Detect and assess device instances in chest X-rays	Device fragments, instance paths, tip mean, tip covariance, placement label
Text extraction	Extract report findings, recommendations, and events	Report spans, device mentions, action recommendations, temporal cues
Graph construction	Link visual, textual, and longitudinal entities	Device nodes, study nodes, report nodes, procedure nodes, temporal edges
Reasoning	Answer clinical and quality-improvement queries	Persistence, correction, uncertainty triage, report-image conflict

Table 1: Core planes in the uncertainty-aware clinical knowledge graph system.

Panel of Figure 1 — Figure 1: Architecture and deployment flow. Offline services preserve visual, text, and context observations before materializing a versioned evidence graph and immutable provenance store. The online path retrieves evidence from a frozen graph snapshot, applies abstention and conflict policies, and presents source-linked results for human review. The visual service is evaluated in this paper; report and longitudinal services require a report-bearing cohort.

Assertion	Required evidence	Mandatory provenance attributes
Device instance	Fragment set or instance mask; family distribution	image UID, model version, coordinate frame, confidence
Tip estimate	Mean location and covariance	contributing fragments, fusion rule, calibration set/version
Report event	Exact source span; assertion type	report UID, report version, extractor version, confidence
Grounded report finding	Text event and candidate device instance	alignment features, threshold, abstention state
Persistent malposition	Two assessments and cross-study link	study times, linkage probability, query version
Follow-up completed	Recommendation, action, and follow-up evidence	source spans/records, temporal window, matching rule

Table 2: Minimum evidence contract for high-value graph assertions. “Required” denotes attributes that must be present before a system may expose the assertion as an automated finding.

Metric	B2 baseline	B7 UCompCXR
Matched instances / false negatives / false positives	11,660 / 5,649 / 93,992	14,721 / 2,588 / 23,854
Detection sensitivity	0.674	0.851
Median tip error, % image diagonal	0.883	1.725
Materialized graph nodes / edges	718,351 / 688,268	914,632 / 884,549
Predicted-device nodes	344,134	118,647
Median nodes per study [P10, P90]	23 [9, 41]	25 [19, 52]

Table 3: Visual prediction and graph-materialization results on the saved RANZCR CLiP test archive. B2 is the direct multi-task baseline and B7 is UCompCXR. Counts are exact across five non-overlapping outer folds; this post-hoc analysis is descriptive rather than confirmatory.

Figure 2: Materialized evidence graphs from saved RANZCR CLiP test predictions. (A) B7 adds fragment-proposal and placement-assessment evidence while maintaining a comparable study-level graph size. (B) All B7 device nodes contain the four fields required for auditable visual evidence; B2 contains none. This is a post-hoc descriptive analysis of existing test predictions.

Service	Responsibility	Failure behavior
Ingestion	Validate identifiers, time zones, de-identification, and source versions	Quarantine invalid records; do not create partial patient links
Visual inference	Emit device observations and calibration attributes	Emit “model unavailable” or “indeterminate”; preserve no fabricated output
Text extraction	Emit span-grounded events and assertion attributes	Retain raw report; mark extraction unavailable or low confidence
Graph builder	Validate contracts, produce edges, retain provenance	Reject invalid edge; retain upstream observations for repair
Reasoning API	Run versioned graph queries and return evidence paths	Return incomplete-evidence state rather than an inferred clinical answer
Review interface	Present evidence and accept adjudications	Store adjudication as a new, attributed observation

Table 4: Reference service boundaries and failure handling.

Component	Task	Metrics
Visual detection	Device instance detection and family classification	device F1, false positives per image, family accuracy
Tip localization	Tip mean and covariance quality	mean error, catastrophic error rate, 95% coverage, calibration error
Placement assessment	Per-device normal/borderline/abnormal status	per-device AUROC, F1, sensitivity at fixed specificity
Report extraction	Findings, recommendations, uncertainty, temporal cues	entity F1, relation F1, span support, negation accuracy
Grounding	Link report events to device instances	grounding accuracy, top-k accuracy, abstention precision
Longitudinal graph	Link devices and events across studies	temporal edge F1, trajectory accuracy, correction-path accuracy

Table 5: Evaluation matrix for uncertainty-aware clinical knowledge graphs.

Explore similar work

CheXthought: A global multimodal dataset of clinical chain-of-thought reasoning and visual attention for chest X-ray interpretation
Apr 29, 2026Sonali Sharma, Jin Long, George Shih +7Chest X-Ray Classification Medical Image Analysis
Center for Artificial Intelligence in Medicine and Imaging, Stanford University, Stanford, CA, USA · Department of Radiology, Stanford University, Stanford, CA, USA · Department of Pediatrics, Stanford University School of Medicine, Stanford, California, USA +6
Vision-language models for chest radiography do not always need the image
Jun 16, 2026Mahshad Lotfinia, Sebastian Ziegelmayer, Lisa Adams +4VLM Evaluation Medical VQA
Pattern Recognition Lab, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany · Department of Diagnostic and Interventional Radiology, TUM University Clinic, School of Medicine and Health, Klinikum rechts der Isar, Technical University of Munich, Munich, Germany · Lab for AI in Medicine, RWTH Aachen University, Aachen, Germany +1
Rethinking Visual Attribution for Chest X-ray Reasoning in Large Vision Language Models
May 19, 2026Guangzhi Xiong, Qiao Jin, Sanchit Sinha +2Medical VQA VLM Interpretability
University of Virginia · National Institutes of Health

Apr 29, 2026, cs.CVJ/K move · Enter open · S save

CheXthought: A global multimodal dataset of clinical chain-of-thought reasoning and visual attention for chest X-ray interpretation

Sonali Sharma, Jin Long, George Shih, Sarah Eid+6

Center for Artificial Intelligence in Medicine and Imaging, Stanford University, Stanford, CA, USA · Department of Radiology, Stanford University, Stanford, CA, USA · Department of Pediatrics, Stanford University School of Medicine, Stanford, California, USA+6

Chest X-Ray Classification Medical Image Analysis Medical VLMs Clinical Reasoning Multimodal CoT Reasoning

Open paper AskPDF arXiv

Chest X-ray interpretation is one of the most frequently performed diagnostic tasks in medicine and a primary target for AI development, yet current vision-language models are primarily trained on datasets of paired images and reports, not the cognitive processes and visual attention that underlie clinical reasoning. Here, we present CheXthought, a global, multimodal resource containing 103,592 chain-of-thought reasoning traces and 6,609,082 synchronized visual attention annotations across 50,312 multi-read chest X-rays from 501 radiologists in 71 countries. Our analysis reveals clinical reasoning patterns in how experts deploy distinct visual search strategies, integrate clinical context, and communicate uncertainty. We demonstrate the clinical utility of CheXthought across four dimensions. First, CheXthought reasoning significantly outperforms state-of-the-art vision-language model chain-of-thought in factual accuracy and spatial grounding. Second, visual attention data used as an inference-time hint recovers missed findings and significantly reduces hallucinations. Third, vision-language models trained on CheXthought data achieve significantly stronger pathology classification, visual faithfulness, temporal reasoning and uncertainty communication. Fourth, leveraging CheXthought's multi-reader annotations, we predict both human-human and human-AI disagreement directly from an image, enabling transparent communication of case difficulty, uncertainty and model reliability. These findings establish CheXthought as a resource for advancing multimodal clinical reasoning and the development of more transparent, interpretable vision-language models.

Abstract

Explore similar work

CheXthought: A global multimodal dataset of clinical chain-of-thought reasoning and visual attention for chest X-ray interpretation

Vision-language models for chest radiography do not always need the image

Rethinking Visual Attribution for Chest X-ray Reasoning in Large Vision Language Models