BEACON-SP: Ontology-Grounded GraphRAG Framework for Clinical Suicide Risk Assessment

Kemal Davaslioglu; Nathan Conger; Sastry Kompella; Yalin E. Sagduyu; Nathaniel D. Bastian

cs.AIOct 6, 2026

BEACON-SP: Ontology-Grounded GraphRAG Framework for Clinical Suicide Risk Assessment

Authors: Kemal Davaslioglu, Nathan Conger, Sastry Kompella, Yalin E. Sagduyu, Nathaniel D. Bastian

Organizations: Nexcepta, Gaithersburg, MD, USA · United States Military Academy, West Point, NY, USA

Abstract

We present BEACON-SP, an ontology-grounded Graph Retrieval-Augmented Generation (GraphRAG) framework for clinician-facing decision support in behavioral health settings such as suicide prevention, where effective assessment requires integrating heterogeneous clinical, behavioral, social, and temporal evidence. BEACON-SP combines patient knowledge graphs with ontology-guided retrieval to support multi-hop reasoning across diagnoses, medications, risk and protective factors, life events, and temporal relationships. The framework is enabled by a comprehensive suicide prevention ontology that integrates the Three-Step Theory, the Integrated Motivational-Volitional Model, and the Suicide Social Determinants of Health Ontology into a unified representation of patient risk factors. We construct ontology-grounded patient knowledge graphs and evaluate BEACON-SP for clinician-facing question answering. Compared with a vector-based retrieval-augmented generation (RAG) baseline on a 1,500-query benchmark spanning 15 clinical categories and 100 patients, BEACON-SP improves completeness, clinical relevance, and evidence grounding under a corrected comparative evaluation protocol, with a small gain on factual accuracy. In paired criterion-level comparisons, GraphRAG is preferred in 76.4% of cases. These results demonstrate the potential of ontology-guided GraphRAG to provide structured, contextualized patient evidence for clinical decision support.

Figures & tables

Fig. 1 : BEACON-SP Ontology components.

Fig. 2 : BEACON-SP GraphRAG system architecture.

Panel of Figure 3 — Fig. 3 : GraphRAG and Vector-based RAG responses to psychiatric history and progression.

System	Factual Accuracy	Comple- teness	Clinical Relevance	Evidence Grounding
RAG	0.424	0.348	0.193	0.432
GraphRAG	0.231	0.342	0.359	0.200

TABLE I : Overall mean absolute judge scores (1,500 queries, 100 patients, 15 tasks). Best score per criterion in bold.

Outcome	Count / Rate
GraphRAG wins	4,482
RAG wins	238
Ties	1,144
Skipped	136
Total comparisons	5,864
GraphRAG win rate	76.4%

TABLE II : Paired comparative evaluation (1,500 queries

\times

4 criteria, minus skipped pairs).

Criterion	GR Wins	RAG Wins	Ties	GR Win Rate
Factual Accuracy	716	70	680	48.8%
Completeness	1,231	100	135	84.0%
Clinical Relevance	1,377	23	66	93.9%
Evidence Grounding	1,158	45	263	79.0%

TABLE III : GraphRAG win rate by criterion under the comparative judge.

	RAG				GraphRAG
Task	Factual Accuracy	Comple- teness	Clinical Relevance	Evidence Grounding	Factual Accuracy	Comple- teness	Clinical Relevance	Evidence Grounding
Comorbidity	0.060	0.070	0.000	0.035	0.260	0.260	0.485	0.195
Diagnosis	0.315	0.310	0.320	0.280	0.140	0.305	0.360	0.135
Lethal Means Assessment	0.625	0.620	0.145	0.520	0.425	0.390	0.220	0.255
Medication	0.580	0.620	0.565	0.640	0.240	0.390	0.495	0.210
Ontology	0.000	0.045	0.000	0.000	0.030	0.155	0.005	0.010

TABLE IV : Mean absolute judge scores per task (100 patients). Best score per RAG–GraphRAG pair bolded.

Task	GraphRAG Win Rate
Risk Factors	91.5%
Ontology	89.4%
Comorbidity	85.1%
Risk Construct Inference	84.7%
Protective vs. Risk Balance	84.5%
Lethal Means Assessment	83.0%

TABLE V : GraphRAG paired win rate by task using the comparative judge (100 patients).

Explore similar work

Explainable Suicide Risk Assessment on Social Media with Multi-Task QLoRA
Sep 30, 2026Xuan Zhong Feng, Geoffrey Martin, Hexin Dong +1Suicide Risk Risk Stratification
Department of Population Health Sciences, Weill Cornell Medicine, New York, NY, USA · Systems Engineering, Cornell University, New York, NY, USA
Bag of Tricks or Bag of Myths? Reducing Modeling Complexity with Task Knowledge in Explainable Suicide Risk Assessment
Sep 7, 2026Shlok Shelat, Shrey Salvi, Souvik Roy +2Suicide Risk Risk Stratification
Indian AI Research Organisation · Ahmedabad University · University of Maryland, Baltimore County
ChronoMedKG: A Temporally-Grounded Biomedical Knowledge Graph and Benchmark for Clinical Reasoning
May 21, 2026Md Shamim Ahmed, Farzaneh Firoozbakht, Lukas Galke Poech +2Biomedical Knowledge Graph Clinical Reasoning Training
University of Southern Denmark · University of Hamburg

Sep 30, 2026, cs.CLJ/K move · Enter open · S save

Explainable Suicide Risk Assessment on Social Media with Multi-Task QLoRA

Xuan Zhong Feng, Geoffrey Martin, Hexin Dong, Yifan Peng

Department of Population Health Sciences, Weill Cornell Medicine, New York, NY, USA · Systems Engineering, Cornell University, New York, NY, USA

Suicide Risk Risk Stratification Large Language Model Safety Risk Social Media Exploratory Factor Analysis+1

Open paper AskPDF arXiv

Explainable suicide-risk assessment requires models not only to estimate risk severity, but also to identify supporting language and the risk and protective factors expressed in a post. We present our system for the IEEE BigData 2026 Cup on Explainable Suicide Risk Assessment on Social Media, which addresses three tasks: risk-level classification, evidence phrase extraction, and multi-label factor identification. Our approach adapts Qwen2.5-Instruct models using quantized low-rank adaptation (QLoRA) and an answer-masked causal language-model objective. We jointly train across all three tasks for risk classification, jointly train on Tasks~~1a and 1b for evidence extraction, and adapt Task~~2 separately for factor identification. We also tailor aggregation to each output: we average risk-level probabilities from the 32B and 72B models, combine evidence phrases through cross-fold consensus, and calibrate factor-specific decisions through rate matching based on out-of-fold operating points. On the official leaderboard, the final system achieved a composite score of 0.7738, with 0.8089 on Task~~1 and 0.6919 on Task~~2. Across the evaluated configurations, three-task training performed best for Task~~1a, joint training on Tasks~~1a and 1b performed best for Task~~1b, and task-specific training performed best for Task~~2. Probability averaging further improved Task~1a when component models had complementary errors. These findings highlight the value of tailoring both training objectives and aggregation strategies to the output structure of each task within a unified language-model framework.

Abstract

Explore similar work

Explainable Suicide Risk Assessment on Social Media with Multi-Task QLoRA

Bag of Tricks or Bag of Myths? Reducing Modeling Complexity with Task Knowledge in Explainable Suicide Risk Assessment

ChronoMedKG: A Temporally-Grounded Biomedical Knowledge Graph and Benchmark for Clinical Reasoning