NGDBench: Towards Neural Graph Data Management

Yufei Li; Yisen Gao; Jiaxuan Xiong; Jiaxin Bai; Shijie Zhong; Haoyu Huang; Zhongwei Xie; Hong Ting Tsang

cs.DBFeb 27, 2026

NGDBench: Towards Neural Graph Data Management

Authors: Yufei Li, Yisen Gao, Jiaxuan Xiong, Jiaxin Bai, Shijie Zhong, Haoyu Huang, Zhongwei Xie, Hong Ting Tsang, +1 more

Organizations: HKUST · Beijing Institute of Technology · Hong Kong Baptist University · Guangdong University of Technology

Abstract

Data critical to real-world decision-making is increasingly found within organizations. Such data is heterogeneous, constantly evolving, and only imperfectly captured. However, current data management systems remain largely passive, retrieving what is explicitly stored while offering limited support for uncovering implicit structure or reasoning under noise, incompleteness, and continuous updates. We argue that next-generation data management requires neural capabilities, which can uncover complex latent relationships, distinguish reliable signals from noise, and remain consistent as the underlying data state evolves. To support this direction, we introduce NGDBench, a benchmark across five domains that unifies structured and unstructured sources. NGDBench adopts a graph view because graphs provide a flexible abstraction for modeling complex systems, capturing latent relationships, and subsuming structured formats such as relational tables. Each instance pairs a clean latent graph with a realistically perturbed observed graph. NGDBench supports full Cypher queries and dynamic data management operations. Evaluations of state-of-the-art Text-to-Cypher by LLMs and GraphRAG pipelines reveal that current neural query methods remain sensitive to noise and struggle with dynamic state tracking, highlighting the need for resilient, inference-capable data management. Our code is available at https://github.com/HKUST-KnowComp/NGDBench.

Figures & tables

Benchmark / Dataset	Data Format	Operators	Multi-variable	Aggregation	Dynamic	General Graph Query
			Query	Support	Updates	(e.g., Cypher)
Q2B Ren et al. (2020)	Id-based Triples	EPFO (AND, OR, $\exists$ )	No	No	No	No
LitCQD Demir et al. (2023)	Id-based Triples & Numbers	EPFO + Numeric Operators	No	Yes	No	No
NRN Bai et al. (2023)	Id-based Triples & Numbers	FOL + Numeric Operators	No	No	No	No
$\text{EFO}_{1}$ -CQA Wang et al. ()	Id-based Triples	FOL (AND, OR, NOT $\exists$ )	No	No	No	No
$\text{EFO}_{k}$ -CQA Yin et al. (2025)	Id-based Triples	FOL (AND, OR, NOT $\exists$ )	Yes	No	No	No

Table 1: Comparison of Recent Neural Graph Reasoning Benchmarks. Existing datasets focus on static reasoning with limited logic (FOL/EPFO), whereas our NGDB-Dataset supports full database capabilities including Cypher queries, aggregations, and dynamic updates.

Figure 1: Motivation and Vision: Overcoming the limitations of passive data access through robust graph-based neural data reasoning.

Figure 2: The NGDBench Framework Pipeline.(1) NGDBench unifies diverse data into LPG graphs, (2) constructs paired clean/noisy graphs via perturbation, and samples Cypher queries from template libraries. (3) Systems are evaluated on robust analytical QA and dynamic graph management under noisy observations.

Category		NGD-BI	NGD-Fin	NGD-Prime	NGD-MCP	NGD-Econ	Summary
Graph Structures (Groundtruth)	#Nodes	2,997,352	10,865	129,312	181,689	33,164	–
	#Edges	17,196,776	57,818	8,100,498	351,263	63,757	–
Graph Structures (Perturbed)	#Nodes	2,997,352	10,865	129,312	181,689	33,164	–
	#Edges	17,971,365	60,476	8,465,124	351,263	63,757	–
Query Categories	#Analytical (No Agg)	17,880	4,264	14,043	8,336	1,707	58,321
	#Analytical (Agg.)	9,580	4,412	6,494	4,167	1,050	25,703

Table 2: Detailed statistics of graph structures (groundtruth and perturbed graphs) and query categories across different domains.

Baseline	NGD-Fin					NGD-BI					NGD-Prime
	NoAgg		Agg			NoAgg		Agg			NoAgg		Agg
	Jaccard	F1	MdRE	MSLE	sMAPE	Jaccard	F1	MdRE	MSLE	sMAPE	Jaccard	F1	MdRE	MSLE	sMAPE
Oracle Cypher	0.448	0.471	0.039	0.649	0.119	0.887	0.911	0.355	8.388	0.783	0.736	0.790	0.203	7.990	0.562
Neo4j-Text2Cypher	0.319	0.319	0.660	31.877	0.975	0.212	0.213	——	45.582	1.681	0.208	0.195	——	40.760	1.481
GPT5.1-Codex	0.358	0.359	0.111	12.425	0.394	0.555	0.571	0.987	15.921	1.089	0.510	0.537	0.206	12.250	0.743
DeepseekV3.2	0.345	0.345	0.102	5.636	0.334	0.394	0.404	0.992	20.862	1.182	0.426	0.446	0.409	12.547	0.835

Table 3: Performance comparison across three datasets under non-aggregation and aggregation settings. (“——” indicates values exceeding 10,000. Bold and underlined values denote the best and second-best results respectively.)

Baseline	NGD-Fin			NGD-BI			NGD-Prime
	MLRE	MSLE	sMAPE	MLRE	MSLE	sMAPE	MLRE	MSLE	sMAPE
Neo4j-Text2Cypher	8.065	17.038	1.137	0.759	3.323	0.280	3.667	16.072	0.337
GPT5.1-Codex	3.036	11.727	0.771	0.516	2.327	0.233	3.342	15.138	0.360
DeepseekV3.2	2.778	14.940	0.716	0.455	2.330	0.171	3.665	16.130	0.352
Qwen3-Coder	2.676	4.619	0.610	0.562	2.656	0.239	3.403	15.619	0.309
GraphRAG	5.129	17.510	1.518	9.043	81.106	1.979	6.337	41.776	1.958

Table 4: Performance comparison on management queries across different datasets.

Figure 3: The left figure is the evaluation of boolean queries on three datasets. The right figure is the evaluation of dynamic steps on NGD-Prime, where lower MLRE indicates better performance.

Baseline	NGD-MCP		NGD-Econ
Baseline	Agg $\downarrow$	NoAgg $\uparrow$	Agg $\downarrow$	NoAgg $\uparrow$
GraphRAG	0.169	0.194	0.673	0.242
HippoRAG2	1.050	0.022	2.138	0.040
ToG-RAG	0.359	0.013	3.809	0.212

Table 5: Performance of graph-based RAG methods on unstructured datasets. Agg reports MSLE (

\downarrow

), and NoAgg reports F1 (

\uparrow

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Noise Family	What It Simulates	Typical Manifestation	Default Ratio
Incomplete edges	Missing facts or partial records	Removed relations in transactional, social, or biomedical graphs	0.10
False edges	Spurious joins or extraction mistakes	Incorrect links introduced by integration or inference pipelines	0.15
Relation-type noise	Ambiguous relation semantics	A valid edge assigned the wrong relation label	0.20
Node-type noise	Entity typing ambiguity	A node assigned an incorrect semantic type	0.20
Attribute noise	Local textual or numerical corruption	Typos, OCR artifacts, malformed values, or numeric deviations	0.25

Appendix

Table 6: Perturbation families used in NGDBench and their default injection ratios.

Source	Erroneous Attribute	Incompleteness	Type Inconsistency	Erroneous Links
Real-world datasets	0.55%–26.9% in reported medical settings	Over 70% missing born_in ; about 90% missing ethnicity in Freebase-style settings	8.0%–38.5% across reported semantic domains	10%–25% in reported financial settings
Prior synthetic injection	10%–50%	2.9% and 10.4% in representative studies	10%–50%	10%, 20%, and 30%

Appendix

Table 7: Survey summary of noise characteristics reported in real-world datasets and prior synthetic-noise studies, compiled from Yang et al. (2022) ; Shi and Weninger (2018) ; Guo et al. (2023) ; Qian et al. (2023) ; Bryzgalova et al. (2025) . The ranges show that realistic noise levels vary substantially across domains and noise families.

Category	Core Operators	Function Description
Data Access	Scan , Seek	Entry operators (e.g., node scan, index seek).
Traversal & Matching	Expand , VarLengthExpand , ShortestPath , Repeat , Optional , Anti , Triadic	Topology navigation, variable-length paths, and pattern matching (incl. anti/optional).
Join & Set Operations	HashJoin , Apply , CartesianProduct , Argument , Union	Combine intermediate results (hash / nested loop); includes unions.
Filter & Projection	Filter , Project , Distinct	Predicates, expressions, field selection, and deduplication.
Aggregation & Control	Aggregate , Unwind , Sort , Limit , Skip , Top	Grouping, list flattening, and output control (order/limit).
Data Modification	Create , Merge , Delete , Set , Foreach	Graph mutations: creation, deletion, property updates.

Appendix

Table 8: Classification and descriptions of the supported Cypher core operators serving as the benchmark’s functional primitives.

Baseline	NGD-Fin			NGD-BI			NGD-Prime
	MdRE	MSLE	sMAPE	MdRE	MSLE	sMAPE	MdRE	MSLE	sMAPE
Neo4j-Text2Cypher	0.000	0.055	0.021	0.000	0.000	0.001	0.965	9.495	1.155
GPT5.1-Codex	0.000	0.001	0.001	0.000	3.060	0.615	0.000	11.476	0.541
DeepseekV3.2	0.000	0.001	0.003	0.000	5.148	0.763	0.000	8.115	0.614
Qwen3-Coder	0.000	0.048	0.018	0.000	2.801	0.756	0.000	7.623	0.639
GraphRAG	0.000	0.588	0.904	——	0.480	1.998	0.875	2.400	1.201

Appendix

Table 9: Performance comparison on aggregation queries, where execution on the noisy graph is equivalent to the clean graph.

Baseline	NGD-Fin			NGD-BI			NGD-Prime
	MdRE	MSLE	sMAPE	MdRE	MSLE	sMAPE	MdRE	MSLE	sMAPE
Neo4j-Text2Cypher	0.179	27.846	0.810	——	16.082	16.509	——	46.272	1.541
GPT5.1-Codex	0.131	14.940	0.771	0.992	19.820	1.233	0.237	12.341	0.766
DeepseekV3.2	0.113	7.629	0.451	0.995	25.755	1.312	0.582	13.345	0.875
Qwen3-Coder	0.116	8.231	0.446	1.000	28.878	1.337	0.578	14.185	0.891
GraphRAG	254.333	36.515	1.777	——	61.688	1.979	993.563	47.204	1.814

Appendix

Table 10: Performance comparison on aggregation queries, where execution on the noisy graph is inequivalent to the clean graph.

Baseline	NGD-MCP							NGD-Econ
	Ex.	$\pm$ 1	Null	RMSE	MdRE	MSLE	sMAPE	Ex.	$\pm$ 1	Null	RMSE	MdRE	MSLE	sMAPE
HippoRAG2	0.088	0.250	0.619	53.012	1.000	1.051	1.196	0.012	0.038	0.097	158.308	1.000	2.138	0.926
GraphRAG	0.493	0.624	0.047	1.795	0.000	0.170	0.396	0.241	0.458	0.154	363623.864	0.667	0.673	0.637
TOG	0.266	0.342	0.044	16.169	0.000	0.359	0.380	0.260	0.323	0.125	18730.315	0.000	3.809	0.371

Appendix

Table 11: Performance comparison of RAG methods on aggregation queries. Exact measures exact-match accuracy,

\pm

1 measures the proportion of predictions within one unit of the gold answer, and Null denotes invalid or empty outputs; RMSE, MdRE, MSLE, and sMAPE measure numerical errors. Higher is better for Exact and

\pm

1, while lower is better for Null and all error metrics.

Explore similar work

Feb 25, 2026

NGDB-Zoo: Towards Efficient and Scalable Neural Graph Databases Training

Neural Graph Databases (NGDBs) facilitate complex logical reasoning over incomplete knowledge structures, yet their training efficiency and expressivity are constrained by rigid query-level batching and structure-exclusive embeddings. We present NGDB-Zoo, a unified framework that resolves these bottlenecks by synergizing operator-level training with semantic augmentation. By decoupling logical operators from query topologies, NGDB-Zoo transforms the training loop into a dynamically scheduled data-flow execution, enabling multi-stream parallelism and achieving a

1.8\times

6.8\times

throughput compared to baselines. Furthermore, we formalize a decoupled architecture to integrate high-dimensional semantic priors from Pre-trained Text Encoders (PTEs) without triggering I/O stalls or memory overflows. Extensive evaluations on six benchmarks, including massive graphs like ogbl-wikikg2 and ATLAS-Wiki, demonstrate that NGDB-Zoo maintains high GPU utilization across diverse logical patterns and significantly mitigates representation friction in hybrid neuro-symbolic reasoning.

Zhongwei Xie, Jiaxin Bai, Shujie Liu +5

The Hong Kong University of Science and Technology · Microsoft Research Asia

Graph Reasoning Neuro-Symbolic Reasoning Graph Neural Networks

Ask PDF

Apr 21, 2026

DW-Bench: Benchmarking LLMs on Data Warehouse Graph Topology Reasoning

This paper introduces DW-Bench, a new benchmark that evaluates large language models (LLMs) on graph-topology reasoning over data warehouse schemas, explicitly integrating both foreign-key (FK) and data-lineage edges. The benchmark comprises 1,046 automatically generated, verifiably correct questions across five schemas. Experiments show that tool-augmented methods substantially outperform static approaches but plateau on hard compositional subtypes.

Ahmed G. A. H Ahmed, C. Okan Sakar

Innosol / Bahçe¸sehir University · Department of Computer Engineering, Bahcesehir University, Turkey

LLM Evaluation LLM Reasoning with Graphs Graph Reasoning

Ask PDF

May 6, 2026

A Unified Benchmark for Evaluating Knowledge Graph Construction Methods and Graph Neural Networks

Knowledge graphs automatically constructed from text are increasingly used in real-world applications. However, their inherent noise, fragmentation, and semantic inconsistencies significantly affect the performance of Graph Neural Networks (GNNs) on downstream tasks. Assessing their performance and robustness remains difficult, as it is often unclear whether observed results stem from the learning model or from the quality of the constructed graph itself. In this work, we introduce a dual-purpose benchmark designed to jointly evaluate (i) the performance of GNNs on noisy, text-derived graphs and (ii) the effectiveness of graph construction methods on a downstream task. The benchmark is built in the biomedical domain from a single textual corpus and includes two automatically constructed graphs generated using different extraction methods, alongside a high-quality reference graph curated by experts that serves as an upper performance bound. This design enables controlled comparison of construction methods and systematic evaluation of GNN robustness through semi-supervised node classification. We further provide a standardized, reproducible, and extensible evaluation framework, facilitating the integration of new graph extraction methods and learning models.

Othmane Kabal, Mounira Harzallah, Fabrice Guillet +2

Nantes University, LS2N, Nantes, 44300, France · National Institute of Informatics, Chiyoda-ku, Tokyo, 101-8430, Japan · Institute of Science Tokyo, Tokyo, 152-8550, Japan

KG Construction Graph Neural Networks Node Classification Biomedical KGs

Ask PDF