cs.DBSep 20, 2026

Graph Memory for LLM Agents: At What Cost? A Comparative Evaluation of Query, Ingest, and Update Performance Across Graph Database Engines

Authors: Donald Nguyen, Gurbinder Gill, Hadi Ahmadi, Christopher J. Rossbach

Organizations: Corvic AI Research

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Graph databases are frequently positioned as categorically necessary for connected-data workloads, yet the systems dimension along which they actually differ - query planning, indexing, and data-readiness cost - is rarely isolated from vendor framing. We construct a synthetic, biomedical-shaped property graph (1.02 million nodes, 5.34 million total node and edge rows) and a twenty-query workload spanning neighborhood lookups, bounded paths, set intersections, anti-joins, grouped aggregation, top-k ranking, temporal filters, full scans, and relational joins. We benchmark Corvic AI - a purpose-built columnar query engine underlying Corvic's ontology management layer ("memories")- against seven purpose-built or graph-extension database systems (LoraDB, Ladybug, DuckPGQ, Memgraph, Neo4j, HugeGraph, and FalkorDB) at three graph scales spanning three orders of magnitude. We report query latency geomeans, bulk-ingest throughput, point-update latency, and answer correctness for each system, and we derive a simple total-cost-of-ownership model that expresses the ingest/query trade-off as a function of query volume. Our central finding is that no system in this sample is categorically fastest: a native graph engine (Ladybug) outperforms Corvic AI on narrow, bounded-neighborhood shapes, while Corvic AI is faster on shapes that scan or join a large fraction of the graph, and a system implementing graph query syntax via SQL/PGQ (DuckPGQ) is measurably slower purely due to query-plan choice. The dominant cost differential in our data is not query latency but the cost of making data queryable at all: bulk-ingest throughput varies by three orders of magnitude across engines (5.0k-4.3M rows/s), a gap that a simple crossover-point calculation shows dominates total cost for any workload with fewer than roughly 105 queries per data refresh.

Figures & tables

Figure 1: Query-latency geomean at 1.02M nodes (Eq. 1 ), all eight engines, lower is better. Corvic AI shown in purple.
Figure 2: Query-latency geomean versus graph scale (y log, x categorical), all eight engines. Corvic AI drawn with a heavier line.
Figure 3: Bulk-ingest throughput at 1.02M nodes, rows/s, log scale, higher is better.
Figure 4: Point-update latency at 1.02M nodes, ms, log scale, lower is better. The only workload in this study where Corvic AI does not lead.
Figure 5: Total wall-clock cost (Eq. 5 ) versus number of point-update operations issued after a fresh bulk load, Corvic AI versus Memgraph. Dashed line marks the crossover q∗q^{*} (Eq. 6 ).
Figure 6: Slowdown factor Si,sS_{i,s} relative to Corvic AI (Eq. 3 ), by engine and scale. 1.0 ×\times (Corvic AI’s own row) is the reference.
Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

EngineGM@1KGM@10K[email protected]CorrectIngestPointBatch
 (ms)(ms)(ms)@1.02M(rows/s)upd. (ms)(rows/s)
Corvic AI1.161.042.1920/204,300,0002.4714,800,000
LoraDB0.590.642.6018/205,000247.001,500,000
Ladybug (copy)1.651.564.0820/20476,0000.962,400,000
DuckPGQ1.371.564.1820/203,500,0003.1016,700,000
Memgraph0.690.964.7720/2087,0000.90974,000
Appendix
Table 1: Full per-engine results. Geomean columns per Eq. 1 ; correctness per Eq. 4 ; ingest per Eq. 2 . Corvic AI row highlighted.

Explore similar work

CardsList
Sep 1, 2026

Relational-Core Graph Analytics Querying graphs at SQL scale, and why the node/edge model is a performance tax, not a truer picture of connected data

A durable assumption holds that graph analytics requires a purpose-built graph engine, and that relational systems are ill-suited to connected data. We argue the opposite for the workloads enterprises actually run. A columnar relational engine fronted by a graph query language matches or exceeds native graph engines on analytical graph queries, and - decisively - scales past the point where in-memory graph engines fail. We further argue that the node/edge property graph is not a more faithful model of connected data but a re-encoding of relationships that already exist explicitly in relational tables; reconstructing them at query time is pure overhead. We present ClickGraph and its Databricks-dialect sibling DeltaGraph, systems that translate Cypher directly onto the native relational schema - the tables, columns, and foreign keys as they already exist - and execute in place on ClickHouse, Databricks, or in-process on lakehouse files, with no import and no separate cluster. Because the output is ordinary SQL, an underperforming query is an open optimization surface: it can be rewritten, and the engine itself extended. We support the argument with a peer system's own published benchmark, in which a columnar engine outruns Neo4j by two-to-four orders of magnitude, and with reproducible measurements across the LDBC Social Network Benchmark suite.

Gene Zhang

System: ClickGraph / DeltaGraph

Relational Databases
AskPDF
Mar 2, 2026

GenDB: The Next Generation of Query Processing -- Synthesized, Not Engineered

Traditional query processing relies on engines that are carefully optimized and engineered by many experts. However, new techniques and user requirements evolve rapidly, and existing systems often cannot keep pace. At the same time, these systems are difficult to extend due to their internal complexity, and developing new systems requires substantial engineering effort and cost. In this paper, we argue that recent advances in Large Language Models (LLMs) are starting to shape the next generation of query processing systems. We propose using LLMs to synthesize execution code for each incoming query, instead of continuously building, extending, and maintaining complex query processing engines. As a proof of concept, we present GenDB, an LLM-powered agentic system that generates instance-optimized and customized query execution code tailored to specific data, workloads, and hardware resources. We implemented an early prototype of GenDB that uses Claude Code Agent as the underlying component in the multi-agent system, and we evaluate it on OLAP workloads. We use queries from the well-known TPC-H benchmark and also construct a new benchmark designed to reduce potential data leakage from LLM training data. We compare GenDB with state-of-the-art query engines, including DuckDB, Umbra, MonetDB, ClickHouse, and PostgreSQL. GenDB achieves significantly better performance than these systems. Finally, we discuss the current limitations of GenDB and outline future extensions and related research challenges.

Jiale Lao, Immanuel Trummer

Cornell University

LLM Code Generation
AskPDF
Apr 21, 2026

DW-Bench: Benchmarking LLMs on Data Warehouse Graph Topology Reasoning

This paper introduces DW-Bench, a new benchmark that evaluates large language models (LLMs) on graph-topology reasoning over data warehouse schemas, explicitly integrating both foreign-key (FK) and data-lineage edges. The benchmark comprises 1,046 automatically generated, verifiably correct questions across five schemas. Experiments show that tool-augmented methods substantially outperform static approaches but plateau on hard compositional subtypes.

Ahmed G. A. H Ahmed, C. Okan Sakar

Innosol / Bahçe¸sehir University · Department of Computer Engineering, Bahcesehir University, Turkey

LLM EvaluationLLM Reasoning with GraphsGraph Reasoning
AskPDF