Structure-Grounded Knowledge Retrieval via Code Dependencies for Multi-Step Data Reasoning

Xinyi Huang

cs.CLApr 12, 2026

Structure-Grounded Knowledge Retrieval via Code Dependencies for Multi-Step Data Reasoning

Authors: Xinyi Huang

Organizations: Simon Fraser University

Abstract

Selecting the right knowledge is critical when using large language models (LLMs) to solve domain-specific data analysis tasks. However, most retrieval-augmented approaches rely primarily on lexical or embedding similarity, which is often a weak proxy for the task-critical knowledge needed for multi-step reasoning. In many such tasks, the relevant knowledge is not merely textually related to the query, but is instead grounded in executable code and the dependency structure through which computations are carried out. To address this mismatch, we propose SGKR (Structure-Grounded Knowledge Retrieval), a retrieval framework that organizes domain knowledge with a graph induced by function-call dependencies. Given a question, SGKR extracts semantic input and output tags, identifies dependency paths connecting them, and constructs a task-relevant subgraph. The associated knowledge and corresponding function implementations are then assembled as a structured context for LLM-based code generation. Experiments on multi-step data analysis benchmarks show that SGKR consistently improves solution correctness over no-retrieval and similarity-based retrieval baselines for both vanilla LLMs and coding agents.

Figures & tables

Figure 1: Comparison among (1) embedding-based similarity between the new task and candidate code snippets, (2) LLM (GPT-5.2) predictions of whether code snippets are required, and (3) ground-truth annotations indicating whether code snippets are actually needed. For similarity computation, we use the openai/text-embedding-3-large model, embedding both function definitions and their natural-language summaries.

Figure 2: Framework of SGKR . Framework of the structure-grounded code retrieval system. In the offline phase, code is parsed via AST to build a semantic graph and a function-call dependency graph, while external knowledge is inserted into relevant functions as comments. Extracted sementic input–output (I/O) tags are added as nodes to augment the graph. At inference time, I/O tags from a query guide the retrieval of relevant subgraphs through dependency paths, and the retrieved functions and knowledge are provided to LLMs for reasoning.

Method	DABstep-hard		ConvFinQA-hard		FinQA-hard
Method	Acc.	Avg. Nodes	Acc.	Avg. Nodes	Acc.	Avg. Nodes
vanilla	6.88%	–	52.00%	–	47.50%	–
few-shot	11.64%	–	63.00%	–	57.78%	–
bge-large-en-v1.5-RAG	10.58%	5	71.33%	2	63.61%	2
GraphSAGE-RAG	6.35%	5	71.67%	2	62.78%	2
CodeBERT-RAG	8.73%	5	68.33%	2	61.94%	2

Table 1: Accuracy comparison across datasets and average number of retrieved nodes (semantic I/O nodes are not counted).

Method	Acc.
No Retrieval	6.88%
SGKR w/ Fixed Nodes	11.64%
SGKR w/o Knowledge	8.73%
SGKR w/o Merging	12.43%
Full SGKR	24.34%

Table 2: Ablation study of SGKR . Each variant removes one key component of the proposed framework.

knowledge & function	Ground Truth	SGKR	GraphSAGE-RAG	CodeBERT-RAG	dense-RAG
rule_applies	needed	✓	✓	✓
find_all_mccs	needed	✓
sum_fee	needed	✓
most_expensive	needed	✓	✓		✓
compute_fee	needed	✓	✓
average_fee	unneeded		✓	✓	✓

Table 3: Retrieved knowledge-code nodes by different methods. ✓means the knowledge is selected by the method.

Knowledge Augmentation for Language Models

Knowledge augmentation for language models adds external or newly learned information to improve factuality, reasoning, and domain performance.

See all 171 papers

DomAgent: Leveraging Knowledge Graphs and Case-Based Reasoning for Domain-Specific Code Generation
Mar 22, 2026Shuai Wang, Dhasarathy Parthasarathy, Robert Feldt +1Case-Based Reasoning Code Generation
Chalmers University of Technology, Gothenburg, Sweden · Volvo Group, Gothenburg, Sweden
GraphSkill: Documentation-Guided Hierarchical Retrieval-Augmented Coding for Complex Graph Reasoning
Feb 20, 2026Fali Wang, Chenglin Weng, Xianren Zhang +3LLM Reasoning with Graphs Graph Reasoning
The Pennsylvania State University, University Park, Pennsylvania, USA · Michigan State University, East Lansing, Michigan, USA
Code-on-Graph: Iterative Programmatic Reasoning via Large Language Models on Knowledge Graphs
Jun 2, 2026Weiwei Ding, Zixuan Li, Long Bai +7LLM Grounding LLM Reasoning with Graphs
1Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences · 2Shandong University · 3Shandong University-Weihai Research Institute of Industrial Technology

Mar 22, 2026J/K move · Enter open · S save

DomAgent: Leveraging Knowledge Graphs and Case-Based Reasoning for Domain-Specific Code Generation

Shuai Wang, Dhasarathy Parthasarathy, Robert Feldt, Yinan Yu

Chalmers University of Technology, Gothenburg, Sweden · Volvo Group, Gothenburg, Sweden

Case-Based Reasoning Code Generation Agentic Code Generation Domain Adaptation for LLMs Knowledge Augmentation for Language Models LLM Reasoning with Graphs

Open paper AskPDF arXiv

Large language models (LLMs) have shown impressive capabilities in code generation. However, because most LLMs are trained on public domain corpora, directly applying them to real-world software development often yields low success rates, as these scenarios frequently require domain-specific knowledge. In particular, domain-specific tasks usually demand highly specialized solutions, which are often underrepresented or entirely absent in the training data of generic LLMs. To address this challenge, we propose DomAgent, an autonomous coding agent that bridges this gap by enabling LLMs to generate domain-adapted code through structured reasoning and targeted retrieval. A core component of DomAgent is DomRetriever, a novel retrieval module that emulates how humans learn domain-specific knowledge, by combining conceptual understanding with experiential examples. It dynamically integrates top-down knowledge-graph reasoning with bottom-up case-based reasoning, enabling iterative retrieval and synthesis of structured knowledge and representative cases to ensure contextual relevance and broad task coverage. DomRetriever can operate as part of DomAgent or independently with any LLM for flexible domain adaptation. We evaluate DomAgent on an open benchmark dataset in the data science domain (DS-1000) and further apply it to real-world truck software development tasks. Experimental results show that DomAgent significantly enhances domain-specific code generation, enabling small open-source models to close much of the performance gap with large proprietary LLMs in complex, real-world applications. The code is available at: https://github.com/Wangshuaiia/DomAgent.

Abstract

Knowledge Augmentation for Language Models

Explore similar work

DomAgent: Leveraging Knowledge Graphs and Case-Based Reasoning for Domain-Specific Code Generation

GraphSkill: Documentation-Guided Hierarchical Retrieval-Augmented Coding for Complex Graph Reasoning

Code-on-Graph: Iterative Programmatic Reasoning via Large Language Models on Knowledge Graphs