LLM Reasoning with Graphs

The field studies how large language models can use graphs to represent relationships, retrieve evidence, and organize multi-step reasoning, especially when answers depend on several connected facts. Common approaches include retrieving paths or subgraphs from knowledge graphs for tasks such as WebQSP and CWQ question answering, giving models tools to navigate graphs, and integrating graph neural networks, symbolic rules, or executable code with language-model reasoning. Other work represents reasoning itself as a chain, tree, or directed graph, and trains models with supervised traces or reinforcement learning to explore paths and verify intermediate steps. These methods can improve multi-hop accuracy and make evidence easier to inspect, but benchmarks also show that graph information is not always used correctly; handling incomplete or noisy evidence, scaling to larger graphs, and demonstrating genuine structural understanding remain open challenges.