FRACTION: Fraud Explanation Generation via Agentic Graph Context Navigation
Organizations: National University of Singapore · Hefei University of Technology · ByteDance Inc. · Virginia Tech
Abstract
Graph fraud explanation generation emerges as an important task, which requires jointly leveraging graph structure and content information to produce human-readable explanations for a given risk label. An intuitive approach is to adapt existing graph-enhanced large language models (LLMs), which inject graph structure into LLMs via graph-conditioned instruction tuning or structure-aware embeddings. However, existing methods face two key challenges in graph fraud explanation generation: a lack of evaluation protocols tailored to explanation quality, and a mismatch between the granularity of graph information they provide and what explanation generation requires. In this paper, we propose FRaud Explanation Generation via Agentic Graph Context NavigaTION (FRACTION), a framework for graph fraud explanation generation. To evaluate explanation quality, we introduce a task-specific rubric that assesses faithfulness, graph grounding, helpfulness, consistency, and conciseness, and validate it against human judgments. To focus on fine-grained details of relevant neighbors without reading the entire neighborhood, we equip a language model with graph tools that progressively disclose neighbor records. To train our graph agent, we propose a rubric-guided trajectory distillation strategy through which the agent learns both tool-use behavior and structured explanation generation. Experimental results on three fraud datasets demonstrate consistent improvements over state-of-the-art baselines under a human-aligned rubric.
Figures & tables
| Dataset | Node Type | # Nodes | # Edges | # Edge |
|---|---|---|---|---|
| Types | ||||
| YelpReviews | Service Review | 67,395 | 17,486,608 | 3 |
| AmazonVideo | Product Review | 37,126 | 9,883,406 | 3 |
| AdGov | Company Snapshot | 2,126,463 | 24,240,638 | 39 |
| Method | Model | Final score | Pass rate | Faithfulness | Graph grounding | Helpfulness | Consistency | Conciseness |
|---|---|---|---|---|---|---|---|---|
| Dataset: YelpReviews | ||||||||
| Zero-shot target-only | Qwen3.6-27B | 2.0615 | 0.00% | 1.9300 | 1.0000 | 2.0425 | 3.9425 | 4.6225 |
| Zero-shot target-only | DeepSeek-v4-Pro | 2.2420 | 0.00% | 2.1200 | 1.0000 | 2.3450 | 4.2375 | 4.8000 |
| TLG | Qwen3.6-27B | 1.9266 | 0.00% | 1.7075 | 1.0025 | 1.8975 | 3.7025 | 4.6750 |
| TLG | DeepSeek-v4-Pro | 1.9545 | 0.00% | 1.7500 | 1.0025 | 1.9225 | 3.7825 | 4.6500 |
| GraphText | Qwen3.6-27B | 2.0855 | 1.75% | 1.8800 | 1.1625 | 2.0800 | 3.8500 | 4.6375 |
| Dataset | Spearman | -value | Mean diff. | MAE | RMSE | Pass agree. | |
|---|---|---|---|---|---|---|---|
| YelpReviews | 20 | 0.7205 | 0.0003 | 0.0500 | 0.2500 | 0.3256 | 95.00% |
| AmazonVideo | 20 | 0.4558 | 0.0434 | 0.2475 | 0.3625 | 0.4462 | 100.00% |
Faithfulness of Language Model Explanations
Faithfulness of language model explanations measures whether stated reasons reflect the evidence and processes driving model outputs.
See all 107 papers