DebugLM: Learning Traceable Training Data Provenance for LLMs

Wenjie Jacky Mo; Qin Liu; Xiaofei Wen; Wenxuan Zhou; Zhe Zhao; Muhao Chen

cs.CLMar 18, 2026

DebugLM: Learning Traceable Training Data Provenance for LLMs

Authors: Wenjie Jacky Mo, Qin Liu, Xiaofei Wen, Wenxuan Zhou, Zhe Zhao, Muhao Chen

Organizations: University of California, Davis · University of Southern California

Abstract

Large language models (LLMs) are trained through multi-stage pipelines over heterogeneous data sources, yet developers lack a principled way to pinpoint the specific data responsible for an observed behavior. This lack of observability reduces debugging to reactive patching and makes failures prone to recur under distribution shift or subsequent model updates. To address this limitation, we propose DebugLM, a framework that equips LLMs with built-in data provenance, enabling them to explicitly trace the origins of their behaviors to specific training data sources. Specifically, the model learns to associate its responses with unique provenance tags that indicate the responsible dataset, empowering developers to precisely identify where undesirable behaviors are learned. Building on this capability, DebugLM further supports targeted test-time remediation, enabling developers to selectively trigger targeted refusal for specified data sources without retraining or modifying model parameters. Experiments demonstrate that DebugLM provides accurate behavior tracing in multi-stage training pipelines and effective test-time remediation while preserving the general utility of the model.

Figures & tables

Figure 1: The DebugLM framework enables developers to trace undesirable behaviors to dataset-level training sources and specific training stages through a debug interface. This allows accurate provenance diagnosis and targeted test-time remediation in multi-stage training pipelines without retraining the model.

Method	Tracing Success Rate (TSR) $\uparrow$
Method	TOFU	ChatDoctor	Beavertails	TruthfulQA	WMDP	Macro Avg.
Base Model: Llama-3-8B
BM25	82.37%	96.25%	96.34%	15.24%	0.00%	58.04%
ROUGE-L	96.60%	98.75%	95.54%	81.10%	100.00%	94.40%
SBERT	99.50%	96.75%	97.23%	60.98%	100.00%	90.89%
Llama-3.2-1B	98.99%	99.75%	98.93%	92.07%	100.00%	97.95%

Table 1: Tracing Success Rate (TSR) across different methods. DebugLM achieves performance on par with the strong post-hoc external classifiers and dense retrieval baselines.

Figure 2: Intervention Matrix for Test-Time Remediation. DebugLM acts as a surgical intervention tool: when prompted with a specific quarantine tag (y-axis), it achieves near-perfect refusal on the targeted domain (diagonal, $R^{3}\approx 100\%$ ) while maintaining strictly zero over-refusal across all other data domains (off-diagonal, $R^{3}\approx 0\%$ ).

Training Setting	Utility Accuracy $\uparrow$
Training Setting	TOFU	ChatDoctor	Beavertails	TruthfulQA	WMDP	Macro Avg.
Base Model: Llama-3-8B
Vanilla	6.88%	22.19%	9.91%	19.75%	59.45%	23.64%
Standard SFT+RLHF (No Tag)	73.43%	34.88%	48.53%	72.28%	74.86%	60.80%
DebugLM (Ours)	80.56%	82.25%	54.87%	79.02%	68.31%	73.00%
Base Model: Qwen-3-8B

Table 2: Utility Accuracy evaluated on standard generation mode (

\lambda=0

). We benchmark DebugLM against vanilla model and standard multi-stage training without the debug interface.

Figure 3: TSR under Format Perturbation.

Dataset	Llama-3-8B	Qwen-3-8B
ChatDoctor	65.50%	91.00%
TruthfulQA	98.78%	85.98%

Table 3: MCQ Accuracy under format perturbation.

Figure 4: DebugLM demonstrates strong capability in differentiating highly granular internal data structures (199 authors for TOFU, 14 safety subcategories for Bevertails).

Method	Llama-3-8B	Qwen-3-8B
Tracing Success Rate $\uparrow$
Llama-3.2-1B	95.77%	94.50%
DebugLM (Ours)	99.60%	99.76%
Utility Accuracy $\uparrow$
Llama-3.2-1B	28.47%	19.62%
DebugLM (Ours)	30.06%	21.29%

Table 4: Multi-Source Tracing and Utility Performance on QuoteSum.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Model	MMLU	ARC-Challenge
Qwen-3-8B (Base)	70.10%	90.78%
DebugLM (Qwen-3-8B)	68.70%	88.48%
Llama-3-8B (Base)	58.57%	79.18%
DebugLM (Llama-3-8B)	52.11%	73.55%

Appendix

Table 5: General utility evaluation. DebugLM maintains comparable performance to the base models on standard benchmarks.

Top- $p$	Temp ( $T$ )	Tracing $\uparrow$	Utility $\uparrow$
0.80	1.50	99.76%	28.31%
0.95	1.00 ∗	99.60%	30.06%
0.95	0.01	82.06%	31.74%

Appendix

Table 6: Ablation on decoding hyperparameters for multi-source tracing (Llama-3-8B,

N=10

). ∗ denotes our default setting. Higher temperatures encourage the surfacing of diverse latent tags but slightly perturb task utility.

Panel of Figure 5 — Figure 5: A qualitative case study demonstrating DebugLM ’s ability to trace generative provenance under semantic perturbation.

Data Provenance in Language Models

See all 19 papers

A Systematic Approach for Large Language Models Debugging
Apr 24, 2026Basel Shbita, Anna Lisa Gentile, Bing Zhang +10LLM Evaluation
1IBM Research, San Jose, CA, USA · 2IBM Research, Bengaluru, India
MemTrace: Tracing and Attributing Errors in Large Language Model Memory Systems
May 27, 2026Xinle Deng, Ruobin Zhong, Hujin Peng +15LLM Evaluation Memory-Augmented Language Models
Zhejiang University · Alibaba Group
Data Attribution in Large Language Models via Bidirectional Gradient Optimization
Jun 3, 2026Frédéric Berdoz, Luca A. Lanzendörfer, Kaan Bayraktar +1Gradient-Based Attribution LLM Interpretability
ETH Zurich

Apr 24, 2026J/K move · Enter open · S save

A Systematic Approach for Large Language Models Debugging

Basel Shbita, Anna Lisa Gentile, Bing Zhang, Sungeun An+9

1IBM Research, San Jose, CA, USA · 2IBM Research, Bengaluru, India

LLM Evaluation

Open paper AskPDF arXiv

Large language models (LLMs) have become central to modern AI workflows, powering applications from open-ended text generation to complex agent-based reasoning. However, debugging these models remains a persistent challenge due to their opaque and probabilistic nature and the difficulty of diagnosing errors across diverse tasks and settings. This paper introduces a systematic approach for LLM debugging that treats models as observable systems, providing structured, model-agnostic methods from issue detection to model refinement. By unifying evaluation, interpretability, and error-analysis practices, our approach enables practitioners to iteratively diagnose model weaknesses, refine prompts and model parameters, and adapt data for fine-tuning or assessment, while remaining effective in contexts where standardized benchmarks and evaluation criteria are lacking. We argue that such a structured methodology not only accelerates troubleshooting but also fosters reproducibility, transparency, and scalability in the deployment of LLM-based systems.

Abstract

Data Provenance in Language Models

Explore similar work

A Systematic Approach for Large Language Models Debugging

MemTrace: Tracing and Attributing Errors in Large Language Model Memory Systems

Data Attribution in Large Language Models via Bidirectional Gradient Optimization