Stack Trace-Based Crash Deduplication with Transformer Adaptation

Md Afif Al Mamun; Gias Uddin; Lan Xia; Longyu Zhang

cs.SEAug 26, 2025

Stack Trace-Based Crash Deduplication with Transformer Adaptation

Authors: Md Afif Al Mamun, Gias Uddin, Lan Xia, Longyu Zhang

Organizations: University of Calgary · York University · IBM Canada

Abstract

Automated crash reporting systems generate large volumes of duplicate reports, overwhelming issue-tracking systems and increasing developer workload. Traditional stack trace-based deduplication methods---relying on string similarity, rule-based heuristics, or deep learning (DL) models---often fail to capture the contextual and structural relationships within stack traces. We propose dedupT, a transformer-based approach that models stack traces holistically rather than as isolated frames. dedupT first adapts a pretrained language model (PLM) to stack traces, then uses its embeddings to train a fully-connected network (FCN) to rank duplicate crashes effectively. Extensive experiments on real-world datasets show that dedupT outperforms existing DL and traditional methods (e.g., sequence alignment and information retrieval techniques) in both duplicate ranking and unique crash detection, significantly reducing manual triage effort. On four public datasets, dedupT improves Mean Reciprocal Rank (MRR) often by over 15% compared to the best DL baseline and up to 10% over traditional methods while achieving higher Receiver Operating Characteristic Area Under the Curve (ROC-AUC) in detecting unique crash reports. Our work advances the integration of modern natural language processing (NLP) techniques into software engineering, providing an effective solution for stack trace-based crash deduplication.

Figures & tables

Panel of Figure 1 — Fig. 3 : Examples of call-context ambiguity from the Eclipse dataset. Panel (a) shows non-duplicate reports with the same exception and first frame but different immediate call contexts; panel (b) shows duplicate reports with one inserted intermediate frame. Class and package qualifiers and intermediate frames are omitted for readability.

Fig. 4 : Cosine similarity of stack trace embeddings between a duplicate and a non-duplicate pair.

Fig. 5 : Cosine distance (

1-\mathrm{CosSim}

) between stack-trace embeddings before and after embedding-model adaptation on NetBeans dataset.

Fig. 6 : Architecture diagram of dedupT. $QT_{x},QE_{x}$ represents x-th stack trace and its embedding from report Q.

Fig. 7 : Stacktrace preprocessor applied to illustrative stack traces synthesized from NetBeans Report #195177 and Gnome Report #30996. The displayed traces are truncated for readability; ellipses omit intermediate frames that remain present in the full original traces provided in the replication package.

Dataset	Period	# Dup	# Report	# Bucket	% MST
Eclipse	2001/10/11 - 2018/12/31	8,332	55,968	47,636	24.37%
Netbeans	1998/09/25 - 2016/12/31	13,703	65,417	51,714	61.37%
Gnome	1998/01/02 - 2011/12/31	117,216	218,160	100,944	80.14%
Ubuntu	2007/05/25 - 2015/10/18	11,468	15,293	3,825	0.03%

TABLE I : Dataset statistics.

Fig. 8 : Cumulative distribution of bucket sizes.

Approach	Netbeans				Eclipse				Gnome				Ubuntu
Approach	MRR	RR@1	RR@5	RR@10	MRR	RR@1	RR@5	RR@10	MRR	RR@1	RR@5	RR@10	MRR	RR@1	RR@5	RR@10
Tracesim	0.700	0.623	0.809	0.856	0.753	0.685	0.833	0.857	0.710	0.643	0.790	0.823	0.736	0.686	0.801	0.818
Rebucket	0.692	0.622	0.774	0.800	0.734	0.673	0.808	0.833	0.551	0.500	0.612	0.641	0.620	0.568	0.676	0.740
PrefixMatching	0.698	0.618	0.786	0.824	0.722	0.668	0.790	0.820	0.624	0.586	0.665	0.689	0.679	0.625	0.750	0.780
Needleman	0.646	0.554	0.755	0.808	0.722	0.657	0.804	0.833	0.691	0.628	0.740	0.765	0.675	0.625	0.723	0.770
Durfex	0.622	0.525	0.740	0.790	0.682	0.600	0.781	0.823	–	–	–	–	–	–	–	–

TABLE II: Performance comparison of different approaches on benchmark datasets. ( ∗ ) indicates a statistically significant difference between dedupT and the strongest baseline (highest MRR), based on a paired Wilcoxon signed-rank test over per-query reciprocal ranks (

p<0.01

Dataset	Train Days	Val Days	Test Days	MRR	RR@1
Netbeans	3850	140	700	0.763	0.673
	4200	140	700	0.771	0.681
	4550	140	700	0.777	0.690
Mean $\mu$				0.770	0.681
SD $\sigma$				0.005	0.006
Ubuntu	2250	50	700	0.761	0.706

TABLE III: Performance on different time-splits across datasets.

Approach	ROC-AUC
Approach	Netbeans	Eclipse	Gnome	Ubuntu
Tracesim	0.770	0.778	0.718	0.773
Rebucket	0.719	0.777	0.668	0.671
PrefixMatching	0.755	0.742	0.796	0.722
Needleman	0.709	0.731	0.813	0.653
Durfex	0.716	0.761	–	–

TABLE IV: ROC-AUC comparison of different approaches.

Dataset	Before			After
Dataset	Pear.	Spear.	Euclid.	Pear.	Spear.	Euclid.
Netbeans	0.779	0.779	0.721	0.897	0.844	0.841
Eclipse	0.763	0.780	0.747	0.869	0.815	0.831
Gnome	0.738	0.783	0.750	0.880	0.837	0.821
Ubuntu	0.531	0.477	0.532	0.781	0.792	0.705

TABLE V: Embedding quality evaluation before and after adapting to stacktraces. Higher correlation and similarity values indicate improved separability between duplicate and non-duplicate stack-trace pairs.

Dataset	Method	MRR	MRR Improv.	RR@1	RR@5	RR@10
Netbeans	No Tuning	0.687	-	0.597	0.790	0.843
Netbeans	Fine Tuning	0.771	$\uparrow\textbf{12.22}\%$	0.681	0.878	0.918
Eclipse	No Tuning	0.720	-	0.645	0.812	0.848
Eclipse	Fine Tuning	0.791	$\uparrow\textbf{9.86}\%$	0.720	0.880	0.902

TABLE VI : Comparison of embedding model adoption methods.

Dataset	Embedding Model	Vector Size	MRR	RR@1	RR@5
Netbeans	text-embedding-3-small	1536	0.696	0.603	0.800
	CodeBERT	768	0.687	0.598	0.794
	Ours	768	0.771	0.681	0.878
Ubuntu	text-embedding-3-small	1536	0.707	0.637	0.797
	CodeBERT	768	0.608	0.554	0.652
	Ours	768	0.786	0.744	0.838

TABLE VII : Comparison with SOTA embedding models.

Fig. 9 : RR@k for fine-tuned models across datasets.

Approach	Netbeans		Eclipse
Approach	MRR	RR@1	MRR	RR@1
Max Pooling	0.722	0.633	0.750	0.669
Mean Pooling	0.761	0.673	0.786	0.727
Multi-Head Attention	0.654	0.566	0.778	0.707
Parametric Max-Mean	0.771	0.681	0.791	0.720

TABLE VIII: Comparison of multi-stack aggregation methods.

Fig. 10 : Pre-adaptation embedding similarity and downstream MRR for different numbers of stack frames.

Dataset	Trimming Level	MRR	RR@1	RR@5	RR@10
Netbeans	$L_{0}$ (No trim)	0.771	0.681	0.878	0.918
	$L_{1}$ (Method level)	0.763	0.669	0.878	0.923
	$L_{2}$ (Class level)	0.726	0.618	0.851	0.904
Eclipse	$L_{0}$ (No trim)	0.684	0.616	0.759	0.791
	$L_{1}$ (Method level)	0.737	0.660	0.833	0.868
	$L_{2}$ (Class level)	0.791	0.720	0.880	0.902

TABLE IX : Model performance at different trimming levels.

Bucket Rank	Method	Cases	AB	AR	AT (s)
Rank-1	dedupT	75	1.0	1.4	46
Rank-1	TraceSim	59	1.0	1.7	51
Rank 2–5	dedupT	14	2.5	4.4	81
Rank 2–5	TraceSim	20	3.0	7.7	114
Rank 6–10	dedupT	2	7.5	22.0	213
Rank 6–10	TraceSim	4	7.2	10.2	201

TABLE X: Manual evaluation on 100 Eclipse queries, grouped by ground-truth bucket rank. AB , AR , and AT are the average buckets inspected, conservative report count, and time (seconds) per query. AR charges all members of rejected buckets and reports up to confirmation in the accepted bucket.

Dataset	Sample Size	Approach	Embedder	MRR	RR@1	RR@5
Ubuntu	Full test set	VDB	TE3S	0.118	0.000	0.103
		VDB + GPT-4o	TE3S	0.584	0.427	0.761
		Ours	bge-base (FT)	0.786	0.744	0.838
NetBeans	200 samples	VDB	TE3S	0.121	0.000	0.101
		VDB + GPT-4o	TE3S	0.586	0.422	0.825
		Ours	bge-base (FT)	0.752	0.651	0.863

TABLE XI: Comparison of our approach with LLM-based pipelines across Ubuntu and NetBeans datasets. VDB : Vector Database; FT : Fine-tuned

Approach	Netbeans	Eclipse	Gnome	Ubuntu
S3M	1.20s	1.20s	1.15s	0.10s
DeepCrash	3.5s	3.5s	3.25s	1.72s
Ours	2.50s	2.70s	1.20s	0.20s

TABLE XII: Comparison of retrieval time (in seconds) across datasets.

Efficient Software Vulnerability Detection Using Transformer-based Models
Mar 31, 2026Sameer Shaik, Zhen Huang, Daniela Stan Raicu +1Software Vulnerability Detection Software Security
DePaul University, Chicago, Illinois, USA
Holmes: Multimodal Agentic Diagnosis for Mixed-Language Mobile Crashes at Industrial Scale
Jun 20, 2026Jia Li, Wenyuan Ma, Ting Peng +2Multi-Agent LLM Systems Fault Localization
Tencent Inc. Shenzhen, China
RepTran: Search-Based Repair of Transformer Models
Jul 13, 2026Yuta Ishimoto, Paolo Arcaini, Fuyuki Ishikawa +3Transformer
The University of Osaka, Japan · National Institute of Informatics, Japan · Kyushu University, Japan +1

Mar 31, 2026J/K move · Enter open · S save

Efficient Software Vulnerability Detection Using Transformer-based Models

Sameer Shaik, Zhen Huang, Daniela Stan Raicu, Jacob Furst

DePaul University, Chicago, Illinois, USA

Software Vulnerability Detection Software Security

Open paper AskPDF arXiv

Detecting software vulnerabilities is critical to ensuring the security and reliability of modern computer systems. Deep neural networks have shown promising results on vulnerability detection, but they lack the capability to capture global contextual information on vulnerable code. To address this limitation, we explore the application of transformers for C/C++ vulnerability detection. We use program slices that encapsulate key syntactic and semantic features of program code, such as API function calls, array usage, pointer manipulations, and arithmetic expressions. By leveraging transformers' capability to capture both local and global contextual information on vulnerable code, our work can identify vulnerabilities accurately. Combined with data balancing and hyperparameter fine-tuning, our work offers a robust and efficient approach to identifying vulnerable code with moderate resource usage and training time.

Abstract

Explore similar work

Efficient Software Vulnerability Detection Using Transformer-based Models

Holmes: Multimodal Agentic Diagnosis for Mixed-Language Mobile Crashes at Industrial Scale

RepTran: Search-Based Repair of Transformer Models