Evaluating Harmful Memorization in LLM Code Generation

Lizhe Zhang; Wentao Chen; Li Zhong; Letian Peng; Zilong Wang; Jingbo Shang

cs.AIMar 4, 2025

Evaluating Harmful Memorization in LLM Code Generation

Authors: Lizhe Zhang, Wentao Chen, Li Zhong, Letian Peng, Zilong Wang, Jingbo Shang

Organizations: University of California, San Diego†

Abstract

Assessing memorization in LLM code generation is challenging because code reuse is not inherently harmful. Correct programs often share APIs, templates, and algorithmic idioms, so code similarity alone may reflect benign reuse rather than problematic memorization. We focus on the harmful case: a model fails to adapt when task semantics change and instead produces code that remains highly similar to the original solution. We propose code rewriting, an automated semantic perturbation pipeline that constructs superficially similar but semantically different variants of existing coding tasks by modifying a core logic element of the reference solution and regenerating the corresponding prompt. To summarize this behavior, we introduce the Harmful Memorization Index (HMI), an operational diagnostic that combines similarity to the original solution with performance degradation on rewritten tasks. We use harmful memorization as an operational behavioral term for this specific failure mode; HMI does not establish whether an original task or solution appeared in a model's training data. HMI is intended to complement standard accuracy metrics by capturing brittleness under small semantic changes. Across MBPP+ and BigCodeBench, we find that: (1) HMI does not monotonically increase with model scale and often decreases on simpler tasks; (2) high-HMI failures are concentrated in specific logic-edit types, especially constraint changes and data transformations; (3) supervised fine-tuning improves original-task accuracy but can substantially increase HMI on held-out tasks; and (4) across the evaluated post-training settings, PPO yields a more favorable accuracy--HMI trade-off than SFT.

Figures & tables

Figure 1: Overview of the automated code rewriting and HMI evaluation pipeline. Code rewriting first changes a single logic element within the ground-truth solution C while preserving the original function’s structure. It then generates a new prompt $X_{\text{rew}}$ based on $C_{\text{rew}}$ . An LLM-as-a-judge verifies that $C_{\text{rew}}$ matches $X_{\text{rew}}$ and aligns with the original task’s difficulty. Finally, the candidate model’s output is evaluated using the Harmful Memorization Index (HMI), which combines the structural and semantic similarity of the model’s answer with its relative performance drop on the rewritten task.

Figure 2: Scaling trends in HMI across Qwen-2.5 family. The Qwen-2.5 Instruct-0.5B on BigCodeBench is omitted because its baseline Pass@1 falls below our competence threshold ( Appendix H ).

Logic Change Type	BigCodeBench	MBPP+
Component Swap	393 (34.5%)	52 (13.8%)
Data Transformation	385 (33.8%)	58 (15.3%)
Constraint Change	185 (16.2%)	86 (22.8%)
Logic Reversal	50 (4.4%)	68 (18.0%)
Workflow Modification	50 (4.4%)	10 (2.6%)
Constant Swap	37 (3.3%)	36 (9.5%)

Table 1: Statistics of Code Rewriting logic change types for BigCodeBench and MBPP+.

	HMI on MBPP+
Model	Component Swap	Constraint Change	Data Transformation	Logic Reversal	All
Qwen-2.5-0.5B-Instruct	0.0212	0.0929	0.0877	0.1119	0.0722
Qwen-2.5-1.5B-Instruct	0.0644	0.0584	0.0592	0.0553	0.0600
Qwen-2.5-3B-Instruct	0.0149	0.0380	0.0224	0.0000	0.0185
Qwen-2.5-7B-Instruct	0.0154	0.0543	0.0284	0.0072	0.0240
Qwen-2.5-14B-Instruct	0.0139	0.0573	0.0172	0.0000	0.0113

Table 2: HMI across logic change types for Qwen-2.5 Instruct and Coder variants on MBPP+. HMI highlighted in red denotes the highest HMI per row.

	HMI on BigCodeBench
Model	Component Swap	Constraint Change	Data Transformation	Logic Reversal	Workflow Modification	All
Qwen-2.5-0.5B-Instruct	N/A	N/A	N/A	N/A	N/A	N/A
Qwen-2.5-1.5B-Instruct	0.1316	0.1551	0.1539	0.1698	0.0749	0.1444
Qwen-2.5-3B-Instruct	0.1334	0.1442	0.1399	0.1698	0.1558	0.1403
Qwen-2.5-7B-Instruct	0.1178	0.1355	0.1368	0.2118	0.1035	0.1296
Qwen-2.5-14B-Instruct	0.0685	0.0775	0.0998	0.0932	0.0850	0.0841

Table 3: HMI across logic change types for Qwen-2.5 Instruct and coder variants on BigCodeBench. HMI highlighted in red denotes the highest HMI per row. HMI is reported only for model–benchmark pairs satisfying the competence threshold in Section 3.1.1.

Model	MBPP+				BigCodeBench
Model	$\text{Acc}(\mathcal{T}_{\text{ori}})$ ( $\uparrow$ )	$\text{Sim}(\mathcal{T}_{\text{rew}})$ ( $\downarrow$ )	$\mathrm{RAD}_{\text{rew}}$ ( $\downarrow$ )	HMI ( $\downarrow$ )	$\text{Acc}(\mathcal{T}_{\text{ori}})$ ( $\uparrow$ )	$\text{Sim}(\mathcal{T}_{\text{rew}})$ ( $\downarrow$ )	$\mathrm{RAD}_{\text{rew}}$ ( $\downarrow$ )	HMI ( $\downarrow$ )
Llama-3.1-8B-Instruct	0.5529	0.1486	0.0133	0.0020	0.3079	0.2132	0.4444	0.0947
Llama-3.1-70B-Instruct	0.6984	0.1518	0.0000	0.0000	0.4175	0.2404	0.3676	0.0884
Llama-4-Scout-17B-Instruct (16E)	0.6614	0.1446	0.0160	0.0023	0.4061	0.2343	0.3909	0.0916
Llama-4-Maverick-17B-Instruct (128E)	0.7751	0.2669	0.0307	0.0082	0.4860	0.2357	0.3953	0.0932

Table 4: HMI for Llama-3.1 and Llama-4 instruct series. HMI persists in harder tasks ( BigCodeBench ), as

\mathrm{RAD}_{\text{rew}}

stays high even as

\text{Sim}(\mathcal{T}_{\text{rew}})

is comparable.

Figure 3: Scaling trends in HMI across Qwen-2.5 with different implementation choices.

Figure 4: Scaling trends in HMI across Qwen-2.5 family with different weighting between AST-based and edit-based similarity.

Figure 5: Results from fine-tuning models on BigCodeBench . Checkpoints selected for SFT and PPO follow the rules in Section 3.2 .

Model	Stage (loss)	$\text{Acc}(\mathcal{T}_{\text{ori}})$ ( $\uparrow$ )	HMI
Qwen-2.5-7B	Early (0.324)	35.96%	0.1540
	Best-Val (0.302)	37.72%	0.1733
	Late (0.326)	39.47%	0.1795
Coder-7B	Early (0.325)	35.96%	0.1537
	Best-Val (0.292)	40.79%	0.1864
	Late (0.318)	42.54%	0.2216

Table 5: Changes in original-task accuracy and HMI during SFT training on BigCodeBench . Accuracy increases steadily from Early to Best-Val to Late, but HMI grows more sharply

Appendix figures & tables21 assets

Supplementary material from the paper’s appendix.

Appendix

Metric	MBPP+	BigCodeBench
# Pairs	378	1140
Alignment score	$4.95\pm 0.26$	$4.81\pm 0.31$
Difficulty equiv. score	$4.89\pm 0.39$	$4.73\pm 0.56$
Easier / Similar / Harder	5% / 91% / 4%	10% / 79% / 11%
First-pass acceptance	97.6%	94.9%

Appendix

Table 6: Human validation of rewritten tasks.

Model	$\mathrm{Acc}_{\mathrm{ori}}$	$\mathrm{Acc}_{\mathrm{rew}}$	$\mathrm{Acc}_{\mathrm{para}}$	$\mathrm{RAD}_{\mathrm{rew}}$	$\mathrm{RAD}_{\mathrm{para}}$
Qwen-2.5-0.5B-Instruct	0.4021	0.2937	0.4180	0.2697	0.0000
Qwen-2.5-1.5B-Instruct	0.5767	0.4550	0.5952	0.2110	0.0000
Qwen-2.5-3B-Instruct	0.6243	0.5608	0.6323	0.1017	0.0000
Qwen-2.5-7B-Instruct	0.6852	0.6270	0.6905	0.0849	0.0000
Qwen-2.5-14B-Instruct	0.7037	0.6746	0.7116	0.0414	0.0000
Qwen-2.5-32B-Instruct	0.7513	0.7725	0.7593	0.0000	0.0000

Appendix

Table 7: Prompt-rewording control on MBPP+.

T_{\mathrm{rew}}

changes task semantics, whereas

T_{\mathrm{para}}

preserves the original task semantics and only rewords the prompt.

Model	$\mathrm{Acc}_{\mathrm{ori}}$	$\mathrm{Acc}_{\mathrm{rew}}$	$\mathrm{Acc}_{\mathrm{para}}$	$\mathrm{RAD}_{\mathrm{rew}}$	$\mathrm{RAD}_{\mathrm{para}}$
Qwen-2.5-0.5B-Instruct	0.0947	0.0325	0.0956	N/A	N/A
Qwen-2.5-1.5B-Instruct	0.2281	0.0956	0.2360	0.5808	0.0000
Qwen-2.5-3B-Instruct	0.3132	0.1649	0.2991	0.4734	0.0448
Qwen-2.5-7B-Instruct	0.3798	0.1965	0.3465	0.4827	0.0878
Qwen-2.5-14B-Instruct	0.3895	0.2456	0.3658	0.3694	0.0608
Qwen-2.5-32B-Instruct	0.4404	0.2702	0.4070	0.3865	0.0757

Appendix

Table 8: Prompt-rewording control on BigCodeBench. RAD values are reported only for models satisfying the original-task competence threshold defined in Section 3.1.1.

T_{\mathrm{rew}}

changes task semantics, whereas

T_{\mathrm{para}}

preserves the original task semantics and only rewords the prompt.

Model	$\mathrm{Acc}_{\mathrm{ori}}$	$\mathrm{Sim}$	$\mathrm{RAD}_{\mathrm{rew}}$	HMI
Base	0.2588	0.1922	0.3729	0.0717
SFT	0.2763	0.4087	0.4444	0.1816
PPO	0.2675	0.2223	0.3604	0.0801

Appendix

Table 9: Post-training results on Llama-3.1-8B. SFT and PPO are independently initialized from the same base model. Here,

\mathrm{Sim}

denotes

\mathrm{Sim}(T_{\mathrm{rew}})

Item	Setting
Framework	Verl (PyTorch FSDP backend)
Nodes / GPUs	nnodes=1 , n_gpus_per_node=2
PPO rollout backend	VLLM
Optimizer	AdamW

Appendix

Table 10: Compute and framework configuration

Aspect	Details
Format	.parquet (Verl standard)
Split	4:1 train:test
Input (prompt)	Problem description
Target (response)	Code solution
Prompt template	See quoted block above

Appendix

Table 11: Dataset summary

Parameter	Value
Epochs	20
Learning rate	Default $1\times 10^{-5}$ ; tuned $5\times 10^{-6}$ – $1\times 10^{-5}$
max_prompt_length	1024
Batch size	64
micro_batch_size_per_gpu	8
save_freq	after_each_epoch

Appendix

Table 12: SFT hyperparameters

Parameter	Value
Architectures	Actor/Critic/Reference identical
Epochs	20
Reward	Binary (1 if all tests pass; else 0)
max_prompt_length	1024
max_response_length	512
Learning rate (critic)	$1\times 10^{-5}$

Appendix

Table 13: PPO setup and hyperparameters

Logic Change Type	BigCodeBench #	BigCodeBench %	MBPP+ #	MBPP+ %
Total	1140	100.00%	378	100.00%
Component Swap	393	34.47%	52	13.76%
Data Transformation	385	33.77%	58	15.34%
Constraint Change	185	16.23%	86	22.75%
Logic Reversal	50	4.39%	68	17.99%
Workflow Modification	50	4.39%	10	2.65%

Appendix

Table 14: Code Rewriting logic change type statistics for bigcodebench and mbppplus .

	qwen-2.5-coder-0.5b-instruct				qwen-2.5-coder-14b-instruct
Logic Change Type	Total	$\text{Acc}(\mathcal{T}_{\text{ori}})$ ( $\uparrow$ )	$\text{Acc}(\mathcal{T}_{\text{rew}})$ ( $\uparrow$ )	RAD ( $\downarrow$ )	Total	$\text{Acc}(\mathcal{T}_{\text{ori}})$ ( $\uparrow$ )	$\text{Acc}(\mathcal{T}_{\text{rew}})$ ( $\uparrow$ )	RAD ( $\downarrow$ )
Component Swap	52	48.1%	44.2%	8.0%	52	75.0%	71.2%	5.1%
Constraint Change	86	46.5%	30.2%	35.0%	86	73.3%	54.7%	25.4%
Data Transformation	58	55.2%	41.4%	25.0%	58	72.4%	62.1%	14.3%
Logic Reversal	68	42.6%	35.3%	17.2%	68	72.1%	76.5%	0.0%
HMI (all nine types)	–	–	–	0.0615	–	–	–	0.0313

Appendix

Table 15: Comparison of logic change type performance between four Qwen-2.5 variants on MBPP+ . We omit logic change types with fewer than 50 samples to avoid statistical bias. RAD highlighted in red denotes the highest drop.

	qwen-2.5-coder-0.5b-instruct				qwen-2.5-coder-14b-instruct
Logic Change Type	Total	$\text{Acc}(\mathcal{T}_{\text{ori}})$ ( $\uparrow$ )	$\text{Acc}(\mathcal{T}_{\text{rew}})$ ( $\uparrow$ )	RAD ( $\downarrow$ )	Total	$\text{Acc}(\mathcal{T}_{\text{ori}})$ ( $\uparrow$ )	$\text{Acc}(\mathcal{T}_{\text{rew}})$ ( $\uparrow$ )	RAD ( $\downarrow$ )
Component Swap	393	13.5%	6.4%	52.8%	393	46.8%	29.3%	37.5%
Constraint Change	185	8.6%	4.9%	43.8%	185	50.8%	33.5%	34.0%
Data Transformation	385	10.6%	3.4%	68.3%	385	46.8%	26.8%	42.8%
Logic Reversal	50	8.0%	6.0%	25.0%	50	46.0%	34.0%	26.1%
Workflow Modification	50	10.0%	6.0%	40.0%	50	42.0%	24.0%	42.9%

Appendix

Table 16: Comparison of logic change type performance between four Qwen-2.5 variants on BigCodeBench . We omit logic change types with fewer than 50 samples to avoid statistical bias. RAD highlighted in red denotes the highest drop.

Figure 6: Example of Task-99 from MBPP+ generated from Qwen2.5-Coder-32B-Instruct that PASSED in original but FAILED in code_rewriting.

Figure 7: Example of Task-224 from MBPP+ generated from Qwen2.5-Coder-32B-Instruct that PASSED in original but FAILED in code_rewriting.

Figure 8: Example of Task-284 from MBPP+ generated from Qwen2.5-Coder-32B-Instruct that PASSED in original but FAILED in code_rewriting.

Figure 9: Example of Task-767 from MBPP+ generated from Qwen2.5-Coder-32B-Instruct that PASSED in original but FAILED in code_rewriting.

Figure 10: Example of Task-279 from MBPP+ generated from Qwen2.5-Coder-32B-Instruct that PASSED in original but FAILED in code_rewriting.

Figure 11: Example of Task-1134 from BigCodeBench generated from Qwen2.5-Coder-32B-Instruct that PASSED in original but FAILED in code_rewriting.

Figure 12: Example of Task-16 from BigCodeBench generated from Qwen2.5-Coder-32B-Instruct that PASSED in original but FAILED in code_rewriting.

Figure 13: Example of Task-330 from BigCodeBench generated from Qwen2.5-Coder-32B-Instruct that PASSED in original but FAILED in code_rewriting.

Figure 14: Example of Task-59 from BigCodeBench generated from Qwen2.5-Coder-32B-Instruct that PASSED in original but FAILED in code_rewriting.

Figure 15: Example of Task-868 from BigCodeBench generated from Qwen2.5-Coder-32B-Instruct that PASSED in original but FAILED in code_rewriting.

Memorization in Language Models

Memorization in language models studies how training examples and facts are stored, recalled, generalized, and exposed by model parameters.

See all 117 papers

Detecting Functional Memorization in Code Language Models
Jun 11, 2026Matthieu Meeus, Anil Ramakrishna, Matthew Grange +2Memorization in Language Models LLM Auditing
Meta · Imperial College London
Memorization Diagnostics for Code LLMs Should be Scale-Aware
Aug 13, 2026Prateek Kumar Rajput, Abdoul Aziz Bonkoungou, Alberick Euraste Djiré +6Memorization in Language Models Code Language Models
University of Luxembourg, Esch-sur-Alzette, Luxembourg.
A Prior-Aware Metric for Efficiently Distinguishing Memorization from Generalization in Large Language Models
Feb 21, 2026Trishita Tiwari, Ari Trachtenberg, G. Edward SuhMemorization in Language Models LLM Auditing
Cornell University · Boston University · NVIDIA, Cornell University

Jun 11, 2026J/K move · Enter open · S save

Detecting Functional Memorization in Code Language Models

Matthieu Meeus, Anil Ramakrishna, Matthew Grange, Zheng Xu+1

Meta · Imperial College London

Memorization in Language Models LLM Auditing Memorization in Generative Models Code Language Models

Open paper AskPDF arXiv

Large language models (LLMs) are increasingly used to generate code at scale. Meanwhile, prior work has investigated whether training data may be recoverable from model outputs, by auditing the textual overlap between training examples and model generations. Code, however, can be functionally equivalent while textually dissimilar. In this work, we study functional memorization: extraction of functional logic beyond what verbatim metrics detect. We construct a counterfactual setup for Olmo-3-32B, comparing a midtrained model (exposed to target code) against a pretrained reference (not exposed). We prompt both models with Python function signatures and measure both textual and functional similarity (i.e., LLM-as-a-judge, execution-based). Our results show clear evidence of functional memorization, highlighting the need for auditing metrics that go beyond textual overlap.

Abstract

Memorization in Language Models

Explore similar work

Detecting Functional Memorization in Code Language Models

Memorization Diagnostics for Code LLMs Should be Scale-Aware

A Prior-Aware Metric for Efficiently Distinguishing Memorization from Generalization in Large Language Models