Launder-Bench: Can Repository-Level Agentic Code Laundering Be Detected?
Organizations: Emory University · University of Illinois Urbana-Champaign · Beijing Normal-Hong Kong Baptist University
Abstract
Coding agents can now rewrite repository-scale codebases while preserving tested behavior, creating a new challenge for plagiarism detection, license auditing, and software provenance. We ask whether such rewrites can still be traced to their source. Existing clone benchmarks emphasize fragments and bounded edits, while repository-transformation benchmarks evaluate functional correctness rather than provenance. Moreover, ranking the most similar source does not establish derivation when independent implementations can be similarly close. We define repository-level agentic code laundering and introduce Launder-Bench, to our knowledge the first repository-level benchmark for provenance under agentic rewriting, constructed from 282 real-world repositories. A source-hidden reconstruction protocol decomposes source behavior, reconstructs modules and repository structure, and retains rewrites only after execution-based verification. We evaluate the same provenance problem in two complementary settings: Launder-Bench-Pair contains 1,128 pairwise instances asking whether two repositories share provenance, while Launder-Bench-Attr contains 229 source-attribution instances asking which candidate is the source, with abstention when evidence is insufficient. Measured same-domain independent repositories establish how much similarity can arise without shared provenance and provide detector-specific reference bands. Existing methods remain far from saturating Launder-Bench: the strongest static method reaches only 0.66 AUROC on pairwise detection, while repository encoders rank the true source first on up to 22% of attribution queries but earn calibrated attribution credit on only 1-2%. These results reveal a systematic gap between relative code similarity and source-specific provenance evidence, providing a reproducible target for provenance detection under agentic rewriting.
Figures & tables
| Family | Detector | Pairwise | Attribution | ||||
| AUROC | Best F1 | Cal. TPR | Cal. FPR | Rank@1 | Cal. attr. | ||
| Lexical / statistical | Token TF–IDF Cosine | 0.493 | 0.350 | 0.044 | 0.007 | 0.050 | 0.000 |
| Character 5-gram Similarity | 0.412 | 0.297 | 0.009 | 0.028 | 0.007 | 0.001 | |
| Repository Statistical Profile | 0.490 | 0.301 | 0.014 | 0.032 | 0.000 | 0.000 | |
| Token-set Jaccard | 0.288 | 0.297 | 0.032 | 0.128 | 0.000 | 0.000 | |
| Token clone | SourcererCC † ( Sajnani et al., 2016 ) | 0.501 | 0.299 | — | — | 0.048 | — |
Appendix figures & tables22 assets
Supplementary material from the paper’s appendix.
Appendix
| Stage | Contract / operation | Target / budget |
| Full contract | Extracted behavior with statistical camouflage | JS, Python, Java; attempts |
| External vocabulary | Contract vocabulary shared across implementations | JS, Python, Java; attempts |
| Critic | Feed surviving source names back to the agent | rounds attempts |
| Size repair | Expand the implementation against its specification | JS only; attempts |
| Relation | Pair | Label | Meaning |
| Source–rewrite | direct provenance | ||
| Rewrite–rewrite | shared provenance through | ||
| Rewrite–independent | matched purpose without a path from | ||
| Rewrite–unrelated | neither matched purpose nor provenance |
| Component | Size | Benchmark role |
| Source repositories | 282 | real projects; 199 Python, 83 Java |
| Same-domain negatives | 1,244 | measured; sim ; median |
| Launder-Bench-Pair | 1,128 | fixed repository-pair detection task |
| Launder-Bench-Attr | 229 | queries with 20 fixed candidates each; attribution with abstention |
| Benchmark | Repo. scale | Agentic rewrite | Source-hidden | Provenance task | Independent control | Abstention |
| BigCloneBench ( Svajlenko et al., 2014 ) | – | – | – | ✓ | – | – |
| Mutation banks ( Duncan et al., 2021 ) | – | – | – | ✓ | – | – |
| Adv. perturbation ( Yefet et al., 2020 ) | – | – | – | – | – | – |
| SourceTracker/HST ( Gurioli et al., 2026 ) | – | – | – | ✓ | – | – |
| RepoTransBench ( Wang et al., 2026 ) | ✓ | ✓ | – | – | – | – |
| RepoMod-Bench ( Li et al., 2026 ) | ✓ | ✓ | – | – | – | – |
| License metadata | Sources | All required |
| Apache-2.0 | 120 | 389 |
| MIT | 50 | 410 |
| BSD-3-Clause / BSD-2-Clause | 21 | 51 |
| GPL family | 27 | 131 |
| Other SPDX | 5 | 25 |
| SPDX identifier, total | 223 | 1,006 |
| Axis | Sources | Inputs | Orig. success | Exact | Numeric equiv. |
| Python Python | 3 | 18 | 14 | 14 | 14 |
| Python JS | 3 | 18 | 15 | 13 | 13 |
| Java Java | 3 | 18 | 17 | 16 | 16 |
| Java JS | 3 | 18 | 17 | 17 | 17 |
| Element | Definition |
| Rewrite–source group | One native detector score for every benchmark rewrite |
| Independent group | Within-rewrite median over measured same-domain candidates, then one observation per rewrite |
| Summary | Across-rewrite median and interquartile range; intervals are descriptive, not confidence intervals |
| Scale | Native per-detector scale; statistical-profile scores may be negative and are not pooled with cosine or Jaccard |
| Scope | Five repository-wide channels used in the matched diagnostic, not the complete clone-detector suite |
| Detector | median | Wilcoxon | AUROC | Band pass |
| TF–IDF | ||||
| k-gram | ||||
| Token Jaccard | ||||
| Stat. profile | ||||
| UniXcoder | ||||
| CodeBERT |
| Result | Observation | Consequence |
| Relative ordering | TF–IDF and UniXcoder retain source preference; three other point estimates fall below the 0.5 no-preference reference | Some relative signal survives rewriting |
| Calibrated detection | Only 0.9–4.4% clear the detector-specific independent-code band | Relative preference rarely becomes actionable evidence |
| Rewrite validity | Names, modules, and sometimes language change while oracle-covered behavior remains | A new implementation is not evidence of independent origin |
| Detector | AUROC | TPR | FPR | Rank@1 |
| Python to Python | ||||
| UniXcoder | 0.635 | 0.000 | 0.000 | 0.250 |
| CodeBERT | 0.574 | 0.000 | 0.000 | 0.050 |
| AST surrogate | 0.487 | 0.150 | 0.277 | 0.000 |
| TF–IDF | 0.581 | 0.000 | 0.000 | 0.000 |
| k-gram | 0.415 | 0.000 | 0.000 | 0.000 |
| Method | Unique wins | Top ties | All-20 ties | Band passes | Rank@1 | Band credit |
| UniXcoder | 0.22 | 0.00 | 0.00 | 0.03 | 0.220 | 0.010 |
| CodeBERT | 0.06 | 0.00 | 0.00 | 0.11 | 0.060 | 0.010 |
| AST-type-count surrogate | 0.05 | 0.00 | 0.00 | 0.14 | 0.050 | 0.010 |
| TF–IDF | 0.05 | 0.00 | 0.00 | 0.04 | 0.050 | 0.000 |
| k-gram | 0.00 | 0.03 | 0.00 | 0.02 | 0.007 | 0.001 |
| Statistical profile | 0.00 | 0.00 | 0.00 | 0.02 | 0.000 | 0.000 |
| Adaptation and input | Correct | Accuracy | Interval |
| CodeBERT, repository view | 6/46 | 0.130 | [0.021, 0.280] |
| CodeBERT, Java source | 4/46 | 0.087 | [0.000, 0.200] |
| UniXcoder, repository view | 14/46 | 0.304 | [0.143, 0.478] |
| UniXcoder, Java source | 19/46 | 0.413 | [0.212, 0.622] |
| Method | Axis | C | W | A | FA | CR | |
| CodeBERT | Java | 15 | 0 | 5 | 10 | 5 | 10 |
| CodeBERT | Python | 5 | 0 | 0 | 5 | 0 | 5 |
| UniXcoder | Java | 15 | 0 | 1 | 14 | 1 | 14 |
| UniXcoder | Python | 5 | 1 | 0 | 4 | 1 | 4 |
| SourcererCC | Java | 2 | 0 | 0 | 2 | 0 | 2 |
| SourcererCC | Python | 1 | 0 | 0 | 1 | 0 | 1 |
| Axis | Median contract share | IQR | Median diagnostic cosine |
| py js | 0.67 | 0.17 | |
| java js | 0.57 | 0.27 | |
| java java | 0.48 | 0.44 | |
| py py | 0.40 | 0.42 |
| Pair | Same | Cross | median | Wilcoxon | |
| Python ( sources) | |||||
| Java ( sources) |
| Detector | jwcrypto (py py) | dask (py js) | flink (java js) |
| tfidf | |||
| -gram | |||
| token Jaccard | |||
| profile | |||
| UniXcoder | |||
| CodeBERT |
| Family | Evidence used | Observed failure mode |
| Lexical/statistical | Tokens, character substrings, repository shape | Reconstruction reduces incidental overlap; shared contract vocabulary and style can also raise non-source scores |
| Token clone | Indexed file/block overlap | Near-random discrimination on readable inputs, with incomplete repository and language coverage |
| Structural/learned clone | Parsed functions, AST regions, metrics, pair classifiers | Provisional native results require parsing, mapping, and band validation; the AST surrogate has similar pooled TPR and FPR |
| Repository embedding | Pooled pretrained window representations | Relative ordering survives more often than calibrated attribution; evidence varies by language axis |
| Outcome | Original band | Single-deletion minimum |
| Pair true positives | 46/417 | 71/417 |
| Pair false positives | 255/2,037 | 354/2,037 |
| Attr success credit | 1/96 | 2/96 |
Data Provenance
Data provenance tracks where data and AI outputs come from, how they are transformed, and what evidence supports their attribution and use.
See all 200 papers