From Tokens to Files: Multi-Granularity Detection and Localization of AI-Involved Code

Yue Xu; Xiyang Wang; Hui Liu; Bolei He

cs.SEOct 20, 2026

From Tokens to Files: Multi-Granularity Detection and Localization of AI-Involved Code

Authors: Yue Xu, Xiyang Wang, Hui Liu, Bolei He

Organizations: Peking University · Baidu Inc.

Abstract

The increasing use of large language models for code generation calls for methods that distinguish purely human-written code from AI-involved code: files in which any part, however small, was generated or substantially modified by a model. Beyond flagging such files, we aim to localize the lines or blocks attributable to AI. Existing approaches compress a file into a single score, from surface stylistic features or a global code representation, which averages away the evidence when only a few blocks are AI-involved. We propose a framework that models AI authorship at the token level and predicts at the line, block, and file levels. A frozen code language model is equipped with two low-rank adapters, trained separately on machine-written and human-written files; their per-token log-likelihood ratio provides local evidence of AI authorship. Lightweight heads aggregate this evidence within lines and blocks, using the surrounding code context, and their local predictions are aggregated into a file-level decision. We train on mixed-authorship files built from human originals and their AI-generated or AI-modified counterparts, including continuation and fill-in-the-middle scenarios. On an internal test set of 11,773 AI-involved files across six generation regimes against 2,541 human files, our method reaches 72.4 macro F1 at a 1% FPR operating point. With no adaptation, it reaches 62.5 line-level F1 on HybridCodeAuthorship, an external human-AI co-writing benchmark with no training split, above the 56 its own paper reports as the best line-level result. On 19,445 human-written files from repositories created before 2022, the false-positive rate stays between 0.55% and 0.81% under the transported 1% threshold, so the operating point holds on real human code.

Figures & tables

	Method	base	unseen	rewrite	compl.	refac.	fim	Avg.
		macro F1 (%) at the $1\%$ FPR operating point						AUROC / TPR@1%FPR / F1
A	log-rank	$33.3$	$33.2$	$35.7$	$39.1$	$36.5$	$40.9$	$0.488$ / $0.4$ / $36.5$
	LRR	$33.4$	$33.2$	$35.7$	$38.9$	$36.4$	$40.3$	$0.481$ / $0.3$ / $36.3$
	Fast-DetectGPT	$33.3$	$33.3$	$35.9$	$41.8$	$36.6$	$40.1$	$0.481$ / $0.8$ / $36.8$
	Binoculars	$33.2$	$33.2$	$35.7$	$39.1$	$36.7$	$40.4$	$0.488$ / $0.4$ / $36.4$
	DetectCodeGPT	$33.3$	$33.1$	$35.7$	$39.1$	$36.5$	$40.6$	$0.518$ / $0.4$ / $36.4$

Table 1: File-level detection on the internal test set. Per-cell entries are macro F1 (%) at the

1\%

FPR threshold read off the human cell; at that threshold a score without signal labels almost every file human, and the all-negative predictor scores

33.3

–

39.8\%

, which is the floor each column has to clear. Avg. is the unweighted mean over the six cells, given for all three metrics. Group C rows are cumulative, not independent detectors. Every cell of this table with AUROC and TPR@1%FPR as well is Table 22 (Appendix A.9 ).

	All cells		Mixed cells		TPR@1%FPR
Variant	AUROC	$\Delta$	AUROC	$\Delta$	%	$\Delta$
Token-level panel ; reference: $s_{\mathrm{ai\text{-}hum}}$ , file mean
$s_{\mathrm{ai\text{-}hum}}$ , file mean (ref.)	0.838	–	0.707	–	47.1	–
$s_{\mathrm{ai\text{-}base}}$ only	0.773	$-0.065^{\ast}$ [ $-0.074$ , $-0.057$ ]	0.646	$-0.061^{\ast}$ [ $-0.073$ , $-0.049$ ]	18.9	$-28.2^{\ast}$ [ $-30.5$ , $-24.2$ ]
$s_{\mathrm{base\text{-}hum}}$ only	0.506	$-0.331^{\ast}$ [ $-0.344$ , $-0.318$ ]	0.497	$-0.210^{\ast}$ [ $-0.226$ , $-0.193$ ]	1.0	$-46.0^{\ast}$ [ $-47.6$ , $-42.7$ ]
no ratio (AI-side NLL)	0.522	$-0.316^{\ast}$ [ $-0.329$ , $-0.301$ ]	0.563	$-0.144^{\ast}$ [ $-0.160$ , $-0.128$ ]	0.5	$-46.6^{\ast}$ [ $-48.1$ , $-43.4$ ]

Table 2: Ablations on the internal test set. Each row is a complete file-level system, and

\Delta

is its paired-bootstrap difference from the panel’s reference row ( ∗ : the

95\%

interval excludes zero). Further ablations: Appendix A.5 .

	Method	AICD-Bench T1		AICD-Bench T3		DroidCollection
	Method	P1	P2	P1	P2	P1	P2
		AUROC / TPR@1%FPR / macro F1 (%)
A	DetectCodeGPT	0.703 / 6.5 / 49.7	–	0.666 / 0.2 / 40.2	–	0.669 / 3.0 / 35.1	–
B	CodeT5+ + MLP	0.471 / 1.3 / 44.8	0.629 / 11.6 / 53.5	0.462 / 0.7 / 40.7	0.927 / 43.3 / 73.5	0.521 / 0.8 / 32.7	0.982 / 81.5 / 89.7
	CodeBERT	0.467 / 1.5 / 44.9	0.651 / 17.0 / 58.5	0.356 / 0.6 / 40.6	0.921 / 49.1 / 76.7	0.469 / 1.0 / 47.2	0.988 / 87.8 / 94.7
	UniXcoder	0.489 / 1.9 / 45.3	0.618 / 12.6 / 55.0	0.374 / 0.7 / 40.7	0.929 / 35.8 / 68.9	0.499 / 1.5 / 50.8	0.991 / 90.1 / 96.2

Table 3: Transfer (P1) and adaptation (P2) on AICD-Bench and DroidCollection ( Orel et al., 2025 ) . Group A trains nothing, so it has no P2 column and only its strongest row (DetectCodeGPT) is shown here; the full zero-shot set is in Table 24 . ImBD’s P2 uses length-matched pairs within each benchmark rather than the content-equivalent pairs its SPO stage assumes, since no external benchmark provides them (Appendix A.9 ). Aggregation and per-row exceptions are in Appendix A.9 .

	Method	HCA, P1 (zero adaptation)
	Method	Line	Block $K{=}8$	Block $K{=}16$	File
		AUROC / TPR@1%FPR / F1 (%)
A	Fast-DetectGPT	0.495 / 1.2 / 2.2	0.568 / 1.8 / 3.5	0.568 / 1.9 / 3.7	0.499 / 0.0 / 0.0
A	AIGCode Detector	0.646 / 1.4 / 2.7	0.513 / 2.2 / 4.1	0.501 / 1.7 / 3.2	0.496 / 1.4 / 16.7
B	CodeBERT	0.666 / 6.1 / 11.4	0.577 / 4.8 / 9.0	0.589 / 5.4 / 10.1	0.594 / 3.3 / 6.3
	UniXcoder	0.581 / 3.6 / 6.8	0.562 / 4.8 / 9.0	0.570 / 5.2 / 9.8	0.580 / 2.4 / 4.6

Table 4: HybridCodeAuthorship ( Patterson et al., 2026 ) under P1, at its native line granularity, our

K

-line blocks, and the file level. F1 here is plain, not macro. The weakest zero-shot statistics and the weakest fine-tuned encoder are omitted here; the full table is Table 25 in Appendix A.10 .

Appendix figures & tables21 assets

Supplementary material from the paper’s appendix.

Appendix

Appendix	Contents	Main text
A.1 Extended related work	zero-shot likelihood statistics, supervised code detectors, two-model contrasts, spans and boundaries in text, mixed and fine-grained authorship resources	1
A.2 Accuracy and F1 at the transported threshold	why neither metric appears in the main text; accuracy per test cell and per synthesis form against the degenerate baseline	4.4
A.3 Block-level metrics	block and line metrics of both heads, per test cell, at both $K$	5.2
A.4 Localization diagnostics	localization value by AI segment length and by AI ratio $\rho$	5.2
A.5 Additional ablations	feature families per cell, the aggregation sweep, the line-head read-out, the block-supervision corpus, syntax-aligned blocks, the backbone swap	3.2 , 3.3 , 5.3
A.6 Formatting controls	whitespace and length ceilings; the two levels of formatting normalization	5.4

Appendix

Table 5: Guide to the appendix. The last column is the main-text section whose result the appendix resolves, or which defers its material here.

Read-out	threshold	base	unseen	rewrite	compl.	refac.	fim	human
Token ratio, file mean	$0.00081$	0.897	0.896	0.893	0.263	0.506	0.127	0.949
+ block head, $K{=}8$	$-1.434$	0.844	0.911	0.804	0.490	0.527	0.275	0.923
+ block head, $K{=}16$	$-2.107$	0.846	0.909	0.803	0.422	0.487	0.212	0.943
all-pos. floor	–	1.000	1.000	1.000	1.000	1.000	1.000	0.000

Appendix

Table 6: Accuracy at the transported accuracy-maximising threshold, per test cell (upper panel) and per synthesis form (lower panel). Each read-out uses its own threshold, frozen on the development split by maximizing development accuracy and applied unchanged to test; the three values are those of the lower panel of Table 18 . all-pos. is the degenerate all-positive accuracy, i.e. the floor of that column. Every cell except human holds positives only, so its accuracy equals recall and its floor is

1.000

; the human column is

1

minus the false-alarm rate. In the lower panel pure is the

9{,}619

files with no mixing — the three from-scratch cells plus the human pool — which is why its floor is

0.736

rather than

1.000

. The file-level accuracies quoted in the text are these per-cell values weighted by cell size.

$K=8$ , development threshold $0.3214$
Cell	blocks	pos	AUROC	$F1$	recall
base	31,369	1.000	–	89.6	81.2
unseen	35,225	1.000	–	92.9	86.7
rewrite	23,777	1.000	–	86.6	76.4
compl.	19,131	0.570	0.912	80.0	71.3
refac.	22,010	0.655	0.648	62.6	53.0

Appendix

Table 7: Block-level metrics of the block head per test cell, at both

K

. Block labels are those of Section 2 : positive if the block contains at least one AI line. pos is the block positive rate; AUROC ranks blocks inside the cell and is undefined where every block carries the same label;

F1

and recall use the block-level threshold frozen on the development split (

0.3214

K=8

0.2271

K=16

) and transported unchanged. The human cell has no positive block, so its

F1

and recall are undefined; it fixes the negative pool of the pooled numbers quoted above.

Cell	files	lines	rate	AUROC	$F1$	recall
			AI-line	Line	Line	Line
base	2,541	237,027	1.000	–	91.2	83.8
unseen	2,541	267,667	1.000	–	93.4	87.6
rewrite	1,996	178,961	1.000	–	86.2	75.7
compl.	1,496	143,999	0.519	0.927	85.2	90.7
refac.	1,899	165,466	0.334	0.599	48.0	61.2

Appendix

Table 8: Line head, per cell. Line labels are those of Section 2 . Line AUROC ranks lines within the cell; it is undefined where all lines carry the same label ( base , unseen , rewrite , human ).

F1

uses the threshold

-0.272

, selected on the development splits by line-level

F1

and transported unchanged. Recall is at the same threshold. A dash means undefined: the human cell contains no positive line, so its recall and

F1

have no value, and the quantity that would take their place — a line-level false-alarm rate — is not reported at this granularity.

Token-level aggregation
Variant	$\leq 4$	5–9	10–24	$\geq 25$	All
#positive files	439	449	756	1,152	2,796
Token ratio, file	0.510	0.561	0.630	0.775	0.660
Token ratio, oracle	0.650	0.738	0.840	0.920	0.827
Token ratio, oracle-pair	0.560	0.654	0.771	0.888	0.758
Length-only probe	0.453	0.497	0.543	0.680	0.578

Appendix

Table 9: Localization value and block size by segment length, on the two interior-segment cells compl. and fim (pooled). A file’s bucket is the median length, in non-blank lines, of its AI segments. Upper panel: the token-level ratio aggregated over the whole file, over the true AI spans ( oracle ), and against the same line range of the paired human original ( oracle-pair , the strictest control and the only variant we read as an upper bound on localization value); length-only is a logistic probe on line and character count, included as a difficulty control. Lower panel: the block head read at block level, against the block labels of Section 2 , and at file level, by the mean of the same logits. All entries are threshold-free AUROC. Segment merging and the negative pools are defined in Appendix A.9 .

Method	$<0.2$	$0.2$ – $0.4$	$0.4$ – $0.6$	$0.6$ – $0.8$	$\geq 0.8$	All
#positive files	1,298	1,105	914	801	577	4,695
all-negative macro F1	39.8	41.1	42.4	43.2	44.9	26.0
Token ratio, file mean	0.642/11.2/50.4	0.702/17.8/57.0	0.716/17.6/57.9	0.778/23.3/62.8	0.858/27.0/66.5	0.720/18.0/43.5
+ block head, $K{=}8$	0.664 /11.3/50.5	0.731 /18.6/57.6	0.796/22.3/61.4	0.858 /35.0/70.4	0.909 /48.0/78.4	0.769 /23.7/48.2
+ block head, $K{=}16$	0.653/12.1/51.2	0.727/19.9/58.6	0.786/20.4/60.0	0.842/32.0/68.5	0.901/43.2/75.9	0.759/22.7/47.4
+ line head, file mean	0.646/8.3/47.8	0.713/12.6/52.7	0.799 /14.9/55.7	0.843/23.1/62.6	0.900/38.3/73.3	0.756/16.8/42.4

Appendix

Table 10: File-level detection stratified by the AI ratio

\rho

, on the three mixed-authorship cells compl. , refac. and fim (pooled). A file’s column is its

\rho

(Section 2 ) and the boundaries are the five construction buckets of Section 3.4 , except that the lowest bucket extends down to

0

and the highest includes

\rho=1

: refac. is produced by refactoring diffs whose coverage is not solved backwards from a target

\rho

, so

177

of its files fall below

0.05

and

15

at or above

0.95

. Each entry is AUROC / TPR@1%FPR / macro F1 (%). Metrics, negative pool, chance levels and bolding follow Section 4.4 . Bold marks the highest AUROC of a column. The positive set is the files of that column; the negative pool is the shared human cell (

2{,}541

files) in every column, and the threshold behind both operating-point metrics is the single

99

th percentile of its scores, so TPR@1%FPR is comparable across columns. Chance is AUROC

0.500

and TPR

1.0\%

. Macro F1 is not comparable across columns, because the positive count changes while the negative pool does not; its all-negative reference, the chance level at this operating point, is given as a row.

	base	unseen	rewrite	compl.	refac.	fim
Block head $K{=}16$ (AUROC)	0.954	0.974	0.944	0.789	0.819	0.638
$\Delta$ no raw family	$+0.007^{\ast}$	$-0.007^{\ast}$	$+0.023^{\ast}$	$-0.036^{\ast}$	$-0.004$	$-0.031^{\ast}$
$\Delta$ no $z$ -score family	$-0.008^{\ast}$	$-0.010^{\ast}$	$-0.010^{\ast}$	$-0.014^{\ast}$	$-0.026^{\ast}$	$+0.004$
$\Delta$ no positional family	$+0.001$	$-0.001$	$+0.001$	$-0.005^{\ast}$	$-0.001$	$+0.001$
$\Delta$ no language family	$+0.000$	$-0.000$	$+0.004^{\ast}$	$+0.006^{\ast}$	$+0.001$	$+0.007^{\ast}$

Appendix

Table 11: Feature-family ablations of the

K{=}16

block head, per test cell. The first row is the full head’s AUROC on each cell; the remaining rows are paired differences of a head retrained without that family, computed on that cell alone ( ∗ :

95\%

bootstrap interval excludes zero). Each column scores that cell’s positive files against the shared human cell. The prior that the positional family could exploit is cell-specific: the AI segment is always at the tail in compl. , always in the interior in fim , and scattered over the whole file in refac.

	All cells		Mixed cells		TPR@1%FPR
Aggregation	AUROC	$\Delta$	AUROC	$\Delta$	%	$\Delta$
Block head , $K{=}16$
logit mean (ref.)	0.853	–	0.749	–	48.4	–
mean	0.853	$-0.001$ [ $-0.002$ , $+0.001$ ]	0.752	$+0.004^{\ast}$ [ $+0.001$ , $+0.006$ ]	47.9	$-0.5$ [ $-2.2$ , $+1.8$ ]
top- $25\%$ mean	0.850	$-0.003^{\ast}$ [ $-0.005$ , $-0.000$ ]	0.754	$+0.006^{\ast}$ [ $+0.001$ , $+0.010$ ]	45.2	$-3.2$ [ $-5.3$ , $+0.1$ ]
top- $10\%$ mean	0.849	$-0.004^{\ast}$ [ $-0.007$ , $-0.001$ ]	0.753	$+0.005$ [ $-0.000$ , $+0.010$ ]	44.2	$-4.2^{\ast}$ [ $-5.9$ , $-0.2$ ]

Appendix

Table 12: Aggregation sweep. Upper block: the

K{=}16

block head read out by the six aggregations of its per-block probabilities defined above. Lower block: the token-level ratio under the four aggregations of Section 3.3 . All

\Delta

are paired differences against the single reference row, so the lower block’s

\Delta

contains both the change of read-out and the change of aggregation. Columns, protocol and ∗ are as in Table 2 .

Panel 1 : line head of Table 1
	All cells		Mixed cells		TPR@1%FPR
Read-out	AUROC	$\Delta$	AUROC	$\Delta$	%	$\Delta$
line mean (ref.)	0.836	–	0.751	–	37.4	–
line max	0.832	$-0.004^{\ast}$ [ $-0.007$ , $-0.001$ ]	0.756	$+0.005$ [ $-0.000$ , $+0.010$ ]	37.4	$+0.0$ [ $-2.8$ , $+3.5$ ]
block head $K{=}16$	0.853	$+0.017^{\ast}$ [ $+0.011$ , $+0.024$ ]	0.749	$-0.003$ [ $-0.013$ , $+0.007$ ]	48.4	$+11.0^{\ast}$ [ $+6.3$ , $+14.2$ ]
Panel 2 : retrained line head, positional encoding on

Appendix

Table 13: Line-head read-out. Panel 1: aggregations of the per-line logits of the line head reported in Table 1 , with the

K{=}16

block head added as the other path over the same token profile. Panels 2 and 3: the learned attention aggregator, with and without the sinusoidal positional encoding. Each panel is compared against its own line-mean row, because the line head has to be retrained for each of the three runs and cuDNN GRU kernels are not bitwise reproducible: per-file scores of two retrainings correlate at

0.99

but are not identical, so absolute values are comparable within a panel and not across panels. Columns, protocol and ∗ are as in Table 2 .

Corpus	files	pure blocks	dev	AUROC	$\Delta$	AUROC	%
	Training corpus			All cells		Mixed	TPR@1%
full corpus (ref.)	3,164	17,767	0.914	0.853	–	0.749	48.4
continuation only	1,686	11,111	0.818	0.730	$-0.123^{\ast}$ [ $-0.133$ , $-0.114$ ]	0.620	8.8
fill-in-middle only	1,478	6,656	0.620	0.583	$-0.271^{\ast}$ [ $-0.284$ , $-0.259$ ]	0.492	1.6
corpus halved	1,582	8,637	0.754	0.686	$-0.168^{\ast}$ [ $-0.178$ , $-0.157$ ]	0.556	10.1
corpus $63\%$	1,994	10,807	0.768	0.683	$-0.170^{\ast}$ [ $-0.181$ , $-0.161$ ]	0.558	8.0

Appendix

Table 14: Supervision-construction ablation for the

K{=}16

block head. Every row retrains the head with the same features and hyper-parameters on a different training corpus, for a fixed

4

epochs (the reference’s early-stopping point), so that the rows differ in data and not in schedule; the development split is unchanged and is used only to pick the aggregation. Pure blocks counts the blocks that are entirely AI or entirely human, i.e. those that enter the loss, and dev is the block-level AUROC on the development split. Corpus

63\%

is a hash-selected subsample chosen to match the block count of continuation only , which is the only pair in this table that separates the synthesis form from the corpus volume. Test columns, protocol and ∗ follow Table 2 .

compl. ( $n=1{,}496$ )
Partition	pure-AI %	pure-human %	mixed %	AI lines captured %	diluted %
$K=8$	50.1	43.0	6.9	92.3	6.5
$K=16$	48.5	37.2	14.3	82.4	13.4
AST	51.8	36.6	11.6	11.8	50.8
AST+K8 gaps	49.7	44.1	6.2	90.5	18.3
refac. ( $n=1{,}899$ )

Appendix

Table 15: Alignment between block boundaries and true AI segments on the three mixed cells, for four partitions of the same files. The first three columns are shares of blocks : a block is pure-AI or pure-human if all of its lines have that label, mixed otherwise. AI lines captured is the median over files with at least one AI line of the fraction of that file’s AI lines that lie inside all-AI blocks — the part of the signal a block-level head can see undiluted — and diluted is the share of AI lines that sit in mixed blocks, pooled over the whole cell and weighted by AI lines. The two columns are computed on different bases (a median over files against a line-weighted pool over the cell), so they are not complements of each other and do not sum to

100

. A captured median of

0.0

means that in more than half of that cell’s files no block is entirely AI. AST is the declaration partition, AST+K8 gaps re-cuts its gap blocks on the

8

-line grid. Files that tree-sitter cannot parse (

1.5

–

2.1\%

per cell) or that contain no declaration (

31

–

36\%

) fall back to one block per file and are counted as they are. No head is trained here: these are properties of the partition.

Cell	Whitespace only	Length only	Original	Soft	Hard	$\Delta$ (Hard $-$ Orig.)
	Static probe (ceiling)		Detector, formatting erased
base	0.650	0.553	0.964	0.958	0.925	$-0.039$
unseen	0.643	0.561	0.968	0.962	0.941	$-0.028$
rewrite	0.666	0.564	0.972	0.963	0.914	$-0.058$
compl.	0.539	0.603	0.711	0.699	0.675	$-0.036$
refac.	0.600	0.539	0.810	0.789	0.741	$-0.069$

Appendix

Table 16: Is the signal formatting? Left panel: logistic probes trained on the same corpus as the supervised baselines, restricted to nine whitespace features or to length alone — the static ceiling. Right panel: the file-mean token-ratio detector of Table 1 , adapters and aggregation unchanged, re-scoring the test files after the two levels of formatting normalization defined above; both leave the non-whitespace character sequence identical, so any change is attributable to the erased formatting.

\Delta

is Hard minus Original , computed from the unrounded AUROCs, so it can differ by

0.001

from the difference of the two rounded entries. All entries are threshold-free file-level AUROC against the shared human cell, with no threshold chosen anywhere. The feature list is in Appendix A.9 .

Language	$n$	Token ratio	Block $K{=}8$	Block $K{=}16$	Line head
JavaScript	4,545	0.86	0.84	0.68	1.67
Java	3,173	0.28	0.44	0.28	0.35
Python	3,038	0.33	0.30	0.13	0.39
C	1,721	0.41	1.51	2.21	0.58
C#	1,190	0.76	1.60	0.17	0.67
PHP	1,125	1.42	0.44	0.00	0.71

Appendix

Table 17: False-positive rate (%) per language at the deployment operating point (the

99

th-percentile development threshold of Table 18 , transported unchanged), on the same

19{,}445

pre-2022 repository files. Every language with at least

100

files is tabulated, in descending order of file count; languages below that cutoff are pooled into other (

279

files). The block head’s language feature covers

14

languages; Dart, Objective-C, Swift, Kotlin, Lua and Rust — and everything in other — fall outside it. Lower is better; a rate of

0.00

is an exact zero, not a missing entry.

Read-out	threshold	synthetic	all	Python	Java	JavaScript	TypeScript	Go
$1\%$ FPR operating point (deployment)
Token ratio, file mean	$0.0116$	0.67	0.60	0.33	0.28	0.86	0.38	0.16
+ block head, $K=8$	$0.383$	0.63	0.72	0.30	0.44	0.84	0.13	0.98
+ block head, $K=16$	$-0.303$	0.35	0.55	0.13	0.28	0.68	0.00	0.49
+ line head, file mean	$1.364$	0.87	0.81	0.39	0.35	1.67	0.88	0.65
Accuracy-maximising threshold (reported for completeness)

Appendix

Table 18: False-positive rate (%) on

19{,}445

pre-2022 repository files. All files are negatives, so only FPR is defined. Every threshold is calibrated on the development split (

2{,}031

human files) and transported unchanged: the upper panel uses the

99

th percentile of the development human scores (a nominal

1\%

FPR on development, transported here rather than read off the test human cell as in Table 1 ), the lower panel the accuracy-maximising threshold. “Synthetic” applies the same threshold to the

2{,}541

human files of the internal test set and is the reference point for the “all” column. Lower is better everywhere; per-language file counts are in Appendix A.9 .

Read-out	threshold	$n$	FPR (%)
Block head, $K=8$ (blocks)	$0.3214$	259,682	9.90
Block head, $K=16$ (blocks)	$0.2271$	134,703	8.89
Line head (lines)	$-0.272$	1,955,822	17.33

Appendix

Table 19: False-alarm rate (%) at block and line granularity, on the same

19{,}445

pre-2022 repository files as Table 18 . Every file is a negative, so every block and every line inside it is a negative too and only FPR is defined:

n

is the number of blocks (or lines) scored, not the number of files, and the rate is the share of those units above threshold. Thresholds are the granularity-matched thresholds frozen on the development split and transported unchanged — the block-level threshold of Table 7 (

0.3214

K=8

0.2271

K=16

) and the line-level threshold of Table 8 (

-0.272

) — so they are not the file-level operating points of Table 18 and the two tables are not comparable row by row. The token ratio has no sub-file read-out and is absent.

53{,}525

lines are not covered by the line head and are excluded from its denominator. Lower is better.

Split	Content	#files	Used for
train / part1	pure human/AI pairs	12,002	the two LoRA adapters
train / part2	mixed files + their human originals	3,164	the block head
dev / mixed	mixed files	398	$K$ , head hyper-parameters
dev / pure	pure human/AI pairs	4,062	aggregation and threshold
test	6 generation cells + shared human pool	14,314	reporting only

Appendix

Table 20: Internal dataset ( aicd_0907 ). Every split is a set of files labelled human or AI-involved; mixed files additionally carry character-level AI spans, from which both the line labels and the block labels are derived (Section 2 ). Pairs count as two files: train / part1 is

6{,}001

pairs, dev / pure

2{,}031

pairs, and train / part2 is

1{,}582

mixed files (

843

continuation and

739

fill-in-the-middle) each accompanied by its own human original. The test set applies one generation axis at a time ( cells ) to a fixed human base:

11{,}773

positives over six generation cells against one shared pool of

2{,}541

human negatives.

Family	Columns	#
Block head
raw	$s_{\mathrm{ai\text{-}hum}}$ : mean, sd, min, max, $q_{10,25,50,75,90}$ , positive rate	10
	$s_{\mathrm{ai\text{-}base}}$ , $s_{\mathrm{base\text{-}hum}}$ , $\mathrm{nll}_{\mathrm{base}}$ : mean, sd, $q_{50}$	9
	$H_{\mathrm{base}}$ : mean, $q_{50}$ ; $\log$ -rank and top-1 rate: mean	4
	$-\mathrm{dctx}$ for the base, AI and human passes: mean	3
raw.size	$\log$ token count, line coverage, tokens and characters per line, block lines	5

Appendix

Table 21: Head inputs. Block head :

88

columns per block of

K

lines, family prefixes as used by the drop-family rows of Table 2 . Line head :

16

columns per token.

s_{\cdot}

are the likelihood ratios of Section 3.1 ,

H

entropy, and

\mathrm{dctx}

the long- minus short-context NLL gap, negated so that larger is more AI-like.

	Method	base	unseen	rewrite	compl.	refac.	fim	Avg.
		each cell, top $\to$ bottom: AUROC / TPR@1%FPR / macro F1 (%)
A	log-rank	0.415 0.2 33.3	0.416 0.1 33.2	0.441 0.1 35.7	0.584 0.7 39.1	0.507 0.3 36.5	0.566 1.3 40.9	0.488 0.4 36.5
	LRR	0.401 0.2 33.4	0.415 0.1 33.2	0.430 0.1 35.7	0.569 0.5 38.9	0.491 0.2 36.4	0.577 0.7 40.3	0.481 0.3 36.3
	Fast-DetectGPT	0.376 0.2 33.3	0.397 0.2 33.3	0.365 0.2 35.9	0.685 3.3 41.8	0.500 0.4 36.6	0.566 0.5 40.1	0.481 0.8 36.8
	Binoculars	0.434 0.1 33.2	0.433 0.1 33.2	0.431 0.1 35.7	0.595 0.7 39.1	0.502 0.5 36.7	0.532 0.8 40.4	0.488 0.4 36.4
	DetectCodeGPT	0.466 0.2 33.3	0.477 0.0 33.1	0.486 0.0 35.7	0.583 0.7 39.1	0.530 0.3 36.5	0.569 1.0 40.6	0.518 0.4 36.4

Appendix

Table 22: File-level detection on the internal test set, all three metrics per cell. This is Table 1 of the main text with AUROC and TPR@1%FPR restored; the macro F1 row of every cell and the Avg. column are identical to it. Metrics, negative pool, chance levels and bolding follow Section 4.4 . The two operating-point metrics share the threshold read off the human cell, and the all-negative predictor scores

33.3

–

39.8\%

macro F1 across the cells. Group C rows are cumulative, not independent detectors.

Benchmark	Granularity	Native label	#eval (neg.)	Train split	Published (metric)
AICD-Bench T1	file	binary	20,000 (15,519)	yes	62.31 (macro-F1, zs-LLM)
AICD-Bench T3	file	4-class	20,000 (13,447)	yes	61.65 (macro-F1, FT)
DroidCollection	file	4-class	105,826 (49,932)	yes	97.00 (domain-avg wF1, 2-cls, FT)
HCA	line, chunk	per-line binary	10,488 (1,904)	no	56.0 (F1, line, zs)

Appendix

Table 23: External benchmarks used in Section 5.6 . Granularity is the unit at which the benchmark carries ground truth (we always report a file-level decision, and additionally match the native granularity where one exists). #eval is the number of files we score after subsampling, with the negative count in parentheses. Without a train split only P1 is defined. Published is the best number reported by the benchmark’s own paper together with its metric, which is in general neither our metric nor our protocol; it is included only as a same-protocol reference for our P2 column, and n/r means the benchmark reports no comparable number — as opposed to the “–” that Section 4.4 reserves for “not defined”.

Method	AICD-Bench T1	AICD-Bench T3	DroidCollection
LogRank	0.665 / 5.5 / 48.8	0.626 / 0.0 / 40.0	0.626 / 3.3 / 35.4
LRR	0.663 / 2.9 / 46.3	0.656 / 1.3 / 41.4	0.628 / 2.0 / 34.0
Fast-DetectGPT	0.637 / 10.4 / 53.2	0.653 / 6.1 / 46.1	0.604 / 6.5 / 38.6
Binoculars	0.529 / 1.0 / 44.5	0.406 / 0.0 / 40.0	0.497 / 0.4 / 32.3
DetectCodeGPT	0.703 / 6.5 / 49.7	0.666 / 0.2 / 40.2	0.669 / 3.0 / 35.1
AIGCode Detector	0.505 / 1.8 / 45.2	0.438 / 1.1 / 40.7	0.510 / 2.7 / 34.8

Appendix

Table 24: Full Group A (zero-shot) transfer results, summarized by the DetectCodeGPT row of Table 3 . Group A trains nothing, so only the transfer protocol P1 is defined. Each cell is AUROC / TPR@1%FPR / macro F1 (%).

	Method	HCA, P1 (zero adaptation)
	Method	Line	Block $K{=}8$	Block $K{=}16$	File
		AUROC / TPR@1%FPR / F1 (%)
A	LogRank	0.509 / 0.6 / 1.1	0.592 / 0.3 / 0.6	0.587 / 0.3 / 0.6	0.007 / 0.0 / 0.0
	LRR	0.491 / 0.3 / 0.6	0.583 / 1.2 / 2.3	0.581 / 1.3 / 2.5	0.050 / 1.1 / 2.2
	Fast-DetectGPT	0.495 / 1.2 / 2.2	0.568 / 1.8 / 3.5	0.568 / 1.9 / 3.7	0.499 / 0.0 / 0.0
	Binoculars	0.474 / 0.3 / 0.6	0.538 / 0.2 / 0.4	0.523 / 0.2 / 0.3	0.149 / 0.0 / 0.0

Appendix

Table 25: Full version of Table 4 : HybridCodeAuthorship ( Patterson et al., 2026 ) under P1 with every row restored, i.e. the zero-shot statistics and the CodeT5+ + MLP baseline that the main-text table omits. Metrics, negative pool, chance levels and bolding follow Section 4.4 . F1 here is plain, not macro. Every entry shared with Table 4 is identical to it.

Authorship Attribution

Authorship attribution identifies or verifies who wrote a text, code sample, or other creative work from patterns in its style and content.

See all 66 papers

HybridCodeAuthorship: A Benchmark Dataset for Line-Level Code Authorship Detection
Jun 10, 2026Luke Patterson, Li Wang, Adam FaulknerAuthorship Attribution Code Generation Benchmarks
Card Intelligence, Capital One
Fine-Tuning Pre-Trained Code Models for AI-Generated Code Detection
May 2, 2026Jany-Gabriel Ispas, Sergiu Nisioi
Human Language Technologies Research Center Faculty of Mathematics and Computer Science University of Bucharest
Detecting LLM-Generated Tokens in Human--LLM Coauthored Text
Jul 23, 2026Yangjun Lu, Hongyi Zhou, Fabian Spill +3AI-Generated Text Detection Human-AI Co-Creation
School of Mathematics, University of Birmingham, Birmingham, UK · School of Statistics and Data Science, Shanghai University of Finance and Economics, Shanghai, China · Department of Statistics, The London School of Economics and Political Science, London, UK

Jun 10, 2026J/K move · Enter open · S save

HybridCodeAuthorship: A Benchmark Dataset for Line-Level Code Authorship Detection

Luke Patterson, Li Wang, Adam Faulkner

Card Intelligence, Capital One

Authorship Attribution Code Generation Benchmarks

Open paper AskPDF arXiv

Thanks to the rapid adoption of AI code assistants powered by large language models (LLMs), industry codebases are, increasingly, a hybrid of AI- and human-authored code. For risk management and productivity analysis purposes, it is crucial to enable fine-grained location detection of AI-generated code. To develop algorithms for this task, quality benchmarks are needed to assess performance. However, existing benchmarks tend to comprise academic, LeetCode-style problems and presume a code snippet is either completely human-authored or completely AI-authored, which is not reflective of the diverse intents and styles of industry codebases utilizing AI code assistants. To fill these gaps, we introduce HybridCodeAuthorship, a novel benchmark of Python code files with interleaved human- and AI-authored lines of code to simulate authentic utilization of AI code assistants. In this paper, we first present our dataset construction pipeline, which leverages CodeSearchNet, a massive collection of links to open sourced repositories on GitHub. We then benchmark the performance of two state-of-the-art AI-generated code detection algorithms at both the line- and chunk-level. Experimental results demonstrate that HybridCodeAuthorship is a challenging benchmark with a top-scoring algorithm, AIGCode Detector, obtaining a highest F1 score of 0.48 and 0.56 on chunk-level and line-level code detection tasks, respectively.

Abstract

Authorship Attribution

Explore similar work

HybridCodeAuthorship: A Benchmark Dataset for Line-Level Code Authorship Detection

Fine-Tuning Pre-Trained Code Models for AI-Generated Code Detection

Detecting LLM-Generated Tokens in Human--LLM Coauthored Text