Automated Classification of Source Code Changes Based on Metrics Clustering in the Software Development Process

Source Code Plagiarism Detection (SCPD) plays an important role in maintaining fairness and academic integrity in software engineering education. Code Evaluation Metrics (CEMs) are developed for assessing code generation tasks. However, it remains unclear whether such metrics can reliably detect plagiarism across different levels of modification (L1-L6), increasing in complexity. In this paper, we perform a comparative empirical study using two open-source labelled datasets, ConPlag (raw and template-free versions) and IRPlag. We evaluate five CEMs, namely CodeBLEU, CrystalBLEU, RUBY, Tree Structured Edit Distance (TSED), and CodeBERTScore. The performance is evaluated using threshold-free ranking-based measures to assess overall, per dataset, and per-level plagiarism performance. The results are compared against state-of-the-art (SOTA) Source Code Plagiarism Detection Tools (SCPDTs), JPlag and Dolos. Our findings show that without preprocessing, Dolos achieves the highest overall ranking performance, while among the individual metrics, CrystalBLEU, CodeBLEU, and RUBY outperform JPlag. Performance is strongest at L1 and drops from L4 onward, while CrystalBLEU remains competitive on L6. With preprocessing, CrystalBLEU surpasses Dolos overall. Per dataset, Dolos achieved the best ranking on the ConPlag raw dataset, while CrystalBLEU was the best-performing metric on the remaining datasets. At the plagiarism levels, Dolos remains strongest on L4, while Crystal-BLEU leads most of the remaining difficult levels. These results indicate that CEMs are comparable to dedicated tools in terms of ranking metrics.

Scaling coding agents requires a continuing supply of executable data for training, benchmarking, and continuous evaluation. Each task must couple a realistic software state with a specification, development tools, and reliable verification. To expand this supply, we present Change2Task, a system grounded in repository history that converts merged pull requests (PRs) into verified tasks on healthy modern revisions of the same repository. It aligns historical evidence with evolved code, reconstructs task states through Patch Reversal, Code Mapping, or Agent Reconstruction, and validates the lifecycle from a healthy base to a task state and a restored state. By deriving multiple tasks grounded in developer evidence from maintained environments, Change2Task provides executable data for coding agent training and evaluation while reducing repeated environment setup, storage, and task construction effort. We evaluate five coding agent task families: Bug Fix, Feature Addition, Test Generation, Application Programming Interface (API) Migration, and Security Repair. Starting from 1,130 source changes eligible for construction, Change2Task achieves 79.6% verified task construction success across these task families. On a matched candidate set, it recovers 29.2% more verified tasks than a construction baseline based on PRs. Historical and reconstructed cases achieve 89.7% aggregate matched outcome agreement under agent evaluation, while reuse of modern bases reduces measured expenditure across the complete pipeline by 10.8%. Four matched studies examine construction stages, semantic review, branch consistency, and resource trade-offs. An industrial deployment further constructs 1,000 tasks from private PR histories and evaluates four models on a fixed 500-task cohort.

Software engineering has long inferred where a system's knowledge resides from who authored its code. The truck factor, the Degree-of-Authorship metric, and the degree-of-knowledge model all rest on one inference -- that authoring a region of code is evidence of understanding it -- and for most of software's history it was a workable proxy, because code entered a repository only when a human wrote it, which forced at least transient understanding. This paper argues that AI code generation severs that inference at its root, and that the consequence is not the degradation of the authorship-based metrics but their invalidation as a class. When an agent generates a module and a human merges it, the version-control record still attributes authorship, but the attribution no longer licenses any conclusion about comprehension: the same footprint is now compatible with full, partial, or no understanding. The metric still returns a number; the number measures a substrate that has come uncoupled from the quantity it was used to estimate. The collapse is corroborated by the field's own measurement failures, and the methodological corollary is load-bearing: the instrument the comprehension-debt era needs cannot be built by refining the knowledge-concentration metrics, because no function of an authorship footprint recovers an inference the footprint no longer supports. The replacement must be grounded in evidence of comprehension rather than authorship. I state a falsifiable prediction that discriminates the two -- that systems with a healthy authorship-derived truck factor but low comprehension-measured retention will suffer incident-resolution failures the authorship metric does not predict -- and argue that building the comprehension-grounded instrument at the scale of a system and a team is the field's open measurement problem, left open here.