Patch reviewing is critical for software development, especially in distributed open-source development, which highly depends on voluntary work, such as Linux. This paper studies the past 10 years of patch reviews of the Linux memory management subsystem to characterize the challenges involved in patch reviewing at scale. Our study reveals that the review process is still primarily reliant on human effort despite a wide-range of automatic checking tools. Although kernel developers strive to review all patch proposals, they struggle to keep up with the increasing volume of submissions and depend significantly on a few developers for these reviews. To help scale the patch review process, we introduce FLINT, a patch validation system framework that synthesizes insights from past discussions among developers and automatically analyzes patch proposals for compliance. FLINT employs a rule-based analysis informed by past discussions among developers and an LLM that does not require training or fine-tuning on new data, and can continuously improve with minimum human effort. FLINT uses a multi-stage approach to efficiently distill the essential information from past discussions. Later, when a patch proposal needs review, FLINT retrieves the relevant validation rules for validation and generates a reference-backed report that developers can easily interpret and validate. FLINT targets bugs that traditional tools find hard to detect, ranging from maintainability issues, e.g., design choices and naming conventions, to complex concurrency issues, e.g., deadlocks and data races. FLINT detected 2 new issues in Linux v6.18 development cycle and 7 issues in previous versions. FLINT achieves 21% and 14% of higher ground-truth coverage on concurrency bugs than the baseline with LLM only. Moreover, FLINT achieves a 35% false positive rate, which is lower than the baseline.
Figures & tables
Figure 1: Emails exchanged in the Linux memory management mailing list from 2015 to November 2025.
Figure 2: Number of patches and without any reply in the mailing list of the memory management subsystem from 2015 to Nov 2025, with random sampling of the 100 threads for each year. The number in the figure for each year is the percentage of the patches without any reply.
Figure 3: Distribution of the feedback came from. The dataset is a random sampling of 20 reply emails for each year.
Total
Threads
228
Emails
5,717
Replies
3,950
Patches
1,633
Table 1: LKML 2024 sampled dataset properties.
Figure 4: System Overview.
Figure 5: A simplified rule generated by FLINT .
Figure 6: A simplified report from about a race condition [ 38 , 2 ] reported by FLINT .
Stage
Rule Category
Numbers
1. Extraction
5211
2. Filter
3003
3. Categorization
Logic
1825
Convention
1178
4. Consolidation
Logic
592
Convention
437
Table 2: Number of rules after each rule generation stage.
Figure 7: An example of comments providing more details.
Figure 8: Abbreviated example of an evaluated issue [ 38 , 2 ] by CoT-verifier.
Model
Knowledge cutoff
Thinking
Gemini-2.5-Pro
January, 2025 [ 13 ]
Dynamic
Gemini-2.5-Flash ∗
Gemma3:4b
August, 2024 [ 41 ]
No
Gemma3:12b
Qwen3-coder:30b ∗
May, 2025 [ 47 , 24 ] (release date)
Yes
Table 3: Information of models in evaluation. We use qwen3-coder:30b-a3b-q4_K_M as Qwen3-coder:30b, and gemini-2.5-flash-preview-09-2025 as Gemini-2.5-Flash in the system.
Figure 9: Filtered, merged, and collected code logic and convention rules as threads are progressively analyzed.
Gemini-2.5-Pro
Gemini-2.5-Flash
Metric
Full Set
Full Set
High Conf.
( N=80 )
( N=80 )
( >90 , N=41 )
Cohen’s Kappa
0.925
0.475
0.809
Precision
0.950
0.500
0.750
Recall
0.974
0.952
1.000
F1 Score
0.962
0.658
0.857
Table 4: Performance and inter-rater reliability of the CoT-verifier against human review. The evaluation covers 80 judgments balanced across correctness labels (10 true and 10 false per criterion). We compare Gemini-2.5-Pro (verifying Gemini-2.5-Pro) against Gemini-2.5-Flash (verifying Gemma3:4b). Full Set includes all 80 judgments; High Conf. refers to the subset where the confidence score is >90 .
Score
GA3:4b
GA3:12b
Q3C:30b
G2.5F
G2.5P
GCS
0.18
0.35
0.51
0.63
0.65
0.43
0.41
0.47
0.71
0.63
H-GCS
0.27
0.40
0.75
0.72
0.67
0.52
0.59
0.78
0.77
0.75
Table 5: The true positives analysis. Overall GCS and H-GCS with Best-of- 10 and majority voting, K=3 . FLINT R/S is gray color, and FLINT LLM is white color. Gemini-2.5-Pro (G2.5P) and Gemini-2.5-Flash (G2.5F)’s CoT-verifier are Gemini-2.5-Pro, others, Gemma3 (GA3:4b and GA3:12b) and Qwen3-coder:30b (Q3C:30b), CoT-verifier is Gemini-2.5-Flash. Higher is better.
Figure 10: The true positives analysis. GCS and H-GCS with Best-of-10 and majority voting, K=3 , of Gemini-2.5-Pro.
System
Total
TP
FP
FPR
Prec.
FLINT R/S
20
13
7
35%
65%
FLINT LLM
20
9
11
55%
45%
Table 6: False report rate and precision of FLINT R/S versus FLINT LLM with Gemini-2.5-Pro.
Historical Linux kernel patches capture defect knowledge that applies beyond their original repair sites. Recent work has shown that large language models (LLMs) can generate static-analysis checkers from historical patches and use them to uncover new kernel bugs. However, complete-checker generation requires the model both to recover the defect semantics expressed by a patch and to implement sophisticated program-analysis machinery, including object tracking, alias analysis, path-state maintenance, and interprocedural propagation. Coupling these responsibilities in a single end-to-end code-generation task can turn a simple defect rule into an unstable and expensive analyzer-implementation problem. To address this problem, we present TyPatch, which decouples patch-specific defect semantics from analyzer implementation. An LLM translates each patch into a typestate rule specifying its tracked object, actions, guards, transitions, and violations. A shared backend then executes these rules, binding their actions to program events, tracking object identity across aliases, propagating typestate along program paths, and producing reports for all rules. On Linux v6.16, TyPatch finds 559 distinct bugs, 121 of which have been confirmed by kernel developers. In a matched 38-patch comparison with the state-of-the-art complete-checker construction workflow, TyPatch uses 88.3-90.1% fewer generation tokens, while its initial report pools achieve 3.42-14.95× the precision of those produced by that workflow.
Ruoyu Wang, Tuo Li, Jia Li
College of AI Tsinghua University Beijing, China · The University of Hong Kong Hong Kong, China · Tsinghua University Beijing, China
Code review is credited with substantially changing a patch's code between its first submission and the version that eventually lands. However, prior work typically studied only the final merged patch without comparing it to the first submission. We present a function-level measurement that tracks 10,117 trajectories (each function followed across the numbered revisions of one patch series) through the patch history of the Linux IIO subsystem, comparing similarity scores against unrelated function pairs as a baseline. A naive reading yields near-total similarity, but this is largely an artifact of composition: 75.3% of tracked trajectories are never textually modified between versions, contributing a trivial 100% similarity that inflates the headline. Restricting to the trajectories with a real edit, semantic purpose is still largely preserved (mean similarity 0.990 vs. a 0.909 baseline), but drift appears to concentrate in the first review round mainly because later rounds contain more functions that nobody touched, not because edits become more conservative over time. After controlling for it, a statistically detectable but small residual effect remains. This points to an open question: whether near-ceiling similarity reflects preserved purpose or a measurement tool that cannot detect the significance of small, localized edits. We present this work as a first look and outline next steps.
Lucas Ciziks, Paulo Meirelles, Marco Aurélio Gerosa
Universidade de São Paulo (IME-USP) São Paulo, Brazil · Northern Arizona University (NAU) Flagstaff, AZ, USA
LLM-assisted defect discovery has a precision crisis: plausible-but-wrong reports overwhelm maintainers and degrade credibility for real findings. We present Refute-or-Promote, an inference-time reliability pattern combining Stratified Context Hunting (SCH) for candidate generation, adversarial kill mandates, context asymmetry, and a Cross-Model Critic (CMC). Adversarial agents attempt to disprove candidates at each promotion gate; cold-start reviewers are intended to reduce anchoring cascades; cross-family review can catch correlated blind spots that same-family review misses. Over a 31-day campaign across 7 targets (security libraries, the ISO C++ standard, major compilers), the pipeline killed roughly 79% of 171 candidates before advancing to disclosure (retrospective aggregate); on a consolidated-protocol subset (lcms2, wolfSSL; n=30), the prospective kill rate was 83%. Outcomes: 4 CVEs (3 public, 1 embargoed); LWG 4549 accepted to the C++ working paper; 5 merged C++ editorial PRs; 3 compiler conformance bugs; 8 merged security-related fixes without CVE; an RFC 9000 errata filed under committee review; and 1+ FIPS 140-3 normative compliance issues under coordinated disclosure -- all evaluated by external acceptance, not benchmarks. The most instructive failure: ten dedicated reviewers unanimously endorsed a non-existent Bleichenbacher padding oracle in OpenSSL's CMS module; it was killed only by a single empirical test, motivating the mandatory empirical gate. No vulnerability was discovered autonomously; the contribution is external structure that filters LLM agents' persistent false positives. As a preliminary transfer test beyond defect discovery, a simplified cross-family critique variant also solved five previously unsolved SymPy instances on SWE-bench Verified and one SWE-rebench hard task.