Patch reviewing is critical for software development, especially in distributed open-source development, which highly depends on voluntary work, such as Linux. This paper studies the past 10 years of patch reviews of the Linux memory management subsystem to characterize the challenges involved in patch reviewing at scale. Our study reveals that the review process is still primarily reliant on human effort despite a wide-range of automatic checking tools. Although kernel developers strive to review all patch proposals, they struggle to keep up with the increasing volume of submissions and depend significantly on a few developers for these reviews. To help scale the patch review process, we introduce FLINT, a patch validation system framework that synthesizes insights from past discussions among developers and automatically analyzes patch proposals for compliance. FLINT employs a rule-based analysis informed by past discussions among developers and an LLM that does not require training or fine-tuning on new data, and can continuously improve with minimum human effort. FLINT uses a multi-stage approach to efficiently distill the essential information from past discussions. Later, when a patch proposal needs review, FLINT retrieves the relevant validation rules for validation and generates a reference-backed report that developers can easily interpret and validate. FLINT targets bugs that traditional tools find hard to detect, ranging from maintainability issues, e.g., design choices and naming conventions, to complex concurrency issues, e.g., deadlocks and data races. FLINT detected 2 new issues in Linux v6.18 development cycle and 7 issues in previous versions. FLINT achieves 21% and 14% of higher ground-truth coverage on concurrency bugs than the baseline with LLM only. Moreover, FLINT achieves a 35% false positive rate, which is lower than the baseline.
Figures & tables
Figure 1: Emails exchanged in the Linux memory management mailing list from 2015 to November 2025.
Figure 2: Number of patches and without any reply in the mailing list of the memory management subsystem from 2015 to Nov 2025, with random sampling of the 100 threads for each year. The number in the figure for each year is the percentage of the patches without any reply.
Figure 3: Distribution of the feedback came from. The dataset is a random sampling of 20 reply emails for each year.
Total
Threads
228
Emails
5,717
Replies
3,950
Patches
1,633
Table 1: LKML 2024 sampled dataset properties.
Figure 4: System Overview.
Figure 5: A simplified rule generated by FLINT .
Figure 6: A simplified report from about a race condition [ 38 , 2 ] reported by FLINT .
Stage
Rule Category
Numbers
1. Extraction
5211
2. Filter
3003
3. Categorization
Logic
1825
Convention
1178
4. Consolidation
Logic
592
Convention
437
Table 2: Number of rules after each rule generation stage.
Figure 7: An example of comments providing more details.
Figure 8: Abbreviated example of an evaluated issue [ 38 , 2 ] by CoT-verifier.
Model
Knowledge cutoff
Thinking
Gemini-2.5-Pro
January, 2025 [ 13 ]
Dynamic
Gemini-2.5-Flash ∗
Gemma3:4b
August, 2024 [ 41 ]
No
Gemma3:12b
Qwen3-coder:30b ∗
May, 2025 [ 47 , 24 ] (release date)
Yes
Table 3: Information of models in evaluation. We use qwen3-coder:30b-a3b-q4_K_M as Qwen3-coder:30b, and gemini-2.5-flash-preview-09-2025 as Gemini-2.5-Flash in the system.
Figure 9: Filtered, merged, and collected code logic and convention rules as threads are progressively analyzed.
Gemini-2.5-Pro
Gemini-2.5-Flash
Metric
Full Set
Full Set
High Conf.
( N=80 )
( N=80 )
( >90 , N=41 )
Cohen’s Kappa
0.925
0.475
0.809
Precision
0.950
0.500
0.750
Recall
0.974
0.952
1.000
F1 Score
0.962
0.658
0.857
Table 4: Performance and inter-rater reliability of the CoT-verifier against human review. The evaluation covers 80 judgments balanced across correctness labels (10 true and 10 false per criterion). We compare Gemini-2.5-Pro (verifying Gemini-2.5-Pro) against Gemini-2.5-Flash (verifying Gemma3:4b). Full Set includes all 80 judgments; High Conf. refers to the subset where the confidence score is >90 .
Score
GA3:4b
GA3:12b
Q3C:30b
G2.5F
G2.5P
GCS
0.18
0.35
0.51
0.63
0.65
0.43
0.41
0.47
0.71
0.63
H-GCS
0.27
0.40
0.75
0.72
0.67
0.52
0.59
0.78
0.77
0.75
Table 5: The true positives analysis. Overall GCS and H-GCS with Best-of- 10 and majority voting, K=3 . FLINT R/S is gray color, and FLINT LLM is white color. Gemini-2.5-Pro (G2.5P) and Gemini-2.5-Flash (G2.5F)’s CoT-verifier are Gemini-2.5-Pro, others, Gemma3 (GA3:4b and GA3:12b) and Qwen3-coder:30b (Q3C:30b), CoT-verifier is Gemini-2.5-Flash. Higher is better.
Figure 10: The true positives analysis. GCS and H-GCS with Best-of-10 and majority voting, K=3 , of Gemini-2.5-Pro.
System
Total
TP
FP
FPR
Prec.
FLINT R/S
20
13
7
35%
65%
FLINT LLM
20
9
11
55%
45%
Table 6: False report rate and precision of FLINT R/S versus FLINT LLM with Gemini-2.5-Pro.