Are LLMs Reliable Code Reviewers? Systematic Overcorrection in Requirement Conformance Judgement

Haolin Jin; Huaming Chen

cs.SEFeb 28, 2026

Are LLMs Reliable Code Reviewers? Systematic Overcorrection in Requirement Conformance Judgement

Authors: Haolin Jin, Huaming Chen

Organizations: University of Sydney, Sydney, NSW, Australia.

Abstract

Large language models (LLMs) have become essential tools in software development, widely used for requirements engineering, code generation and review tasks. Software engineers often rely on LLMs to verify if code implementation satisfy task requirements, thereby ensuring code robustness and accuracy. However, it remains unclear whether LLMs can reliably determine code against the given task descriptions, which is usually in a form of natural language specifications. In this paper, we uncover a systematic failure of LLMs in matching code to natural language requirements. Specifically, with widely adopted benchmarks and unified prompts design, we demonstrate that LLMs frequently misclassify correct code implementation as non-compliant or defective. Surprisingly, we find that more detailed prompt design, particularly with those requiring explanations and proposed corrections, leads to higher misjudgment rates, highlighting critical reliability issues for LLM-based code assistants. We further analyze the mechanisms driving these failures and evaluate the reliability of rationale-required judgments. Building on these findings, we propose a Fix-guided Verification Filter that treats the model proposed fix as executable counterfactual evidence, and validates the original and revised implementations using benchmark tests and spec-constrained augmented tests. Our results expose previously under-explored limitations in LLM-based code review capabilities, and provide practical guidance for integrating LLM-based reviewers with safeguards in automated review and development pipelines.

Figures & tables

Figure 1: Workflow of evaluating LLM code conformance on canonical and buggy solutions under three prompting modes, with outputs logged for downstream scoring.

Figure 2: The Full prompt template used in our experiments. It provides the requirement and code, then requests a three-step response.

Category	Explanation
Misread Spec	The model misreads or overlooks key requirement semantics (e.g., interpreting constraint A as constraint B).
Added Requirement	The model introduces constraints not stated in the requirement and rejects the code for violating these hallucinated requirements.
Overthink Edge	The model overemphasizes extreme or unspecified edge cases, despite the requirement not mandating such handling (or the code already covering it).
Assumed Type	The model assumes stricter (or different) input types/formats and rejects the implementation based on those assumptions.
Imagined Runtime	The model speculates about runtime errors (e.g., index error, None handling, exceptions) without concrete evidence from the given code and requirement.
Performance Nitpick	The model treats efficiency/complexity as a hard requirement even when not mentioned, and rejects correct code due to perceived inefficiency.

Table 1: FN reason taxonomy for normalizing the main rationale patterns in those judgments.

Model	Approch	HumanEval		MBPP		Quixbugs
		FPR	FNR	FPR	FNR	FPR	FNR
GPT-4o	Direct	2.44	26.2	3.70	35.9	10.9	35.0
	Direct + Explain	0.00	58.5	0.00	74.1	5.00	45.0
	Full	0.00	73.2	0.20	87.9	5.00	60.0
Gemini-2.0-flash	Direct	8.54	25.6	10.3	34.7	22.5	25.0
	Direct + Explain	7.32	23.2	11.1	35.1	22.5	22.5

Table 2: False-positive rate (FPR, %) and false-negative rate (FNR, %) of five LLMs on three benchmarks (HumanEval, MBPP, and QuixBugs) under three prompting approaches ( Direct , Direct+Explain , and Full ). FNR reflects over-correction (rejecting correct implementations), while FPR reflects unsafe acceptance (accepting buggy implementations). Green highlights relatively lower error rates within each model benchmark block, whereas red highlights relatively higher error rates.

Figure 3: Absolute FN and FP counts across prompt settings for GPT-4o and Llama-3.1-8B.

Figure 4: Distribution of perceived fault types in FN rationales (left) and top-4 perceived errors by model (right). Categories include Misread_Spec (spec misinterpretation), Added_Requirement (unstated constraints), Overthink_Edge (boundary overconcern), Assumed_Type (format assumption), Imagined_Runtime (unsupported runtime speculation), Perf_Nitpick (performance critique), Read_Nitpick (style critique), Precision_Error (numeric precision), Logic_Error (algorithmic flaw claim), and Vague_Description (vague unsupported reasoning).

Figure 5: Counts of inconsistent rationales (labeled as contradiction or unclear by a GPT-4o evaluator) under rationale-enabled prompts. Bars compare Direct+Explain vs. Full for each model across HumanEval, MBPP, and QuixBugs, report absolute counts.

Figure 6: Directional breakdown of contradiction cases by model. Each chart shows the fraction of Contradiction_NO (verdict=NO but rationale positive) vs. Contradiction_YES (verdict=YES but rationale negative), center labels report percentages and the total number of contradictions (

n

Model	HumanEval		MBPP		QuixBugs
	BugMatch	SymMatch	BugMatch	SymMatch	BugMatch	SymMatch
GPT-4o	59.1	98.2	70.8	94.7	58.3	100.0
Claude-4.5	65.0	97.5	75.2	93.3	57.9	92.1
Gemini-2.0	52.0	98.7	67.2	95.8	38.2	91.2
Llama-3.1	44.2	96.8	52.2	96.0	41.7	100.0
Mistral-3.1	50.0	96.8	65.7	94.5	58.3	95.8

Table 3: A2 fault-awareness on buggy instances when the model verdict is correct. We report match rates (%) between the rationale and ground-truth bug type matched and failure symptoms matched.

Figure 7: Example of a symptom is correct but bug type is not matched, the model correctly states the failure symptom, but frames the root cause as Missing Logic (highlighted in red) instead of the ground-truth Value Misuse.

Figure 8: bug-type alignment on TN buggy instances, Bars report the distribution of rationale labels (match, mismatch, not mentioned, unclear) with respect to ground-truth bug types.

Figure 9: Overview of the Fix-guided Verification Filter. When the judge outputs NO with a proposed fix, we execute both the original and fixed implementations on the benchmark tests and generated spec-constrained tests, and update the final verdict according to four executable cases.

Model	Setting	HumanEval		MBPP		QuixBugs
		FPR	FNR	FPR	FNR	FPR	FNR
GPT-4o	Full (Before)	0.0	70.7	0.0	88.7	5.0	57.5
	Full+Filter (After)	1.8	23.2 $\downarrow$ ( 47.6 )	0.4	40.0 $\downarrow$ ( 48.8 )	7.5	27.5 $\downarrow$ ( 30.0 )
Gemini-2.0-flash	Full (Before)	7.8	25.0	11.1	30.1	22.5	20.0
	Full+Filter (After)	6.4	5.8 $\downarrow$ ( 19.2 )	10.2	10.2 $\downarrow$ ( 19.9 )	22.5	12.5 $\downarrow$ ( 7.5 )
Claude-4-5-sonnet	Full (Before)	1.9	46.0	5.3	61.2	5.0	50.0

Table 4: False-positive rate (FPR, %) and false-negative rate (FNR, %) before vs. after applying the Fix-guided Verification Filter under Full prompting. For each Full+Filter row, the FNR cell additionally reports the absolute reduction (percentage points) from the corresponding Full baseline as

\downarrow(\Delta)

Automated Code Review

Automated code review uses software analysis and AI to find defects, assess changes, and support pull-request decisions.

See all 48 papers

Inferring Code Correctness from Specification
May 28, 2026Tambon Florian, Papadakis MikeAutomated Software Testing Code Generation Evaluation
University of Luxembourg Luxembourg, Luxembourg
Bias in the Loop: Auditing LLM-as-a-Judge for Software Engineering
Apr 18, 2026Zixiao Zhao, Amirreza Esmaeili, Fatemeh FardSoftware Engineering LLM-as-a-Judge
University of British Columbia, Canada
From Natural Language to Verified Code: Toward AI Assisted Problem-to-Code Generation with Dafny-Based Formal Verification
Apr 24, 2026Md Erfan, Md Kamal Hossain Chowdhury, Ahmed Ryan +1LLM Prompting Code Generation
Department of Computer Science, The University of Alabama, Tuscaloosa, USA · Alabama Water Institute, The University of Alabama, Tuscaloosa, USA

May 28, 2026J/K move · Enter open · S save

Inferring Code Correctness from Specification

Tambon Florian, Papadakis Mike

University of Luxembourg Luxembourg, Luxembourg

Automated Software Testing Code Generation Evaluation Specification-Based Testing LLM Code Generation

Open paper AskPDF arXiv

Large language models (LLMs) have become integral to modern software development, enabling automated code generation at scale. However, validating the correctness of LLM-generated code remains a critical and largely unsolved challenge. Existing approaches either rely on dynamic consensus across multiple code candidates - making them costly and difficult to scale - or on static reasoning that is susceptible to dynamic bugs and order bias. In this paper, we propose TRAILS~ (Targeted Reasoning Agreement via Inputs and Specifications), an approach that grounds LLM reasoning with concrete (input, output) pairs. TRAILS~ first generates diverse test inputs via category partitioning based on the specification, then executes them against the candidate code and prompts LLMs to assess whether the resulting input-output pairs conform to the specification - without ever reasoning over the code itself. Scores are aggregated across inputs, to determines whether the program is likely correct. We evaluate TRAILS~ on two datasets, LiveCodeBench and CoCoClaNeL, across three LLMs (Qwen3Coder-30B, Devstral-Small-24B, and Olmo3.1-Instruct), comparing against HoarePrompt and a Zero-Shot Chain-of-Thought baseline. TRAILS~ improves Matthew Correlation Coefficient by up to 39% relative to Zero-Shot COT and consistently outperforms HoarePrompt. Beyond accuracy, TRAILS~ demonstrates greater stability across seeded runs, reducing sensitivity to LLM non-determinism, and assigns correct labels to a larger set of unique code samples than competing approaches.

Abstract

Automated Code Review

Explore similar work

Inferring Code Correctness from Specification

Bias in the Loop: Auditing LLM-as-a-Judge for Software Engineering

From Natural Language to Verified Code: Toward AI Assisted Problem-to-Code Generation with Dafny-Based Formal Verification