Trusting the Tool: Asymmetric Deference of LLM Vulnerability Auditors to Static-Analysis Evidence

ZhiHong Liu; Shuyao Gao

cs.SEOct 23, 2026

Trusting the Tool: Asymmetric Deference of LLM Vulnerability Auditors to Static-Analysis Evidence

Authors: ZhiHong Liu, Shuyao Gao

Organizations: Seoul School of Integrated Sciences and Technologies (aSSIST University), Seoul, Republic of Korea · Liaoning Mechanical & Electrical College of Technology, Dandong, Liaoning, China

Abstract

Pipelines that combine large language models (LLMs) with static analysis let the model consume the analyser's output as evidence, but whether the model distinguishes true from false evidence, and whether its scrutiny depends on what the evidence claims, has not been measured. We built an intraprocedural analyser for C/C++ that produces proof obligations and audited vulnerable and patched functions from PrimeVul under six evidence conditions that differ only in the notes, including fabricated assurances that a missing check exists and fabricated alarms that an added check is missing. The study covers 916 pairs in four disjoint sets, eleven model configurations from four vendors and 16,572 audits; no model is trained or fine-tuned. True evidence barely improved accuracy (+0.02). False evidence acted asymmetrically: relative to true notes about the same operations, false assurances lowered the detection of real vulnerabilities by up to 19 points, whereas false alarms never significantly lowered accuracy and were disputed far more often. An internally registered replication on seven new models confirmed that false assurances move verdicts towards their claim more than false alarms do (mean asymmetry +0.16, 95% CI +0.10 to +0.24). Instructions in the audit prompt did not reliably reduce deference; the best one failed to replicate. Verifying each claim in a separate call and showing the refuted claims raised accuracy under false assurances by 12 points in a pre-specified test (p=0.002) without loss under true evidence; for a second model the gain was smaller and not significant. Pipelines should therefore verify a tool's guarantees before the model relies on them, and studies should measure deference in a direction-aware way.

Figures & tables

Figure 1: Study overview. For each PrimeVul pair, an intraprocedural analyser produces proof obligations for both versions; the fix commit decides which obligations are relevant and how false evidence is built. The same function is audited under every evidence condition and, for RQ4, under different presentations of the same notes.

	Evidence	Content	Functions
C0	none	plain audit	374
C1	all, true	every obligation the analyser reports	374
C2	relevant, true	only patch-relevant obligations	189
C3	irrelevant, true	only obligations away from the patch that carry a concern	183
C4	false assurance	vulnerable versions: relevant concerns replaced by fabricated, refutable guarantees	83
C5	false alarm	patched versions: concerns the patch removed, guards deleted	97

Table 1: Evidence conditions. Functions: number of study-set functions to which a condition applies.

Name in this paper	API model identifier	R	Vendor	Sets (conditions)
deepseek-chat	deepseek-chat		DeepSeek	S (all); H, D (C1, C2, C4, C5); T (C2, C4)
deepseek-flash (R)	deepseek-flash	✓	DeepSeek	S (all)
deepseek-v4-pro (R)	deepseek-v4-pro	✓	DeepSeek	S, 60 pairs (C0, C2, C4, C5)
qwen-plus	qwen-plus-2025-12-01		Alibaba	S (C2, C4, C5); T (C2, C4)
Registered replication (H3), via Alibaba Cloud Bailian
glm-5.2	glm-5.2		Zhipu AI	S (C2, C4, C5)

Table 2: Model configurations. R: reasoning enabled (effort low ). Sets: S study, H held-out, D development, T test. Conditions in brackets were run on a subset only.

Figure 3: Order of the RQ4 evaluations. After the strategy selected on the study set failed to replicate, strategies were compared on a separate development set only, and the frozen strategy was tested once on unseen pairs with pre-specified hypotheses. Values are accuracy changes under false assurances relative to the plain presentation (uncorrected McNemar

p

Figure 4: Accuracy per evidence condition on the study set, for vulnerable versions (left) and patched versions (right). Conditions apply to different subsets of functions ( Table 1 ), so the figure is descriptive; paired comparisons are in Fig. 5 . False evidence exists only for one version: false assurances ( C4 ) for vulnerable versions and false alarms ( C5 ) for patched ones.

Figure 5: Effect of false evidence relative to true evidence about the same operations, with 95% bootstrap CIs. Orange: false assurance on vulnerable versions ( C4

-

C2 ); blue: false alarm on patched versions ( C5

-

C2 ). The upper block shows the four exploratory configurations and the lower block the seven models of the registered replication ( Section 6.4 ). *

p<.05

, **

p<.01

(exact McNemar). † 60-pair subset.

Model	Contrast	$\boldsymbol{n}$	True $\rightarrow$ false	Discordant	$\boldsymbol{p}$	Difference (95% CI)
deepseek-chat	C4 vs C2	83	0.75 $\rightarrow$ 0.64	10 / 1	0.012	$-0.11$ ( $-0.18$ , $-0.04$ )
	C5 vs C2	67	0.67 $\rightarrow$ 0.79	0 / 8	0.008	$+0.12$ ( $+0.04$ , $+0.19$ )
deepseek-flash (R)	C4 vs C2	83	0.87 $\rightarrow$ 0.90	3 / 6	0.508	$+0.04$ ( $-0.04$ , $+0.11$ )
	C5 vs C2	67	0.37 $\rightarrow$ 0.55	3 / 15	0.008	$+0.18$ ( $+0.06$ , $+0.30$ )
deepseek-v4-pro (R) †	C4 vs C2	29	0.97 $\rightarrow$ 0.83	4 / 0	0.125	$-0.14$ ( $-0.28$ , $-0.03$ )
	C5 vs C2	28	0.29 $\rightarrow$ 0.32	4 / 3	1.000	$+0.04$ ( $-0.14$ , $+0.21$ )

Table 3: Matched contrasts on the study set.

n

: functions with both conditions. Discordant: functions correct only under the true notes / only under the false notes.

Claims	$\boldsymbol{n}$	Refuted	Confirmed	Unclear
C4 false assurances	251	0.76 (0.70, 0.81)	0.10	0.14
C5 false alarms	428	0.79 (0.75, 0.82)	0.06	0.15
C2 true holds-here	203	0.02 (0.01, 0.06)	0.90	0.08
C2 true concerns	488	0.45 (0.41, 0.50)	0.31	0.24

Table 4: Verification of true and false claims when the model is asked to check them ( deepseek-chat , study set). Shares of claims; refuted with Wilson 95% CI.

Figure 6: (a) Share of answers that dispute a note according to the lexical cue, by model and type of false evidence (Wilson 95% CIs). (b) Share of answers that reject the false note according to each annotator, reweighted from the 80-answer stratified sample to all C4 / C5 answers (bootstrap 95% CIs).

Figure 7: How far false evidence moves verdicts towards its own claim, relative to true notes about the same operations (left), and the asymmetry between false assurances and false alarms (right), with 95% bootstrap CIs over pairs. Top: the four configurations from which the measure was derived; bottom: the seven models of the registered replication. Diamonds: mean over the models of a group. † 60-pair subset.

Strategy	Dimension	Change relative to plain
quote	framing	instruction to treat notes as unverified claims and to quote the code before relying on one
alarms	content	“holds here” assurances removed; operations and concerns kept
reframe	framing	assurances recast as claims to verify (“to verify: the tool reports …”)
provenance	source	the source text of every line a note cites is printed next to the citation
verify	checking	a separate call checks each claim against the code; refuted claims are withheld
verify-show	checking	as verify, but refuted claims are listed separately as contradicted by the code

Table 5: Presentation strategies. All use the same notes; only their presentation changes.

Figure 8: Verify-show on a false assurance (illustrative, simplified from a study-set pair). A first call checks each claim of the notes against the code and quotes the deciding line; the audit then sees the confirmed notes and, separately, the claims that the code contradicts.

Figure 9: Every presentation strategy evaluated, in the order of evaluation: accuracy change relative to the plain presentation on vulnerable versions under false assurances (left) and under true relevant evidence (right). Hollow markers: exploratory comparisons; filled markers: the pre-specified test, run once. Blue: deepseek-chat ; pink: qwen-plus .

LLM Auditing

Large language model auditing tests models and deployed systems for bias, safety, privacy, reliability, and provenance using behavioral and internal evidence.

See all 634 papers

Correct Verdicts, Flawed Reasoning: Structured Auditing of LLM-based Vulnerability Reasoning
Oct 5, 2026Boyue Caroline Hu, Kaivalya Ahir, Ronghao Ni +1LLM Auditing Software Vulnerability Detection
Carnegie Mellon University, Pittsburgh, Pennsilvanya, USA
PromptAudit: Auditing Prompt Sensitivity in LLM-Based Vulnerability Detection
May 22, 2026Steffen J. Camarato, Yahya Hmaiti, Mandana Ghadamian +1Prompt Sensitivity LLM Auditing
University of Central Florida Orlando, Florida, USA
Beyond Refusal: A Same-Lineage Study of Aligned and Abliterated LLMs for Vulnerability Analysis
Jul 7, 2026Mingchen Li, Meikang Qiu, Zifan Peng +4Software Vulnerability Detection LLMs for Cybersecurity
University of North Texas, USA · Augusta University, USA · Newcastle University, United Kingdom

Oct 5, 2026J/K move · Enter open · S save

Correct Verdicts, Flawed Reasoning: Structured Auditing of LLM-based Vulnerability Reasoning

Boyue Caroline Hu, Kaivalya Ahir, Ronghao Ni, Limin Jia

Carnegie Mellon University, Pittsburgh, Pennsilvanya, USA

LLM Auditing Software Vulnerability Detection Faithfulness of Language Model Reasoning Structured Reasoning LLMs for Cybersecurity

Open paper AskPDF arXiv

Large Language Models (LLMs) are increasingly deployed for automated software vulnerability analysis. Binary classification alone is insufficient; practitioners need explanations to triage bugs and engineer patches. Standard practice relies on Chain-of-Thought (CoT) prompting, but free-form reasoning allows models to obscure logical leaps, hallucinated execution steps, and internal inconsistencies behind plausible prose. Our manual audit reveals that approximately 60% of correct vulnerability verdicts are accompanied by fabricated or unverifiable claims, and free-form explanations allow reasoning errors to evade LLM-as-a-judge evaluation. We present Vulnerability Explanation Reasoning Auditor (VERA), an automated framework for auditing LLM vulnerability reasoning. Rather than accepting free-form text, VERA asks models to output a Structured Reasoning Record (SRR) encoding tracked pointers, memory operations, and state transitions in machine-readable fields. A multi-stage judge audits each SRR against eight reasoning failure modes using deterministic checks, with LLM calls reserved for semantic interpretation. The standardized SRR schema also enables automated mutation testing to benchmark judges at scale without human annotation. Our evaluation shows reasoning flaws occur in correct verdicts just as frequently as incorrect ones, and VERA exposes 87% of reasoning errors that free-form LLM-as-judge systematically miss.

Abstract

LLM Auditing

Explore similar work

Correct Verdicts, Flawed Reasoning: Structured Auditing of LLM-based Vulnerability Reasoning

PromptAudit: Auditing Prompt Sensitivity in LLM-Based Vulnerability Detection

Beyond Refusal: A Same-Lineage Study of Aligned and Abliterated LLMs for Vulnerability Analysis