Reviewer Capability Governs Rejection Targeting, Not Repair Skill: Evidence from LLM Execute-Review-Revise Pipelines

Faizan Tanveer

cs.SESep 2, 2026

Reviewer Capability Governs Rejection Targeting, Not Repair Skill: Evidence from LLM Execute-Review-Revise Pipelines

Authors: Faizan Tanveer

Organizations: National University of Computer and Emerging Sciences (FAST NUCES)

Abstract

Multi-agent LLM pipelines increasingly assign roles, including execution and verification, to models of different capability tiers. This is done because running a flagship model at every stage is expensive. Previous literature has established that verification stages are not always beneficial, but holds reviewer capability roughly fixed relative to the executor. We vary it. We replace the reviewer with models spanning a capability range down to one that cannot solve the problems at all, and measure the outcome of every individual rejection. This is done across a constant set of 100 olympiad mathematics problems. A cross-family mid-tier reviewer improves final accuracy by 12 percentage points, from 52 to 64 percent (p = 0.0005), with zero damaged answers. Same-model self-review attains the highest error-detection rate of any condition (0.85 recall) yet yields no significant gain: it rejects 2.1 times as often for a third the repair rate (15 against 43 percent, p = 0.0074) and falsely rejects 35 percent of its own correct answers against 2 percent for the cross-family reviewer (paired p = 0.000015). The low damage rate of self-review proves to be an artifact of revision inertia rather than reviewer quality: of 18 falsely rejected correct answers, the three where the executor complied all became wrong, while the fifteen it ignored survived unchanged. Below a capability floor the role becomes inert: our weakest reviewer changed zero of 100 final answers while doubling token cost. These findings describe a single executor-reviewer configuration on 100 problems and should be read as a controlled pilot rather than a general claim about verification stages.

Figures & tables

Condition	Accuracy	vs. C0	McNemar $p$
C0, no review	0.52	—	—
C1, self-review	0.58	$+6$ pp	0.146, ns
C2, cross-family	0.64	$+12$ pp	0.0005
C3, weak	0.52	0	1.000

Table 1: Final accuracy by condition. Exact McNemar on paired per-problem outcomes (

N=100

Condition	Recall	False-rej. rate	Rejections
C1, self-review	0.85	18/52, 35%	59
C2, cross-family	0.56	1/52, 2%	28
C3, weak	0.04	0/52, 0%	2

Table 2: Detection behaviour. Recall is

P(\textsc{reject}\mid A_{0}

wrong

)

; the false-rejection rate is

P(\textsc{reject}\mid A_{0}

correct

)

, over the 52 problems where

A_{0}

was correct.

Condition	Rejections	Repair	Damage	No-change ( $A_{0}$ wrong)	No-change ( $A_{0}$ right)	Miscorrection
C1	59	9, 15%	3, 5%	9, 15%	15, 25%	23, 39%
C2	28	12, 43%	0, 0%	3, 11%	1, 4%	12, 43%
C3	2	0	0	2	0	0

Table 3: Every rejection classified by outcome. No-change is split by whether

A_{0}

was correct, because a no-change on a correct answer is a false rejection that happened to be harmless. Percentages are of that condition’s rejections.

	False-rej. rate	Repair $\mid$ change
	(targeting)	(skill)
C1, self-review	18/52, 35%	9/35, 26%
C2, cross-family	1/52, 2%	12/24, 50%
Test	McNemar	Fisher
$p$	0.000015	0.096

Table 4: Targeting versus repair skill. Repair skill conditions on the executor having actually changed the answer, excluding no-change cases.

Cond.	Tokens/task	$\Delta$ tokens	Tokens per extra correct
C0	668	—	—
C1	2,207	$+1{,}539$	$\approx 25{,}600$
C2	3,511	$+2{,}844$	$\approx 23{,}700$
C3	1,353	$+685$	undefined

Table 5: Token cost. Counts are exact provider usage metadata, not estimates.

Explore similar work

Review Rarely Beats the Stronger Model: Separating Verification from Capability Transfer in LLM Pipelines
Oct 11, 2026Yirui Zhang, Yash Raj ShresthaLLM Answer Verification LLM Evaluation
University of Lausanne
When Does a Second Model Help? Cross-Model Review in LLM Verification
Oct 1, 2026Tae-Eun SongLLM Evaluation Language Model Error Detection
Daejeon Jungang Cheonggua Co., Ltd.
LLMs as a Jury: Cross-Model Consensus Can Outperform Process Reward Models for LLM Reasoning
Jul 11, 2026Ning LiuLLM Answer Verification Test-Time Scaling
Independent Researcher

Oct 11, 2026J/K move · Enter open · S save

Review Rarely Beats the Stronger Model: Separating Verification from Capability Transfer in LLM Pipelines

Yirui Zhang, Yash Raj Shrestha

University of Lausanne

LLM Answer Verification LLM Evaluation LLM-as-a-Judge

Open paper AskPDF arXiv

Qwen3-8B reviewing Llama-3.1-8B in math adds 33.2 points, yet Qwen3-8B alone is 6.4 points more accurate than the result. Reviewing another model's answer is usually judged by its gain over the generator. We show that this confuses verification, which adds accuracy beyond either model alone, with capability transfer, where a stronger reviewer supplies answers it could have given on its own. Measured against the stronger model alone, across 48 pairs of 12 open-weight models (3B to 72B) in math, knowledge, code and reasoning, review beats it in only 9 of 96 review directions, and does worse the larger the capability gap. The reason is anchoring: the reviewer mostly follows the generator, discarding its own correct answers. Review costs 2.6 to 2.8 times as much as the stronger model alone, while a selector using the same two calls is 2.7 to 4.2 points more accurate than review and cheaper. Studies of verification should report the stronger model alone, not only the gain over the generator.

Abstract

Explore similar work

Review Rarely Beats the Stronger Model: Separating Verification from Capability Transfer in LLM Pipelines

When Does a Second Model Help? Cross-Model Review in LLM Verification

LLMs as a Jury: Cross-Model Consensus Can Outperform Process Reward Models for LLM Reasoning