cs.AIOct 15, 2026

ClaimScope: Label-Blind Witness Verification for Mathematical Claim Auditing

Authors: Mushir Akhtar

Abstract

Mathematical agents are commonly evaluated by answer accuracy or proof acceptance. We introduce ClaimScope, a 72-task benchmark for auditing proposed claims through a verdict, scope, counterexample, and repair. Tasks cover optimizer-state transport, finite-horizon affine recurrences, and exact 2×22\times2 linear algebra. We evaluate Qwen3-8B with direct generation and four matched two-pass conditions, each starting from the same draft. A witness checker evaluates only the model's submitted state, without consulting reference verdicts or supplying a replacement witness. Its feedback still conveys information about validity: a verified refutation establishes that the claim is false, and rejection reasons can suggest structural fixes. In this evaluation, verifier feedback yields 23/3923/39 accepted counterexamples on invalid claims and 25/7225/72 safe completions, compared with 15/3915/39 and 20/7220/72 for checker-free generic feedback. Strict completion remains 17/7217/72; verdict accuracy changes from 47/7247/72 to 46/7246/72. The claim-grouped safe-completion difference is .094.094, with a bootstrap 95% interval of [.000,.198][.000,.198]. All seven task-level gains follow horizon/input-length diagnostics. A post-hoc schema-only correction of the direct outputs reaches the same 25 safe tasks without another model call. These results separate claim classification from executable evidence, while locating the observed gains in witness-structure correction rather than demonstrating broader reasoning improvement.

Figures & tables

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. ThinknCheck: Grounded Claim Verification with Compact, Reasoning-Driven, and Interpretable Models

    Apr 2, 2026Delip Rao, Feijiang Han, Chris Callison-BurchClaim VerificationLLM Answer Verification

  2. What Do Claim Verification Datasets Actually Test? A Reasoning Trace Analysis

    Apr 2, 2026Delip Rao, Chris Callison-BurchClaim VerificationLLM Evaluation