StrictPRMBench: A Replayable Evaluation Benchmark for Auditing Fixed-Score PRM Aggregation
Organizations: Independent Researcher, Hangzhou, China
Abstract
Process reward models (PRMs) score reasoning steps, but best-of-N selection also requires a rule that aggregates those scores. We study that rule as a separate evaluation object. StrictPRMBench fixes mathematical reasoning pools, scorer evidence, and the source trace of a deletion intervention, so that different reducers face the same recorded selection problem. Across seven scoring checkpoints and two benchmarks, the stored N=32 outcomes give higher deletion-induced selection instability for product than geometric mean (GM) in all 14 cells. The equal-cell difference is +0.0965; a shared-problem bootstrap, conditional on these checkpoints and frozen pools, gives a 95% interval of [0.0846,0.1086]. A pre-injection-defined sensitivity excludes replacement slots that are either reducer's clean winner and retains 4,459 of 4,622 cell-problem records. The retained difference is +0.0986 [0.0875,0.1098]. Effects are heterogeneous: the two Qwen cells have positive percentile-bootstrap intervals on retained support but two-sided exact paired p-values of 0.0625 and 0.0703. Clean-pool Qwen accuracy does not detect a product advantage, but this is not an equivalence result. We provide versioned outcome records, CPU reanalysis, and an explicit evidence-availability contract. The contribution is a diagnostic evaluation protocol and a reproducible audit of recorded selector behavior, not a new PRM, a universally optimal reducer, or evidence of deployment-wide harm. Raw-score replay and outcome-record reanalysis are distinguished: missing primary edited-score vectors limit independent evaluation of new reducers under the same stress.
Figures & tables
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
| Analysis | Records | Meaning of denominator |
|---|---|---|
| Full product–GM | 4,622 | 14 cells over 847 distinct benchmark–question IDs; all available paired selector records, not an assertion that every clean winner is correct. |
| Comparison-safe GM | 4,459 | Same source records, excluding replacement of either compared clean winner; 845 distinct questions. |
| Comparison-safe min | 4,502 | Comparator-specific retained support; not the same sample as safe GM. |
| Comparison-safe last | 4,470 | Comparator-specific retained support. |
| Qwen natural utility | 1,000 | 500 unedited pools per benchmark, independent of stress eligibility. |
| MATH level analysis | 1,926 | 376 questions with both compared clean winners recorded as correct; not a comparison-safe filter. |
| Subset | Cards | GM | No preference | Product | Split |
|---|---|---|---|---|---|
| InternLM2 / GSM8K | 50 | 37 | 12 | 1 | 0 |
| InternLM2 / MATH | 50 | 25 | 25 | 0 | 0 |
| Skywork / GSM8K | 50 | 30 | 20 | 0 | 0 |
| Skywork / MATH | 50 | 21 | 27 | 0 | 2 |
| All cards | 200 | 113 | 84 | 1 | 2 |
| Input in anc/inputs/ | Evidence role |
|---|---|
| selection_records.csv | Recorded clean/injected identities and paired flip outcomes; all three comparators. |
| qwen_natural_records.csv | Recorded Qwen clean selections and correctness for 500 pools per benchmark. |
| math_level_records.csv | Level-stratification records, clean-correct flags, and original-candidate checks. |
| math500_level_map.csv | One-to-one problem/level mapping and source-question hashes. |
| qwen72b_records.csv | Public 143-row GSM8K pilot. |
| historical_ni_summary.csv | Published 14-cell summary; recount target for the NI estimates. |
| Checkpoint label | Base family/scale | Role in this benchmark |
|---|---|---|
| Math-Shepherd | Mistral, 7B | Process scorer; stepwise verification [ 2 ] . |
| Skywork-7B | Qwen, 7B | Process scorer [ 41 ] . |
| Skywork-1.5B | Qwen, 1.5B | Process scorer [ 41 ] . |
| Qwen-7B | Qwen, 7B | Qwen2.5-Math PRM; documented product aggregation [ 14 , 40 ] . |
| RLHFlow-8B | Llama, 8B | MC-estimation process reward implementation, not a 7B RLHF checkpoint [ 42 ] . |
| InternLM2-7B | InternLM2, 7B | Scalar reward checkpoint adapted by prefix scoring [ 43 ] . |
Benchmark Auditing
Benchmark auditing checks whether evaluation datasets, metrics, and infrastructure measure the capabilities and risks they claim to assess.
See all 109 papers