PaperCritic: Training AI Reviewers to Assess the Soundness of Papers
Organizations: KAIST AI · Carnegie Mellon University Language Technologies Institute · LG AI Research
Abstract
As AI conference submissions grow rapidly, AI reviewers are being considered as an alternative. However, simply imitating human reviewing is problematic because reviewing serves multiple purposes, including evaluating papers' soundness, interestingness, and clarity, the last two of which are subjective. In this work, we focus solely on evaluating soundness, the most important and objective criterion, and define the learning objective as jointly maximizing (1) precision, based on the soundness and substantiveness of review points, and (2) recall, the coverage of high-quality reference review points. Under this problem formulation, we construct a quality-labeled review point dataset, PaperSound-23K, using 1K ICLR submissions and 23K human-written and LLM-generated review points. Using this dataset, we train an AI reviewer model, PaperCritic, through reinforcement learning with the F1 score of precision and recall as the reward. Our experiments show that PaperCritic outperforms frontier models and models trained with other AI reviewer training methods on an in-domain test set. Surprisingly, learning to review ICLR submissions also transfers to reviewing Nature-family papers and even to out-of-domain reviewing tasks (e.g., code review, legal contract review, and feedback on story writing).
Figures & tables
| Human-Model | Human-Human | |
| Soundness | 0.654 | 0.640 |
| Claim Linkage | 0.611 | 0.606 |
| Severity | 0.563 | 0.557 |
| High-quality (binary) | 0.624 | 0.597 |
| Train | Test | Total | |
| # Papers | 889 | 100 | 989 |
| # Review points | 20,933 | 2,282 | 23,215 |
| Avg. review points / paper | 23.55 | 22.82 | 23.47 |
| Std. review points / paper | 7.04 | 6.12 | 6.95 |
| % Human review points | 62.6% | 61.2% | 62.4% |
| Statistics of Quality Labels | |||
| Precision | Recall | F1 | |
| Qwen3 larger/reasoning models | |||
| Qwen3-32B (non-thinking) | 14.8 | 15.5 | 11.8 |
| Qwen3-235B (non-thinking) | 17.5 | 21.2 | 10.7 |
| Qwen3-32B (thinking) | 16.5 | 22.6 | 13.5 |
| Qwen3-235B (thinking) | 19.5 | 34.3 | 19.2 |
| Frontier models | |||
| PeerReview Bench | |||
| Precision | Recall | F1 | |
| Qwen3-4B-Instruct-2507 | 34.0 | 30.0 | 22.7 |
| SFT | 40.9 | 33.3 | 26.8 |
| +RL (PaperCritic-4B) | 76.3 | 49.7 | 58.7 |
| SWE-PRBench | CLAUSE-Legal | StoryFeedback | |||||
| Precision | Recall | F1 | Precision | Recall | F1 | Recall | |
| Qwen3-4B-Instruct-2507 | 7.7 | 8.9 | 7.2 | 13.7 | 15.0 | 13.8 | 78.0 |
| SFT | 3.3 | 1.0 | 1.5 | 9.0 | 11.5 | 8.9 | 74.0 |
| +RL (PaperCritic-4B) | 8.3 | 17.0 | 10.5 | 14.7 | 37.3 | 20.1 | 90.0 |
| Precision | Recall | F1 | |
| PaperCritic-4B | 47.4 | 51.4 | 45.1 |
| w/o Matcher | 65.0 | 10.3 | 13.2 |
| w/o Soundness RM | 13.1 | 53.0 | 17.6 |
| w/o Substantiveness RM | 52.8 | 35.2 | 36.1 |
| w/ Matching-based Precision | 22.3 | 47.5 | 25.7 |
| Precision | Recall | F1 | |
| DeepSeek-V4-Flash-0731 | 41.5 | 34.1 | 33.3 |
| Claude Opus 5 | 27.3 | 54.2 | 33.1 |
| PaperCritic-4B | 48.2 | 50.3 | 44.4 |
| Precision | Recall | F1 | |
| Qwen3-8B | 13.0 | 4.9 | 4.2 |
| SFT | 26.0 | 28.7 | 20.0 |
| +RL (PaperCritic-8B) | 38.8 | 45.1 | 34.9 |
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
| Dimension | Category | Papers | Percentage (%) |
| Year | 2020 | 167 | 16.89 |
| 2021 | 192 | 19.41 | |
| 2022 | 233 | 23.56 | |
| 2023 | 397 | 40.14 | |
| Acceptance | Accepted | 392 | 39.64 |
| Rejected | 597 | 60.36 |
| Hyperparameter | SFT | DPO | RL |
| Learning rate | |||
| Paper epochs | 3 | 2 | 2 |
| Effective batch size | 16 reviews | 16 pairs | 30 reviews |
| Warmup ratio | 0.05 | 0.03 | 0.03 |
| LR schedule | Constant after warmup | Linear decay | Linear decay |
| Completion | Points/paper | Visible tokens/paper | |||
| Model | cap | Parsed | Retained | Full response | Retained points |
| Qwen3-4B-Instruct-2507 | 16,384 | 8.00 | 8.00 | 852.2 | 836.2 |
| SFT | 2,048 | 6.58 | 5.80 | 620.0 | 544.1 |
| +DPO | 2,048 | 6.02 | 6.02 | 538.8 | 526.7 |
| +RL w/ preference reward | 2,048 | 5.02 | 5.02 | 506.0 | 495.9 |
| +RL w/ similarity reward | 2,048 | 8.74 | 7.96 | 566.6 | 505.6 |
| Candidate models | Sound- ness | Substan- tiveness | Ref.- Matching |
| Qwen3-4B-Instruct-2507 | -0.059 | — | — |
| Qwen3-14B | 0.107 | 0.574 | 0.429 |
| Qwen3.6-35B-A3B | 0.244 | 0.596 | 0.613 |
| Distilled Qwen3-4B-Instruct-2507 | 0.407 | N/A | N/A |
| Model | F1 (%) | 95% CI | F1 (pp) | Paired 95% CI |
| PaperCritic-4B | 45.14 | [39.20, 50.81] | — | — |
| DeepSeek-V4-Flash-0731 | 39.32 | [33.56, 44.82] | [ , ] | |
| Claude Opus 5 | 39.10 | [34.10, 44.07] | [ , ] | |
| Gemini 3 Flash | 28.67 | [24.09, 33.12] | [ , ] | |
| Gemini 3.1 Pro | 25.04 | [19.38, 30.74] | [ , ] | |
| Claude Sonnet 5 | 28.86 | [23.76, 33.93] | [ , ] |
| Freq. | Quality Assoc. | |||
| Pattern | SFT | RL | Sound | Ref.-matched |
| “Can you” | 0.0 | 71.9 | +15.5 | +6.0 |
| “But it is unclear that” | 0.3 | 41.9 | +10.8 | +7.1 |
| Notation inconsistency | 0.3 | 9.5 | –25.1 | –12.5 |
| Soundness RM | Precision | Recall | F1 |
| Current RM | 47.4 | 51.4 | 45.1 |
| Qwen3-4B-Instruct-2507 | 20.4 | 33.6 | 19.7 |
| Precision | Recall | F1 | |
| Recursive generation | |||
| SFT | 18.5 | 30.6 | 17.4 |
| +RL w/ point-wise reward | 42.3 | 42.8 | 37.1 |
| One-shot generation (Ours) | |||
| SFT | 23.0 | 25.6 | 16.6 |
| +RL w/ F1 reward | 47.4 | 51.4 | 45.1 |
Automated Peer Review
Automated peer review uses computational systems, especially language models, to assess research papers, generate feedback, and support editorial decisions.
See all 121 papers