Overview

Automated peer review studies how computational systems can evaluate manuscripts, identify weaknesses, check claims, and assist tasks such as reviewer matching, rebuttal, and meta-review. Common approaches include prompting and fine-tuning language models, retrieval-augmented and multi-agent review, rubric-based scoring, and tool-assisted checks of references, code, or experiments. Evaluation uses conference-paper datasets, expert annotations, outcome prediction, and injected-error or claim-verification benchmarks to test dimensions such as correctness, coverage, calibration, and actionability. Systems can provide useful and sometimes complementary feedback, but unreliable judgments, hallucinations, bias, and vulnerability to strategic rewriting or prompt injection leave human oversight and robust evaluation central concerns.

Latest papers

CardsList

122 papers11 papers in the last four weeks, up 10% on the four weeks before. 0.1% of all new papers.

  1. When Figures Disagree with Text: Evaluating Autonomous Visual Evidence Verification in Scientific Papers

    Nov 18, 2026Guijia Zhang, Hongbo Qiu, Jiajun Zha +10Multimodal Model EvaluationDocument Image Understanding

  2. PaperCritic: Training AI Reviewers to Assess the Soundness of Papers

    Nov 13, 2026Hyun Ryu, Ji Yong Cho, Eunho Yang +4Automated Peer ReviewScholarly Peer Review

  3. AI Reviews Are Biased Towards the Appendix

    Nov 13, 2026Saptarshi Ghosh, Mohammad Saim, Tianyu JiangAutomated Peer ReviewBias in LLM-as-a-Judge

  4. Auditing AI peer reviewers: a benchmark on real scientific papers

    Nov 12, 2026Íñigo Zubeldia, Boris Bolliet, Francisco Villaescusa-Navarro +2Automated Peer ReviewLanguage Model Error Detection

  5. ReviewMap: Mapping Evaluative Acts and Automation Boundaries in Peer-Review Comments

    Nov 5, 2026Kihyun Na, Injung KimScholarly Peer ReviewAutomated Peer Review

  6. Ranking Agreement Does Not Reveal Paragraph-Edit Agreement in Two Released Paper Acceptance Predictors

    Nov 4, 2026Shaoliang Yang, Jun WangAutomated EvaluationAutomated Peer Review

  7. Can AI Reviewers See Your Efforts to Improve Your Paper? A Large-Scale Study of Paper Revision Trajectories

    Nov 1, 2026Yaochen Wang, Chenguang Wang, Ming Li +8Scholarly Peer ReviewAutomated Peer Review

  8. Comparative Evaluation of LLM-Based Review Paradigms for Academic Paper Quality Assessment

    Oct 30, 2026Fan Huang, Hongyu ZhouAutomated Peer ReviewLLM-as-a-Judge

  9. Style over Substance? Linguistic Bias in LLM-based Peer Review

    Oct 26, 2026Sai Suresh Macharla Vasu, Sumedh Santosh Joshi, Philipp Christmann +6Linguistic Bias in Language ModelsBias in LLM-as-a-Judge

  10. PaperScorer: How Predictable Is Paper Acceptance?

    Oct 22, 2026Sachin Konan, Jonathan Liu, Zhuang LiuAutomated Peer Review

  11. Major Revision, Minor Accountability: LLMs as Reviewers of Published Articles in Top-Tier Journals

    Oct 22, 2026Bastien Le Guellec, Su Hwan Kim, Raphael Bentegeac +1Scholarly Peer ReviewAutomated Peer Review

  12. Author-in-the-loop Agentic Reproducibility Auditing for a Machine Learning Conference

    Oct 15, 2026Yukun Dong, Linjun Zhang, Tian ZhengML ReproducibilityScientific Reproducibility

  13. PremiseCheck: Efficient Fact-Checking of Human and AI Peer Reviews

    Oct 15, 2026Dingjie Song, Zhiling Yan, Ziqian Zhang +2Automated Fact-CheckingFactual Consistency Evaluation

  14. Can We Believe AI Peer Review?

    Oct 14, 2026Zongrui Yang, Haoyuan Li, Zhongsheng Wang +2Scholarly Peer ReviewAutomated Peer Review

  15. How Well Do LLMs Perform in Peer Review? Evidence from Finance and Accounting

    Oct 13, 2026Hai Lu, Yichen Luo, Luping YuAutomated Peer ReviewLLM-as-a-Judge

  16. How Is Automated Research Evaluated? A Survey of Benchmarks and Evaluation Practices

    Oct 8, 2026Liulei Zhang, Dejing Zhou, Chuyue Huang +5AI-Assisted Scientific ResearchBenchmark Design

  17. Beyond Imitation: A Framework and Benchmark for LLM-Assisted Peer Review

    Oct 8, 2026Rachel S. Y. Teo, Yutaro Yamada, Shashank Kotyan +2Automated Peer ReviewLLM Evaluation

  18. A Missing Piece for Trustworthy AI Reviewers: From Benchmarking Rhetorical Robustness to SciCore Review

    Sep 30, 2026Chenguang Wang, Ming Li, Chengrui Fan +4Automated Peer ReviewLanguage Model Robustness

  19. When Evidence Conflicts: Reliability-aware Meta-review Generation

    Sep 21, 2026Xinzhe Wang, Fei Tao, Jiang Xie +2Scholarly Peer ReviewEvidence-Grounded Generation

  20. Judging a Review by its Cover: A Reliability Analysis of LLM-based Peer Review Evaluation Metrics

    Sep 20, 2026Shakiba Amirshahi, Sajad Ebrahimi, Hai Son Le +2Automated Peer ReviewLLM-as-a-Judge

  21. One Reference or Many? Rethinking Multi-Reference Training for Intermediate-Entropy Text Generation

    Sep 19, 2026Maitreya Prafulla Chitale, Ketaki Mangesh Shetye, Yash More +4Text GenerationAutomated Peer Review

  22. When AI Reviews Train AI Reviewers: Scientific-Judgment Collapse and Mitigation

    Sep 17, 2026Sy-Tuyen Ho, Minghui Liu, Hieu Tran +1Automated Peer ReviewLLM Reliability

  23. PaperDoctor: Evidence-Grounded and Actionable Feedback for Scientific Papers in Progress

    Sep 15, 2026Kevin Qinghong Lin, Siyuan Hu, Pan Lu +14AI-Assisted Scientific ResearchScientific Reproducibility

  24. AppliedScientist: Automated Scientific Revision Through Iterative AI Reviewing

    Sep 13, 2026Vidushee Vats, Karun Sharma, Shengzhi Li +1AI-Assisted Scientific ResearchAutomated Peer Review

  25. ActReview: Rebuttal-Guided Training Data and Rubric Rewards for Actionable Peer Review Generation

    Sep 8, 2026Yiling Ma, Yilun Zhao, Sihong Wu +3LLM Post-TrainingAutomated Peer Review

  26. The Emerging AI Paper-Review Arms Race: Adversarial Co-Evolution in Scholarly Publishing

    Sep 7, 2026Chenguang Wang, Ming Li, Adebayo Braimah +6AI-Assisted Scientific ResearchScholarly Peer Review