Automated Peer Review

Follow this topic Ask about this topic

Momentum

5 papers in the last four weeks, level with the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 66

All topics

Gaming AI-Assisted Peer Reviews Poses New Risks to the Scientific Community
Jun 8, 2026Lin Li, Qi Zhang, Xander Davies +2Adversarial Attacks Adversarial Attacks on LLMs
OATML, University of Oxford · MBZUAI
EGTR-Review: Efficient Evidence-Grounded Scientific Peer Review Generation via Multi-Agent Teacher Distillation
Jun 4, 2026Xinpeng Qiu, Wang Yihu, Zhifeng Liu +2Multi-Agent LLM Systems Evidence-Grounded Generation
Department of Information Management, Peking University · PKU-WUHAN Institute for Artificial Intelligence, Peking University
Can AI Refute Economic Theory? Evidence from Beyond the Knowledge Cutoff
Jun 3, 2026Alexis Akira TodaHuman-in-the-Loop AI Automated Peer Review
Department of Economics, Emory University.
Can AI Review Improve Paper Drafting? An Empirical Study on 20 Computer Architecture Submissions
May 31, 2026Di WuScholarly Peer Review Automated Peer Review
University of Central Florida Department of ECE Orlando, FL, USA
ReviewGuard: Aligning LLM-Assisted Peer Review with Long-Term Scientific Impact
May 29, 2026Abdur Rasool, Xiaohui Huang, Yanqing Hu +1Scholarly Peer Review Automated Peer Review
Department of Statistics and Data Science Southern University of Science and Technology Shenzhen 518055, China
SoundnessBench: Can Your AI Scientist Really Tell Good Research Ideas from Bad Ones?
May 28, 2026Sy-Tuyen Ho, Minghui Liu, Huy Nghiem +1AI for Science LLM Evaluation
University of Maryland, College Park
Feedback-to-Rubrics: Can We Learn Expert Criteria from Inline Comments?
May 28, 2026Kotaro Yoshida, So Kuroki, Yuki Imajuku +4Rubric-Based Evaluation Automated Peer Review
Sakana AI · Institute of Science Tokyo
PRAIB: Peer Review AI Benchmark of Behaviour of LLM-Assisted Reviewing
May 28, 2026Krzysztof Żurawicki, Julia Farganus, Arkadiusz Gaweł +2LLM Evaluation Scholarly Peer Review
Department of Artificial Intelligence Wrocław University of Science and Technology
Review Arcade: On the Human Alignment and Gameability of LLM Reviews
May 27, 2026Hans Ole Hatzel, Sebastian Steindl, Jan StrichLLM-as-a-Judge Human-AI Agreement
Language Technology Group, University of Hamburg, Germany · OTH Amberg-Weiden, Germany · Hub of Computing and Data Science (HCDS), University of Hamburg, Germany
GraphReview: Scientific Paper Evaluation via LLM-Based Graph Message Passing
May 26, 2026Pujun Zheng, Wanying Ren, Jiacheng Yao +2Scholarly Peer Review Automated Peer Review
School of Economics and Management, East China Normal University · Institute of Big Data, Fudan University
E3: Issue-Level Backtesting for Automated Research Critique
May 26, 2026Yashwardhan Chaudhuri, Sanyam Jain, Paridhi MundraAutomated Evaluation Automated Peer Review
TADDLE: A Tool-Augmented Agent for Detecting Deficient LLM-Generated Peer Reviews
May 26, 2026Hanqi Duan, Xiang LiTool-Augmented Language Model Agents LLM Agents
East China Normal University
PRISM: A Multi-Dimensional Benchmark for Evaluating LLM Peer Reviewers
May 26, 2026Ngoc Phan Phuoc Loc, Toan Huynh La Viet, Thanh Tran Khanh +8LLM Evaluation Scholarly Peer Review
VinUniversity · University of Illinois, Urbana-Champaign · University of Notre Dame +1
LLM-as-a-Reviewer: Benchmarking Their Ability, Divergence, and Prompt Injection Resistance as Paper Reviewers
May 25, 2026Lingyao Li, Junjie Xiong, Changjia Zhu +5LLM Evaluation LLM-as-a-Judge
University of South Florida · Missouri University of Science and Technology · University of Alabama +3
Rejoinder: The ICML 2023 Ranking Experiment: Examining Author Self-Assessment in ML/AI Peer Review
May 24, 2026Buxin Su, Jiayao Zhang, Natalie Collina +6Mechanism Design Scholarly Peer Review
*University of Pennsylvania. · †University of Wisconsin–Madison. · ‡Associate Chair of ICML 2023. University of North Carolina at Chapel Hill. +2
On the limits and opportunities of AI reviewers: Reviewing the reviews of Nature-family papers with 45 expert scientists
May 20, 2026Seungone Kim, Dongkeun Yoon, Kiril Gashteovski +55Scholarly Peer Review Automated Peer Review
To Trust or Not to Trust: Authors' Response to AI-based Reviews
May 15, 2026César Leblanc, Lukas PicekScholarly Peer Review Trust in AI
École Normale Supérieure, Paris, France. · Sorbonne University, Paris, France. · University of West Bohemia, Plzeň, Czech Republic. +1
When Reviews Disagree: Fine-Grained Contradiction Analysis in Scientific Peer Reviews
May 11, 2026Sandeep Kumar, Yash Kamdar, Abid Hossain +3Scholarly Peer Review Automated Peer Review
Department of Computer Science and Engineering, Indian Institute of Technology Patna, India · School of Computer Engineering, KIIT Deemed to be University, Bhubaneswar, India
CoCoReviewBench: A Completeness- and Correctness-Oriented Benchmark for AI Reviewers
May 8, 2026Hexuan Deng, Xiaopeng Ke, Yichen Li +6LLM Evaluation Automated Peer Review
Institute of Computing and Intelligence, Harbin Institute of Technology, Shenzhen, China · Zhongguancun Academy, Beijing, China · Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, China +2
Shattering the Echo Chamber: Hidden Safeguards in Manuscripts Against the AI Takeover of Peer Review
May 6, 2026Oubo Ma, Ruixiao Lin, Jiahao Chen +3LLM Security Indirect Prompt Injection
Zhejiang University
Stop Automating Peer Review Without Rigorous Evaluation
May 4, 2026Joachim Baumann, Jiaxin Pei, Sanmi Koyejo +1LLM-as-a-Judge Scholarly Peer Review
Stanford University · Bocconi University
ARA: Agentic Reproducibility Assessment For Scalable Support Of Scientific Peer-Review
May 4, 2026Kevin Riehl, Andres L. Marin, Nikofors Zacharof +6AI-Assisted Scientific Research Scientific Reproducibility
European Commission, Joint Research Centre · Ideas Forward
Can AI Be a Good Peer Reviewer? A Survey of Peer Review Process, Evaluation, and the Future
Apr 30, 2026Sihong Wu, Owen Jiang, Yilun Zhao +5LLM Evaluation Scholarly Peer Review
Yale University · New York University · TCS Research
SafeReview: Defending LLM-based Review Systems Against Adversarial Hidden Prompts
Apr 29, 2026Yuan Xin, Yixuan Weng, Minjun Zhu +5Adversarial Training Adversarial Prompt Generation
1CISPA · 2Westlake University · 3Southern University of Science and Technology +1
Making AI-Assisted Grant Evaluation Auditable without Exposing the Model
Apr 28, 2026Kemal BicakciLLM Auditing Privacy-Preserving ML
Informatics Institute, Istanbul Technical University, Istanbul, Türkiye · Securify Information Technology and Security Training Consulting Inc., Ankara, Türkiye
The Last Human-Written Paper: Agent-Native Research Artifacts
Apr 27, 2026Jiachen Liu, Jiaxin Pei, Jintao Huang +34AI-Assisted Scientific Research Scientific Communication
University of Michigan · Stanford University · Ohio State University +22
PeeriScope: A Multi-Faceted Framework for Evaluating Peer Review Quality
Apr 27, 2026Sajad Ebrahimi, Soroush Sadeghian, Ali Ghorbanpour +6Scholarly Peer Review Automated Peer Review
Reviewerly · Reviewerly, UC Berkeley · Reviewerly, University of Toronto
When AI reviews science: Can we trust the referee?
Apr 26, 2026Jialiang Wang, Yuchen Liu, Hang Xu +7LLM Auditing Adversarial Attacks on LLMs
Department of Computer Science and Engineering, The Hong Kong University of Science and Technology, Hong Kong SAR 999077, China · Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications, School of Computer Science and Engeering, Southeast University, Nanjing 210000, China · These authors contributed equally +2
What Makes a Good AI Review? Concern-Level Diagnostics for AI Peer Review
Apr 21, 2026Ming JinAutomated Peer Review Generative AI Evaluation
Bradley Department of Electrical & Computer Engineering Virginia Tech
Beyond Rating: A Comprehensive Evaluation and Benchmark for AI Reviews
Apr 21, 2026Bowen Li, Haochen Ma, Yuxin Wang +5LLM Evaluation Rubric-Based Evaluation
College of Computer Science and Artificial Intelligence, Fudan University, Shanghai, China · Shanghai Innovation Institute, Shanghai, China

Jun 8, 2026, cs.CLJ/K move · Enter open · S save

Gaming AI-Assisted Peer Reviews Poses New Risks to the Scientific Community

Lin Li, Qi Zhang, Xander Davies, Jianing Qiu+1

OATML, University of Oxford · MBZUAI

Adversarial Attacks Adversarial Attacks on LLMs Scholarly Peer Review Automated Peer Review Generative AI Evaluation

Open paper AskPDF arXiv

AI is increasingly used to support scientific peer review, from manuscript screening, reviewer assistance to editorial triage. Although such systems promise to reduce reviewer burden and accelerate publication, their robustness to strategic manipulation remains poorly understood. Here we show that AI-mediated peer review is vulnerable to a simple, low-cost manipulation: superficial rephrasing of the manuscript abstract. Without changing the underlying scientific content and communication, and even without knowledge of the reviewing model, adversarially rewritten abstracts substantially improve AI review outcomes. We see this across disciplines and publication venues, for both human-written and AI-generated papers. Our strongest attack achieves an attack-success-rate of about 38%, increasing acceptance ratings by +1.31 for Gemini 3 Flash reviewers and by +0.88 for GPT 5.4 Mini reviewers on a 10-point scale. When the original AI review suggests 'reject', the success rate rises to more than 50%. This effect extends beyond overall score inflation, increasing review confidence and scores on core scientific criteria such as soundness, significance and perceived contribution. The attack is practical, requiring only about 5 minutes and $1 for a 10-page AI conference submission, and is hard to distinguish from ordinary scientific editing. Inflated AI reviews could bias downstream human decision-making, shifting editorial recommendations from rejection towards acceptance. These findings reveal a general vulnerability in AI-assisted scientific evaluation: when AI-generated review influence editorial decisions, authors may be incentivized to optimize manuscripts for AI judgment rather than scientific merit. Our results suggest that AI tools should not be treated as neutral evaluators in high-stakes peer review without systematic robustness testing, transparent safeguards and careful human oversight.

Momentum

Related topics

Latest papers 66