Peer Review
Peer review, the process of evaluating scholarly work by experts, aims to ensure research quality and integrity. Current research focuses on automating aspects of peer review using machine learning, particularly large language models (LLMs) and techniques like transfer learning and knowledge distillation, to address challenges such as bias, efficiency, and the increasing volume of submissions. These advancements offer the potential to improve the speed and fairness of the review process, ultimately enhancing the reliability and impact of scientific publications. However, ongoing research also addresses ethical concerns surrounding the use of AI in this critical process, including the detection of AI-generated reviews and the potential for bias amplification.
Papers
Enhancing Peer Review in Astronomy: A Machine Learning and Optimization Approach to Reviewer Assignments for ALMA
John M. Carpenter, Andrea Corvillón, Nihar B. Shah
'Quis custodiet ipsos custodes?' Who will watch the watchmen? On Detecting AI-generated peer-reviews
Sandeep Kumar, Mohit Sahu, Vardhan Gacche, Tirthankar Ghosal, Asif Ekbal
AI-Driven Review Systems: Evaluating LLMs in Scalable and Bias-Aware Academic Reviews
Keith Tyser, Ben Segev, Gaston Longhitano, Xin-Yu Zhang, Zachary Meeks, Jason Lee, Uday Garg, Nicholas Belsten, Avi Shporer, Madeleine Udell, Dov Te'eni, Iddo Drori
Can an unsupervised clustering algorithm reproduce a categorization system?
Nathalia Castellanos, Dhruv Desai, Sebastian Frank, Stefano Pasquali, Dhagash Mehta