Organizations: Department of Statistics and Data Science, University of Pennsylvania · Department of Computer Science, University of Illinois at Urbana-Champaign · Center for Computational and Genomic Medicine, Children’s Hospital of Philadelphia · Department of Biomedical and Health Informatics, Children’s Hospital of Philadelphia · Department of Pathology and Laboratory Medicine, University of Pennsylvania · Department of Genetics, University of North Carolina at Chapel Hill · Department of Biomedical Data Science, Stanford University · School of Management, Yale University · Department of Biostatistics, University of North Carolina at Chapel Hill · Department of Medicine, University of Pennsylvania
Publishing a research manuscript is a routine yet demanding part of scientific life: time-consuming, stressful, and often uncertain in outcome. Recent advances in large language model (LLM)-based agentic AI have shown promise across a range of scientific tasks, and here we ask whether agentic AI can help researchers navigate the publication process itself by reliably inferring a manuscript's eventual publication venue from its content and literature context. We introduce PASS (Publication-oriented Agentic Scientific System), an agentic system that understands manuscripts within their domain-specific literature context and predicts top-matched publication venues. PASS positions each manuscript within its surrounding literature landscape by reconstructing its local scientific neighborhood, tracing its topic trajectory, and reasoning over field-specific journal spaces. Evaluated on a leakage-audited benchmark of over 2,000 preprints across 16 biomedical fields, PASS achieved Top-1 accuracy of 50.3% and Top-5 accuracy of 86.1%, outperforming state-of-the-art LLM baselines and established journal-selection tools. PASS-produced quality scores, such as impact potential and novelty, aligned with independent measures of publication outcome. We also found that the designed literature retrieval module is the strongest performance contributor, particularly for positioning manuscripts relative to nearby work, and that PASS maintained near-full performance from the abstract alone, whereas LLM baselines required the full manuscript text. An independent human evaluation found strong researcher agreement with PASS's manuscript understanding and recommendation rationale. PASS has been released as a public platform (https://ratemypaper.ai/) for broad researcher access.
Figures & tables
Figure 1: Overview of PASS. a , Taking a paper as input, the manuscript-understanding agent analyzes the title, abstract, and main sections to construct a compact summary of the paper’s storyline, contribution, and limitations, producing manuscript-derived signals. b , The literature-retrieval agent converts the manuscript profile into targeted queries and uses retrieved related papers to assess overlap, novelty and positioning relative to prior work, while review and benchmark papers provide broader field trends. c , The journal-fit reasoning agent integrates task-specific instructions, manuscript metadata and literature evidence to jointly generate a ranked list of candidate journals with rationales and manuscript-level assessments of novelty, impact, soundness and writing quality. The final output provides best-fit journals, evidence-grounded rationales and feedback for strengthening the manuscript before submission.
Figure 2: PASS improves journal-landing prediction across biomedical fields. Top-K accuracy of naive GPT-5, task-engineered GPT-5, and PASS across the 15 largest bioRxiv subject categories and a combined medRxiv evaluation set.
Figure 3: PASS scores provide interpretable manuscript-level signals associated with eventual publication venue. a, Representative PASS reports for high- and low-scoring manuscripts. The green check indicates the true published journal. b, Distributions of PASS scores across 2,184 evaluated manuscripts. Scores are reported on a 0–10 scale for the overall assessment and four component dimensions. c, Association between the overall manuscript score and three journal-level metrics: SJR rank, CiteScore and journal H-index. Journals are grouped into metric bins, with higher-prestige or higher-visibility bins shown on the left. P -values are from a Wilcoxon rank-sum test across the highest and lowest metric bins. d, Dimension-specific score differences between the highest and lowest journal-metric bins. ⋆⋆⋆ indicates P -value <0.001 from a Wilcoxon rank-sum test across the highest and lowest metric bins.
Figure 4: Ablation analysis identifies complementary signals underlying PASS performance. a, Ablation design. Ablation variants remove one component at a time. b, Mean Top-K accuracy drop after removing each component. c, Top-K accuracy curve comparing the full system with the ablation variant without literature evidence. d, Retrieval-source ablation showing the effect of removing related studies, review and benchmark papers, or both. e, Distribution of the number of retrieved related papers by manuscript type. f, Mean Top-K accuracy drop after removing literature evidence, stratified by manuscript type.
Figure 5: Agentic reasoning preserves journal-landing signal from section-subset manuscripts. a, Evaluation design for PASS using the full manuscript and section-subset manuscripts. b, Top-K accuracy for three models across manuscript input settings. c, Mean Top-K accuracy across (K=1,…,5), normalized to full-text performance.
Figure 6: Human evaluation supports the accuracy and usefulness of PASS reports. a, Researcher assessment of PASS journal recommendations. b, Researcher agreement with PASS report quality across paper understanding, topic identification, and improvement roadmap. c, Distribution of the number of manuscript-improvement comments generated per report; the dashed line indicates the mean of 12.1 comments. d, Fraction of comments judged useful by researchers for each report; points indicate individual responses and the diamond indicates the mean of 80.2%. e, Researcher agreement with PASS manuscript-level assessments across impact potential, novelty, conceptual soundness, and writing quality, evaluated for overall assessment, strengths, and weaknesses.
1Max-Planck-Institut für Quantenoptik, Garching, Germany · 2Munich Center for Quantum Science and Technology (MCQST), Schellingstrasse 4, D-80799 München, Germany · 3Leinweber Institute for Theoretical Physics, Stanford University, Stanford, CA, USA
School of Interdisciplinary Engineering and Sciences (SINES), National University of Sciences and Technology (NUST), Islamabad, Pakistan · College of Electrical and Mechanical Engineering (CEME), NUST, Rawalpindi, Pakistan