cs.IRFeb 9, 2026

OmniReview: A Large-scale Benchmark and LLM-enhanced Framework for Realistic Reviewer Recommendation

Authors: Yehua Huang, Penglei Sun, Zebin Chen, Zhenheng Tang, Xiaowen Chu

Organizations: The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China · The Hong Kong University of Science and Technology, Hong Kong, China

Abstract

Academic peer review remains the cornerstone of scholarly validation, yet the field faces some challenges in data and methods. From the data perspective, existing research is hindered by the scarcity of large-scale, verified benchmarks and oversimplified evaluation metrics that fail to reflect real-world editorial workflows. To bridge this gap, we present OmniReview, a comprehensive dataset constructed by integrating multi-source academic platforms encompassing comprehensive scholarly profiles through the disambiguation pipeline, yielding 202, 756 verified review records. Based on this data, we introduce a three-tier hierarchical evaluaion framework to assess recommendations from recall to precise expert identification. From the method perspective, existing embedding-based approaches suffer from the information bottleneck of semantic compression and limited interpretability. To resolve these method limitations, we propose Profiling Scholars with Multi-gate Mixture-of-Experts (Pro-MMoE), a novel framework that synergizes Large Language Models (LLMs) with Multi-task Learning. Specifically, it utilizes LLM-generated semantic profiles to preserve fine-grained expertise nuances and interpretability, while employing a Task-Adaptive MMoE architecture to dynamically balance conflicting evaluation goals. Comprehensive experiments demonstrate that Pro-MMoE achieves state-of-the-art performance across six of seven metrics, establishing a new benchmark for realistic reviewer recommendation.

Figures & tables

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Beyond Paper-to-Paper: Structured Profiling and Rubric Scoring for Paper-Reviewer Matching

    Apr 7, 2026Yicheng Pan, Zhiyuan Ning, Ludi Wang +1Automated Peer Review

  2. MERIT: Matching Expertise via Rubric-Informed Training for Reviewer Assignment

    May 27, 2026Zixuan Yang, Yibo Zhao, Weicong Liu +1Automated Peer Review