The IJCNN 2025 Review Process

Michele Scarpiniti; Danilo Comminiello

cs.DLFeb 19, 2026

The IJCNN 2025 Review Process

Authors: Michele Scarpiniti, Danilo Comminiello

Organizations: Department of Information Engineering, Electronics and Telecommunications, Sapienza University of Rome, via Eudossiana 18, 00184 Rome, Italy

Abstract

The International Joint Conference on Neural Networks (IJCNN) is the premier international conference in the area of neural networks theory, analysis, and applications. The 2025 edition of the conference comprised 5,526 paper submissions, 7,877 active reviewers, 426 area chairs, 2,152 accepted papers, and more than 2,300 attendees. This represents a growth of about 100% in terms of submissions, 200% in terms of reviewers, and over 50% in terms of attendees as compared to the previous edition. In this paper, we describe several key aspects of the whole review process, including a strategy for ranking the scores provided by the reviewers by evaluating a score index and a calibrated version used experimentally to remove reviewer-specific bias from reviews.

Figures & tables

Group	Tag	Number	Completed
Group 1	Rev 1	3,702	3,174
Group 2	Rev 2	4,064	3,351
Group 3	Rev 3	1,272	1,059
Group 3	Rev 4	164	152
Total		9,202	7,736

TABLE I: Reviewer distribution at IJCNN 2025.

Track	Number of Reviewers	Reports
Main	7,527	18,427
Competition	10	17
Position Papers	73	130
Workshops	267	432
Total	7,877	18,996

TABLE II: Reviewers per track and related received reports.

Track	Tag	Number
Regular	RegAC	262
Special Sessions	SpeSAC	120
Special Track	STrAC	2
Competitions	CompAC	4
Position Papers	PosPAC	1
J2C Papers	J2CAC	1

TABLE III: Area Chair (AC) distribution at IJCNN 2025.

Track	Submissions	Accepted
Regular	4,225	1,533
Special Sessions	1,049	478
Special Track	30	12
Competitions	11	7
Position Papers	66	15
Workshops	145	107

TABLE IV: Submission distribution at IJCNN 2025.

Option	$\mathbf{C_{1}}$ – $\mathbf{C_{4}}$	$\mathbf{R_{c}}$
Excellent	$~1.0$	1.2
Very good	$~0.5$	1.1
Good	$~0.0$	1.0
Fair	$-0.5$	0.9
Poor	$-1.0$	0.8

TABLE V: Weights assigned to the quality criteria

C_{1}

–

C_{4}

and the Reviewer’s confidence

R_{c}

Option	$\mathbf{S}$
Strong Accept	$~3$
Accept	$~2$
Weak Accept	$~1$
Borderline	$~0$
Weak Reject	$-1$
Reject	$-2$

TABLE VI: Weights assigned to the overall recommendation provided by reviewers and meta-reviewers.

Panel of Figure 7 — Fig. 1: Calibrate reviewers’ scores: (a) histogram of the calibrated scores $\mathbf{y}$ in ( 18 ), (b) scatterplot of the original vs. calibrated reviewers’ scores.

Fig. 2: The per-paper acceptance probability with respect to the calibrated score provided by reviewers.

Fig. 3: The scatterplot of the original vs. calibrated meta-reviewers’ scores.

Scholarly Peer Review

Scholarly peer review studies how experts evaluate research, make publication decisions, and improve the fairness, reliability, and transparency of those processes.

See all 78 papers

The Review Lottery: Benchmarking an Observational Estimator of Peer-Review Noise (ICLR 2017-2025)
Oct 5, 2026Feilian HuangInter-Rater Reliability Annotator Disagreement
Independent researcher
Is Peer Review Really in Decline? Analyzing Review Quality across Venues and Time
Jan 21, 2026Ilia Kuznetsov, Rohan Nayak, Alla Rozovskaya +1Scholarly Peer Review
Ubiquitous Knowledge Processing Lab (UKP Lab), Department of Computer Science, Technical University of Darmstadt and National Research Center for Applied Cybersecurity ATHENE · Department of Computer Science at Queens College, City University of New York (CUNY)
Can We Believe AI Peer Review?
Oct 14, 2026Zongrui Yang, Haoyuan Li, Zhongsheng Wang +2Scholarly Peer Review Automated Peer Review
University of Auckland Auckland, New Zealand

Oct 5, 2026J/K move · Enter open · S save

The Review Lottery: Benchmarking an Observational Estimator of Peer-Review Noise (ICLR 2017-2025)

Feilian Huang

Independent researcher

Inter-Rater Reliability Annotator Disagreement Scholarly Peer Review

Open paper AskPDF arXiv

How much of a conference accept/reject decision would change if the same paper were reviewed by a different set of reviewers? Running a second independent program committee is the gold standard for answering this, but it is prohibitively expensive: done only twice (NeurIPS 2014 and 2021). We build an observational estimator of this quantity from public review data alone, calibrate it twice, and apply it to nine years of ICLR (2017-2025; 36,113 papers, 134,912 reviews). The estimator decomposes scores with a Bayesian ordered-probit model into paper quality and reviewer noise, maps scores to decisions with a logistic model, and simulates two independent committees (posterior draws B=1,000; committee sizes k=2,3,4). Estimated disagreement rates are 23-30% at k=2 and 18-24% at k=4; 30-50% of accepted papers would be rejected. External calibration: at the NeurIPS 2021 reviewer-count caliber (k=3), the simulated 2021 disagreement rate is 23.3% [21.7%, 25.0%] vs. reported 23.0% (bias +0.3pp); accept precision and committee correlation agree within 5pp and 0.04. Internal calibration: on 18,740 papers with 4+ reviews, random model-free 2+2 reviewer splits agree with the k=2 simulation within 1pp in 2018 and 2021-2025. Longitudinally, we find no robust time trend in reviewer noise over 2017-2025. The high accepted-paper flip rates of 2020 and 2021 have distinct mechanisms: the 2020 four-point scale compressed scores (23.7% of papers had zero within-paper variance), and a counterfactual shows coarsening the scale raises disagreement by about 7pp; 2021 instead combined the lowest signal-to-noise ratio in the sample with the most threshold-crowded acceptances. For the LLM era, a 2023 breakpoint test on within-paper score variance finds no break, but the design has almost no power, and no post-2022 review text or confidence data exist, so no LLM attribution is attempted.

Abstract

Scholarly Peer Review

Explore similar work

The Review Lottery: Benchmarking an Observational Estimator of Peer-Review Noise (ICLR 2017-2025)

Is Peer Review Really in Decline? Analyzing Review Quality across Venues and Time

Can We Believe AI Peer Review?