Sep 17, 2026 · cs.DLJ/K move · Enter open · S save
Justin Payan, Bálint Gyevnár, Atoosa Kasirzadeh, Nihar B. Shah
Machine Learning Department, Carnegie Mellon University · Institute for Complex Social Dynamics, Carnegie Mellon University · Departments of Philosophy & Software and Societal Systems, Carnegie Mellon University · Computer Science Department, Carnegie Mellon University
Conferences, journals, funders, schools, and universities are struggling with a surge of potentially AI-generated submissions from ostensibly human authors, who may not have exercised sufficient human oversight for their manuscripts. In turn, institutions evaluating submissions can no longer reliably credit expertise based solely on authors' names on submitted work. To address this problem, we propose greCAPTCHA, a proctored assessment approach that measures authors' understanding of research manuscripts via the construct of capacity to verify, which we define as the knowledge and reasoning required to critically assess the contents underlying one's contributions to a manuscript. greCAPTCHA generates questions assessing multiple levels of understanding and provides an evaluative report based on authors' responses. Using a prototype implementation, we conduct a user study and semi-structured interviews with
31 researchers to evaluate greCAPTCHA. Its automated scores predict which papers were or were not authored by study participants with an AUC of
0.90. Participants reported positive overall experiences with the system and remarked on the appropriate construct validity for author understanding, while also suggesting important changes to be made before deployment. Our results provide initial evidence that greCAPTCHA can assess manuscript-specific understanding under proctored conditions.