Trust, but verify, audit and attribute: a human-in-the-loop framework for checking AI-produced research results
Authors: György Eigner, Patrik P. Süli, Roland Hollós
Organizations: Biomatics and Applied Artificial Intelligence Institute, John von Neumann Faculty of Informatics, Obuda University, Budapest, Hungary · Physiological Controls Research Center, Obuda University, Budapest, Hungary · Doctoral School of Applied Informatics and Applied Mathematics, Obuda University, Budapest, Hungary · HUN-REN Centre for Agricultural Research, Brunszvik u. 2., 2462 Martonvásár, Hungary
Artificial intelligence (AI) systems now produce research results in bulk: in October 2026 a single laboratory released more than 700 AI-generated mathematics manuscripts, only part of them with machine-checked proofs. Such volumes will be checked mostly by AI agents that call specialised software. A passing check certifies only the artefact that was checked. For proofs translated into the proof assistant Lean, it has been shown that this artefact may state something other than the author's text, and that deciding whether a translation is faithful is undecidable in general. The human researcher therefore stays in the loop, and human and AI contributions must be recorded. We propose a domain-independent, human-in-the-loop framework for checking AI-produced results. Validated, agent-usable tools with different grounds of trust check the results; a fidelity audit asks whether each check addressed the claim as stated; the human decides at fixed points; and a dialog report attributes the work to the human and to the AI. We describe two automatic implementations of the tool layer, for mathematics (Wolfram Language, MATLAB, Lean) and statistics (Python and R), and run the mathematical one on a two-claim example. An ambiguous claim was split into two readings, one refuted and one proved.
Figures & tables
Figure 1: The two gaps in every verification step. A validated tool closes the gap between the artefact and its verdict (solid arrow). Nothing in the tool closes the gap between the author’s claim and the artefact (dashed arrow); this is the gap the fidelity audit examines.
Figure 2: The framework. Double frames (blue): human roles; single frames (orange): AI roles; shaded frames (grey): Stage-1 tools, chosen to rest on different grounds of trust. Filled circles H1–H4: the four human decision points of section 5 . Dashed arrow: deviations returned for a faithful retry, an amended target or a negative verdict. Arrows into Stage 3: records that feed the dialog report.
Class
Examples
A verdict establishes
It does not establish
Formal proof
Lean 4 + Mathlib [ 16 , 17 ] ; Rocq [ 18 ]
The formal statement follows from the axioms
That the formal statement is the author’s claim
Symbolic computation
Wolfram Language [ 19 ]
Identities and decisions under stated assumptions
General results from instance checks; hidden branch choices
Numerical computation
MATLAB [ 20 ] , GNU Octave [ 21 ]
Numbers within stated tolerances
Cases near the tolerance; shared formulation errors
Statistical computation
R [ 22 ] ; SciPy [ 23 ] , statsmodels [ 24 ]
Estimates and tests for the data and model given
That data, model and test are those stated
Physical simulation
COMSOL [ 25 ]
Solutions of the discretised model
That model, parameters and mesh are those stated
Table 1: Classes of validated, agent-usable tools by ground of trust.
Figure 3: Wiring of the two examples. Single frames (orange): the AI agent with the skill and its scripts; shaded frames (grey): Stage-1 tools behind MCP servers, with the version used in this paper; numbers: the fixed order of the tool runs; dashed outline (blue): tools that run on the researcher’s computer; heavy frames (green): the validated record that goes to the human (double frames, decision point H2).
Design choice
Example 1: math-prove-tool
Example 2: stat-prove-tool
1. Tools and grounds of trust
Wolfram Language (symbolic decision), MATLAB (symbolic and numerical), Lean 4 + Mathlib (formal proof)
Python stat-validated server and R: two independent statistical implementations