Plato's Cave: A Human-Centered Research Verification System

Matheus Kunzler Maldaner; Raul Valle; Junsung Kim; Tonuka Sultan; Pranav Bhargava; Matthew Maloni; John Courtney; Hoang Nguyen

cs.CLMar 6, 2026

Plato's Cave: A Human-Centered Research Verification System

Authors: Matheus Kunzler Maldaner, Raul Valle, Junsung Kim, Tonuka Sultan, Pranav Bhargava, Matthew Maloni, John Courtney, Hoang Nguyen,

Organizations: University of Florida · Georgia Institute of Technology

Abstract

The growing publication rate of research papers has created an urgent need for better ways to fact-check information, assess writing quality, and identify unverifiable claims. We present Plato's Cave as an open-source, human-centered research verification system that (i) creates a directed acyclic graph (DAG) from a document, (ii) leverages web agents to assign credibility scores to nodes and edges from the DAG, and (iii) gives a final score by interpreting and evaluating the paper's argumentative structure. We report the system implementation and results on a collected dataset of 104 research papers.

Figures & tables

Figure 1: System Overview. (a) The user provides a PDF, URL, or natural-language query. (b) For URLs or queries, a web-surfer agent browses for the specific paper. (c) All text and images from the document are extracted and stored. (d) The content is passed into an LLM and converted into a role-labeled DAG serialized as JSON (nodes and directed dependencies). (e) The frontend parses the DAG JSON and renders an interactive graph. (f) Web-surfer agents verify each node sequentially using external sources to produce normalized verification metrics. (g) Node qualities influence downstream nodes via trust-gated propagation along dependency edges. (h) The scorer aggregates node and edge signals into an overall paper-level score.

Figure 2: Plato’s Cave interface showing a finalized run with the visualized DAG and Integrity Score

Figure 3: System Architecture for Plato’s Cave

Figure 4: Paper-level mean score grouped by spreadsheet rating under the selected calibrated setting. We use a discrete, non-smoothed summary to avoid overstating distributional structure under weak supervision.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Edge feature	Wt.	Paper component	Wt.
role prior	0.30	Bridge	0.25
parent qual	0.20	Best Path	0.25
child qual	0.20	Redundancy	0.15
alignment	0.10	Fragility	$-$ 0.15
synergy	0.20	Coherence	0.10
		Coverage	0.10

Appendix

Table 1: Default weights for edge features and paper-level components.

Role	cred.	rel.	evid.	rigor	repr.	cite
Hypothesis	0.50	0.50	0.00	0.00	0.00	0.00
Conclusion	0.60	0.40	0.00	0.00	0.00	0.00
Claim	0.45	0.35	0.20	0.00	0.00	0.00
Evidence	0.20	0.00	0.50	0.00	0.00	0.30
Method	0.10	0.00	0.00	0.60	0.30	0.00
Result	0.40	0.30	0.30	0.00	0.00	0.00

Appendix

Table 2: Default role-specific node-quality weights. Columns correspond to (credibility, relevance, evidence strength, method rigor, reproducibility, citation support).

Parameter	Default	Meaning
enabled	True	enable/disable trust propagation
agg	min	parent aggregation ( min , mean , softmin , dampmin )
$\alpha$	1.0	exponent on parent trust (negative values are clamped to 0)
$\eta$	$2^{-1/8}\approx 0.917$	floor on the edge-gating factor
$\beta$	6.0	softmin sharpness (higher $\Rightarrow$ closer to min )
$\lambda$	0.35	dampmin mixing (higher $\Rightarrow$ closer to mean )

Appendix

Table 7

Parent role	Child role	Prior
Hypothesis	Claim	0.75
Hypothesis	Evidence	0.75
Hypothesis	Method	0.50
Hypothesis	Result	0.25
Hypothesis	Conclusion	0.25
Evidence	Result	1.00

Appendix

Table 3: Non-default role transition priors

r_{\rho_{u}\to\rho_{v}}

Role pair $(\rho_{u}\!\to\!\rho_{v})$	Parent mix	Child mix
Evi $\to$ Clm	evid:.5, cite:.3, cred:.2	cred:.6, rel:.4
Evi $\to$ Res	evid:.5, cite:.3, cred:.2	cred:.5, rel:.5
Evi $\to$ Con	evid:.5, cite:.4, cred:.1	cred:.7, rel:.3
Met $\to$ Res	rigor:.6, repr:.3, cred:.1	cred:.6, rel:.4
Hyp $\to$ Clm	cred:.3, rel:.7	cred:.6, rel:.4
Hyp $\to$ Evi	cred:.4, rel:.6	cred:.5, rel:.5

Appendix

Table 4: Role-pair synergy specifications (parent/child metric mixes)

Role	Definition
Hypothesis	Testable proposition or research question the paper investigates.
Evidence	Factual observations, data, or citations from prior work that support claims.
Claim	Assertions derived from evidence or reasoning.
Method	Descriptions of procedures, algorithms, or experimental designs.
Result	Outcomes obtained from applying Methods.
Conclusion	High-level takeaways synthesizing Results and answering Hypotheses.

Appendix

Table 5: Semantic role ontology. Each node must be assigned exactly one role.

Scientific Claim Verification

Scientific claim verification checks whether research statements are supported, contradicted, or left uncertain by relevant evidence.

See all 54 papers

ScientistOne: Towards Human-Level Autonomous Research via Chain-of-Evidence
May 25, 2026Rui Meng, Bhavana Dalvi Mishra, Jiefeng Chen +10Data Provenance Citation Verification
Google Cloud AI Research
ResearchQA: Benchmarking Citation-Grounded Question-Answering on Scientific Papers
Jul 13, 2026Saba Imran, Debanjum Singh SolankyLLM Evaluation Citation Verification
Khoj Inc.
Plato-Bio: verification-first biological novelty screening with temporal rediscovery and structural benchmarks
Jul 27, 2026Stefan G. CreadoreData Provenance Scientific Reproducibility
Praxa Labs, an independent open-source research initiative, United States

May 25, 2026J/K move · Enter open · S save

ScientistOne: Towards Human-Level Autonomous Research via Chain-of-Evidence

Rui Meng, Bhavana Dalvi Mishra, Jiefeng Chen, Chun-Liang Li+9

Google Cloud AI Research

Data Provenance Citation Verification Evidence-Grounded Generation Scientific Reproducibility Autonomous Research Agents Scientific Claim Verification

Open paper AskPDF arXiv

Autonomous research agents produce competitive solutions and professional-looking manuscripts, yet their outputs contain verifiability failures undetectable by surface-level evaluation: fabricated citations, unreproducible scores, and method descriptions that diverge from the implementation. We address this through three contributions. First, Chain-of-Evidence (CoE), a verifiability framework requiring every claim to be traceable to its evidence source. Second, ScientistOne, an end-to-end autonomous research system that maintains evidence chains by construction throughout literature review, solution discovery, and paper writing. Third, CoE Audit, a post-hoc audit whose four integrity checks -- score verification, specification violation, reference verification, and method-code alignment -- apply uniformly to all systems. Across 75 papers spanning five systems and five frontier research tasks, every baseline exhibits at least one systematic failure mode: hallucinated reference rates reach 21%, score verification passes in as few as 42% of papers, and method-code alignment ranges from 20% to 80%. ScientistOne achieves zero hallucinated references (0/337), perfect score verification (12/12), and the highest method-code alignment (14/15), while matching or exceeding human expert performance on all five tasks. ScientistOne further generalizes to six additional tasks spanning medical imaging, fine-grained recognition, 3D perception, and language modeling, achieving state-of-the-art on Parameter Golf and gold medals on MLE-Bench tasks where baselines fail entirely.

Abstract

Scientific Claim Verification

Explore similar work

ScientistOne: Towards Human-Level Autonomous Research via Chain-of-Evidence

ResearchQA: Benchmarking Citation-Grounded Question-Answering on Scientific Papers

Plato-Bio: verification-first biological novelty screening with temporal rediscovery and structural benchmarks