cs.AIAug 18, 2026

SGHA: Evidence-Grounded Research Problem Discovery with Local Language Models

Authors: Sarvesh Gharat, Junpei Komiyama

Organizations: C-MInDS, IIT Bombay · Machine Learning Department, MBZUAI

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Recent efforts toward fully automated AI scientists have demonstrated that language-model agents can generate hypotheses, execute experiments, and draft scientific manuscripts. However, during the early stages of research, when research problems are formulated, these AI scientists often rely heavily on proprietary frontier models. Their proposals are shaped by opaque parametric knowledge and by literature searches conditioned on the proposals themselves. Such knowledge is effectively a black box, and this dependence makes the evidential basis and validity of generated research problems difficult to audit and leaves the process vulnerable to model-specific hallucinations and biases. Furthermore, if proprietary research materials are transmitted to external APIs, the use of these models creates confidentiality, privacy, and data-governance concerns. We introduce the Structural Gap Hypothesis Agent (SGHA), a fully automated, corpus-first research-problem discovery system that runs entirely on a local LLM. SGHA structures a scientific literature corpus into evidence-linked paper objects and a typed evidence graph, detects unresolved structural patterns across papers, screens candidate gaps before formulation, and produces traceable research-problem families. In particular, it is able to output assumptions, objectives, success criteria, and remaining ambiguities. All LLM-based components of SGHA are executed using a locally served open-weight 9B language model, without requiring proprietary frontier-model APIs. We compare SGHA with the AI Scientist-v2 idea formulation module in five machine-learning domains. Our results suggest that explicit corpus structure and evidence-constrained reasoning can support promising, inspectable research-problem formulation without relying on frontier models during generation or verification.

Figures & tables

Figure 1: Overview of the SGHA pipeline. “Deterministic” denotes non-LLM stages. The evolutionary branch is optional and is reported separately from the verification-gated family outputs. Evolutionary operations are iterated multiple times and fed into the problem synthesis stage.
Figure 2: Evidence graph view of the Bandits worked example. The left side shows the surrounding literature context and the two source papers used in the worked example. The right side shows the typed local evidence graph: dTS is linked to linearity and bounded-context assumptions, an open issue around nonlinear diffusion models, and a nonlinear score-function failure condition. This local graph supports the assumption–failure candidate gap used later in the qualitative result.
Figure 3: Examples of structural motifs used for candidate-gap discovery. Each mini-panel shows a deterministic graph pattern queried over the typed evidence graph. A motif creates a candidate gap, not a final research problem. SGHA later screens, verifies, formulates, and consolidates these candidates before they can enter the final output.
Figure 4: Graph-mutation view of the optional evolutionary exploration branch. Starting from a literature-grounded seed gap, the branch proposes variants by changing assumptions, mechanisms, objectives, or regimes. Variants are critiqued, selected, and refined before a new exploratory hypothesis is retained. This branch is used for breadth and is kept separate from the main verification-gated family-report path unless stated otherwise.
More in this topic

AI Agents for Scientific Discovery

AI agents for scientific discovery use language models, tools, and experiments to generate, test, and refine scientific knowledge.

See all 288 papers

Explore similar work

CardsList
Jul 10, 2026

Toward Auditable AI Scientists: A Hypothesis Evolution Protocol for LLM Agents

Large language model (LLM) agents are increasingly expected to play a central role in AI-driven scientific discovery. Equipped with broad knowledge, flexible reasoning, and tool use, they have the potential to autonomously explore and solve scientific problems by repeatedly proposing hypotheses, testing them, and revising their beliefs in the light of the evidence. In current agents, however, these hypotheses, tests, and belief updates are buried in unstructured logs, and no mechanism lets the agent or the human researcher audit that process. Here we propose the Hypothesis Evolution Protocol (HEP), an agent harness that provides hypothesis generation, evaluation, and evolution as explicit, auditable operations. On materials-science research tasks, a HEP-equipped agent operates the hypothesis--test--evidence--belief cycle that planning-style agents lack, generalizes across research questions, and exploits the protocol more fully as the base LLM becomes more capable. These results mark a step toward auditable AI scientists, whose scientific reasoning can be inspected, verified, and built upon.

Izumi Takahara, Teruyasu Mizoguchi

Institute of Industrial Science, The University of Tokyo, Tokyo, Japan 153-8505

AI Agents for Scientific DiscoveryLLM AuditingScientific Hypothesis GenerationLLM Agent Harnesses
AskPDF
Jul 6, 2026

FirstResearch: Auditable Question Formation for LLM Scientific Discovery Agents

LLM systems for scientific discovery increasingly assist with ideation, literature synthesis, experiment planning, and report generation, but the first research question they propose can remain difficult to audit: it may sound plausible without exposing the mechanism, falsifier, or assumption that a scientist should inspect. We introduce FirstResearch, a first-principles research-question formation framework for scientific LLM agents whose core artifact is a structured Research Question Certificate. The certificate records primitive definitions, assumptions, a mechanism model, a tension or contradiction, a falsifiable hypothesis, a minimal decisive test, and a failure update rule, making the proposed question inspectable before downstream execution. On ten LLM-agent research topics, FirstResearch outperforms controlled prompt-level baselines inspired by AI co-scientist, Agent Laboratory, and AI Scientist-v2 under a primary DeepSeek-blind-judge protocol. A Gemini-2.5-Flash independent-judge rescore of the same 40 baseline packages preserves the system-level ranking, with FirstResearch scoring 4.86/5 versus 4.38/5 for the strongest baseline and Pearson agreement of 0.865 on average score. A one-repeat ablation checkpoint further suggests that the certificate-centered core is the strongest component: certificate-only scoring reaches 4.90/5 under DeepSeek and 4.88/5 under Gemini, while removing certificates drops below 1/5 under both judges. These results are preliminary and use LLM judges rather than human domain experts, but they support a narrow scientific-discovery claim: explicit derivation constraints are a promising mechanism for making LLM-generated scientific questions more auditable. Code, prompts, saved outputs, and reproduction scripts are available at https://github.com/louiswang524/FirstResearch.

Yufeng Wang

Scientific Hypothesis GenerationLarge Language Model-Guided Scientific Discovery
AskPDF
Mar 25, 2026

AI-Supervisor: Autonomous AI Research Supervision via a Persistent Research World Model

Existing automated research systems operate as stateless, linear pipelines -- generating outputs without maintaining any persistent understanding of the research landscape they navigate. They process papers sequentially, propose ideas without structured gap analysis, and lack mechanisms for agents to verify, challenge, or refine each other's findings. We present AI-Supervisor, a multi-agent orchestration framework where specialized agents provide end-to-end AI research supervision driven by human interests -- from literature review through gap discovery, method development, evaluation, and paper writing -- through autonomous exploration and self-correcting updates of research knowledge. Unlike sequential pipelines, AI-Supervisor maintains a continuously evolving Research World Model, implemented as a Knowledge Graph, that captures methods, benchmarks, known limitations, and unexplored gaps, serving as shared memory across all agents and enabling agents to explore and build upon a structured understanding of the research landscape. The framework introduces three architectural contributions: (1) structured gap discovery that decomposes methods into core modules, validates their performance across benchmarks, and maps the specific gaps each module creates; (2) self-correcting discovery loops that probe why modules succeed on certain problems and fail on others, whether benchmarks carry hidden biases, and whether evaluation protocols remain adequate for emerging challenges; and (3) self-improving development loops governed by cross-domain mechanism search that iteratively targets failing modules by finding solutions from other scientific fields. All agents operate under a consensus mechanism where independent findings are corroborated before being committed to the Research World Model.

Yunbo Long

Autonomous Research AgentsAI Agents for Scientific DiscoveryMulti-Agent LLM Systems
AskPDF