cs.AIAug 27, 2026
SaveAccelerating Scientific Research with Gemini in the Real-World
Organizations: Google DeepMind · Duke University · Columbia University · Google Research ·
Abstract
We present an extension and comprehensive real-world validation of Co-Scientist, a Gemini-based multi-agent system designed to accelerate end-to-end scientific research across hypothesis generation, experimentation, and manuscript generation. Moving beyond in silico hypothesis generation, this specialized configuration transitions Co-Scientist into an execution-grounded research partner advancing closed-loop scientific workflows across materials science, biology, and computer science. In materials science, Co-Scientist interfaced with a semi-automated chemical vapor deposition reactor to design a safe precursor route for MXenes; experimental execution produced a lamellar 2D material sharing key structural similarities with the Ti3C2Tx MXene lattice, although further experiments are needed to confirm the atomic structure. Leveraging Gemini 3 Deep Think for rapid, lab-in-the-loop execution, it also tailored growth recipes to laboratory constraints in minutes, enabling single-attempt growth of monolayer MoS2, MoSe2, and WS2 semiconductors. In biology, Co-Scientist predicted emergent swarming phenotypes of engineered E. coli across inducer (IPTG) gradients from sparse imaging data, quantitatively matching unpublished wet-lab morphological measurements. In computer science, Co-Scientist autonomously discovered an inference-time scaling architecture that outperformed six frontier models on HealthBench (Hard and Professional) while reducing potential clinical harm under blinded physician evaluation. Finally, a double-blind study of end-to-end generated papers with 30 domain experts across 450 reviews demonstrates that Co-Scientist's reliability modules reduce hallucination and plagiarism while improving research safety. Together, these results demonstrate progress toward closed-loop multi-agent scientific AI systems capable of accelerating real-world scientific discovery.
Figures & tables
Figure 1: The extended Co-Scientist architecture and overview of scientific contributions. To bridge the gap between computational ideation and physical validation, we extend Co-Scientist to integrate iterative reasoning with autonomous code execution, and empirical verification, adapting human–AI collaboration to the constraints of each domain. a–c, Co-Scientist multi-agent architecture. a, Ideation: given a research directive and constraints, Co-Scientist explores and refines a hypothesis set based on safety, novelty, plausibility, testability and guided by Bayesian-exploration. b, Experimentation: the top-ranked hypothesis is converted into an experiment plan, which the system follows to produce research code, initially scaffolded using minimal data, then expanded to full-scale execution. c, Paper writing: code and execution logs are synthesized into a manuscript with plagiarism checks and cross-verification of claims. d, 2D Materials synthesis: Co-Scientist designs safe CVD protocols that human experts execute and optimize, yielding 2D layered structures exhibiting similar characteristics as the MXene (atomic confirmation pending) and achieving single-attempt monolayer transition metal dichalcogenides (TMDs) growth. e, Phenotypic prediction: a vision pipeline predicts E. coli swarming morphologies across an IPTG gradient (0–10 mM), with predictions showing quantitative concordance with wet-lab measurements and correctly capturing the absence of a dose-response in the control strain. f, Agent architecture discovery: without human intervention, Co-Scientist discovers Agent_H , which outperforms frontier models on length-adjusted HealthBench Hard and Professional and reduces clinical harm under blinded physician evaluation. g, The studies span a continuum from human-executed synthesis ( d ) through expert–AI collaboration ( e ) to fully autonomous discovery ( f ) and end-to-end manuscript generation, where a double-blind study (30 experts, 450 reviews) shows that Co-Scientist’s reliability modules reduce severe hallucination and plagiarism.
| Directive & Constraints | Level of Autonomy | Key Discovery & Validation |
| Materials Synthesis — CVD Protocol Design | ||
| Directive: Identify safe solid-state precursor chemistry for bottom-up MXene synthesis and generate growth recipes for monolayer 2D TMDs on a custom CVD system. Constraints: Home-built 1-inch quartz tube reactor; solid precursors. | Co-Scientist reasoned over reaction kinetics to substitute toxic precursors with a safe chemical and generated parameterized furnace recipes including machine-level execution code. Human operators loaded samples, ran growth cycles, and performed characterization. | Synthesized 2D layered structures exhibiting crystallographic and morphological signatures consistent with MXene lattice spacing, further experiments needed to confirm atomic phase assignment. Single-attempt monolayer synthesis of , , and via direct hardware control. |
| Biology — Phenotypic Prediction | ||
| Directive: Build an agentic architecture to predict macroscale E. coli swarming colony morphologies at unseen inducer concentrations from sparse experimental images. Constraints: Unpublished 400 dpi colony scans of pLac- rpoS and pLac- gfp control strains at boundary IPTG concentrations; Gemini vision model APIs. | Co-Scientist implemented and optimized an end-to-end vision pipeline from human directives, including leave-one-out interpolation strategy, and Best-of-N rejection sampling. Domain experts refined high-level task framing between rounds and conducted wet-lab plating, imaging, and quantitative feature extraction. | Predicted held-out colony morphologies with quantitative concordance across 3 of 4 morphological metrics and correctly predicted no dose-response in the negative control strain. |
| Computer Science — Agent Architecture Discovery | ||
| Directive: Discover an agent architecture for improving scores on health benchmark. Constraints: Synthetic training corpus; minimal scaffolding (LLM inference API and guideline retrieval). | No human intervention after providing the initial research directive. Co-Scientist autonomously generated, tested, and iterated the entire codebase. | Discovered Agent_H, an inference-time scaling architecture that outperforms six frontier models on HealthBench Hard and Professional (length adjusted) while significantly reducing potential clinical harm under blinded physician evaluation. |
Table 1: Overview of Co-Scientist evaluated across three real-world scientific domains. The studies span a spectrum of autonomy: AI-designed synthesis recipes executed and adapted by human operators in materials science, AI-designed computational pipelines with iterative expert feedback in biology, autonomous program synthesis in computer science, and end-to-end paper generation evaluated by expert peer review.
Figure 2: Co-Scientist-guided chemical vapor deposition (CVD) synthesis and multiscale characterization of a new 2D crystal. a, Schematic of the end-to-end discovery workflow, combining Co-Scientist’s evolutionary ideation with expert human oversight to identify safer precursor routes. b, Experimental CVD system configuration and reaction mechanism hypothesized by the model, showing solid , Ti powder, and forming gas reacting in the hot zone to generate intermediates that carbonize the downstream Ti foil substrate. c, X-ray diffraction (XRD) pattern of the as-grown 2D crystal, exhibiting the characteristic reflection at corresponding to 1.13 nm -spacing, which is close to the interlayer spacing of previously reported MXene. d, Scanning electron microscopy (SEM) image and corresponding energy dispersive X-ray spectroscopy (EDS) elemental mapping showing 2D layered structures, and co-localized Ti , C , and Cl signals, indicating the potential formation of MXene with chloride surface termination ( ). e, Scanning transmission electron microscopy (STEM) image of isolated 2D flakes. f, High-magnification HAADF-STEM image resolving the atomic crystal lattice (from the highlighted region in e ) and its corresponding fast Fourier transform (FFT) pattern, confirming an in-plane -spacing of .
Figure 3: Semi-autonomous CVD protocol design, TMD characterization, and the speed-quality trade-off. a, End-to-end validation workflow: laboratory constraints of our custom CVD system are provided to Co-Scientist, which generates machine-executable growth protocols. b, Optical microscopy images of MoS 2 grown with diverse target morphologies: triangular flakes with edge lengths exceeding 50 m, irregular flakes, and continuous films ( ). c, Raman spectra confirming monolayer thickness ( / separation 21 cm -1 ). d, Optical microscopy images and Raman spectra of three types of TMDs (MoS 2 , WS 2 , MoSe 2 ) synthesized on the first attempt across two compute regimes: rapid inference with direct hardware integration (via Gemini 3 Deep Think) vs. extensive evolutionary ideation.
Figure 4: Workflow for experiment outcome prediction of E. coli swarming behavior. a, Swarming assay and imaging workflow. Swarming agar plates were prepared, inoculated at the center with bacterial cultures, incubated at for 24 hours, and imaged using a high-resolution flatbed scanner. b, Schematic of the experimental system, where the chemical inducer IPTG modulates expression of a swarming regulator, such as rpoS , via an inducible pLac promoter. E. coli engineered with such genetic circuits produce distinct macroscale colony morphologies on swarming medium supplemented with varying IPTG concentrations. Co-Scientist-generated computational pipeline, in which experimental swarm images are preprocessed and provided to Gemini 3 Pro Image using a leave-one-out interpolation strategy. For each target condition, the model generates candidate predictions, which are scored by a secondary evaluator, with the highest-scoring prediction selected as the final output.
Figure 5: Comparison of ground-truth and Co-Scientist-predicted swarm colonies. a, Representative images of 24-hour swarm colonies formed by E. coli pLac- rpoS (yellow) and pLac- gfp (control, green) on agar supplemented with select IPTG concentrations. Top row: ground-truth; bottom row: Co-Scientist-predicted. b, Morphological features extracted from both datasets using an identical segmentation and feature-extraction pipeline. pLac- rpoS colonies show IPTG-dependent morphological changes that are largely captured by the generated images, while pLac- gfp features remain largely unchanged across conditions in both datasets. Plotted points represent the mean of n = 4–5 biological replicates (swarm colonies) for both the ground-truth and Co-Scientist-generated datasets; error bars represent standard errors of the mean (SEM).
Figure 6: Autonomous inference-time scaling architecture for real-world medical response generation. The eight-phase pipeline ( Agent_H ) discovered by Co-Scientist: (1) Triage and adaptive compute allocation: Multi-axis input classification across medical specialty, audience, intent, and complexity, coupled with adversarial risk detection (identifying false medical premises, unsafe dosing prompts, and fabrication bait) and context-gap analysis. (2) Query decomposition: Conditional execution for complex clinical queries, splitting multi-part inquiries into modular sub-questions with dependency mapping. (3) Parallel candidate search: Parallel generation of 28–48 diverse clinical candidate responses spanning domain-specific role personas across stochastic temperature regimes ( ). (4) Tournament selection and consensus: Single-elimination pairwise tournament evaluated on clinical safety, completeness, accuracy, and utility, finalized by a 3-judge ensemble majority vote among finalists. (5) Iterative critique-and-refinement: Multi-turn clinical auditor-editor loop assessing fabrication severity and guideline alignment, iteratively applying targeted corrections while preserving structural integrity. (6) Meta-cognitive verification: Explicit verification ensuring that all triage-identified clinical sub-questions and context gaps have been addressed. (7) Citation audit: Grounding of named clinical practice guidelines, contraindications, and medication dosages against retrieved guideline summaries. (8) Length optimization: Calibrated length compression for the final response targeting the 2,000-character optimal length envelope to preserve essential clinical content while eliminating verbosity penalties. Total compute cost per query ranges from 40 to 80 LLM calls.
| HealthBench Hard | HealthBench Professional | |||
| Model | No length adj. | Length adj. | No length adj. | Length adj. |
| Judge: Gemini 3.5 Flash | ||||
| Agent_H | 0.420 [0.397, 0.443] | 0.377 [0.353, 0.400] | 0.645 [0.610, 0.681] | 0.643 [0.608, 0.679] |
| Claude Opus 5 | 0.390 [0.370, 0.409] | 0.281 [0.259, 0.303] | 0.697 [0.657, 0.735] | 0.572 [0.532, 0.611] |
| Claude Fable 5 | 0.283 [0.262, 0.303] | 0.300 [0.280, 0.320] | 0.610 [0.567, 0.651] | 0.581 [0.539, 0.620] |
| GPT-5.6 Sol | 0.331 [0.312, 0.350] | 0.293 [0.272, 0.313] | 0.664 [0.622, 0.704] | 0.614 [0.573, 0.653] |
Table 2: Performance of Agent_H and frontier model baselines on HealthBench Hard and Professional. Values report mean rubric scores with 95% confidence intervals in brackets, aggregated across 8 independent grading runs per prompt using two independent automated judges (Gemini 3.5 Flash and GPT-5.4). Length-adjusted scores penalize verbosity relative to a 2,000-character pivot using benchmark-specific length-adjustment coefficients ( for Hard; for Professional). For context, Hicks et al. (2026) reported that ChatGPT for Clinicians delivered the strongest overall performance among prior systems, scoring 0.590. Claude Fable 5 had an overall refusal rate of on Hard and on Professional across 8 runs. Note that Agent_H utilizes an agent workflow requiring approximately 40–80 LLM calls per query, whereas all six frontier model baselines operate in a standard single-call inference regime.
Figure 7: Human evaluation results of Agent_H and Gemini 3.1 Pro baseline. a, Preference ranking between Agent_H and base Gemini 3.1 Pro answers across nine rating dimensions. Agent_H demonstrated a statistically significant reduction in the likelihood of harm compared to Gemini 3.1 Pro ( ). Differences across the remaining eight dimensions were not statistically significant. The evaluation involved 106 questions from HealthBench Hard ( ) and Professional ( ), each rated by a single clinician. Stacked bars represent the proportion of answers for which clinicians preferred Agent_H (yellow), Gemini 3.1 Pro (green), or rated them as a tie (light gray). Error bars reflect 95% confidence intervals as determined by bootstrapping, centered on preference rates for Agent_H and Gemini 3.1 Pro, respectively. b, Agreement between physician raters and the Gemini 3.5 Flash autorater. The green dotted line ( ) indicates good agreement. The autorater shows low alignment with clinical raters. Error bars reflect 95% confidence intervals as determined by bootstrap, centered on the mean Randolph’s marginal kappa value for each axis.
Figure 8: Human expert evaluation of Co-Scientist’s autonomously generated research manuscripts. We compare Co-Scientist (yellow), an ablated version without reliability modules (green), and the Agent Laboratory baseline (blue) ( manuscripts each, three reviews per manuscript). Error bars denote 95% CIs; significance is indicated by * ( ), ** ( ), and *** ( ). a, b, Result hallucination. For hallucinations with severity (denoting findings that invalidate the paper), Co-Scientist achieves a rate of 4%, representing a reduction vs. Agent Laboratory ( ) and a reduction vs. the ablation ( ). Co-Scientist prevents extreme hallucinations (severity ) entirely (0%). c, d, Methodological hallucination. For severe hallucinations (score ), Co-Scientist (24% [11.7, 36.2]) significantly outperforms both the ablated model (52%; , ) and the baseline (100%; , ). Extreme hallucinations (score ) are nearly eliminated in Co-Scientist (2%), whereas the Agent Laboratory baseline exhibits a 74% rate ( , ). e, f, Plagiarism mitigation and attribution. Severe plagiarism (score ) decreases to 16% in Co-Scientist, a reduction vs. baseline ( ). The ablated model (50%) shows no significant improvement over the baseline. Proper attribution increases to 39.4% with Co-Scientist ( vs. baseline; ), while the ablation (17.7%) yields no significant gain. g, Safety architecture performance. Ethical oversight modules significantly enhance safety, increasing the proportion of expert-rated safe experiment plans by 24.3 percentage points to 96.7% and research ideas to 96.3%. These improvements ( ) were evaluated by independent expert reviewers, with inter-rater agreement averaging for planning and for ideation across conditions ( within the safety condition; Section D.5 ).
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Figure A1 : Co-Scientist end-to-end workflow. (1) Ideation: A human scientist defines the initial goal. The ideation module consists of Generation, Ethics Review, Reflection, Ranking, and Evolution agents that iteratively propose, critique, and evolve hypotheses using crossover and mutation. The most promising candidate is chosen via tournament selection based on the highest Upper Confidence Bound (UCB) score. (2) Experimentation (computational): The highest-rated hypothesis is translated into a dynamic research plan and code repository. Code is first tested on a small data subset (scaffold experiment) and safely transitioned (scaffold transition) before full-scale execution (experiment). An LLM-based reward model evaluates the runs to produce verified execution logs, with real-time feedback updating the research plan. (3) Paper writing: The system synthesizes the final hypothesis, literature context, source code, and execution logs into an initial scaffold. The manuscript then undergoes iterative refinement and joint optimization, constrained by strict hallucination-clipping and plagiarism checks, to produce the final scientific paper.
Figure A2 : Co-Scientist experimentation module. An LLM-based reward model serves as the fitness function, assigning a scalar score by evaluating the concordance between the program’s output, the original research plan, and predefined criteria for scientific rigor. The framework incorporates two forms of self-correction to improve robustness. Upon runtime failure, a reflection mechanism is triggered, prompting an LLM to analyze the error trace and execution history to propose a targeted corrective action. The system also prompts an LLM to synthesize generalizable insights from the highest-scoring code variants in the population, and these reflections are used to guide future evolutionary steps. Furthermore, the agent can dynamically adapt its research plan if it determines, based on experimental history, that the initial objectives are infeasible, thereby ensuring the research direction remains viable.
Figure A3 : Co-Scientist paper writing module. The system generates a manuscript in three phases. The Figure Generation phase (left) creates visuals through an iterative cycle of code generation and vision-based feedback. In the Scaffold Building phase (center), an initial draft is structured from the research plan, results, and citations. Finally, during Paper Writing (right), the manuscript undergoes cycles of edits, where a multi-objective reward model scores and selects each variation.
Figure A4 : Code safety module. Overview of the pre-execution code analysis pipeline. Candidate code is first classified for harmful intent against safety criteria. Code passing standard protocols proceeds to execution; code flagged as potentially harmful undergoes a sanitization routine that removes malicious logic while preserving the experimental objectives, ensuring the broader research workflow is not disrupted.
Figure A5 : Growth results for the synthesized 2D structure and SEM characterization after MILD treatment . a, Optical images of the growth substrate before (as-grown substrate) and after (scraped substrate) removing the floating dark solids and products across different regions. b, Map sum spectrum from SEM-EDS spectra showing Si (substrate for characterization), Ti, C, O (oxidation), Cl (possible surface termination groups), Fe (resulting from the razor blade) elements, with no detectable N. c, SEM measurements and the corresponding EDS elemental mapping showing 2D layered structures after MILD treatment. SEM-EDS spectra confirm Si (substrate for characterization), Ti, C, O (oxidation), Cl (possible surface termination groups), F (possible surface terminations introduced during MILD treatment) elements with no detectable N.
Figure A6 : The synthesized 2D structure characterizations for yield analysis . a, STEM image of 2D flakes and EDS elemental mapping of Ti, C, Cl, N, and O elements. b, Raman spectroscopy acquired on the as-grown sample. c, Ti 2p XPS spectra of the synthesized 2D crystals.
| Model | Average Similarity | Median Similarity | Exact | High ( 0.95) |
| HealthBench Hard | ||||
| Agent_H | 0.748 [0.739, 0.758] | 0.779 | 0 | 0 |
| Claude Opus 5 | 0.750 [0.739, 0.760] | 0.790 | 0 | 0 |
| Claude Fable 5 | 0.746 [0.737, 0.755] | 0.778 | 0 | 1 |
| GPT-5.6 Sol | 0.748 [0.739, 0.758] | 0.780 | 0 | 1 |
| GPT-5 | 0.734 [0.725, 0.743] | 0.768 | 0 | 1 |
Table A1 : Decontamination analysis: cosine similarity between model responses and HealthBench ground-truth completions using the Universal Sentence Encoder ( Cer et al., 2018 ) (mean, 95% CI). The agent’s similarity profile is comparable to other frontier models across both benchmarks, indicating no data leakage.
| Clinical Evaluation Dimension | Gemini 3.5 Flash vs. Clinician | GPT-5.4 Low Reasoning vs. Clinician |
| Better reflects consensus | 0.186 [0.043, 0.329] | 0.186 [0.043, 0.329] |
| Better reading comprehension | 0.071 [-0.071, 0.214] | 0.114 [-0.029, 0.257] |
| Better knowledge recall | 0.157 [0.014, 0.300] | 0.114 [-0.014, 0.257] |
| Better reasoning | 0.243 [0.100, 0.386] | 0.200 [0.057, 0.343] |
| More inaccurate / irrelevant info | 0.200 [0.057, 0.343] | 0.143 [0.000, 0.286] |
| Omits more information | 0.157 [0.014, 0.300] | 0.086 [-0.043, 0.229] |
Table A2: Free-marginal multirater agreement between autoraters and human clinicians across nine clinical evaluation dimensions (mean, 95% bootstrap CI, 5,000 iterations).
| Category | Metric / Range | Value |
| General Profile | Total Participants | 30 |
| PhD or Post-doctoral | 29 | |
| Experience (years) | Mean (Median) | 11 (11) |
| Highest Freq. (10–14) | ||
| Second Highest (5–9) | ||
| Bibliometrics | Publications [Mean (Max)] | 35 (100) |
Table A3: Demographic & bibliometric profile
| Component | Description |
| Models | Gemini 2.5 Pro (ideation, experimentation meta-agent, paper writing, experimentation solver, reviewer), Gemini 2.5 Flash-Lite (rapid inference within generated experiments). |
| Ideation | Evolutionary hypothesis generation with Bayesian ranking (TrueSkill). 15 papers retrieved per literature review. Convergence-based stopping (no fixed step limit). |
| Paper writing | 25 evolutionary steps 4 parallel candidates. Gemini 2.5 Flash-based reviewer (PDF rendered as image input). Ethics enforcement and hallucination reduction modules enabled. |
| Hardware | 2 NVIDIA A100 40 GB (80 GB total), 12 vCPUs, 85 GB RAM, 512 GB storage. |
| Study design | 50 research topics 3 conditions (Co-Scientist, ablated, Agent Laboratory) = 150 manuscripts. 30 expert reviewers 3 reviews per manuscript = 450 blind evaluations. |
Table A4: Experimental setup for the autonomous paper generation evaluation.
Figure A7 : Impact of ethical oversight on research safety and quality. Panels a, b quantify the reduction in unsafe content when ethical oversight is enabled vs. ablated. The oversight mechanism reduces the total count of unsafe research ideas and eliminates “Clearly Malicious” experiment plans (dark blue), shifting the remaining risk profile primarily toward “Dual-Use” concerns (medium blue). Panel c compares the expert-rated quality of ideas (5-point Likert scale) between the two groups. Blue bars represent ideas identified as safe by experts; red bars represent unsafe ideas. No statistically significant difference in quality was observed between oversight-enabled and ablated conditions, indicating that safety constraints do not compromise scientific rigor. Error bars denote standard error.
| Evaluation Target | Condition | |
| Research Ideas | Co-Scientist Safety | 0.43 |
| Research Ideas | Ablation | 0.63 |
| Research Plans | Co-Scientist Safety | 0.38 |
| Research Plans | Ablation | 0.80 |
Table A5 : Inter-rater agreement (Cohen’s Kappa) for safety evaluation components.
More in this topic
AI for Science
AI for Science applies machine learning and AI agents to scientific data, models, experiments, and discovery workflows.
See all 157 papers