Research Assistant: AstraZeneca's Agentic System for R&D

Piotr Grabowski; Mohamed Alameen; Jorge Bretones; Sabina Cardell; Miguel Carmona; Gavin Edwards; Ben Grainger; Sameh Hassan

cs.AIAug 6, 2026

Research Assistant: AstraZeneca's Agentic System for R&D

Authors: Piotr Grabowski, Mohamed Alameen, Jorge Bretones, Sabina Cardell, Miguel Carmona, Gavin Edwards, Ben Grainger, Sameh Hassan, +11 more

Abstract

We describe Research Assistant, an internal LLM-based system developed at AstraZeneca to help scientists and clinicians explore biomedical questions across a broad range of data sources. The system provides a chat-style interface that brings together evidence from scientific literature, knowledge graphs, chemistry, clinical trials, safety resources, expression data, and internal experimental systems. It supports both a fast mode for direct question answering and a multi-step mode for more complex research tasks. Responses are grounded in retrieved evidence and linked back to the original sources, allowing users to review and further explore the underlying data. In this technical note, we outline the system architecture, the main design choices behind the product, and lessons learned from deploying it at scale to support day-to-day R&D workflows across AstraZeneca.

Explore similar work

Designing a Robust LLM-Based Evaluation System for Agentic AI in Drug Discovery Through Human Alignment
Aug 21, 2026Emma Granqvist, Rocío Mercado, Samuel GenhedenLlm-As-A-Judge Llm-Based Agent
DeepResearch Agent System
Jul 30, 2026Yong Huang, Yulu Huang, for the team CollaborationDeep Research Deepseek
BioResearcher: Scenario-Guided Multi-Agent for Translational Medicine
May 7, 2026Remigiusz Kinas, Joanna Krawczyk, Rafał Powalski +6Biomedical Knowledge Life Sciences Research

Aug 21, 2026 · cs.LGJ/K move · Enter open · S save

Designing a Robust LLM-Based Evaluation System for Agentic AI in Drug Discovery Through Human Alignment

Emma Granqvist, Rocío Mercado, Samuel Genheden

Agentic large language model (LLM) systems are reshaping scientific workflows in chemistry and drug discovery, but evaluating their open-ended, tool-augmented outputs remains a fundamental bottleneck. The LLM-as-a-Judge paradigm has emerged as a scalable alternative, but existing drug discovery benchmarks deploy LLM judges without validating their alignment with human experts. In this work, we present an LLM-as-a-Judge evaluation framework for ChatInvent, an agentic drug discovery assistant deployed at AstraZeneca, with five contributions. First, we define four output-quality evaluation dimensions---Completeness, Relevancy, Structural Clarity, and Scope Adherence---alongside deterministic Tool Call Correctness checks. Second, we validate the judge through a human alignment study with five expert annotators, comparing Gemini 3.1 Pro, Claude Opus 4.7, GPT-5, and Llama 3.1 70B as candidate judges. Third, we optimize the best-performing judge using few-shot demonstrations of human-annotated examples, improving alignment with the human majority vote from 0.80 to 0.86. Fourth, applying the optimized judge to 70 held-out questions, we surface concrete limitations and find no strong evidence that informal phrasing degrades output quality; it may, however, still be helpful to have the LLM rewrite the original question before querying the agent. Finally, we extend the framework to 38 adversarial questions that are ambiguous, invalid, out-of-scope or ethically sensitive, and show that the agent's refusal behavior is guided by the stated intent of a request. Our framework provides a reusable template for human-aligned evaluation of agentic systems in scientific domains.

Llm-As-A-Judge Llm-Based Agent Drug Discovery Human Annotators Human Oversight

Open paper Ask about it PDF

Abstract

Explore similar work

Designing a Robust LLM-Based Evaluation System for Agentic AI in Drug Discovery Through Human Alignment

DeepResearch Agent System

BioResearcher: Scenario-Guided Multi-Agent for Translational Medicine