cs.AIAug 21, 2026
SaveVIALS: A Benchmark for Visual Interpretation of Artifacts in the Life Sciences
Organizations: Handshake AI
Abstract
In professional life sciences workflows, scientists routinely interpret visual artifacts (gel blots, microscopy images, plasmid maps, flow cytometry plots, molecular structures, ...) to inform research decisions. We introduce VIALS, a visual question-answering benchmark with 161 such interpretation tasks, spanning the types of artifacts examined throughout experimental workflows in the biotech industry (rather than polished figures from publications and textbooks). While frontier vision-language models can now fluently describe natural images, we find that they are unable to accurately interpret these scientific images, reflecting limitations in domain knowledge and domain-specific visual reasoning capabilities. In contrast, scientists with relevant domain expertise find these visual interpretation tasks straightforward. AI that cannot similarly interpret such images will have limited utility in professional life sciences workflows, where such artifacts are central to how scientists reason, communicate, and make decisions.
Figures & tables
Figure 1: Three tasks from the VIALS benchmark. ( Left ) The Zone of inhibition artifacts depict agar plates containing growing microorganisms, and this task requires identifying the clear zones around the disks that reveal antibiotic activity—a common assessment in antibiotic discovery and clinical microbiology. ( Middle ) The FACS dot plot artifacts visualize the results of a flow cytometry assay to characterize cell populations, and this task requires identifying cell detection events for a particular biomarker—a key step in immunology research and cell therapy development. ( Right ) The Protein-ligand complex artifact is a 3D visualization from structural biology software, showing how a molecule fits into a protein, and this task requires identifying the molecular contacts that enable binding—a key step in structure-based drug discovery. For each task, the correctness of answers from various models is shown along the bottom (GPT-5.6 Sol, Kimi K3, Gemini 3.1 Pro, Claude Opus 5, and Grok 4.6, from left to right).
| Domain | Representative image types | Scientific fields | Share |
| Phylogenetics | Rectangular and circular phylogenetic trees; branch-length and clade diagrams; taxon and clade labels | Evolutionary biology, microbiology, genomics | 14.3% |
| Flow cytometry | Gating hierarchies, density/contour plots, FMO overlays, histograms, and multi-parameter panels | Immunology, biomedical research, cell biology | 14.3% |
| Blotting | Western blots, SDS-PAGE, and 2D gels; lane- and band-level intensity comparisons | Protein biochemistry, molecular biology | 14.3% |
| Cell counting/Quantification | Fluorescence images, colony and plaque images, blood smears, and confluence estimation | Molecular biology, microbiology, cell biology | 14.3% |
| Plasmid maps | Circular and linear plasmid maps; restriction maps; feature annotations and positions | Molecular biology, synthetic biology, virology | 13.0% |
| Protein structure | Protein–ligand binding diagrams, active-site views, 3D structure and residue contact maps | Structural biology, biochemistry, drug discovery | 13.0% |
Table 1: Artifact domains covered by VIALS, their representative image types, the scientific fields in which those artifacts are commonly found, and each domain’s prevalence in the final 161-task benchmark.
Figure 4: Performance of various vision-language models on VIALS (averaged across 3 independent rollouts per task). Error bars show 95% confidence intervals computed by bootstrap resampling across the three rollouts for each task (reflecting variability in accuracy under re-evaluation with one attempt per task). Claude Fable 5 could not be evaluated as its safety guardrails blocked most of the requests from this benchmark.
| Model | Blotting | Cell Count. / Quant. | Flow Cytometry | Molecular Structure | Phylogenetics | Plasmid Maps | Protein Structure | Other |
| GPT-5.6 Sol | 30.4% | 15.9% | 29.0% | 31.7% | 43.5% | 22.2% | 15.9% | 14.3% |
| Gemini 3.7 Flash | 29.0% | 21.7% | 31.9% | 30.0% | 29.0% | 23.8% | 23.8% | 14.3% |
| Claude Opus 5 | 30.4% | 20.3% | 27.5% | 18.3% | 31.9% | 25.4% | 19.0% | 14.3% |
| Muse Spark 1.2 | 15.9% | 2.9% | 21.7% | 15.0% | 34.8% | 38.1% | 33.3% | 14.3% |
| Gemini 3.1 Pro | 24.6% | 18.8% | 14.5% | 20.0% | 31.9% | 25.4% | 17.5% | 14.3% |
| Kimi K3 | 21.7% | 10.1% | 33.3% | 23.3% | 27.5% | 19.0% | 12.7% | 14.3% |
Table 2: Per-domain accuracy on VIALS achieved by each vision-language model. Bold indicates the best performance in each domain.
| Model | Visual quantification error | Incorrect value or feature selection | Overlooked evidence | Representation misinterpretation | Quantitative reasoning error | Misapplied principle | Fabricated finding |
| GPT-5.6 Sol | 41.8% | 14.2% | 15.6% | 14.2% | 6.4% | 3.5% | 4.3% |
| Gemini 3.7 Flash | 35.3% | 18.0% | 18.0% | 16.5% | 8.3% | 3.0% | 0.8% |
| Claude Opus 5 | 36.8% | 16.9% | 17.6% | 14.0% | 10.3% | 4.4% | 0.0% |
| Muse Spark 1.2 | 38.3% | 26.3% | 12.0% | 16.5% | 3.8% | 2.3% | 0.8% |
| Gemini 3.1 Pro | 34.3% | 12.9% | 20.7% | 15.0% | 10.7% | 5.0% | 1.4% |
| Kimi K3 | 33.1% | 20.9% | 15.8% | 16.5% | 7.9% | 4.3% | 1.4% |
Table 3: Distribution of failure modes by model.
Figure 5: Representative model failures beyond visual extraction. Models can identify relevant features in scientific artifacts yet still use them incorrectly in downstream reasoning. Each panel shows the model’s reading in red, and the correct reading supported by the artifact in green.
| Model | Harness | Direct VLM | Tool-Assisted | Tok D | Tok T | Tok | |
| GPT-5.6 Sol | Codex | 26.5 | 49.1 | +22.6 | 4.1k | 165k | 40 |
| GPT-5.6 Sol | OpenCode | 26.5 | 40.4 | +13.9 | 4.1k | 47k | 11 |
| Claude Opus 5 | Claude Code | 24.4 | 64.2 | +39.8 | 5.3k | 959k | 182 |
| Claude Opus 5 | OpenCode | 24.4 | 65.8 | +41.4 | 5.3k | 737k | 139 |
| Gemini 3.1 Pro | OpenCode | 21.5 | 35.2 | +13.7 | 3.5k | 1.0M | 285 |
| Gemini 3.7 Flash | OpenCode | 26.5 | 60.7 | +34.2 | 3.7k | 1.6M | 432 |
Table 4: Performance of tool-assisted agents on VIALS, across seven models run in different harnesses. Direct VLM vs. Tool-Assisted report accuracy (%) under the single/direct VLM call vs. agentic tool-assisted settings, respectively. Values are the average over rollouts. reports the absolute change in accuracy in percentage points. Tok D and Tok T report mean tokens per attempt in the direct-VLM and tool-assisted settings, respectively; Tok reports the ratio of tool-assisted to direct-VLM token usage.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Figure A1 : Mean accuracy (Pass@1), Pass@3, and Pass^3 achieved by different vision-language models. Pass@3 measures whether a task is solved correctly in at least one of three rollouts, while Pass^3 measures whether it is consistently solved across all three.
Figure A2 : Cost and output-token efficiency on VIALS. Each point represents one model; accuracy is plotted against inference cost per attempt ( left ) and output tokens per attempt ( right ).
Figure A3 : Accuracy of model responses amongst those with high vs. low self-reported confidence. Gray points show the accuracy of responses with low confidence ( ), while colored points show the accuracy of responses with high confidence ( ). The right column reports the share of model responses that were self-reported as high-confidence ( ). Models marked with reported too few low-confidence responses for a reliable comparison.
Figure A4 : Representative VLM failure modes (1 of 2). Together with Figure A5 , the seven panels provide one example from each error category in Table 3 . Each panel shows the model’s response in red and the interpretation supported by the artifact in green.
Figure A5 : Representative failure modes (2 of 2), continuing Figure A4 . Together, the two figures provide one example from each of the seven error categories in Table 3 . Each panel shows the model’s response in red and the interpretation supported by the artifact in green.
| Model | Harness | Cost D | Cost T | Cost |
| GPT-5.6 Sol | Codex | $0.037 | $0.271 | 7 |
| GPT-5.6 Sol | OpenCode | $0.037 | $0.155 | 4 |
| Claude Opus 5 | Claude Code | $0.065 | $1.047 | 16 |
| Claude Opus 5 | OpenCode | $0.065 | $0.917 | 14 |
| Gemini 3.1 Pro | OpenCode | $0.024 | $0.716 | 29 |
| Gemini 3.7 Flash | OpenCode | $0.007 | $0.374 | 56 |
Table A1 : Per-attempt inference cost under direct and tool-assisted evaluation. Cost D is calculated from logged direct-evaluation token usage using the corresponding provider API prices. Cost T is the mean per-attempt cost reported by the Harbor agent runtime. Cost is the ratio of tool-assisted to direct cost.
More in this topic
AI for Science
AI for Science applies machine learning and AI agents to scientific data, models, experiments, and discovery workflows.
See all 161 papers