ScienceVision: Can AI Observe and Reason Like a Scientist?
Organizations: UniPat AI · Elorian AI
Abstract
Scientific discovery builds on accurate observations interpreted through domain knowledge and reasoning. For scientific 3D data, this requires understanding structures and spatial relationships, from protein contacts to anatomical organization. Existing multimodal benchmarks largely evaluate text and 2D images, while scientific 3D evaluations vary in representations and protocols across disciplines, limiting cross-disciplinary comparison. To assess these foundational capabilities, we introduce ScienceVision, a cross-domain benchmark for scientific 3D understanding and reasoning through expert-level visual question answering (VQA). We construct 250 questions, with 50 each from Biology, Chemistry, Medical Imaging, Mechanical CAD, and Architecture, through a pipeline combining synthetic question generation with domain-expert review and annotation. Each task pairs a native 3D artifact with a scientific question, a specified response format, and a task-specific reference answer, requiring domain knowledge to be applied to geometric or topological evidence. We evaluate 12 models on the same questions in three settings: visual question answering with native 3D artifacts accessed through an agent harness's file and code-execution tools; the same workflow augmented with expert-inspired viewer inspection and measurement tools exposed through the Model Context Protocol (MCP); and visual question answering from prescribed rendered 2D images using an agent harness. All settings use the same task-specific deterministic final-answer scoring. The inspection environment supports view changes, target isolation, and slice inspection. In this main comparison, the best mean answer score is 0.53 out of 1. Mean scores in the native-artifact workflow without additional MCP tools exceed those in the rendered-image workflow for every model, by 17.2 percentage points on average.
Figures & tables
| Benchmark | Cross-domain science | QA focus a | Visual interaction b | Static-image QA | Deterministic scoring c |
| DeepTumorVQA ( Chen et al., 2026 ) | |||||
| ProtStructQA ( Mandiga et al., 2026 ) | |||||
| OmniCAD ( Wang et al., 2026 ) | |||||
| SciVisAgentBench ( Ai et al., 2026 ) | Partial | Partial | |||
| GLUE3D ( Mariani et al., 2026 ) | Partial | ||||
| ScienceVision (ours) |
Appendix figures & tables22 assets
Supplementary material from the paper’s appendix.
Appendix
| Subdomain | Tasks | Illustrative objective |
| Architecture (50) | ||
| Dimensional compliance | 16 | Check dimensions against a stated requirement. |
| Alignment and stacking | 12 | Identify continuity or alignment of members. |
| Interference and clearance | 12 | Distinguish spatial interference from clearance. |
| Object geometry readings | 10 | Read object dimensions, arrangement, or scene geometry. |
| Biology (50) | ||
| Model | Harness + 3D | Harness + 3D + MCP | Harness + 2D |
| GPT-6 Astra | 0.5291 | 0.5282 | 0.2498 |
| GPT-6.1 Sol | 0.4893 | 0.5110 | 0.2044 |
| Claude Fable 5.1 | 0.4619 | 0.4631 | 0.2965 |
| Claude Opus 5.5 | 0.4539 | 0.4343 | 0.2383 |
| Gemini 4 Argon | 0.4256 | 0.4283 | 0.1462 |
| Qwen3.8-Max | 0.3830 | 0.3736 | 0.2450 |
| Model | Harness |
| GPT-6 Astra | Codex (max thinking effort) |
| GPT-6.1 Sol | Codex (max thinking effort) |
| Claude Fable 5.1 | Claude Code |
| Claude Opus 5.5 | Claude Code |
| Gemini 4 Argon | Claude Code |
| Qwen3.8-Max-0902 | Claude Code |
| Subdomain | GPT-6 Astra (C) | GPT-6.1 Sol (C) | Claude Fable 5.1 (C) | Claude Opus 5.5 (C) | Gemini 4 Argon (C) | Qwen3.8- Max (O) | |
|---|---|---|---|---|---|---|---|
| Architecture | |||||||
| Alignment and stacking | 12 | 0.2639 | 0.2593 | 0.2569 | 0.3125 | 0.2049 | 0.3056 |
| Dimensional compliance | 16 | 0.4071 | 0.3551 | 0.3208 | 0.3781 | 0.3625 | 0.1821 |
| Interference and clearance | 12 | 0.3361 | 0.3222 | 0.3685 | 0.3569 | 0.3451 | 0.1801 |
| Object geometry readings | 10 | 0.9167 | 0.8000 | 0.8167 | 0.7333 | 0.5750 | 0.4833 |
| Domain mean | 50 | 0.4576 | 0.4132 | 0.4161 | 0.4283 | 0.3630 | 0.2715 |
| Subdomain | Qwen3.8 Flash (O) | Kimi K3 (O) | GLM-5.3 Flash (O) | DeepSeek- V4.1- Flash (O) | Doubao Seed Evolving (O) | Hunyuan-4 Preview (O) | |
|---|---|---|---|---|---|---|---|
| Architecture | |||||||
| Alignment and stacking | 12 | 0.2569 | 0.2315 | 0.2116 | 0.1505 | 0.1829 | 0.1319 |
| Dimensional compliance | 16 | 0.2886 | 0.3179 | 0.2585 | 0.2177 | 0.1836 | 0.0275 |
| Interference and clearance | 12 | 0.3551 | 0.3074 | 0.2125 | 0.2014 | 0.1343 | 0.1319 |
| Object geometry readings | 10 | 0.2833 | 0.2167 | 0.1667 | 0.2333 | 0.1167 | 0.1167 |
| Domain mean | 50 | 0.2959 | 0.2744 | 0.2178 | 0.2008 | 0.1582 | 0.0955 |
| Subdomain | GPT-6 Astra (C) | GPT-6.1 Sol (C) | Claude Fable 5.1 (C) | Claude Opus 5.5 (C) | Gemini 4 Argon (C) | Qwen3.8- Max (O) | |
|---|---|---|---|---|---|---|---|
| Architecture | |||||||
| Alignment and stacking | 12 | 0.2708 | 0.2662 | 0.2708 | 0.3125 | 0.2604 | 0.2245 |
| Dimensional compliance | 16 | 0.3923 | 0.3539 | 0.3081 | 0.3921 | 0.3625 | 0.2025 |
| Interference and clearance | 12 | 0.3130 | 0.3361 | 0.3778 | 0.3384 | 0.3535 | 0.2750 |
| Object geometry readings | 10 | 0.8500 | 0.8167 | 0.8500 | 0.7167 | 0.5500 | 0.3167 |
| Domain mean | 50 | 0.4356 | 0.4211 | 0.4243 | 0.4250 | 0.3733 | 0.2480 |
| Subdomain | Qwen3.8 Flash (O) | Kimi K3 (O) | GLM-5.3 Flash (O) | DeepSeek- V4.1- Flash (O) | Doubao Seed Evolving (O) | Hunyuan-4 Preview (O) | |
|---|---|---|---|---|---|---|---|
| Architecture | |||||||
| Alignment and stacking | 12 | 0.2500 | 0.2315 | 0.2245 | 0.2060 | 0.2245 | 0.1134 |
| Dimensional compliance | 16 | 0.3445 | 0.3455 | 0.2512 | 0.1684 | 0.2175 | 0.1033 |
| Interference and clearance | 12 | 0.3778 | 0.3454 | 0.1977 | 0.2000 | 0.1181 | 0.0556 |
| Object geometry readings | 10 | 0.3167 | 0.2667 | 0.2000 | 0.0667 | 0.1167 | 0.0667 |
| Domain mean | 50 | 0.3242 | 0.3023 | 0.2217 | 0.1647 | 0.1751 | 0.0870 |
| Subdomain | GPT-6 Astra (C) | GPT-6.1 Sol (C) | Claude Fable 5.1 (C) | Claude Opus 5.5 (C) | Gemini 4 Argon (C) | Qwen3.8- Max (O) | |
|---|---|---|---|---|---|---|---|
| Architecture | |||||||
| Alignment and stacking | 12 | 0.1528 | 0.1361 | 0.1815 | 0.1894 | 0.0972 | 0.1931 |
| Dimensional compliance | 16 | 0.1251 | 0.1085 | 0.2211 | 0.1851 | 0.0875 | 0.1172 |
| Interference and clearance | 12 | 0.1806 | 0.1713 | 0.2963 | 0.3009 | 0.1644 | 0.2417 |
| Object geometry readings | 10 | 0.1500 | 0.1500 | 0.2667 | 0.2500 | 0.0500 | 0.2333 |
| Domain mean | 50 | 0.1500 | 0.1385 | 0.2387 | 0.2269 | 0.1008 | 0.1885 |
| Subdomain | Qwen3.8 Flash (O) | Kimi K3 (O) | GLM-5.3 Flash (O) | DeepSeek- V4.1- Flash (O) | Doubao Seed Evolving (O) | Hunyuan-4 Preview (O) | |
|---|---|---|---|---|---|---|---|
| Architecture | |||||||
| Alignment and stacking | 12 | 0.1486 | 0.1259 | 0.0810 | 0.1343 | 0.0694 | 0.0139 |
| Dimensional compliance | 16 | 0.1680 | 0.1600 | 0.0568 | 0.0925 | 0.0104 | 0.0000 |
| Interference and clearance | 12 | 0.3083 | 0.2625 | 0.1954 | 0.1574 | 0.1065 | 0.0278 |
| Object geometry readings | 10 | 0.1500 | 0.1167 | 0.1833 | 0.0667 | 0.0167 | 0.0000 |
| Domain mean | 50 | 0.1934 | 0.1678 | 0.1212 | 0.1129 | 0.0489 | 0.0100 |
| Domain | All zero | Partial maximum | Full-score maximum |
| Architecture | 6 | 27 | 17 |
| Biology | 4 | 11 | 35 |
| Chemistry | 6 | 21 | 23 |
| Mechanical CAD | 5 | 10 | 35 |
| Medical Imaging | 13 | 14 | 23 |
| Harness + 3D | Harness + 3D + MCP | Harness + 2D | ||||
| Model | 250 | 242 | 250 | 242 | 250 | 242 |
| GPT-6 Astra | 0.5291 | 0.5356 | 0.5282 | 0.5292 | 0.2498 | 0.2580 |
| GPT-6.1 Sol | 0.4893 | 0.5013 | 0.5110 | 0.5183 | 0.2044 | 0.2097 |
| Claude Fable 5.1 | 0.4619 | 0.4772 | 0.4631 | 0.4784 | 0.2965 | 0.3063 |
| Claude Opus 5.5 | 0.4539 | 0.4689 | 0.4343 | 0.4487 | 0.2383 | 0.2461 |
| Gemini 4 Argon | 0.4256 | 0.4397 | 0.4283 | 0.4425 | 0.1462 | 0.1510 |
| Model | Access | Codex | Claude Code | (pp) |
| GPT-6 Astra | Harness + 3D | 0.5291 | 0.4082 | |
| Harness + 3D + MCP | 0.5282 | 0.3942 | ||
| GPT-6.1 Sol | Harness + 3D | 0.4893 | 0.3865 | |
| Harness + 3D + MCP | 0.5110 | 0.3843 | ||
| Claude Fable 5.1 | Harness + 3D | 0.5075 | 0.4619 | |
| Harness + 3D + MCP | 0.5349 | 0.4631 |
| Camera plan | ||||
| Aligned | 0.1223 | 0.1516 | 0.1529 | 0.1427 |
| Orbit | 0.1171 | 0.1204 | 0.1463 | 0.1463 |
| Cohort ( ) | Access | Score |
| Earlier study | MCP-only | 0.1757 |
| Main evaluation | Harness + 3D | 0.5291 |
| Harness + 3D + MCP | 0.5282 |
| Panel A: earlier study, GPT-6 Astra with Claude Code and LiteLLM | ||||||
| Plan | Mean | Full | Zero | Ineligible | Retried | |
| Aligned | 1 | 0.1223 | 24 | 210 | 8 | 10 |
| Aligned | 3 | 0.1516 | 29 | 198 | 8 | 9 |
| Aligned | 6 | 0.1529 | 29 | 198 | 8 | 6 |
| Aligned | 12 | 0.1427 | 26 | 199 | 8 | 2 |
| Orbit | 1 | 0.1171 | 23 | 211 | 8 | 3 |
| Task | Harness + 3D | Harness + 3D + MCP | Reading |
| GPT-6 Astra with Codex: benchmark scores and archived trials | |||
| Binding geometry (0114) | Mean 0.33; the trial without MCP excluded the metal–histidine pair and reported the next contact at 2.856 Å | Mean 1.00; 10 MCP calls; scene selection and distance measurement returned 2.229 Å | MCP encoded the scoring convention for direct links |
| Conformation (0107) | Mean 1.00; reproduced the Mol* display sign and precision, and | Mean 0.50; 4 MCP calls; copied the raw tool value | Tool returned a number but not the display convention |
| Architecture (exp-02) | Mean 0.50; built a browser-based viewer and returned approximate dimensions | Mean 0.00; 2 MCP calls; scene loading failed for the source format and the viewer was unavailable | Coverage gap; no fallback to source tooling |
| Claude Fable 5.1, Claude Code, 94-task control | |||
| CAD extents (hard-001-copy) | Replicates 0/1/0; ratio 0.625 written as 0.63, 0.62 and 0.63 | Replicates 1/0/0; all three attempted MCP; ratio 0.625 written as 0.62, 0.63 and 0.63 | Same geometry; the score turns on a rounding boundary |
3D Visual Question Answering
Three-dimensional visual question answering uses spatial and geometric evidence to answer questions about scenes, objects, and volumetric images.
See all 53 papers