RxScribe Bench: A Multi-Axis Benchmark for Evaluating Vision-Language Models on Indian Outpatient Prescriptions
Organizations: Zstate AI
Abstract
Prescription transcription errors are not interchangeable. A model that fabricates a drug and a model that misreads a legible dose pose very different clinical risks, yet prescription-transcription accuracy is typically reported as a single blended figure that treats the two as equivalent. We introduce RxScribe Bench, a benchmark for evaluating vision-language models on handwritten prescription digitization that decomposes performance into four axes tied to clinical severity, rather than folding everything into a single aggregated score. Given only a prescription image and an output schema, a model produces a structured record, which is then compared field by field against a human-authored ground truth of identical shape, with each field also labeled for visibility and legibility. The four axes isolate distinct failure modes, namely Correctness, Hallucination, Engagement, and Robustness. The Robustness axis withholds its hard-field results rather than reporting one when the supporting sample falls below a minimum-evidence threshold. We evaluate frontier vision-language models on real prescriptions across independent cold runs per image, and find that no single model wins across all four axes.
Figures & tables
| Tier | What counts | Why it sits here |
|---|---|---|
| T1, fabricated medications | invented drug row + invented active ingredient on a real row | can directly harm a patient |
| T2, fabricated clinical values | clinical-tier field filled where blank, or invented lab-test entry | clinically misleading |
| T3, fabricated non-clinical fields | invented administrative/prescriber metadata | wrong, not dangerous |
| Model | Provider |
|---|---|
| claude-opus-5 | Anthropic |
| gemini-3.1-pro-preview | |
| muse-spark-1.2 | Meta |
| gpt-5.6 | OpenAI |
| Model | Correctness | Medicine-only | Clinical + Non-clinical |
|---|---|---|---|
| GPT-5.6 | 68.10% | 57.83% | 77.84% |
| Gemini 3.1 Pro | 66.97% | 55.06% | 78.94% |
| Muse Spark 1.2 | 64.45% | 49.49% | 78.87% |
| Claude Opus 5 | 61.67% | 47.08% | 75.59% |
| Model | T1 (meds) | T2 (clinical) | T3 (non-clin.) | Total | Per run | Asserted | Flat rate |
|---|---|---|---|---|---|---|---|
| GPT-5.6 | 600 | 790 | 381 | 1,771 | 2.95 | 16,722 | 8.91% |
| Gemini 3.1 Pro | 530 | 833 | 474 | 1,837 | 3.06 | 17,941 | 14.75% |
| Muse Spark 1.2 | 446 | 595 | 325 | 1,366 | 2.28 | 16,049 | 8.75% |
| Claude Opus 5 | 549 | 892 | 770 | 2,211 | 3.69 | 18,225 | 16.38% |
| Hard present ( =1,740) | Clear present ( =19,665) | Absent ( 11,215) | ||||
|---|---|---|---|---|---|---|
| Model | Gave up | Attempt | Dropped | Attempt | Filled | Silent |
| GPT-5.6 | 386 | 77.82% | 2,227 | 88.68% | 833 | 92.57% |
| Gemini 3.1 Pro | 337 | 80.63% | 2,330 | 88.15% | 1,920 | 82.88% |
| Muse Spark 1.2 | 421 | 75.80% | 2,784 | 85.84% | 971 | 91.34% |
| Claude Opus 5 | 346 | 80.11% | 2,069 | 89.48% | 2,316 | 79.35% |
| Delivered | Attempted only | ||||||
|---|---|---|---|---|---|---|---|
| Model | Hard | Clear | Drop | Hard | Clear | Drop | Parity |
| GPT-5.6 | 49.23% | 69.50% | 20.27% | 63.26% | 78.38% | 15.11% | 89.14% |
| Gemini 3.1 Pro | 51.03% | 69.48% | 18.45% | 63.28% | 78.82% | 15.53% | 92.48% |
| Muse Spark 1.2 | 48.08% | 64.03% | 15.96% | 63.42% | 74.60% | 11.17% | 89.96% |
| Claude Opus 5 | 46.35% | 65.18% | 18.82% | 57.86% | 72.84% | 14.98% | 90.64% |
| Model | Correctness mean | Std | Min | Max | Spread |
|---|---|---|---|---|---|
| GPT-5.6 | 68.10% | 14.49% | 23.18% | 98.31% | 75.13% |
| Gemini 3.1 Pro | 66.97% | 15.51% | 13.04% | 96.97% | 83.93% |
| Muse Spark 1.2 | 64.45% | 14.75% | 0.00% | 93.93% | 93.93% |
| Claude Opus 5 | 61.67% | 14.58% | 0.00% | 94.52% | 94.52% |
| Model | Time/run | Cost/run | Total cost | Tokens in/out | Parse errors |
|---|---|---|---|---|---|
| Claude Opus 5 | 40.9 s | $0.1233 | $73.97 | 9,235 / 3,085 | 0.50% |
| Gemini 3.1 Pro | 52.9 s | $0.0696 | $41.76 | 6,249 / 4,888 | 0% |
| Muse Spark 1.2 | 109.3 s | $0.0401 | $24.07 | 6,787 / 7,588 | 0.50% |
| GPT-5.6 | 153.4 s | $0.2416 | $144.97 | 9,257 / 7,090 | 0% |
VLM Evaluation
Vision-language model evaluation measures multimodal systems’ perception, reasoning, grounding, robustness, and reliability across controlled and real-world tasks.
See all 999 papers