ParseBench-2: Severity-Aware Evaluation of Document Parsing
Organizations: LlamaIndex
Abstract
Document-parsing metrics can give high scores to outputs that change important facts and penalize changes that preserve meaning. People can often judge these consequences more easily than they can design a metric to measure them. We introduce a method for developing parsing metrics from severity policies that specify how each error type should affect the score. We express these policies through test scenarios that specify how a metric should behave. We then optimize the metric implementations against these fixed expectations, using coding agents to refine comparison rules, error detectors and weights under human review. This makes stated human priorities an explicit objective of metric design. We apply this method in ParseBench-2, a comprehensive document-parsing benchmark that combines broad capability coverage with metrics calibrated to error severity. It covers 4,317 pages across six dimensions of parsing quality: text, formatting, tables, charts, visual grounding, and forms. We use detailed annotations and targeted difficult cases to check for changed facts, broken structural relationships, and missing content. We take the lower of a parsing score that measures content recovery and a severity cap that penalizes detected errors, so recovering most of a page cannot hide a serious error. Across six severity test suites, our metrics more closely match the expert judgments expressed by the scenario targets than existing metrics and penalize at most 3.4% of the tested benign variations. We conduct a comprehensive evaluation spanning frontier vision-language models, specialized document parsers and open-weight systems, comparing their overall rankings, performance across parsing capabilities and tradeoffs between accuracy and cost.
Figures & tables
| ParseBench-2 | ParseBench-1 [ 69 ] | Best other baseline | |||||
| Test suite | Parsing metric | Severity cap | Final score | Score | Metric | Score | Metric |
| Text | 52.2 | 88.3 | 88.5 | 25.3 | Content Faithfulness | 29.5 | OmniDocBench NED [ 42 ] |
| Formatting | 40.7 | 86.5 | 86.6 | Semantic Formatting | FinTOC harmonic mean (adapted) [ 16 ] | ||
| Tables | 50.0 | 87.5 | 87.8 | GTRM | 58.2 | PulseBench-Tab T-LAG [ 43 ] | |
| Charts | 46.4 | 83.3 | 86.1 | 26.6 | ChartDataPointMatch | 30.6 | EpiCurveBench [ 4 ] |
| Forms | 20.4 | 97.2 | 97.2 | – | – | 37.4 | Generic exact-match field F1 (ours) |
| Scores by dimension | Cost | ||||||||
| System | Type | Overall | Text | Format. | Tables | Charts | Visual grounding | Forms | $/page |
| LlamaParse (Agentic Plus) | Parser | 90.1 | 90.3 | 93.6 | 94.6 | 92.6 | 87.5 | 81.8 | 0.028 |
| LlamaParse (Agentic) | Parser | 86.0 | 87.0 | 92.7 | 89.9 | 77.4 | 88.3 | 80.9 | 0.013 |
| Claude Opus 5.5 | Frontier VLM | 85.7 | 92.7 | 89.4 | 94.1 | 74.1 | 86.2 | 77.8 | 0.065 |
| LlamaParse (Cost Effective) | Parser | 84.6 | 85.5 | 91.9 | 87.2 | 73.1 | 88.6 | 81.2 | 0.004 |
| Gemini 4 argon (low) | Frontier VLM | 84.2 | 89.7 | 89.2 | 92.8 | 68.2 | 88.1 | 77.2 | 0.027 |
Appendix figures & tables27 assets
Supplementary material from the paper’s appendix.
Appendix
| Collection | Role | Weight | Pages |
| TEXT 508 pages | |||
| Text | core | 1 | 508 |
| FORMATTING 861 pages, 508 shared with Text | |||
| Text pages † | core | 508 | |
| Headings | targeted | 100 | |
| Strikethrough ‡ | targeted | 128 | |
| Error types by level, benign variations | ||||||
| Test suite | C | M | m | H | D | Example perturbation (C; M; m; H) |
| Text | 13 | 16 | 5 | 8 | 9 | negation flipped; words dropped; spelling typos; paragraphs run together |
| Formatting | 5 | 8 | 24 | 15 | 3 | strikethrough lost; insertion underline lost on a redline; bold and italic stripped; bold for bold |
| Tables | 27 | 18 | 9 | 5 | 3 | column headers swapped; two rows merged; empty row inserted; extra HTML markup |
| Charts | 22 | 9 | 3 | 14 | 5 | order of magnitude off; series missing; series names abbreviated; units written in cells |
| Forms | 13 | 13 | 8 | 34 | 3 | digit misread in a value; value dropped, caption kept; typo in a text value; fields written as a two-column table |
| Perturbation | Category | Description |
|---|---|---|
| CRITICAL ERRORS | ||
| Negation flipped | Word errors | "not", "no" and "never" are dropped, or "not" is added after is, are, was, will or can. One or two words change and the meaning inverts. |
| Output cut off | Missing or extra content | The output stops early, as with a token limit. |
| Date with day and month swapped | Wrong facts | A date’s day and month are exchanged, such as March 4 April 3 or 10 November 11 October. The resulting date looks plausible but is incorrect. |
| Paragraphs dropped | Missing or extra content | Whole paragraphs disappear, as when a region is missed by detection. |
| Two values swapped between clauses | Wrong facts | Two figures in one paragraph trade places (about 62% … the remaining 38% becomes about 38% … the remaining 62%). Every number is still on the page, each in the wrong clause. |
| Perturbation | Category | Description |
|---|---|---|
| CRITICAL ERRORS | ||
| Strikethrough lost | Strikethrough and scripts | Deleted text loses its strikethrough and reads as live text: a voided clause or an old fee reads as current. |
| Exponent lost (10 9 109) | Strikethrough and scripts | A superscript exponent becomes ordinary digits glued to its base: 10 sup 5 /sup reads as 105, 10 sup -3 /sup as 10-3. |
| Live text struck through | Strikethrough and scripts | A phrase of live text is struck through: the reader takes it as deleted. |
| Digit turned into exponent | Strikethrough and scripts | The last digit of a plain number is set as a superscript: 2026 reads as 202 6 . |
| Exponent written as subscript | Strikethrough and scripts | Exponents are emitted as subscripts: 10 sub 5 /sub . |
| Perturbation | Category | Description |
|---|---|---|
| CRITICAL ERRORS | ||
| Table missing entirely | Missing or extra | The parser returns no table at all, only a placeholder line, as when it fails on a complex layout. |
| Column headers swapped | Values in the wrong place | Two column headers trade places, so both columns are labelled with the other’s period or measure. |
| Decimal point inserted | Numbers | A speck or a line break is read as a decimal point inside a whole number: 97 9.7. |
| Decimal point misplaced | Numbers | The decimal point is lost or moves one place: 13.8 138, 0.41 4.1. |
| Headers shifted one column | Values in the wrong place | The lowest header row slides one column right, so every column is labelled with its left neighbour’s header. |
| Perturbation | Category | Description |
|---|---|---|
| CRITICAL ERRORS | ||
| Negative signs lost | Wrong numbers | Bars below the axis are transcribed as positive values. Only applies to charts with negative values. |
| Order of magnitude off | Wrong numbers | A decimal point is misread, so a value is ten times too large or too small (412.5 41.25). |
| Scale lost | Scale and units | Values are written in the base unit without saying so, 412500000 instead of 412.5 million, or as fractions without a % sign. Nothing in the output tells the reader the scale. |
| Cumulative values read | Wrong numbers | Stacked series are read from the top of each stack instead of by their individual heights. Later series therefore include the values of earlier ones. |
| Digit typo | Values | One digit is wrong in some values, as from a misread label. |
| Perturbation | Category | Description |
|---|---|---|
| CRITICAL ERRORS | ||
| Decimal point lost in an amount | Values | 52,287.47 is written as 5228747. The digits are unchanged, but the amount is one hundred times larger. |
| Digit misread in a value | Values | One digit changes in some numeric values (amount, date, id): the form now states a different number. |
| Two digits of an ID transposed | Values | An identifier with two neighbouring digits swapped (123-45-6789 123-54-6789): every digit is there, the number is wrong. |
| Values of two fields swapped | Values | Two filled fields exchange their values: each value is attached to the wrong caption. |
| Repeated block’s values swapped | Values | Two blocks with the same captions (Applicant / Co-applicant, State line 1 / 2) exchange their values: each value is written under the wrong person or line. |
| Perturbation | Category | Description |
|---|---|---|
| CRITICAL ERRORS | ||
| One page-wide box | One Text box covers the entire page and contains all its text. | |
| Biggest paragraph missed | Paragraph 5 is missing. All other elements are correct. | |
| Five paragraphs in one box | One Text box contains all five paragraphs and also covers the picture and table. | |
| Right box, wrong text | Paragraph 3 has the correct box but contains paragraph 4’s text. | |
| Coarse blocks | Two Text blocks cover large page regions: one from the title through paragraph 3, and one from the icon through paragraph 5. | |
| Metric columns | Source and suite implementation |
|---|---|
| TEXT | |
| ParseBench-2 Text, Move-tolerant fidelity (component average), Next: reading-order component, Ordered critical tokens F1, Bag-of-words recall | Our metric ( Section A.3.1 ), component variants and internal baselines. |
| Content Faithfulness, Order rule pass rate | ParseBench-1 [ 69 ] . The suite generates order rules from consecutive sentences. |
| Character accuracy ( CER), Word accuracy ( WER) | Levenshtein distance [ 28 ] with JiWER-style default normalization [ 25 ] , expressed as accuracy and clipped at zero. |
| ISRI word accuracy | ISRI/ocreval [ 38 ] . The local implementation aligns lowercased Latin-1 words with a longest common subsequence. |
| edit distance, Unique-word F1 | Nougat evaluation [ 6 ] . Character edit similarity and set F1 after removing tables. |
| Perturbation | Sev. | 1. ParseBench-2 Text | 2. Order rule pass rate | 3. Text 1 NED | 4. Ordered critical tokens F1 | 5. Move-tolerant fidelity (component average) | 6. Unit-test pass rate | 7. Content Faithfulness | 8. Text + reading order | 9. CCT accuracy | 10. Bag-of-words recall | 11. Flexible character accuracy | 12. ISRI word accuracy | 13. NID (Indel similarity) | 14. Text 1 NED (MGAM matching) | 15. Next: reading-order component | 16. Word order (Kendall) | 17. Reading order 1 ED | 18. BLEU-4 | 19. Heuristic score | 20. Word accuracy (1 WER) | 21. Word edit distance | 22. METEOR | 23. chrF (character n-gram F) | 24. Unique-word F1 | 25. Tokens found | 26. Word F1 (multiset) | 27. Full-page OCR score | 28. 1 tokens added | 29. 1 edit distance | 30. Character accuracy (1 CER) | 31. Document parsing STEDS |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| CRITICAL | ||||||||||||||||||||||||||||||||
| Output cut off | 90 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ | ✗ | ✗ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ | ✓ | ✓ | ✓ | |
| Negation flipped | 90 | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ||||||
| Paragraphs dropped | 90 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ | ✗ | ✗ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ | ✓ | ✓ | ✓ | ||
| Wrong digits | 90 | ✓ | ✗ | ✓ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ||||||||||||||
| Date with day and month swapped | 90 | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ||||||
| Perturbation | Sev. | 1. ParseBench-2 Formatting | 2. formatting_v2 cap only | 3. formatting_v2 proportional only | 4. Style-token F1 | 5. Heading F1, exact level | 6. Normalized title accuracy | 7. FormatTest pass rate | 8. STEDS | 9. Normalized text styling | 10. Title F1 / level accuracy harmonic mean | 11. All formatting rules pass rate | 12. Heading TEDS | 13. Semantic Formatting | 14. MHS heading tree | 15. Text 1 NED (cleaned) | 16. Tag-sequence 1 NED | 17. Raw Markdown 1 NED |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| CRITICAL | ||||||||||||||||||
| Strikethrough lost | 90 | ✓ | ✓ | ✗ | ✗ | ✗ | ✗ | ✓ | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | ||||
| Exponent lost (10 9 109) | 90 | ✓ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | |||||||||
| Live text struck through | 90 | ✓ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ||
| Digit turned into exponent | 90 | ✓ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ||||
| Exponent written as subscript | 90 | ✓ | ✓ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ | |||||
| Perturbation | Sev. | 1. ParseBench-2 Tables | 2. Table test pass rate | 3. TableRecordMatch | 4. TEDS-S | 5. GriTS-Top | 6. GriTS-Con | 7. Table size match | 8. T-LAG (cell-neighbour graph F1) | 9. GTRM composite | 10. Position accuracy (text adaptation) | 11. Structure BLEU-4 | 12. Merged-cell F1 (MCD) | 13. Composite structure acc | 14. AccCon (exact table) | 15. Adjacency F1 | 16. JSON accuracy | 17. Cell accuracy (TCE) | 18. Row and column F1 (RCE) | 19. Relation F1, exact | 20. TEDS (v1.6 normalization) | 21. TEDS (normalized) | 22. Shifted content accuracy | 23. GriTS-Con | 24. TEDS | 25. Relation F1, chrF | 26. 1 table edit distance | 27. Alignment score | 28. Bigram Jaccard |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| CRITICAL | |||||||||||||||||||||||||||||
| Table missing entirely | 90 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Column headers swapped | 90 | ✓ | ✓ | ✓ | ✗ | ✗ | ✗ | ✓ | ✗ | ✗ | ✓ | ✓ | ✓ | ✗ | ✓ | ✗ | ✗ | ||||||||||||
| Headers shifted one column | 90 | ✓ | ✓ | ✓ | ✗ | ✗ | ✗ | ✓ | ✗ | ✓ | ✓ | ✓ | ✗ | ✓ | |||||||||||||||
| Sign or comparator lost | 90 | ✓ | ✓ | ✗ | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ | ✓ | ✓ | ✗ | ||||||||||||||||
| Decimal point misplaced | 90 | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ||||||||||||
| Perturbation | Sev. | 1. ParseBench-2 Charts | 2. ChartDataPoint pass rate | 3. EpiCurveSimilarity | 4. 1 A-MAPE | 5. SCRM slight | 6. SCRM strict | 7. SCRM high | 8. ChartOCR score | 9. RMS-TBE-F1-Sig | 10. RMS-TBE-F1 | 11. Metric 6b | 12. Point F1 (2% box) | 13. Normalized mapping similarity | 14. RNSS | 15. (1 nMAE) coverage | 16. Relaxed accuracy (QA by lookup) | 17. RMS F1 | 18. Row IoU (medium tolerance) | 19. 0.5 RMS + 0.5 TEDS ′ | 20. Chart parsing TEDS |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| CRITICAL | |||||||||||||||||||||
| Scale lost | 90 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Order of magnitude off | 90 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ||
| Negative signs lost | 90 | ✓ | ✓ | ✓ | ✓ | ✗ | ✗ | ✗ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | |
| Digit typo | 90 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ | ||||
| Series names swapped | 90 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ | ✓ | ✓ | ✓ | ✗ | ✓ | ✗ | ✓ | ✓ | ✓ | ✓ | ✓ | |
| Perturbation | Sev. | 1. ParseBench-2 Forms | 2. forms_v2 cap only | 3. forms_v2 proportional only | 4. Value presence | 5. Field F1 (exact) | 6. ANLS per field | 7. 1 - NED per field | 8. Entity-linking F1 | 9. Checkbox accuracy | 10. Page text 1 - NED | 11. rule_form_field_pass_rate | 12. form_kv_md_core_kv_f1 | 13. form_bench_text_value_f1 | 14. form_bench_text_attribution_f1 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| CRITICAL | |||||||||||||||
| Digit misread in a value | 90 | ✓ | ✓ | ✓ | ✗ | ✗ | ✗ | ✗ | |||||||
| Decimal point lost in an amount | 90 | ✓ | ✓ | ✗ | ✗ | ✗ | ✗ | ||||||||
| Two digits of an ID transposed | 90 | ✓ | ✓ | ✗ | ✗ | ✗ | ✗ | ||||||||
| Values of two fields swapped | 90 | ✓ | ✓ | ✗ | ✓ | ✗ | |||||||||
| Values shifted to the next field | 90 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ | ✓ | ✓ | ✓ | ✓ | |||
| Perturbation | Sev. | 1. ParseBench-2 Visual grounding | 2. EPR v3 draft | 3. EPR v1, no text gate | 4. EPR v1 |
|---|---|---|---|---|---|
| CRITICAL | |||||
| One page-wide box | 90 | ✓ | ✓ | ✓ | ✓ |
| Biggest paragraph missed | 90 | ✓ | ✓ | ||
| Five paragraphs in one box | 90 | ✓ | ✓ | ✓ | ✓ |
| Right box, wrong text | 90 | ✓ | ✓ | ✗ | |
| Coarse blocks | 90 | ✓ | ✓ | ✓ | ✓ |
| Pages limited by the cap (%) | Errors per page | ||||||||
| Pipeline | Text | Fmt | Tab. | Chart | Form | All | Crit. | Maj. | Min. |
| DOCUMENT PARSERS | |||||||||
| LlamaParse (Agentic Plus) | 47 | 37 | 21 | 34 | 75 | 42 | 0.63 | 1.85 | 3.84 |
| LlamaParse (Agentic) | 54 | 33 | 27 | 59 | 79 | 51 | 1.92 | 4.25 | 4.23 |
| LlamaParse (Cost Effective) | 56 | 35 | 31 | 56 | 79 | 52 | 2.65 | 4.77 | 4.24 |
| Reducto | 59 | 33 | 39 | 46 | 78 | 51 | 1.36 | 2.75 | 5.28 |
| # | Pipeline | ParseBench-2 score | Text | Format. | Tables | Charts | Visual grounding | Forms | $/page |
| DOCUMENT PARSERS ranked by ParseBench-2 score; best score per column in bold | |||||||||
| 1 | LlamaParse (Agentic Plus) | 90.1 | 90.3 | 93.6 | 94.6 | 92.6 | 87.5 | 81.8 (82.9) | 0.028 |
| 2 | LlamaParse (Agentic) | 86.0 | 87.0 | 92.7 | 89.9 | 77.4 (87.1) | 88.3 | 80.9 (82.7) | 0.013 |
| 3 | LlamaParse (Cost Effective) | 84.6 | 85.5 | 91.9 | 87.2 | 73.1 (87.2) | 88.6 | 81.2 (82.5) | 0.004 |
| 4 | Reducto | 83.9 | 81.9 | 82.8 | 84.9 | 89.1 | 84.1 | 80.9 | 0.030 |
| 5 | Extend (Parse 2) | 83.2 | 80.2 | 83.3 | 84.2 | 85.1 | 84.8 | 81.5 | 0.025 |
| # | Pipeline | ParseBench-2 score | Text | Format. | Tables | Charts | Visual grounding | Forms | $/page |
| DIRECT VISION MODELS the raw model called with our prompt | |||||||||
| 1 | Claude Opus 5.5 | 85.7 | 92.7 | 89.4 | 94.1 | 74.1 | 86.2 | 77.8 | 0.065 |
| 2 | Gemini 4 argon (low) | 84.2 | 89.7 | 89.2 | 92.8 | 68.2 | 88.1 | 77.2 | 0.027 |
| 3 | GPT-5.6 terra (low) | 77.1 | 81.0 | 87.9 | 79.6 | 76.3 | 62.5 | 75.1 | 0.031 |
| 4 | Gemini 3 Flash (minimal) | 76.6 | 86.7 | 86.1 | 88.9 | 58.0 | 66.3 | 73.8 | 0.009 |
| 5 | Gemini 3.7 Flash (low) | 75.6 | 86.7 | 86.6 | 89.4 | 33.9 | 80.9 | 75.9 | 0.008 |
| Pipeline | Text | Tab. | Chart | Lay. | Form | Hdr. | SERFF | Fin. | Dense | Head. | Strike | Under. | Blank | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Document parsers, ranked | ||||||||||||||
| LlamaParse (Agentic Plus) | 90.3 | 96.1 | 92.6 | 87.5 | 79.8 | 90.3 | 93.5 | 93.7 | 94.8 | 97.1 | 94.5 | 92.4 | 95.7 | 13 |
| LlamaParse (Agentic) | 87.0 | 94.9 | 77.4 | 88.3 | 78.9 | 75.2 | 85.3 | 92.2 | 86.5 | 97.2 | 82.7 | 94.1 | 95.5 | 13 |
| LlamaParse (Cost Effective) | 85.5 | 91.2 | 73.1 | 88.6 | 79.2 | 73.3 | 84.1 | 90.8 | 84.5 | 97.6 | 78.1 | 91.7 | 95.0 | 13 |
| Reducto | 81.9 | 88.6 | 89.1 | 84.1 | 79.0 | 70.2 | 86.4 | 88.5 | 80.2 | 95.3 | 32.6 | 49.5 | 94.0 | 13 |
| Extend (Parse 2) | 80.2 | 87.3 | 85.1 | 84.8 | 79.6 | 74.3 | 84.8 | 88.8 | 76.5 | 95.8 | 85.5 | 34.5 | 94.8 | 13 |
| Dimension | $/page | ParseBench-2 score | |||||||||
| Base configuration | Feature | Dim. | Base | Feature | Core forms | Base | Feat. | Base | w/ feat. | p50 (s) | |
| LlamaParse (Agentic) | specialized charts | Charts | 77.4 | 87.1 | 9.7 | – | 0.013 | 0.013 | 86.0 | 87.7 | 6.5 21.5 |
| LlamaParse (Cost Effective) | specialized charts | Charts | 73.1 | 87.2 | 14.2 | – | 0.004 | 0.004 | 84.6 | 87.0 | 3.6 20.6 |
| Reducto (r-1) | chart agent | Charts | 25.2 | 73.6 * | 48.4 * | – | 0.010 | 0.064 | 76.8 | 84.8 * | 3.0 238.0 |
| Datalab (Accurate) | chart understanding | Charts | 67.8 | 61.8 | 6.1 | – | 0.010 | 0.013 | 81.3 | 80.3 | 7.1 13.7 |
| LlamaParse (Agentic Plus) | enriched forms | Forms | 81.8 | 82.9 | 1.1 | 79.8 81.2 | 0.028 | 0.028 | 90.1 | 90.2 | 25.7 35.8 |