LABEL: Literature Asymmetry between Efficacy and Limits in Drug Knowledge and Language Models
Organizations: Rensselaer Polytechnic Institute, Troy, NY
Abstract
Every medication comes with two types of medical facts: indications, which describe what conditions the drug treats (benefits), and contraindications, which describe who must avoid it to prevent serious harm (safety limits). Although contraindications are critical for patient safety and FDA regulations require listing them as rigorously as indications, we find that contraindications are far less documented than indications in general prose. In this paper, we measure how often these texts contain both facts across 574 drugs from PrimeKG by comparing a drug's indications against its own contraindications. Across Wikipedia articles and PubMed abstracts, we find that reference prose names a drug's benefits about twice as often as its safety limits (within-drug odds ratio 2.19). We call this documentation gap LABEL (Literature Asymmetry between Efficacy and Limits). Language models mirror this gap. As we scale Qwen2.5 from 0.5B to 14B, accuracy on indications rises by 18.3 points, whereas accuracy on contraindications remains flat near random guessing. However, when we provide models with a retrieved passage that states the answer, they use safety facts just as effectively as benefits. Furthermore, routing safety questions to official FDA labels boosts accuracy by 15.8 points whenever the label states the fact. Overall, our findings show that models fail on medication safety not because they cannot reason about risk, but because free-text corpora lack safety facts. Clinical AI systems should therefore route medication-safety questions to official regulatory labels and structured databases rather than relying on general public records. Code and data are available at https://github.com/SMRayeed/LABEL.
Figures & tables
| Source | Item | Count |
| PrimeKG | indication edges | |
| contraindication edges | ||
| off-label use edges | ||
| Hetionet | treats / palliates edges | / |
| MONDO | parent links / diseases | / |
| Text sample | drugs |
| Level | Text contains | Example |
|---|---|---|
| exact name | “type 1 diabetes mellitus” | |
| an alternative name | “juvenile diabetes” | |
| only the family | “diabetes” | |
| nothing | - |
| Corpus | Ind. | Contra. | OR MH | |
|---|---|---|---|---|
| Wikipedia | ||||
| PubMed | ||||
| Both |
| Corpus | Named | Stated | Share (ind./con.) |
|---|---|---|---|
| Wikipedia | / | ||
| PubMed | / | ||
| Whole label | / |
| Text | Ind. | Con. | OR MH | Deg. |
|---|---|---|---|---|
| Wikipedia | ||||
| Wiki + PubMed | ||||
| Label, own section | ind. | |||
| Label, whole | contra. | |||
| All three | none |
| Model | Ind. | Contra. | Gap | Within | |
|---|---|---|---|---|---|
| Qwen2.5-0.5B | |||||
| Qwen2.5-7B | |||||
| OLMo-2-7B | |||||
| Granite-3.1-8B |
| Rel. | In label | Prose | Label | Diff. | |
|---|---|---|---|---|---|
| Contra. | yes | ||||
| no | |||||
| Ind. | yes | ||||
| no |
Appendix figures & tables36 assets
Supplementary material from the paper’s appendix.
Appendix
| Stage | Scripts | What it does |
|---|---|---|
| download | - | Downloads PrimeKG, MONDO, the Disease Ontology and Hetionet, and records their checksums. |
| prepare | prep_primekg, mondo_parents, prep_hetionet | Splits the graphs by relation and builds the disease hierarchy tables. |
| text | gonogo_p1, drug_matched_sample, second_corpus, retier, name_tiers | Draws the samples, fetches Wikipedia and PubMed, and assigns every edge a naming level and a floor. |
| extend | fetch_labels, label_coverage | Fetches FDA labels and measures naming in each label section. |
| models | run_probe_cpu, abstain_probe | Runs the closed-book probe and the harder-option and page tests as cluster jobs. |
| analysis | dilution_test, mh_rbg, w3_stats, w3_extras, w3_failure, w3_summary | Computes every odds ratio, strata table, paired gain and page effect. |
| File | Version | SHA-256 (start) |
|---|---|---|
| PrimeKG kg.csv | Dataverse | 0a5a010fcd88 |
| MONDO mondo.obo | 2026-10-06 | 2631e8e165d4 |
| DOID doid.obo | 2026-09-30 | 51e717b6b9f5 |
| Hetionet edges | v1.0 | 611f0411ac66 |
| Hetionet nodes | v1.0 | 3ad9eb86c031 |
| Explanation | Test | Result |
|---|---|---|
| Famous drugs dominate | Within-drug odds ratio ( Appendix F ) | - in prose |
| Many contraindications per drug | Logistic model, edge-count strata ( Appendix F ) | Gap holds, {10}^{-3}$$ |
| One editorial process | Wikipedia and PubMed separately ( Appendix F ) | and |
| Retrieval favors efficacy | Split PubMed budget ( Appendix C ) | OR both ways |
| Matcher favors a relation | Hetionet, two benefit relations ( Appendix L ) | No gap, 0.16$$ |
| Mention without relation | Judge, four annotators ( Appendix J ) | Prose - , label |
| PrimeKG relation | Edges |
|---|---|
| indication | |
| contraindication | |
| off-label use | |
| disease-disease | |
| all rows in kg.csv |
| MONDO term | Descendants |
|---|---|
| disease or disorder | |
| human disease | |
| disease by body system or component | |
| disease by etiologic mechanism | |
| disease of genetic or genomic mechanism | |
| Mendelian disease |
| Sample | Edges | Exact | Alt. | Family | None |
|---|---|---|---|---|---|
| Text (PrimeKG) | |||||
| Probe (PrimeKG) | |||||
| Hetionet |
| Search | Relation | Wikipedia | PubMed | Union |
|---|---|---|---|---|
| Balanced | contraindication | |||
| indication | ||||
| both | ||||
| Single | contraindication | |||
| indication | ||||
| both |
| Real edge | Floor | |||||
|---|---|---|---|---|---|---|
| Relation | exact | +alt. | +family | exact | +alt. | +family |
| Indication | ||||||
| Off-label | ||||||
| Contraind. | ||||||
| Relation | Exact, corrected | Nomenclature | Granularity |
|---|---|---|---|
| Indication | |||
| Off-label use | |||
| Contraindication | |||
| Mean |
| Drug | Contraindicated disease | Found as |
|---|---|---|
| Isoniazid | acute liver failure | family |
| Entecavir | non-alcoholic steatohepatitis | family |
| Ganciclovir | adult idiopathic neutropenia | family |
| Prednisolone | ocular tuberculosis | family |
| Clonidine | chronic renal failure syndrome | family |
| Sumatriptan | coronary thrombosis | family |
| Corpus | Ind. | Contra. | OR [RBG % CI] | CMH |
| Wikipedia | [ , ] | |||
| PubMed, balanced | [ , ] | |||
| PubMed, sgl. | [ , ] | |||
| Wiki + PubMed | [ , ] | |||
| Wiki + PubMed, sgl. | [ , ] | |||
| Label, own section | [ , ] |
| Corpus | Contra. coef. | log edges | ||
|---|---|---|---|---|
| Wikipedia | ||||
| PubMed, balanced | ||||
| PubMed, sgl. | ||||
| Wiki + PubMed | ||||
| Label, own section | ||||
| Label, whole |
| Edges per drug | - | - | + |
|---|---|---|---|
| Ind. edges ( , prose) | |||
| Contra. edges ( , prose) | |||
| Wikipedia | ∗∗∗ | ∗∗ | |
| PubMed, balanced | ∗∗∗ | ∗∗∗ | |
| PubMed, sgl. | ∗∗∗ | ∗∗∗ | |
| Wiki + PubMed | ∗∗ | ∗∗ |
| Corpus | Relation | - | - | + |
|---|---|---|---|---|
| Wikipedia | Ind. | |||
| Contra. | ||||
| PubMed | Ind. | |||
| Contra. | ||||
| Label, whole | Ind. | |||
| Contra. |
| Corpus | Indication | Contraindication |
|---|---|---|
| Wikipedia | [ , ] | [ , ] |
| Wiki + PubMed | [ , ] | [ , ] |
| Label, own section | [ , ] | [ , ] |
| Label, whole | [ , ] | [ , ] |
| All three | [ , ] | [ , ] |
| Floor, whole label |
| Relation | Missed by prose | Own section | Whole label |
|---|---|---|---|
| Indication | |||
| Contraindication |
| Relation | Prompt stem (X is the drug’s name) |
|---|---|
| Indication | X is a drug used to treat the disease called |
| Contraindication | X is a drug that must not be given to patients who have |
| Off-label use | X is a drug sometimes used off-label to treat |
| Treats (Hetionet) | X is a drug used to treat the disease called |
| Palliates (Hetionet) | X is a drug used to relieve the symptoms of |
| Model | Ind. | Off-label | Contra. | Gap [ % CI] |
|---|---|---|---|---|
| Qwen2.5-0.5B | [ , ] | |||
| Qwen2.5-1.5B | [ , ] | |||
| Qwen2.5-3B | [ , ] | |||
| Qwen2.5-7B | [ , ] | |||
| Qwen2.5-14B | [ , ] | |||
| Wikipedia names it |
| Relation | 0.5B [ % CI] | 7B [ % CI] | Gain |
|---|---|---|---|
| Indication | [ , ] | [ , ] | |
| Off-label | [ , ] | [ , ] | |
| Contraind. | [ , ] | [ , ] |
| Indication | Off-label | Contraind. | ||||
|---|---|---|---|---|---|---|
| Model | named | not | named | not | named | not |
| Qwen2.5-0.5B | ||||||
| Qwen2.5-1.5B | ||||||
| Qwen2.5-3B | ||||||
| Qwen2.5-7B | ||||||
| Qwen2.5-14B | ||||||
| Naming group | Indication | Contraindication |
|---|---|---|
| Names the disease | ||
| Names only its family | ||
| Silent |
| Model | Ind. | Contra. | Gap | Within | (within) |
|---|---|---|---|---|---|
| Qwen2.5-0.5B | |||||
| Qwen2.5-7B | {10}^{-15}$$ | ||||
| OLMo-2-7B | |||||
| Granite-3.1-8B | {10}^{-15}$$ |
| Indication | Contraindication | |||||
|---|---|---|---|---|---|---|
| Model | wrong | decl. | prec. | wrong | decl. | prec. |
| Qwen2.5-0.5B | ||||||
| Qwen2.5-7B | ||||||
| OLMo-2-7B | ||||||
| Granite-3.1-8B | ||||||
| Indication | Contraindication | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Model | Group | Closed | Page | Change [ % CI] | Model | Group | Closed | Page | Change [ % CI] | ||
| Qwen2.5-0.5B | named | [ , ] | Qwen2.5-0.5B | named | [ , ] | ||||||
| family | [ , ] | family | [ , ] | ||||||||
| silent | [ , ] | silent | [ , ] | ||||||||
| Qwen2.5-7B | named | [ , ] | Qwen2.5-7B | named | [ , ] | ||||||
| family | [ , ] | family | [ , ] | ||||||||
| Names it | Family only | Silent | ||||
|---|---|---|---|---|---|---|
| Model | Ind. | Con. | Ind. | Con. | Ind. | Con. |
| Qwen2.5-0.5B | ||||||
| Qwen2.5-7B | ||||||
| OLMo-2-7B | ||||||
| Granite-3.1-8B | ||||||
| Pooled 7-8B | ||||||
| Indication | Contraindication | |||
|---|---|---|---|---|
| Page | closed | page | closed | page |
| Names the disease | ||||
| Names only its family | ||||
| Silent | ||||
| Corpus | Definition | Ind. | Contra. | OR [ % CI] |
|---|---|---|---|---|
| Wikipedia | named | [ , ] | ||
| keyword | [ , ] | |||
| judge | [ , ] | |||
| strict | [ , ] | |||
| PubMed | named | [ , ] | ||
| keyword | [ , ] |
| Relation | Corpus | T | C | W | M | Supports | |
|---|---|---|---|---|---|---|---|
| Indication | Wikipedia | ||||||
| Label | |||||||
| Contra. | Wikipedia | ||||||
| Label |
| Drug, disease | Sentence | Label |
|---|---|---|
| Isocarboxazid, endogenous depression | “It has demonstrated efficacy in various depressive subtypes, including atypical depression, melancholic depression, and endogenous depression.” | T |
| Isocarboxazid, hypertension | “Marplan (isocarboxazid) should not be administered to any patient with a confirmed or suspected cerebrovascular defect or to any patient with cardiovascular disease, hypertension, …” | C |
| Ketorolac, myocardial infarction | “Nonsteroidal anti-inflammatory drugs (NSAIDs) cause an increased risk of serious cardiovascular thrombotic events, including myocardial infarction and stroke, …” | W |
| Diclofenac, rheumatoid arthritis | “Patients with osteoarthritis more often develop symptomatic liver disease than patients with rheumatoid arthritis.” | M |
| Model | Condition | Correct | Wrong | Decline |
|---|---|---|---|---|
| Qwen2.5-7B | prose | |||
| label | ||||
| routed | ||||
| OLMo-2-7B | prose | |||
| label | ||||
| routed |
| Relation | Named | Correct | Wrong | Decline | |
|---|---|---|---|---|---|
| Ind. | yes | [ , ] | |||
| no | [ , ] | ||||
| Contra. | yes | [ , ] | |||
| no | [ , ] |
| treats | palliates | |
| Named in Wikipedia (%) | ( 600$$ ) | ( 385$$ ) |
| Odds ratio, pooled | , 0.70$$ | |
| Within drug ( drugs) | [ , ] | |
| Relation term, edges fixed | 0.16$$ | |
| Gap, - edges | ( 0.06$$ ) | |
| Gap, - edges | ( 0.89$$ ) | |
| Edges per drug | treats | palliates | ||
|---|---|---|---|---|
| - | ||||
| - | ||||
| - |
| Fact | Wikipedia | PubMed | Label |
|---|---|---|---|
| Indications | |||
| infectious diarrheal disease | none | none | none |
| shigellosis | none | named | none |
| conjunctivitis | none | none | named |
| Contraindications | |||
| peripheral neuropathy | none | named | named |
| Drug | Contraindicated condition |
|---|---|
| Ketoconazole | torsades de pointes (QT prolongation) |
| Carbidopa | myocardial infarction |
| Diflunisal | hypertension |
| Desvenlafaxine | angle-closure glaucoma |
| Lisinopril | neonatal jaundice |
| Drug | Contraindicated condition |
|---|---|
| Sumatriptan | Prinzmetal angina |
| Acetylcysteine | asthma |
| Apomorphine | orthostatic hypotension |
| Bimatoprost | uveitis |
| Fenoprofen | kidney disease |
| Levocetirizine | kidney disease |
Medical LLMs
Medical large language models are AI systems adapted to clinical and biomedical data for health reasoning, information extraction, and communication.
See all 428 papers