cs.HCAug 27, 2026

An Offline Smartphone Check for Drug Interactions and Past Reactions in Handwritten Prescriptions: A Technical Feasibility Study in Rural Bangladeshi Outpatient Care

Authors: Shahir Abdullah

Organizations: Department of Computer Science and Engineering, United International University, Dhaka, Bangladesh

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Rural Bangladeshi outpatient clinics keep no patient records, prescriptions are handwritten, and physicians have under a minute per patient, so they easily miss a known adverse drug reaction (ADR) or drug-drug interaction (DDI). We built and evaluated an offline Android application that photographs a handwritten prescription, detects and reads each medicine line, resolves brand names to active ingredients, and retrieves, but never predicts, documented interactions and the patient's recorded reactions. This quantitative technical-feasibility study, with qualitative error analysis, involved no patients or clinicians. Reference data combine two Bangladeshi drug registries (27,880 brands) with 190,421 DDI pairs. A YOLOv8n line detector reached [email protected] = 0.687 on held-out prescriptions but recalled only 60.6% of lines at the deployed cutoff. On-device ML Kit read 11.8% of 780 benchmark handwritten words exactly. We trained a CRNN-CTC reader on public handwriting data, decoded it against the brand lexicon, and fine-tuned it on 660 author-labelled prescription crops. On benchmark words, the reader accepted the correct ingredient for 41.5% and a wrong one for 4.2%, versus 21.0% and 5.3% for ML Kit at its preset cutoff (McNemar p<10^-20). On scorable lines from held-out real prescriptions, the reader accepted the correct ingredient for 36.0% [26.8, 46.3] and a wrong one for 2.2% [0.6, 7.8]. In 62 simulated runs on a mid-range phone, median time per patient was 26-31 s and 59 of 62 runs finished within 60 s, but 62% of medicine lines still needed a manual check. A cross-visit medication history added after the evaluation passed seven automated tests and one two-visit functional test but was not evaluated. The pipeline is feasible within the clinic workflow; reading handwriting, not retrieval, is the bottleneck. Nothing here demonstrates a clinical benefit.

Figures & tables

ApproachADR/DDILMIC fitNo-records infra.Rural BDMain limitation for our setting
EHR allergy/interaction alerts (high-income) [ 5 , 6 ]✓✗✗✗Requires full EHR/CPOE; severe alert fatigue
LMIC point-of-care CDS / mHealth [ 9 , 10 , 11 ]✗✓✓✗Targets other conditions; not medication safety
Bangladeshi prescription-error studies [ 7 , 8 ]∼\sim✓∼\sim✓Measure the problem; no deployed system
Handwritten-prescription reading / OCR [ 19 , 20 ]✗✓∼\sim✓Reading only; no ingredient resolution, history or check
This work✓✓✓✓Simulated use only; reading accuracy is the bottleneck
TABLE I: Comparison of existing approaches against the requirements of the target setting (rural Bangladeshi outpatient care).
Planned axisStatus in this paper
Workflow fit (s/patient)Executed as simulated use by the author: 62 logged runs, two sessions (Sec. VIII-D ).
Component accuracyExecuted : detection, reading, resolution, with CIs (Secs. VIII-A – VIII-C ).
Retrospective detection of known site casesNot executed : no site records were used.
Identity re-linking via phone numberNot executed : dummy numbers only.
Usability / acceptability (e.g. SUS [ 21 ] , interviews)Not executed : no clinicians took part.
Reaction-note reading (history capture)Implemented, not tested : no reaction note was marked in any logged run.
TABLE II: Evaluation plan of the earlier version versus what was executed.
#SourceContentRole
1BdMedex [ 22 ] + DGDA registered products [ 23 ]21,714 + 41,201 brand rowsReference: brand-to-ingredient lookup. Never trained on.
2DrugBank-derived DDI table [ 24 , 25 , 26 ]191,541 drug pairs with textReference: interaction knowledge base. Never trained on.
3Handwritten prescriptions with medicine-line boxes [ 27 ]200 photos, 1,086 boxes (YOLO format)Detector: train 140 / val 30 / test 30 photos (seed 42). The author also added text labels to the boxes (Sec. IV-B ).
4Doctor’s Handwritten Prescription BD [ 20 ]4,680 word images, 78 names (+generic)OCR benchmark only : test split, 780 images. Never trained or tuned on.
5RxHandBD [ 19 ]5,578 word images, 1,509 distinct textsWe train the handwriting reader on 4,463 images and use the test split (1,115) to select checkpoints and the first cutoff. No image appears in both this dataset and Dataset 4, but 33 of Dataset 4’s 78 names appear in Dataset 5’s training labels.
TABLE III: Data sources and their roles. Sources marked “Never trained on” are lookup tables. We never report results on a split used for model selection.
Fig. 2: Implemented pipeline. Light boxes: human steps; mid-grey: on-device learned components; dark: retrieval. Reaction notes are read with ML Kit and stored at ingredient level against the phone number. Check (C) and the visit store were added after the evaluated sessions (Sec. VIII-F ).
#ChangeTriggerEffect
1Ingredient canonicalisation (salt, synonyms)Only 39.3% of dictionary ingredients existed in the DDI table; same drug stored under two spellingsCoverage 60.6%; one spelling per ingredient, so reactions match across brands
2Full-dictionary fuzzy matchScaffold matched only the first 50 brands sharing a 3-letter prefix; OCR gets the first 3 letters right 23% of the time+1.7 pp ingredient-correct (EasyOCR proxy)
3ML Kit →\rightarrow first real testAuthor’s first photographs: “Napa Extend” read as “e. Nopa Euknd”3/3 lines unreadable; correct brand ranked 19th and 39th by fuzzy match
4DGDA merge, vet filter, typo snappingBrands on real prescriptions missing from BdMedex13,611 →\rightarrow 27,880 brands
5CRNN-conv reader + lexicon CTCNeed a handwriting model; free text unrecoverableD4: IC 30.0% / WA 4.6% @0.99 (cutoff read off D4, see #8)
6CRNN-LSTM readerLSTM export to TFLite initially failed; conv fallback overwrote one LSTM runD4 greedy exact 15.1% →\rightarrow 21.3%
TABLE IV: Incremental development. “D4” = Dataset 4 benchmark (780 words, on-device); IC / WA = ingredient-correct / wrong-accepted (% of all words) at the stated cutoff.
[email protected] / [email protected]:0.950.687 / 0.314
Precision / recall (at max-F1 confidence)0.845 / 0.564
Box precision / recall at cutoff 0.15 (IoU ≥\geq 0.5)71.4% / 60.6%
Exact box count per photo16.7% [7.3, 33.6]
Mean absolute count error1.70 boxes
TABLE V: Detection on the Dataset 3 test split (30 photos, 165 boxes).
ReaderExactCutoffIC (%)WA (%)Top-1 IC (%)ms/line
EasyOCR + fuzzy (desktop proxy)—0.8217.9 [15.4, 20.8]4.5 [3.2, 6.2]28.1—
ML Kit + fuzzy (original)11.8%0.8221.0 [18.3, 24.0]5.3 [3.9, 7.1]30.0128
CRNN-conv + lexicon15.1%0.99 †30.0 [26.9, 33.3]4.6 [3.4, 6.3]54.4160
CRNN-LSTM + lexicon (brand posterior)21.3%0.999922.9 [20.1, 26.0]2.1 [1.3, 3.3]62.3207
CRNN-LSTM + lexicon (ingredient-set)21.3%0.999926.8 [23.8, 30.0]2.1 [1.3, 3.3]62.3198
Fine-tuned LSTM + lexicon (ingredient-set), deployed24.5%0.99541.5 [38.1, 45.0]4.2 [3.0, 5.9]63.7200
TABLE VI: Dataset 4 test words ( N=780N=780 ), on-device. Exact = greedy text equals the name. IC/WA at the operating cutoff with 95% Wilson CIs. “Top-1” = always accept the best lexicon/fuzzy match (no cutoff). † Cutoff read off this benchmark (optimistic). ms/line: median on the Galaxy A52 (ML Kit: recognition only; CRNN: including lexicon decoding).
 RxHand-onlyFine-tuned
CER (121 medicine lines)0.630.40
Brand part read exactly4.1 [1.8, 9.3]16.5 [11.0, 24.2]
Top-1 IC (no cutoff)33.7 [24.7, 44.0]58.4 [48.0, 68.1]
IC @0.98 (5% cap on val)18.0 [11.4, 27.2]44.9 [35.0, 55.3]
WA @0.985.6 [2.4, 12.5]7.9 [3.9, 15.4]
IC @0.995 (2% cap, deployed)14.6 [8.7, 23.4]36.0 [26.8, 46.3]
TABLE VII: Real prescription crops, test split: 89 scorable medicine lines (IC/WA, % with 95% CI). Last rows: app behaviour over all 165 boxes at τa=0.995\tau_{a}{=}0.995 , τs=0.9\tau_{s}{=}0.9 .
 S1 ( n=32n{=}32 )S2 ( n=30n{=}30 )
Phone entry6.2 [5.6, 7.2]6.2 [5.6, 7.4]
Photo / pick8.2 [6.7, 8.7]8.7 [8.1, 9.4]
Detection1.1 [1.1, 1.1]1.8 [1.8, 1.8]
Confirm boxes7.6 [6.5, 8.3]11.1 [9.6, 13.6]
Reading (all lines)0.9 [0.8, 1.1]1.1 [0.9, 1.3]
Total per patient26.4 [24.0, 33.3]30.7 [28.3, 33.7]
TABLE VIII: Per-run timing from the application logs (median with 95% bootstrap CI, seconds) and line dispositions. S1/S2 = session 1/2.
 Visit 1Visit 2
History before checknone storedvisit 1 (in window)
Read automaticallyPantosil →\rightarrow pantoprazoleVoltalin D →\rightarrow diclofenac
Manual / “Is this X?”2 manual2 manual; “NAB (diclofenac)?” →\rightarrow No
Resultnot cleared (unreadable)flagged : diclofenac (today) + pantoprazole (visit 1)
Stored for later visitsPantosilVoltalin D (NAB not stored)
Total time200.6 s52.9 s
TABLE IX: Functional test of the cross-visit check: one dummy patient, two visits on the same day (application log and stored history).
More in this topic

Healthcare

Healthcare AI develops and evaluates models and systems for clinical care, medical research, public health, and healthcare operations.

See all 617 papers

Explore similar work

CardsList
Feb 17, 2026

Exploring Drug Safety Through Knowledge Graphs: Protein Kinase Inhibitors as a Case Study

Adverse Drug Reactions (ADRs) are a leading cause of morbidity and mortality. Existing prediction methods rely mainly on chemical similarity, machine learning on structured databases, or isolated target profiles, but often fail to integrate heterogeneous, partly unstructured evidence effectively. We present a knowledge graph-based framework that unifies diverse sources, drug-target data (ChEMBL), clinical trial literature (PubMed), trial metadata (ClinicalTrials.gov), and post-marketing safety reports (FAERS) into a single evidence-weighted bipartite network of drugs and medical conditions. Applied to 400 protein kinase inhibitors, the resulting network enables contextual comparison of efficacy (HR, PFS, OS), phenotypic and target similarity, and ADR prediction via target-to-adverse-event correlations. A non-small cell lung cancer case study correctly highlights established and candidate drugs, target communities (ERbB, ALK, VEGF), and tolerability differences. Designed as an orthogonal, extensible analysis and search tool rather than a replacement for current models, the framework excels at revealing complex patterns, supporting hypothesis generation, and enhancing pharmacovigilance. Code and data are publicly available at https://github.com/davidjackson99/PKI_KG.

David Jackson, Michael Gertz, Jürgen Hesser

Institute of Informatics, University of Amsterdam, Amsterdam, Netherlands · Institute of Computer Science, Heidelberg University, Heidelberg, Germany · Data Analysis and Modeling in Medicine, Mannheim Institute for Intelligent Systems in Medicine (MIISM), Medical Faculty Mannheim, Heidelberg University, Mannheim, Germany

Clinical Risk PredictionBiomedical KGsKG ConstructionDrug Discovery
AskPDF
May 27, 2026

SafeRx-Agent: A Knowledge-Grounded Multi-Agent Framework for Safe and Explainable Medication Recommendation

Medication recommendation predicts medications for patient visits, but existing methods still face two key challenges. At the model level, traditional drug recommendation methods only predict structured drug codes with limited evidence grounding, while LLM agents can use richer clinical context but may lack safety verification and traceability. At the task level, existing benchmarks often use broad medication categories, which ignore subgroup-level safety differences and can lead to risk overestimation. We introduce the first fine-grained medication recommendation setting based on fourth-level ATC code generation. We propose Safe Prescription Agent (SafeRx-Agent), a knowledge-grounded multi-agent framework that uses patient context, external clinical knowledge, and safety verification to recommend traceable medication sets. Experimental results on MIMIC-III and MIMIC-IV datasets show that SafeRx-Agent improves fine-grained medication prediction accuracy while controlling drug interactions, contraindications, and medication set size.

Xinyu Wang, Hanwei Wu, Zhenghan Tai +7

McGill University · McMaster University · University of Toronto +1

Medication RecommendationMulti-Agent LLM SystemsDrug-Drug Interaction PredictionHealthcareClinical Decision SupportLLM Agent Safety
AskPDF
Jul 30, 2026

GRAIN: Molecules Are Not the Right Granularity -- Active-Ingredient Modeling for Safe Medication Recommendation

Medication recommendation from electronic health records must balance predictive accuracy against the risk of adverse drug-drug interactions (DDIs) under polypharmacy. Existing safety-aware recommenders operate at one of two granularities: the drug code, which treats each medication as an indivisible token, or the molecular substructure, which is finer than pharmacological interaction knowledge is actually organized. We argue that the active ingredient is the missing granularity, and introduce GRAIN, a medication recommendation framework built around it. GRAIN encodes longitudinal patient trajectories (diagnoses, procedures, past medications) with a selective state space backbone that handles long, irregular visit sequences in linear time. On top of it we introduce a joint objective unifying three knowledge sources aligned to a common medication vocabulary: a drug-level DDI graph, an ingredient-level DDI graph obtained by normalizing medication codes to active ingredients via RxNorm, and an EHR-derived co-prescription graph. A proportional controller adapts the accuracy-safety trade-off to the observed validation DDI rate rather than fixing it a priori. Under strictly matched settings -- identical preprocessing, cohort, vocabulary, split, and evaluation code -- GRAIN improves over a re-implemented MambaHealth baseline on MIMIC-IV across all standard multi-label metrics (Jaccard 0.4488 to 0.4983, PRAUC 0.6911 to 0.7485, F1 0.5989 to 0.6453) while reducing the drug-level DDI rate from 0.1875 to 0.0948. We further define an ingredient-level DDI rate, a safety measure invisible to drug-code-level evaluation. The results indicate that ingredient-level normalization recovers predictive signal erased by code-level aggregation, and that it is complementary to, rather than in competition with, accurate sequence modeling.

Juao Fan, Jinhan Li, Shengxin Zhu

Guangdong Provincial Key Laboratory of Interdisciplinary Research and Application for Data Science, Beijing Normal–Hong Kong Baptist University, Zhuhai, China

Medication RecommendationClinical Decision SupportDrug-Drug Interaction Prediction
AskPDF