Tool-Augmented Large Language Models for Reliable Medical Calculation from Patient Notes
Organizations: College of Computer Science and Technology, Dalian University of Technology, Dalian, China · Air Force Communications NCO Academy, Dalian, China
Abstract
Medical calculators translate raw patient observations into clinically actionable values and scores. However, applying these calculators directly to free-form patient notes requires precise variable extraction combined with the exact execution of complex formulas, scoring rules, and output conventions. Relying solely on direct large language model (LLM) prompting is unreliable in this context, as rule recall, parameter extraction, arithmetic reasoning, and formatting become entangled within a single generative step. To address these limitations, we propose a tool-augmented LLM framework for reliable medical calculation from patient notes. The framework retrieves and structures calculator specifications, generates reusable executable tools, and uses a constrained extractor to map each patient note to calculator-ready inputs. The tools then perform deterministic calculation, separating clinical language understanding from rule execution and arithmetic. Evaluated on the MedCalc-Bench dataset, our method achieves 86.73% final-answer accuracy, outperforming the one-shot prompting and supervised fine tuning-only baselines by 24.0 and 17.27 percentage points. Ablation and error analyses show that these performance gains are primarily driven by rule-grounded tool synthesis and schema-constrained extraction, whereas remaining failures are concentrated in clinical-semantic extraction and benchmark-specific output contracts. By grounding calculator logic in executable tools, the proposed approach alleviates arithmetic hallucinations and improves the computational reliability of the final clinical outputs.
Figures & tables
| Item | Setting |
|---|---|
| Base models | Qwen3-8B |
| Tuning method | LoRA adapter on frozen base model |
| LoRA rank / alpha / dropout | 16 / 32 / 0.05 |
| Learning rate | |
| Epochs | 5 |
| global batch size | 8 |
| Method | Model | Score (%) |
| MedRaC * [ 23 ] | LLaMA3.1-8B-Instruct | 64.68 |
| Open-book guided prompting [ 14 ] | GLM-4.6V | 82.00 |
| Open-book prompting * [ 14 ] | GLM-4.7 | 85.50 |
| Code agent (Python REPL) [ 31 ] | Claude Sonnet 4.6 | 86.24 |
| Direct prompting | Qwen3-8B | 14.00 |
| Zero-shot CoT | Qwen3-8B | 31.55 |
| Tool setting | Model | Runtime | Tool acc. | Final acc. |
| errors | (%) | (%) | ||
| Direct generation | GPT-5.5 | 0 | 74.45 | 73.27 |
| Retrieval + repair | Qwen3.5-Plus | 21 | 84.45 | 76.18 |
| Retrieval + repair | GPT-5.3-Codex | 4 | 89.00 | 82.82 |
| Retrieval + repair | GPT-5.5 | 0 | 94.91 | 86.73 |
| Note: The last column pairs each tool set with the same Qwen3-8B-SFT extractor. | ||||
| Variant | Final acc. (%) | |
|---|---|---|
| Full workflow | 86.73 | - |
| No extractor SFT | 83.73 | -3.00 |
| No schema/few-shot | 79.18 | -7.55 |
| No tool | 69.46 | -17.27 |
| Builder | Calls | Input | Output | Total |
|---|---|---|---|---|
| Qwen3.5-Plus | 144 | 243,571 | 147,350 | 390,921 |
| GPT-5.3-Codex | 159 | 272,884 | 124,773 | 397,657 |
| GPT-5.5 | 147 | 227,454 | 371,758 | 599,212 |
| Tool library | Total (min) | Seconds/case |
|---|---|---|
| Qwen3.5-Plus: retrieval + repair | 48.15 | 2.626 |
| GPT-5.3-Codex: retrieval + repair | 48.47 | 2.644 |
| GPT-5.5: retrieval + repair | 47.17 | 2.573 |
| GPT-5.5: direct generation | 47.28 | 2.579 |
| Case A | Case B | Case C | |
|---|---|---|---|
| Calculator | Creatinine Clearance, Cockcroft-Gault (lab test) | HEART Score (risk) | Steroid Conversion Calculator (dosage) |
| Patient note (key facts) | 87-year-old man; height 163 cm; weight 48 kg; serum creatinine 1.4 mg/dL; the question asks Cockcroft-Gault clearance with weight selection. | 47-year-old man in the ED; diabetes, hypertension, atorvastatin use, normal ECG, troponin upper limit, no prior atherosclerotic disease, slightly suspicious history. | Patient receives Hydrocortisone PO 162.935 mg; the question asks for the equivalent Dexamethasone PO dose, with the oral route preserved. |
| Question | Creatinine clearance, mL/min, using adjusted body weight if it exceeds ideal. | HEART Score (0–10) at first admission, before treatment. | Equivalent Dexamethasone PO dose in mg. |
| Generated tool pseudocode (written once by GPT-5.5) | ibw = ideal_weight(sex, height) ; w = actual if actual < ibw else adjusted ; crcl = ((140-age)w)/(72scr) . | risk_count = diabetes + hypertension + hypercholesterolemia + ... ; risk_points = 2 if risk_count >= 3 else 1 if risk_count >= 1 else 0 . | if "cortisone" in name: return cortisone ; elif "hydrocortisone" in name: return hydrocortisone ; then convert by steroid-equivalence table. |
| Extracted parameters (Qwen3-8B-SFT) | sex=Male, age=87 y, weight=48 kg, height=163 cm, creatinine=1.4 mg/dL. | age=47, hypertension=True, diabetes=True, hypercholesterolemia=False , ECG=Normal, troponin= , atherosclerotic=False, suspicion=Slightly suspicious. | input steroid=Hydrocortisone PO, dose=162.935 mg; target steroid=Dexamethasone PO. |
| Tool computation | weight 48 ideal 59.6, so uses actual weight; . | only 2 risk factors counted, so risk block scores 1 and total = 4 . | Hydrocortisone is misread as cortisone: 162.935\times 0.75/25={\color[rgb]{1,0,0}4.88805} mg. |
LLM Tool Use
Large language model tool use studies how models select, call, and coordinate external tools to extend their reasoning and take actions.
See all 295 papers