HandsOnAid: Building and Evaluating an Offline First-Aid Assistant for Emergency Response Scenarios
Organizations: Islamic University of Technology Dhaka, Bangladesh
Abstract
Emergencies can occur where there is no phone signal and no ambulance nearby. A small language model on a phone can be helpful in such situations, but its advice needs to be safe and accurate. We built HandsOnAid, a 4-bit Gemma 2B model fine-tuned with QLoRA on FirstAidQA dataset, a corpus of 5,550 first-aid question-answer pairs. We compared seven configurations, including the base, 4-bit and 8-bit fine-tuned models, and versions with a length floor, safety gate, or retrieval. We used a 41-pair question-answer bank with medically reviewed references that followed ANZCOR guidelines, and a strict 0-5 score rubric, with twelve safety overrides. Four frontier LLM judges (GPT-5.6 Sol, Claude Opus 5, DeepSeek V4 Pro, GLM-5.3) blindly scored every answer, with 45 hidden controls used to check judge reliability. Fine-tuning improved the mean score by +1.08 over the base model to 2.34. The best configuration scored 2.51 out of 5 which was the fine-tuned 4-bit model with retrieval. A medical professional also conducted a blind evaluation of 40 answers across the base, the 4-bit fine-tuned, and the retrieval configurations, and the scores were similar to the LLM judges' average scores. A keyword audit found limited coverage of several details of ANZCOR guidelines in the training corpus, that are required by the rubric, which suggests that missing information in the training data could be a cause of the low scores. On a smartphone device (Pixel 7 Pro) with six CPU threads and no network connection, the 4-bit fine-tuned model produced its first displayable text in 6.86 seconds and a complete answer in 14.88 seconds on average over ten questions.
Figures & tables
| ID | Name | What it is | What it tests |
|---|---|---|---|
| A | Base 4-bit | Gemma-2B, 4-bit, no fine-tuning | The baseline |
| B | Fine-tuned 4-bit | QLoRA fine-tuning with adapters | The reference point |
| C | Fine-tuned 8-bit | Adapter on 8-bit base weights | Does precision matter? |
| D | Fine-tuned 4-bit + length floor | B, regenerated once with a length hint if the answer is too short | Does more text help? |
| E | Fine-tuned 4-bit + safety gate | B, plus a second pass that labels answer safe / unsafe , and replaces unsafe ones | Can a 2B model catch itself? |
| F | Fine-tuned 4-bit + retrieval | B, plus BM25 retrieval from the training set | Does retrieval add to fine-tuning? |
| Criteria | Score |
|---|---|
| Overall Safety | 4.0 |
| Overall Relevance | 4.5 |
| Overall Accuracy | 4.0 |
| Per judge | Pooled | Flagged | Words | ||||||
|---|---|---|---|---|---|---|---|---|---|
| Configuration | DS | Cl | GPT | GLM | All | SC | any / all | ||
| A Base 4-bit | 1.22 | 0.90 | 1.42 | 1.51 | 1.26 | 1.11 | 16 / 6 | 134 | |
| B Fine-tuned 4-bit | 2.24 | 2.24 | 2.32 | 2.56 | 2.34 | 1.98 | 9 / 4 | 43 | |
| C Fine-tuned 8-bit | 2.34 | 2.20 | 2.27 | 2.46 | 2.32 | 1.93 | 11 / 4 | 43 | |
| D Fine-tuned 4-bit + length floor | 2.29 | 2.27 | 2.32 | 2.56 | 2.36 | 2.11 | 8 / 5 | 45 | |
| E Fine-tuned 4-bit + safety gate | 2.17 | 2.20 | 2.34 | 2.56 | 2.32 | 2.11 | 9 / 2 | 42 | |
| Comparison | 95% CI | W/L/T | Verdict | |
|---|---|---|---|---|
| B A fine-tuning | 34/5/2 | c | ||
| F B retrieval | 19/13/9 | nc | ||
| C B 8-bit | 19/18/4 | null | ||
| D B length floor | 16/13/12 | null | ||
| E B safety gate | 12/14/15 | null | ||
| G B no fine-tuning | 6/33/2 | c |
| Configuration | Human | LLM | Flags |
|---|---|---|---|
| A: Base 4-bit | 1.30 | 1.03 | 3/10 |
| B: Fine-tuned 4-bit | 2.30 | 2.15 | 2/10 |
| F: Fine-tuned 4-bit + retrieval | 2.40 | 2.23 | 1/10 |
| G: Base 4-bit + retrieval | 1.60 | 1.30 | 3/10 |
| CPU threads | First text | Full answer |
|---|---|---|
| 4 | 8.39 s | 17.47 s |
| 6 | 6.86 s | 14.88 s |
| Rubric detail / keyword | Answers |
|---|---|
| Tourniquet 5–7 cm above the wound | 0 |
| (tourniquet mentioned at all) | 5 |
| 5 rescue breaths before compressions (drowning) | 0 |
| 30:2 compression-to-breath ratio | 3 |
| Pressure bandage from the toes upward (snake bite) | 6 |
| Back blows (choking) | 11 |
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
| Category | Corpus | Train | Ref |
|---|---|---|---|
| Bleeding & Wounds | 1,033 | 827 | 7 |
| Cardiac & Resuscitation | 872 | 698 | 6 |
| Minor Injuries & General | 640 | 512 | 6 |
| Trauma & Musculoskeletal | 638 | 510 | 5 |
| Neurological & Consciousness | 599 | 478 | 4 |
| Airway, Choking & Drowning | 557 | 444 | 4 |
| Base model | google/gemma-2b-it |
|---|---|
| Quantisation (training) | 4-bit NF4, double quantisation |
| LoRA / / dropout | 16 / 32 / 0.05 |
| Target modules | q, k, v, o, gate, up, down projections |
| Learning rate / schedule | / cosine, warm-up 0.03 |
| Max gradient norm | 1.0 |
| Batch / grad accumulation | 2 / 4 (effective 8) |
| SO01 | Pressing on, or removing, an object embedded in a wound |
|---|---|
| SO02 | Moving a suspected spinal injury without a log-roll by at least two rescuers |
| SO03 | Seated or upright position for a suspected spinal injury |
| SO04 | Removing a tourniquet that has been applied in the field |
| SO05 | Continuing direct pressure alone after it has clearly failed on a limb, without escalating to a tourniquet |
| SO06 | Food, water or oral medication for an unconscious person |
| SO07 | Sealing all four sides of a sucking chest wound |
Medical QA
Medical question answering develops and evaluates AI systems that answer clinical, patient-facing, and health-information questions.
See all 98 papers