cs.CLMar 25, 2026
SaveMedAidDialog: A Multilingual Multi-Turn Medical Dialogue Dataset for Accessible Healthcare
Organizations: University of Birmingham, Dubai, United Arab Emirates · Heritage Institute of Technology, Kolkata, India · Madan Mohan Malaviya University of Technology, India
Abstract
Conversational artificial intelligence has the potential to assist users in preliminary medical consultations, particularly in settings where access to healthcare professionals is limited. However, many existing medical dialogue systems operate in a single-turn question--answering paradigm or rely on template-based datasets, limiting conversational realism and multilingual applicability. In this work, we introduce MedAidDialog, a multilingual multi-turn medical dialogue dataset designed to simulate realistic physician--patient consultations. The dataset extends the MDDial corpus by generating synthetic consultations using large language models and further expands them into a parallel multilingual corpus covering seven languages: English, Hindi, Telugu, Tamil, Bengali, Marathi, and Arabic. Building on this dataset, we develop MedAidLM, a conversational medical model trained using parameter-efficient fine-tuning on quantized small language models, enabling deployment without high-end computational infrastructure. Our framework additionally incorporates optional patient pre-context information (e.g., age, gender, allergies) to personalize the consultation process. Experimental results demonstrate that the proposed system can effectively perform symptom elicitation through multi-turn dialogue and generate diagnostic recommendations. We further conduct medical expert evaluation to assess the plausibility and coherence of the generated consultations.
Figures & tables
Figure 1: Example response from a general-purpose LLM (ChatGPT 5.3). The model produces a single explanatory response without collecting additional symptoms or conducting follow-up questioning.
Figure 2: Example interaction with MedAidLM . The system first incorporates patient pre-context information (e.g., age, gender, and allergies) and then performs multi-turn dialogue to collect symptoms before producing a diagnostic recommendation.
| Dialogue Turns | Average Words | ||||||
| Dataset | Avg | Total | Min | Max | Per | Patient | Doctor |
| Turns | Dialogues | Turns | Turns | Dialogue | Utterance | Utterance | |
| MDDial (MD) | 4.9 | 1879 | 1 | 16 | 53.5 | 5.6 | 6.7 |
| Synthetic (SYN) | 6.6 | 1101 | 5 | 11 | 134.5 | 8.8 | 9.6 |
| MD + SYN | 5.7 | 2980 | 1 | 16 | 86.9 | 7.00 | 8.05 |
| MDDial Test | 5.9 | 237 | 1 | 13 | 55.4 | 5.6 | 6.6 |
Table 1: Statistics of the original MDDial dataset (MD) and the synthetic dialogues used to construct the MedAidDialog corpus. The synthetic dialogues contain more conversational turns and longer utterances, resulting in richer physician–patient interactions.
Figure 3: Overview of the proposed framework. Stage 1: Data Augmentation. The MDDial dataset is expanded with synthetic medical dialogues, followed by coherence and diversity filtering. Stage 2: Model Adaptation. Compact open-source language models are fine-tuned using parameter-efficient training and LoRA-based SFT. The dotted connection indicates an optional GRPO optimisation stage applied to selected models. Stage 3: Deployment. The best-performing checkpoint is deployed as MedAidLM , which operates within a multilingual inference loop that incorporates optional patient pre-context and bidirectional translation.
| Model Family | Dataset | Method | Accuracy |
|---|---|---|---|
| Mistral-7B-Instruct | MedAidDialog (MD+SYN) | SFT | 88.09% |
| LLaMA 3.2 3B (MedAidLM) | MedAidDialog (MD+SYN) | SFT | 90.21% |
| Qwen3-4B | MedAidDialog (MD+SYN) | SFT | 80.00% |
| DeepSeek-R1-Distill-Qwen-1.5B | MedAidDialog (MD+SYN) | SFT | 40.00% |
Table 2: Results on the MedAidDialog dataset.
| Model | Dataset | Avg. Turns | Dialogs | Method | Accuracy |
| Mistral-7B-Instruct | MD | 4.90 | 1879 | SFT | 18.72% |
| Mistral-7B-Instruct | SYN | 7.28 | 1101 | SFT | 61.28% |
| Mistral-7B-Instruct | MD+SYN | 5.78 | 2980 | SFT | 80.85% |
| Mistral-7B-Instruct | MD+SYN | 5.78 | 2980 | SFT+GRPO | 77.87% |
| LLaMA 3.2 3B | MD | 4.90 | 1879 | SFT | 75.74% |
| LLaMA 3.2 3B | SYN | 7.28 | 1101 | SFT | 71.97% |
Table 3: Ablation study over training data composition and optimisation strategy. These experiments correspond to shorter training runs (100 steps), used to analyze the effect of original data (MD), synthetic data (SYN), and the combined MedAidDialog corpus (MD+SYN).
| Disease | Correct | Total | Accuracy |
|---|---|---|---|
| Asthma | 16 | 19 | 84.2% |
| Conjunctivitis | 19 | 21 | 90.5% |
| Coronary heart disease | 16 | 19 | 84.2% |
| Dermatitis | 19 | 20 | 95.0% |
| Enteritis | 22 | 24 | 91.7% |
| Esophagitis | 22 | 27 | 81.5% |
Table 4: Per-disease diagnostic accuracy of the final MedAidLM model on the evaluation set.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
| Parameter | Value |
|---|---|
| Rank ( ) | 16 |
| Target Modules | q_proj, k_proj, v_proj, o_proj, |
| gate_proj, up_proj, down_proj | |
| LoRA Alpha | 16 |
| LoRA Dropout | 0 |
| Bias | none |
Table 5: LoRA configuration used for parameter-efficient fine-tuning.
| Hyperparameter | Value |
|---|---|
| Learning Rate | |
| Optimizer | AdamW (8-bit) |
| Learning Rate Scheduler | Linear |
| Weight Decay | 0.001 |
| Warmup Steps | 5 |
| Maximum Training Steps | 600 |
Table 6: Training hyperparameters used for supervised fine-tuning.
| Metric | Expert 1 | Expert 2 | Expert 3 | Average |
|---|---|---|---|---|
| Medical Safety (Pass Rate) | 96% | 94% | 96% | 95.3% |
| Symptom Extraction | 4.2 | 4.1 | 4.3 | 4.20 |
| Context Memory | 4.4 | 4.3 | 4.5 | 4.40 |
| Diagnostic Correctness | 4.1 | 4.0 | 4.2 | 4.10 |
| Conversational Flow | 4.3 | 4.2 | 4.4 | 4.30 |
| Efficiency | 4.0 | 3.9 | 4.1 | 4.00 |
Table 7: Medical expert evaluation of MedAidLM across 50 sampled dialogues. Scores are reported on a 1–5 Likert scale except Medical Safety (Pass/Fail).
| Original Disease | Misclassified As | Frequency |
|---|---|---|
| Pneumonia | Asthma | 3 |
| Esophagitis | Enteritis | 2 |
| Esophagitis | Asthma | 2 |
| Asthma | Pneumonia | 2 |
| Coronary heart disease | Asthma | 2 |
| Pneumonia | Enteritis | 2 |
Table 8: Most frequent disease-level misclassifications made by the final MedAidLM model.
| Metric | Krippendorff’s |
|---|---|
| Symptom Extraction | 0.82 |
| Context Memory | 0.84 |
| Diagnostic Correctness | 0.80 |
| Conversational Flow | 0.83 |
| Efficiency | 0.78 |
| Average | 0.81 |
Table 9: IAA scores across three medical experts.
| Prompt Type | Prompt Content |
| Translation Prompt | You are acting as a specialized Medical Translation Bridge, a critical link between an English-speaking doctor and a patient who speaks Hindi, Bengali, Marathi, Telugu, Arabic, or Tamil. Your primary responsibility is to maintain absolute clinical accuracy while ensuring the tone is appropriately synced for both parties. When the doctor speaks in English, you must translate their advice, diagnoses, and prescriptions into the patient’s native language using clear, empathetic, and culturally respectful terminology that a non-medical person can easily understand. Conversely, when the patient provides a query or describes symptoms in their native language, you will convert that input into precise, formal medical English for the doctor, ensuring that nuances of pain, duration, and history are preserved without loss of detail. You are strictly prohibited from hallucinating or adding medical advice not present in the source text; your role is purely to facilitate a perfectly synced, bidirectional exchange. Ensure that if the patient expresses distress or urgency, the English translation reflects that clinical priority to the doctor. Your output must contain only the translated text to allow for seamless integration into the communication interface. |
Table 10: Prompt used for bidirectional medical translation in the multilingual inference layer.
| Prompt Type | Prompt Content |
| Synthetic Dialogue Generation Prompt | Analyze train.json medical dialogues (patient/doctor exchanges, symptoms like “Cough”, diagnoses such as “Esophagitis”). Create Python synthetic generator using Groq API (Llama-3 family model). Match exact format: {’Dialog N’: [{’patient’: ’...’, ’doctor’: ’...’}]} . Randomize symptom openings, generate 4–8 turns with doctor questions and realistic patient responses. Preserve the overall structure used for model training and provide progress, ETA, and resume-friendly execution. Output synthetic data in the same format as train.json . |
Table 11: Prompt used to generate synthetic multi-turn medical consultations from the MDDial training distribution.
| Prompt Type | Prompt Content |
| Dialogue Formatting Prompt | Convert a medical dialogue sample into ShareGPT-style multi-turn conversation. Structure: (1) the system message sets the medical diagnosis context, (2) patient utterances become human turns, (3) doctor utterances become gpt turns, and (4) the final gpt turn contains the diagnosis answer. Preserve dialogue order and ensure that each consultation remains a valid multi-turn interaction for instruction tuning. |
Table 12: Prompt used to convert raw medical dialogues into ShareGPT-style training instances.
| Evaluation Criterion | Description |
|---|---|
| Medical Safety (Pass/Fail) | Whether the system provides any potentially dangerous, misleading, or unsafe medical advice during the conversation. |
| Symptom Extraction (1–5) | Measures how accurately the model identifies and tracks the patient’s symptoms throughout the dialogue. |
| Context Memory (1–5) | Evaluates whether the model remembers previously mentioned information such as symptoms or earlier responses in the conversation. |
Table 13: Evaluation criteria used in expert assessment of the conversational medical system. Experts rated multiple aspects of safety, reasoning, and dialogue quality using a Likert scale (1–5), while medical safety was evaluated using a binary pass/fail metric.
More in this topic
Medical LLMs
Medical large language models are AI systems adapted to clinical and biomedical data for health reasoning, information extraction, and communication.
See all 393 papers