cs.CLMar 25, 2026

MedAidDialog: A Multilingual Multi-Turn Medical Dialogue Dataset for Accessible Healthcare

Authors: Shubham Kumar Nigam, Suparnojit Sarkar, Piyush Patel

Organizations: University of Birmingham, Dubai, United Arab Emirates · Heritage Institute of Technology, Kolkata, India · Madan Mohan Malaviya University of Technology, India

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Conversational artificial intelligence has the potential to assist users in preliminary medical consultations, particularly in settings where access to healthcare professionals is limited. However, many existing medical dialogue systems operate in a single-turn question--answering paradigm or rely on template-based datasets, limiting conversational realism and multilingual applicability. In this work, we introduce MedAidDialog, a multilingual multi-turn medical dialogue dataset designed to simulate realistic physician--patient consultations. The dataset extends the MDDial corpus by generating synthetic consultations using large language models and further expands them into a parallel multilingual corpus covering seven languages: English, Hindi, Telugu, Tamil, Bengali, Marathi, and Arabic. Building on this dataset, we develop MedAidLM, a conversational medical model trained using parameter-efficient fine-tuning on quantized small language models, enabling deployment without high-end computational infrastructure. Our framework additionally incorporates optional patient pre-context information (e.g., age, gender, allergies) to personalize the consultation process. Experimental results demonstrate that the proposed system can effectively perform symptom elicitation through multi-turn dialogue and generate diagnostic recommendations. We further conduct medical expert evaluation to assess the plausibility and coherence of the generated consultations.

Figures & tables

Figure 1: Example response from a general-purpose LLM (ChatGPT 5.3). The model produces a single explanatory response without collecting additional symptoms or conducting follow-up questioning.
Figure 2: Example interaction with MedAidLM . The system first incorporates patient pre-context information (e.g., age, gender, and allergies) and then performs multi-turn dialogue to collect symptoms before producing a diagnostic recommendation.
 Dialogue TurnsAverage Words
DatasetAvgTotalMinMaxPerPatientDoctor
 TurnsDialoguesTurnsTurnsDialogueUtteranceUtterance
MDDial (MD)4.9187911653.55.66.7
Synthetic (SYN)6.61101511134.58.89.6
MD + SYN5.7298011686.97.008.05
MDDial Test5.923711355.45.66.6
Table 1: Statistics of the original MDDial dataset (MD) and the synthetic dialogues used to construct the MedAidDialog corpus. The synthetic dialogues contain more conversational turns and longer utterances, resulting in richer physician–patient interactions.
Figure 3: Overview of the proposed framework. Stage 1: Data Augmentation. The MDDial dataset is expanded with synthetic medical dialogues, followed by coherence and diversity filtering. Stage 2: Model Adaptation. Compact open-source language models are fine-tuned using parameter-efficient training and LoRA-based SFT. The dotted connection indicates an optional GRPO optimisation stage applied to selected models. Stage 3: Deployment. The best-performing checkpoint is deployed as MedAidLM , which operates within a multilingual inference loop that incorporates optional patient pre-context and bidirectional translation.
Model FamilyDatasetMethodAccuracy
Mistral-7B-InstructMedAidDialog (MD+SYN)SFT88.09%
LLaMA 3.2 3B (MedAidLM)MedAidDialog (MD+SYN)SFT90.21%
Qwen3-4BMedAidDialog (MD+SYN)SFT80.00%
DeepSeek-R1-Distill-Qwen-1.5BMedAidDialog (MD+SYN)SFT40.00%
Table 2: Results on the MedAidDialog dataset.
ModelDatasetAvg. TurnsDialogsMethodAccuracy
Mistral-7B-InstructMD4.901879SFT18.72%
Mistral-7B-InstructSYN7.281101SFT61.28%
Mistral-7B-InstructMD+SYN5.782980SFT80.85%
Mistral-7B-InstructMD+SYN5.782980SFT+GRPO77.87%
LLaMA 3.2 3BMD4.901879SFT75.74%
LLaMA 3.2 3BSYN7.281101SFT71.97%
Table 3: Ablation study over training data composition and optimisation strategy. These experiments correspond to shorter training runs (100 steps), used to analyze the effect of original data (MD), synthetic data (SYN), and the combined MedAidDialog corpus (MD+SYN).
DiseaseCorrectTotalAccuracy
Asthma161984.2%
Conjunctivitis192190.5%
Coronary heart disease161984.2%
Dermatitis192095.0%
Enteritis222491.7%
Esophagitis222781.5%
Table 4: Per-disease diagnostic accuracy of the final MedAidLM model on the evaluation set.
Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

ParameterValue
Rank ( rr )16
Target Modulesq_proj, k_proj, v_proj, o_proj,
 gate_proj, up_proj, down_proj
LoRA Alpha16
LoRA Dropout0
Biasnone
Appendix
Table 5: LoRA configuration used for parameter-efficient fine-tuning.
HyperparameterValue
Learning Rate2×10−42\times 10^{-4}
OptimizerAdamW (8-bit)
Learning Rate SchedulerLinear
Weight Decay0.001
Warmup Steps5
Maximum Training Steps600
Appendix
Table 6: Training hyperparameters used for supervised fine-tuning.
MetricExpert 1Expert 2Expert 3Average
Medical Safety (Pass Rate)96%94%96%95.3%
Symptom Extraction4.24.14.34.20
Context Memory4.44.34.54.40
Diagnostic Correctness4.14.04.24.10
Conversational Flow4.34.24.44.30
Efficiency4.03.94.14.00
Appendix
Table 7: Medical expert evaluation of MedAidLM across 50 sampled dialogues. Scores are reported on a 1–5 Likert scale except Medical Safety (Pass/Fail).
Original DiseaseMisclassified AsFrequency
PneumoniaAsthma3
EsophagitisEnteritis2
EsophagitisAsthma2
AsthmaPneumonia2
Coronary heart diseaseAsthma2
PneumoniaEnteritis2
Appendix
Table 8: Most frequent disease-level misclassifications made by the final MedAidLM model.
MetricKrippendorff’s α\alpha
Symptom Extraction0.82
Context Memory0.84
Diagnostic Correctness0.80
Conversational Flow0.83
Efficiency0.78
Average0.81
Appendix
Table 9: IAA scores across three medical experts.
Prompt TypePrompt Content
Translation PromptYou are acting as a specialized Medical Translation Bridge, a critical link between an English-speaking doctor and a patient who speaks Hindi, Bengali, Marathi, Telugu, Arabic, or Tamil. Your primary responsibility is to maintain absolute clinical accuracy while ensuring the tone is appropriately synced for both parties. When the doctor speaks in English, you must translate their advice, diagnoses, and prescriptions into the patient’s native language using clear, empathetic, and culturally respectful terminology that a non-medical person can easily understand. Conversely, when the patient provides a query or describes symptoms in their native language, you will convert that input into precise, formal medical English for the doctor, ensuring that nuances of pain, duration, and history are preserved without loss of detail. You are strictly prohibited from hallucinating or adding medical advice not present in the source text; your role is purely to facilitate a perfectly synced, bidirectional exchange. Ensure that if the patient expresses distress or urgency, the English translation reflects that clinical priority to the doctor. Your output must contain only the translated text to allow for seamless integration into the communication interface.
Appendix
Table 10: Prompt used for bidirectional medical translation in the multilingual inference layer.
Prompt TypePrompt Content
Synthetic Dialogue Generation PromptAnalyze train.json medical dialogues (patient/doctor exchanges, symptoms like “Cough”, diagnoses such as “Esophagitis”). Create Python synthetic generator using Groq API (Llama-3 family model). Match exact format: {’Dialog N’: [{’patient’: ’...’, ’doctor’: ’...’}]} . Randomize symptom openings, generate 4–8 turns with doctor questions and realistic patient responses. Preserve the overall structure used for model training and provide progress, ETA, and resume-friendly execution. Output synthetic data in the same format as train.json .
Appendix
Table 11: Prompt used to generate synthetic multi-turn medical consultations from the MDDial training distribution.
Prompt TypePrompt Content
Dialogue Formatting PromptConvert a medical dialogue sample into ShareGPT-style multi-turn conversation. Structure: (1) the system message sets the medical diagnosis context, (2) patient utterances become human turns, (3) doctor utterances become gpt turns, and (4) the final gpt turn contains the diagnosis answer. Preserve dialogue order and ensure that each consultation remains a valid multi-turn interaction for instruction tuning.
Appendix
Table 12: Prompt used to convert raw medical dialogues into ShareGPT-style training instances.
Evaluation CriterionDescription
Medical Safety (Pass/Fail)Whether the system provides any potentially dangerous, misleading, or unsafe medical advice during the conversation.
  
Symptom Extraction (1–5)Measures how accurately the model identifies and tracks the patient’s symptoms throughout the dialogue.
  
Context Memory (1–5)Evaluates whether the model remembers previously mentioned information such as symptoms or earlier responses in the conversation.
  
Appendix
Table 13: Evaluation criteria used in expert assessment of the conversational medical system. Experts rated multiple aspects of safety, reasoning, and dialogue quality using a Likert scale (1–5), while medical safety was evaluated using a binary pass/fail metric.
More in this topic

Medical LLMs

Medical large language models are AI systems adapted to clinical and biomedical data for health reasoning, information extraction, and communication.

See all 396 papers

Explore similar work

CardsList
May 13, 2026

IndicMedDialog: A Parallel Multi-Turn Medical Dialogue Dataset for Accessible Healthcare in Indic Languages

Most existing medical dialogue systems operate in a single-turn question--answering paradigm or rely on template-based datasets, limiting conversational realism and multilingual applicability. We introduce IndicMedDialog, a parallel multi-turn medical dialogue dataset spanning English and nine Indic languages: Assamese, Bengali, Gujarati, Hindi, Marathi, Punjabi, Tamil, Telugu, and Urdu. The dataset extends MDDial with LLM-generated synthetic consultations, translated using TranslateGemma, verified by native speakers, and refined through a script-aware post-processing pipeline to correct phonetic, lexical, and character-spacing errors. Building on this dataset, we fine-tune IndicMedLM via parameter-efficient adaptation of a quantized small language model, incorporating optional patient pre-context to personalise multi-turn symptom elicitation. We evaluate against zero-shot multilingual baselines, conduct systematic error analysis across ten languages, and validate clinical plausibility through medical expert evaluation.

Shubham Kumar Nigam, Suparnojit Sarkar, Piyush Patel

University of Birmingham, Dubai, United Arab Emirates · Heritage Institute of Technology, Kolkata, India · Madan Mohan Malaviya University of Technology, India

Multilingual Language ModelsHealthcareClinical NLPMedical LLMs
AskPDF
May 28, 2026

Dial HEALTHDIAL for Advice: A Multilingual and Multi-Parallel Spoken Dialogue Dataset for Knowledge-Grounded Information Seeking

Creating spoken dialogue datasets is methodologically challenging, and these challenges are amplified when the goal is to build multilingual, multi-parallel datasets at scale. This work introduces HEALTHDIAL, a large-scale, multilingual, and multi-parallel dataset for developing and evaluating retrieval-augmented generation (RAG)-based spoken dialogue systems. The dataset comprises 6,000 information-seeking dialogues (1,500 per language) grounded in trusted content from the World Health Organization (WHO) and 163 hours of user speech recorded from native speakers of diverse dialects across four official WHO languages: Arabic, Chinese, English, and Spanish. Each speaker is annotated with demographic (e.g., gender, age) and sociolinguistic (e.g., primary language, region of origin) variables. We report benchmark results across key dialogue tasks, which reveal consistent performance disparities across languages, even among high-resource ones. To support future research, we release the dataset, a prototype system, and a toolkit for data collection and system evaluation.

Songbo Hu, Yinhong Liu, Ej Zhou +5

Language Technology Lab, University of Cambridge, UK · Independent Researcher · School of Computation, Information and Technology, Technical University of Munich, Germany

Spoken Dialogue SystemsRAG Evaluation
AskPDF
May 26, 2026

A Dataset of Robot-Patient and Doctor-Patient Medical Dialogues for Spoken Language Processing Tasks

Large Language Models (LLMs) have brought huge improvements to Artificial Intelligence (AI), which can be applied to general-purpose tasks. However, their application to textual or spoken medical consultations is still an open research problem. This paper proposes MeDial-Speech, a novel speech dataset for training and evaluating Med-AIs that can carry out consultations with patients. It was collected in realistic environments from robot-patient and doctor-patient dialogues, contains 111+ hours of speech data (without data augmentation), and covers four health conditions: Lewy body dementia, heart failure, shoulder pain, and angina. In addition, we propose a dialogue benchmark via sentence selection (with 20 options) to evaluate three state-of-the-art LLMs: GPT-5 mini, DeepSeek-V3, and Claude Sonnet 4. Experimental results reveal that Claude Sonnet 4 is the best in sentence selection, with 71.1% accuracy using manual transcriptions and 74.7% using automatic transcriptions, and that all LLMs are highly overconfident in their probabilistic predictions, regardless of selecting correct or incorrect sentences in medical dialogues. This dataset is free of charge for non-commercial purposes at: https://huggingface.co/datasets/hcuayahu/MeDial-Speech

Heriberto Cuayahuitl, Grace Jang

School of Engineering and Physical Sciences University of Lincoln, UK · Lincoln Medical School Universities of Lincoln and Nothingham, UK

HealthcareSpeech ProcessingClinical Language Model Evaluation
AskPDF