cs.LGFeb 27, 2026
SaveStethoLM: Audio Language Model for Cardiopulmonary Analysis Across Clinical Tasks
Organizations: Eindhoven University of Technology
Abstract
Listening to heart and lung sounds - auscultation - is one of the first and most fundamental steps in a clinical examination. Despite being fast and non-invasive, it demands years of experience to interpret subtle audio cues. Recent deep learning methods have made progress in automating cardiopulmonary sound analysis, yet most are restricted to simple classification and offer little clinical interpretability or decision support. We present StethoLM, the first audio-language model specialized for cardiopulmonary auscultation, capable of performing instruction-driven clinical tasks across the full spectrum of auscultation analysis. StethoLM integrates audio encoding with a medical language model backbone and is trained on StethoBench, a comprehensive benchmark comprising 77,027 instruction-response pairs synthesized from 16,125 labeled cardiopulmonary recordings spanning seven clinical task categories: binary classification, detection, reporting, reasoning, differential diagnosis, comparison, and location-based analysis. Through multi-stage training that combines supervised fine-tuning and direct preference optimization, StethoLM achieves substantial gains in performance and robustness on out-of-distribution data. Our work establishes a foundation for instruction-following AI systems in clinical auscultation.
Figures & tables
Figure 1: Overview of StethoLM and StethoBench. A. Automated benchmark creation pipeline, where off-the-shelf LLMs generate 77,027 task–response pairs from 16,125 cardiopulmonary recordings and associated annotations. B. Distribution of audio type and the examples of disease that StethoLM covers. C. StethoLM architecture integrating audio–text alignment, supervised fine-tuning for diverse clinical tasks.
Figure 2: Diverse clinical tasks supported by StethoLM. Instructions (left) represent realistic clinical queries, while responses (right) provide task-appropriate outputs ranging from binary decisions to complex diagnostic reasoning.
| Dataset | Samples (train/val/test) | Pairs | Minutes | Modality | Application / Task |
| In-Distribution (ID) Datasets | |||||
| ICBHI | 539 / 381 | 9,028 | 329 | Respiratory | Abnormality Detection |
| KAUH | 200 / 69 / 67 | 5,842 | 97 | Mixed | Disease Classification |
| SPR | 1,773 / 1,002 / 48 | 14,183 | 276 | Respiratory | Respiratory Event Detection |
| CoughVID | 1,223 / 830 | 9,844 | 283 | Cough | COVID-19 Detection |
| CovidUK | 2,000 / 1,000 / 1,000 | 13,356 | 378 | Cough | COVID-19 Screening |
Table 1: Summary of datasets included in StethoBench. The upper section lists in-distribution (ID) datasets used for training, validation, and testing. The lower section lists out-of-distribution (OOD) datasets used for generalization evaluation. OOD datasets differ from ID data in recording devices, patient populations, disease prevalence, clinical contexts, or collection protocols, reflecting realistic deployment scenarios. “Samples” indicates the number of audio recordings . “Pairs” refers to the number of corresponding instruction-response pairs .
| Models | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Task | LLM | Pengi | LTU | Gama | Gemma3N | Gemini | Qwen | StethoLM | ||||||||
| BertS | Acc | BertS | Acc | BertS | Acc | BertS | Acc | BertS | Acc | BertS | Acc | BertS | Acc | BertS | Acc | |
| Classification | 48.6 | 4.9 | 33.6 | 7.6 | 49.0 | 2.5 | 46.4 | 7.5 | 51.3 | 31.3 | 49.3 | 49.5 | 58.9 | 31.0 | 75.5 | 66.4 |
| Detection | 43.0 | 2.8 | 30.8 | 4.3 | 43.1 | 3.6 | 44.0 | 4.2 | 47.1 | 11.5 | 45.9 | 19.2 | 52.5 | 22.0 | 70.4 | 47.9 |
| Reporting | 41.1 | 2.2 | 30.7 | 4.1 | 45.1 | 2.0 | 44.7 | 2.0 | 47.5 | 9.2 | 49.8 | 13.2 | 56.7 | 12.7 | 72.8 | 36.2 |
| Reasoning | 41.8 | 0.0 | 27.6 | 1.0 | 43.7 | 4.0 | 46.5 | 5.3 | 45.2 | 9.0 | 47.6 | 22.2 | 55.6 | 29.0 | 71.4 | 44.1 |
Table 2: Performance comparison on the StethoBench benchmark. BertS denotes BERTScore and Acc denotes Accuracy (both in %). The best score for each metric per task is highlighted in bold , and the second-best score is underlined . Note: "Gemini" refers to Gemini-2.5-Flash and "Qwen" refers to Qwen2.5-Omni . Additional baseline comparisons with Qwen3-Omni ( Xu et al., 2025b ) and Audio Flamingo 3 ( Goel et al., 2025 ) are provided in Appendix E . Supplementary metrics (ROUGE-1, METEOR) and relaxed accuracy evaluation for DDx focusing on clinical plausibility are provided in Appendix F .
| Models | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Dataset | LLM | Pengi | LTU | Gama | Gemma3N | Gemini | Qwen | StethoLM | ||||||||
| BertS | Acc | BertS | Acc | BertS | Acc | BertS | Acc | BertS | Acc | BertS | Acc | BertS | Acc | BertS | Acc | |
| TR | 44.0 | 0.5 | 27.7 | 1.6 | 44.2 | 0.6 | 44.0 | 0.3 | 48.4 | 5.6 | 44.8 | 7.5 | 60.7 | 17.7 | 66.2 | 25.7 |
| CinC | 39.5 | 1.1 | 29.4 | 1.3 | 42.3 | 0.4 | 42.6 | 0.7 | 44.5 | 5.2 | 45.9 | 21.5 | 54.0 | 12.4 | 63.3 | 22.2 |
| BMD | 45.0 | 2.5 | 27.3 | 1.4 | 47.2 | 1.2 | 46.7 | 4.4 | 48.2 | 11.7 | 40.7 | 20.9 | 58.6 | 17.3 | 67.3 | 30.4 |
| FluSense | 45.8 | 0.3 | 30.6 | 15.6 | 45.8 | 9.4 | 47.3 | 22.3 | 50.7 | 9.4 | 52.3 | 14.3 | 59.4 | 37.1 | 61.5 | 23.2 |
Table 3: Out-of-Domain (OOD) performance comparison across datasets. BertS denotes BERTScore and Acc denotes Accuracy (both in %). The best score for each metric per dataset is highlighted in bold , and the second-best score is underlined . Additional baseline comparisons with Qwen3-Omni ( Xu et al., 2025b ) and Audio Flamingo 3 ( Goel et al., 2025 ) are provided in Appendix E . Supplementary metrics (ROUGE-1, METEOR) are provided in Appendix F .
Figure 3: Zero-shot classification comparison with classification models. Accuracy across four binary tasks comparing zero-shot inference (StethoLM via report generation + text similarity) against supervised linear probing (AudioMAE, CLAP, Opera-CE). Tasks include COVID-19 detection from exhalation/cough recordings and COPD screening from lung sounds.
Figure 4: Attention patterns revealing audio grounding during response generation. Heatmap visualizes how generated response tokens (vertical axis) attend to key tokens (horizontal axis). Black solid lines mark the audio-instruction boundary; red dashed lines mark where response generation begins.
Figure 5: Qualitative analysis of StethoLM’s clinical reasoning capabilities. Representative outputs demonstrate the model’s ability to generate clinically coherent responses across diverse auscultation tasks.
| Component | Configuration | BertS. | Acc. |
| StethoLM | Full model: Audio+Text, all trainable | 71.8 | 47.8 |
| Modality | Audio removed at inference | 57.8 | 28.5 |
| Audio-text mismatch (shuffled pairing) | 63.6 | 31.9 | |
| Audio Encoder | CLAP | 71.4 | 47.0 |
| AST | 71.8 | 47.5 | |
| LLM Backbone | Llama-3.2-3B-Instruct | 70.4 | 45.3 |
Table 4: Ablation study on StethoLM components. Each variant modifies one component of the StethoLM configuration. BertS denotes BERTScore (%), and Acc denotes accuracy (%). All models are evaluated on the StethoBench benchmark.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
| Ground Truth | Instruction | Generated Response | Error Type |
|---|---|---|---|
| Diagnosis: Bronchitis, Sound: Wheezing | What could be the differential diagnosis? | While the primary diagnosis is bronchitis, differentials could include asthma or pneumonia, given obstructive sounds like wheezing and rhonchi . | Acoustic hallucination (rhonchi added) |
| Diagnosis: Pneumonia (severe), Sounds: Crackles | Explain your reasoning for suspecting severe pneumonia. | The presence of coarse crackles, consistent with fluid accumulation and alveolar collapse , strongly suggests severe pneumonia. | Clinical over-elaboration (pathophysiology added) |
| Murmur: No, Outcome: Abnormal | Provide differential diagnosis. | Possible valvular dysfunction , myocardial disease, or congenital anomaly. | Contradiction (valvular dysfunction typically causes murmurs) |
Table 5: Examples of error patterns in synthetic instruction-response generation.
| Task | Accurate | Acoustic Halluc. | Clinical Over-elab. | Contradict. |
|---|---|---|---|---|
| Classification | 8/10 (80%) | 0 | 2 | 0 |
| Detection | 9/10 (90%) | 0 | 1 | 0 |
| Reporting | 8/10 (80%) | 1 | 1 | 0 |
| Reasoning | 7/10 (70%) | 0 | 2 | 1 |
| DDx | 6/10 (60%) | 1 | 3 | 0 |
| Comparison | 9/10 (90%) | 0 | 1 | 0 |
Table 6: Synthetic data validation results by task type. Based on 10 samples per task (70 total).
| Task | Judge Model Variation | Prompt Variation | ||||
|---|---|---|---|---|---|---|
| GPT-5-mini | Qwen-Plus | Gemini-2.5-Pro | Prompt 1 | Prompt 2 | Prompt 3 | |
| Classification | 69.5 | 73.5 (+4.0) | 81.0 (+11.5) | 69.5 | 64.0 (-5.5) | 66.0 (-3.5) |
| Detection | 46.0 | 46.0 (0.0) | 54.5 (+8.5) | 46.0 | 44.5 (-1.5) | 44.5 (-1.5) |
| Report | 32.0 | 34.0 (+2.0) | 56.5 (+24.5) | 32.0 | 30.5 (-1.5) | 30.0 (-2.0) |
| Reasoning | 45.0 | 51.0 (+6.0) | 64.5 (+19.5) | 45.0 | 45.0 (0.0) | 46.0 (+1.0) |
| DDx | 29.0 | 35.0 (+6.0) | 53.0 (+24.0) | 29.0 | 27.5 (-1.5) | 26.0 (-3.0) |
Table 7: Robustness analysis of clinical accuracy metric. Values in parentheses show deviation from the baseline (GPT-5-mini with Prompt 1). All results on 200 random samples per task.
| Model | In-Domain | Out-of-Domain | ||
|---|---|---|---|---|
| BertS | Acc | BertS | Acc | |
| Qwen2.5-Omni | 56.5 | 21.2 | 58.2 | 21.1 |
| Qwen3-Omni | 56.1 | 23.9 | 57.0 | 20.4 |
| Audio Flamingo 3 | 56.2 | 21.4 | 56.7 | 18.3 |
| StethoLM | 71.8 | 47.8 | 64.8 | 25.2 |
Table 8: Additional baseline comparison. Performance on in-domain and out-of-domain test sets. BertS denotes BERTScore and Acc denotes Accuracy (both in %). The best score is bold , second-best is underlined .
| Models | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Task | LLM | Pengi | LTU | Gama | Gemma3N | Gemini | Qwen | StethoLM | ||||||||
| R-1 | MET | R-1 | MET | R-1 | MET | R-1 | MET | R-1 | MET | R-1 | MET | R-1 | MET | R-1 | MET | |
| Classification | 20.8 | 15.1 | 4.0 | 0.8 | 31.0 | 20.7 | 18.0 | 12.1 | 29.7 | 19.5 | 14.4 | 16.4 | 33.8 | 22.5 | 50.2 | 36.6 |
| Detection | 13.9 | 9.1 | 2.6 | 0.8 | 19.2 | 10.2 | 15.0 | 9.3 | 22.7 | 15.7 | 12.7 | 14.7 | 24.7 | 13.9 | 43.4 | 32.3 |
| Reporting | 15.0 | 9.4 | 3.6 | 0.9 | 18.0 | 9.8 | 14.8 | 8.2 | 20.3 | 11.5 | 19.0 | 15.4 | 25.6 | 14.3 | 50.0 | 39.3 |
| Reasoning | 13.9 | 6.2 | 1.7 | 0.5 | 18.7 | 7.3 | 19.6 | 9.5 | 18.6 | 8.6 | 10.3 | 14.2 | 26.7 | 14.1 | 48.8 | 35.1 |
Table 9: Performance comparison on StethoBench benchmark using ROUGE-1 and METEOR (complementing Table 2 ). R-1 denotes ROUGE-1 and MET denotes METEOR (both in %). The best score for each metric per task is highlighted in bold , and the second-best score is underlined . Note: "Gemini" refers to Gemini-2.5-Flash and "Qwen" refers to Qwen2.5-Omni .
| Models | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Dataset | LLM | Pengi | LTU | Gama | Gemma3N | Gemini | Qwen | StethoLM | ||||||||
| R-1 | MET | R-1 | MET | R-1 | MET | R-1 | MET | R-1 | MET | R-1 | MET | R-1 | MET | R-1 | MET | |
| TR | 17.7 | 10.5 | 1.2 | 0.5 | 24.2 | 13.8 | 17.7 | 10.5 | 24.1 | 15.4 | 11.0 | 14.6 | 30.2 | 20.3 | 35.2 | 21.5 |
| CinC | 13.8 | 6.8 | 1.1 | 0.4 | 19.5 | 8.4 | 16.5 | 8.6 | 20.5 | 9.8 | 5.0 | 5.1 | 24.0 | 12.8 | 32.6 | 18.0 |
| BMD | 18.0 | 11.3 | 1.7 | 0.6 | 22.7 | 14.6 | 19.2 | 10.7 | 23.6 | 13.3 | 12.4 | 16.1 | 27.5 | 16.0 | 37.4 | 25.9 |
| FluSense | 16.3 | 12.5 | 2.5 | 0.6 | 23.8 | 12.9 | 19.2 | 12.2 | 22.8 | 16.2 | 15.8 | 14.0 | 30.5 | 18.4 | 25.4 | 17.6 |
Table 10: Out-of-Domain (OOD) performance using ROUGE-1 and METEOR (complementing Table 3 ). R-1 denotes ROUGE-1 and MET denotes METEOR (both in %). The best score for each metric per dataset is highlighted in bold , and the second-best score is underlined .
Figure 6: Representative failure cases across major error categories. Each example shows the instruction, ground truth, model prediction, and analysis revealing the underlying failure pattern. These failure patterns complement the successful generation examples in Figure 5 , together providing a complete characterization of model capabilities and limitations.