Automating clinical documentation from long-form doctor-patient conversations remains challenging for modern audio-language models. While cascaded ASR systems perform well, end-to-end (E2E) models often struggle with information loss and hallucinations on extended audio. For the BeTraC 2026 challenge, the ASLP team presents a fully E2E multimodal system that generates structured SOAP notes directly from audio, bypassing intermediate transcripts. We constructed a 1.41-million-sample multi-task corpus and applied a multi-stage pipeline: domain pre-training, supervised fine-tuning, and reward optimization. Evaluating the architecture under both Lightweight (3B) and Heavyweight (30B) constraints reveals that each training stage progressively enhances performance. Furthermore, scaling to 30B parameters substantially boosts concept extraction and summarization quality. Ultimately, our E2E systems consistently outperform representative cascaded ASR+LLM baselines, proving the efficacy of direct multimodal optimization for clinical documentation.
Figures & tables
Task Category
Dataset Source
Duration
I. General Long-Form Audio
Long-Form ASR
MLC-SLM Challenge
350.9 h
Dense Audio Captioning
General Web Audio
1,797.9 h
Temporal Grounding
DAC/TAC/TAG
∼ 1,865 h
II. Medical Speech Recognition
Conversational ASR
YFYeung Medical 1 1 1 https://huggingface.co/datasets/yfyeung/medical
∼ 48 h
TABLE I: Summary of the Unified Multi-Task Dataset for Full-Parameter Pre-training
Model
Training Stage
Concept F1
C-Precision
C-Recall
ROUGE-2
ROUGE-3
Words
Lightweight Track ( < 6B)
Qwen2.5-Omni-3B [ 27 ]
Baseline (Zero-Shot)
0.2604
0.2891
0.2450
0.0920
0.0344
380
Qwen2.5-Omni-3B
Best SFT
0.3872
0.3814
0.4093
0.2155
0.1325
410
Qwen2.5-Omni-3B
Best GRPO
0.4082
0.4078
0.4224
0.2323
0.1418
372
Heavyweight Track ( < 36B)
Qwen3-Omni-30B [ 28 ]
Baseline (Zero-Shot)
0.1879
0.1879
0.1964
0.0558
0.0175
351
TABLE II: Summary of Best Evaluation Results per Training Stage on the Dev/Test Set.
Input Prompt (Role: User)
<audio> This is a patient-doctor conversation recording. Generate a clinical SOAP note following the structure below.
1. Subjective
- Chief Complaint (CC)
- History of Present Illness (HPI)
- Review of Systems (ROS), grouped by body system
2. Objective
Fig. 1: Instruction prompt used for end-to-end SOAP note generation.