Neural Signals Generate Clinical Notes in the Wild
Organizations: University of Illinois Urbana-Champaign · SANKEN, Osaka University
Abstract
Generating clinical reports that summarize abnormal patterns, diagnostic findings, and clinical interpretations from long-term EEG recordings remains labor-intensive. We present CELM, the first clinical EEG-to-Language foundation model capable of summarizing long-duration, variable-length EEG recordings and performing end-to-end clinical report generation at multiple scales. CELM integrates pretrained EEG foundation models with language models to enable scalable multimodal learning. We curate a large-scale clinical EEG dataset containing 9,922 reports paired with approximately 11,000 hours of EEG recordings from 9,048 patients to train CELM, and release the benchmark pipeline, with automated report-structuring, to facilitate future research. Experimental results show that CELM consistently outperforms existing methods across all evaluation settings. Importantly, we further conduct human evaluation with clinical experts, demonstrating that CELM generates reports that are more clinically coherent, diagnostically reliable, and better aligned with expert interpretation. We release our model code and benchmark construction pipeline at https://github.com/Jathurshan0330/CELM.
Figures & tables
Appendix figures & tables18 assets
Supplementary material from the paper’s appendix.
Appendix
| Site | # of Paired EEG & Reports | # of Patients | Total Duration (hrs) |
| S0001 | 5049 | 4669 | 6639 |
| S0002 | 4873 | 4379 | 4367 |
| Module | Hyperparameter | Values |
| Linear Projector | Projector Input Dimension (CBraMod Embedding Dimension) | 200 |
| Projector Output Dimension (Qwen-3 4B Embedding Dimension) | 2560 | |
| Bias | True | |
| # Trainable parameters | 522,240 | |
| Perceiver Projector | # Query tokens | |
| Embedding Dimension | 200 |
| Hyperparameter | Values |
| Batch size | 4 |
| Gradient accumulation steps | 4 |
| Optimizer | AdamW |
| Learning rate | 1e-4 |
| Weight decay | 0.01 |
| 0.9 |
| Quartile | Dur. (min) | Range (min) | Proj. | ROUGE-1 | METEOR | BERTScore |
| S0001 | ||||||
| Q1 | 54.4 | 0.7–65.2 | SCA | 0.5522 | 0.4553 | 0.8709 |
| SCC | 0.4031 | 0.2895 | 0.8112 | |||
| Q2 | 71.5 | 65.3–77.8 | SCA | 0.5940 | 0.5089 | 0.8824 |
| SCC | 0.4041 | 0.3081 | 0.8013 | |||
| Q3 | 86.4 | 78.0–94.2 | SCA | 0.5618 | 0.4828 | 0.8758 |
| Dimension | Evaluation criterion and rating scale |
| Clinical Correctness | Correctness of clinical content relative to the ground-truth report. 5: Very good; 4: Good; 3: Acceptable; 2: Poor; 1: Very poor. |
| Findings Coverage | Extent to which key clinical findings in the ground-truth report are identified. 5: All found; 4: Most found; 3: Some found; 2: Few found; 1: Missed. |
| Finding Accuracy | Accuracy of finding attributes, including lateralization, frequency, and morphology. 5: All correct; 4: Minor errors; 3: Some errors; 2: Many errors; 1: Inaccurate. |
| Real-World Readiness | Suitability of the generated report for real-world clinical use. 5: Very good; 4: Good; 3: Acceptable; 2: Poor; 1: Very poor. |
| Completeness | Extent to which the generated report covers important information. 5: Very good; 4: Good; 3: Acceptable; 2: Poor; 1: Very poor. |
| Model Details | |
| Developers | Jathurshan Pradeepkumar, Zheng Chen, and Jimeng Sun (University of Illinois Urbana-Champaign; SANKEN, Osaka University). |
| Model date | April 2026. |
| Model version | v1.0. Two variants: CELM (Sequence Context Alignment, SCA) and CELM-SCC (Sequence Context Compression). Separate models for S0001 and S0002 |
| Model type | EEG-to-language model for structured clinical report generation. A frozen pretrained EEG encoder (CBraMod) produces one token per 10-s epoch (Epoch-Aggregated Tokenization). A trainable sequence-aware alignment module maps these tokens into the embedding space of a frozen instruction-tuned LLM (Qwen3-4B-Instruct-2507). In SCA this module is a linear-attention transformer followed by a linear projection and in SCC it is Perceiver-style cross-attention with learnable queries. The LLM generates report text conditioned on the EEG tokens, a task prompt, and optional clinical context. |
| Inputs and outputs | Input: scalp EEG resampled to 200 Hz, 22-channel 10-20 montage in a fixed channel order, up to 3 hours long. Optional clinical context (reason for study, clinical history). Output: free-text report sections (EEG description/details, impression/interpretation, background activity, epileptiform abnormalities, events/seizures). |
| Training | Supervised fine-tuning with a next-token prediction objective. Only the alignment module is updated and the EEG encoder and LLM stay frozen. Hyperparameters and compute are in Appendix C . No fairness constraints were applied during training. |
Medical Report Generation
Medical report generation turns medical images, clinical records, or conversations into structured, clinically useful reports.
See all 90 papers