Clinical free-text notes contain vital patient information. They are structured into labelled sections; recognizing these sections has been shown to support clinical decision-making and downstream NLP tasks. In this paper, we advance clinical section segmentation through three key contributions. First, we curate a new de-identified, section-labeled obstetrics notes dataset, to supplement the medical domains covered in public corpora such as MIMIC-III, on which most existing segmentation approaches are trained. Second, we systematically evaluate transformer-based supervised models for section segmentation on a curated subset of MIMIC-III (in-domain), and on the new obstetrics dataset (out-of-domain). Third, we conduct the first head-to-head comparison of supervised models for medical section segmentation with zero-shot large language models. Our results show that while supervised models perform strongly in-domain, their performance drops substantially out-of-domain. In contrast, zero-shot models demonstrate robust out-of-domain adaptability once hallucinated section headers are corrected. These findings underscore the importance of developing domain-specific clinical resources and highlight zero-shot segmentation as a promising direction for applying healthcare NLP beyond well-studied corpora, as long as hallucinations are appropriately managed.
Figures & tables
a/p:
"This is 28 y/o G6P4013 presents for repeat cesarean delivery and bilateral tuba ligation. admit to labor and delivery cbc, type and screen iv fluids consented for repeat cesarean delivery, possible hysterectomy, possible blood transfusion, and bilateral tubal ligation. Hgb 11.1 patient alst ate at 2100on <DATE> discussed with the team <NAME> MD Obstetrics and Gynecology Resident Attending note I saw patient prior to scheduled repeat c/section this morning, reviewed cerner and counseled patient r/b/a. Patient with history of prior c/s x 1 here for RCD and BTL. Labs this morning includes Hgb 11.1, plat 207, bl gp O+ve. Singlton cephalic, posterior placenta. Writtent consent obtained, will proceed when OR is ready. Patient voiced understanding of the plan. <NAME> , MD"
Figure 1: Assessment and Plan section from a sample obstetrics note (includes typographical errors and masked identifier tokens).
Section Header
MedSecId
ONC Dataset
<none>
24-hour-events
addendum
allergies
assessment-and-plan
chief-complaint
Table 1: Section Headers in MedSecId vs. ONC( = Present, = Absent).
Section Header
Total Spans
Overall %
social-history
120
7.92
current-medications
114
7.52
allergies
114
7.52
physical-examination
102
6.73
family-history
100
6.60
history-of-present-illness
96
6.33
Table 2: Frequency distribution of section headers in the ONC dataset (excluding <none>).
Model
HL
H%
HS
MedSecId
Mistral-7B-Instruct-v0.3
2,560
22.21%
433
Llama 3.1-8B Instruct
452
3.92%
89
Qwen-2.5-32B-Instruct
497
4.31%
54
Llama 3.3-70B Instruct
404
3.50%
57
ONC
Table 3: Hallucination Analysis on MedSecId and ONC. HL = number of hallucinated lines; H% = hallucination rate; HS = number of hallucinated sections types not in the original label set.
Table 4: Top 5 most frequently hallucinated section headers generated by each model. (Llama 3.3-70B-Instruct produced only four hallucinated headers in total.)
Table 7: Comparison of per-note macro and weighted F1 scores ± 95% bootstrap confidence intervals across 100 obstetric notes for the top two zero-shot LLMs and the best-performing supervised model (BioMedBERT+CRF).
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 3: Proportional distribution of section labeling errors for Llama 3.3-70B-Instruct (Other = 0%, 0 instances).
Text Span
Error Type
Explanation
Review / Management Results review: GBS (group B streptococcal): negative Hepatitis B : negative Syphilis screen: NR x2 Rubella: Immune HIV: NR STDs: Neg Blood type: O+ 1-hr GTT: 94 Genetic Screening Tests (First/Sequential/QUAD) : normal.
Label Confusion
The predicted label "assessment-and-plan" is a valid label, but it clearly differs from the gold label "labs-imaging", as the text span primarily discusses laboratory results, which aligns more closely with "labs-imaging".
In addition patient was incidentally found to have 4cm arrachnoid cyst of her left temporal fossa.
Valid Local Interpretation
The predicted label physical-examination makes sense given the text span, which describes a medical finding, even though the gold label is assessment-and-plan . The sentence could be part of a physical examination section, but in the context of the entire clinical note, it might be more appropriately classified under assessment and plan due to the incidental finding mentioned.
Appendix
Table 8: Representative examples of Llama 3.3-70B-Instruct prediction mismatches categorized by LLM, with associated reasoning.