MedTri: A Platform for Structured Medical Report Normalization to Enhance Vision-Language Pretraining

Yuetan Chu; Xinhua Ma; Xinran Jin; Gongning Luo; Xin Gao

cs.CVFeb 25, 2026

MedTri: A Platform for Structured Medical Report Normalization to Enhance Vision-Language Pretraining

Authors: Yuetan Chu, Xinhua Ma, Xinran Jin, Gongning Luo, Xin Gao

Organizations: King Abdullah University of Science and Technology (KAUST), Thuwal, Saudi Arabia · Faculty of Computing, Harbin, China.

Abstract

Medical vision-language pretraining increasingly relies on medical reports as large-scale supervisory signals; however, raw reports often exhibit substantial stylistic heterogeneity, variable length, and a considerable amount of image-irrelevant content. Although text normalization is frequently adopted as a preprocessing step in prior work, its design principles and empirical impact on vision-language pretraining remain insufficiently and systematically examined. In this study, we present MedTri, a deployable normalization framework for medical vision-language pretraining that converts free-text reports into a unified [Anatomical Entity: Radiologic Description + Diagnosis Category] triplet. This structured, anatomy-grounded normalization preserves essential morphological and spatial information while removing stylistic noise and image-irrelevant content, providing consistent and image-grounded textual supervision at scale. Across multiple datasets spanning both X-ray and computed tomography (CT) modalities, we demonstrate that structured, anatomy-grounded text normalization is an important factor in medical vision-language pretraining quality, yielding consistent improvements over raw reports and existing normalization baselines. In addition, we illustrate how this normalization can easily support modular text-level augmentation strategies, including knowledge enrichment and anatomy-grounded counterfactual supervision, which provide complementary gains in robustness and generalization without altering the core normalization process. Together, our results position structured text normalization as a critical and generalizable preprocessing component for medical vision-language learning, while MedTri provides this normalization platform. Code and data will be released at https://github.com/Arturia-Pendragon-Iris/MedTri.

Figures & tables

Figure 1: Illustration of normalization differences across raw reports, RadGraph, and MedTri. The raw clinical report contains irrelevant content and mixed structures. RadGraph extracts only diagnostic entities with limited imaging descriptions. MedTri produces anatomically anchored, image-grounded triplets that preserve morphological and spatial detail while removing stylistic and clinically irrelevant text.

Dataset	# Reports	Image Modality	Body Region
MIMIC-CXR [ 13 ]	65,000	X-ray	Chest
CT-RATE [ 10 ]	20,000	CT	Chest
CT-INSPECTION [ 11 ]	10,000	CT	Chest
AbdomenAtlas [ 17 ]	5,000	CT	Abdomen
Parrot [ 16 ]	2,658	CT, MRI, X-ray, etc	Head, abdomen, chest, etc

Table 1: Overview of the report dataset used to develop and evaluate the MedTri normalization platform.

Input:	IMPRESSION: 1. No evidence of pulmonary embolism. No acute intrathoracic pathology. 2. Mild bronchial thickening, nonspecific, but may reflect infectious or inflammatory process. 3. Main pulmonary artery is dilated measuring 39 mm, possibly reflecting background pulmonary hypertension.
Prompt:	You will receive one paragraph containing radiology findings (image-based description) and a final diagnosis. Your ONLY task is to convert this information into a structured radiology summary using the specified format. Each line must follow this exact format: [Anatomical Term]: [Radiologic Description + Diagnosis Category]. List only the anatomical regions mentioned in the findings or diagnosis. Maintain the anatomical hierarchy (e.g., lung before airway, heart before aorta) if multiple structures are present. For each anatomical term, include radiologic findings (location, size, attenuation, density, texture, etc.) and diagnosis category if explicitly stated. Use objective imaging language only. Use direct descriptive phrasing. Keep each anatomical line concise and single-sentence, using semicolons to separate multiple findings.
Output:	Lung: No evidence of pulmonary embolism; no acute intrathoracic pathology.
	Bronchi: Mild bronchial thickening; nonspecific; may reflect infectious or inflammatory process.
	Main pulmonary artery: Dilated; measures 39 mm; possibly reflecting background pulmonary hypertension.

Table 2: Example of the standardized ChatGPT-5.1 prompt used to generate structured triplets from free-text reports.

Figure 2: Illustration of MedTri’s optional augmentation modules. MedTri-K enriches each triplet with clinically validated radiologic signatures to improve visual interpretability. MedTri-C generates anatomically inconsistent counterfactuals through controlled perturbations to strengthen fine-grained visual–semantic discrimination.

Metric	ChatGPT-5.1	Qwen2.5	MedTri
Anatomical correctness (1-5)	4.888	3.363	4.838
Image groundedness (1-5)	4.775	2.938	4.763
VRAM usage (GB)	–	15.04	1.50
Time/report (s)	3.25±2.14	1.21±0.67	0.45±0.18
BERT score	–	0.734±0.073	0.906±0.072
BLEU	–	0.097	0.541

Table 3: Comparison of computational efficiency, expert evaluation, and normalization accuracy across different systems.

SwinT	MIMIC						NIH						Pneumonia
	1%		10%		100%		1%		10%		100%		1%		10%		100%
	ACC	F1	ACC	F1	ACC	F1	ACC	F1	ACC	F1	ACC	F1	ACC	F1	ACC	F1	ACC	F1
Raw report	0.612	0.218	0.701	0.263	0.781	0.302	0.655	0.221	0.722	0.266	0.762	0.293	0.812	0.673	0.833	0.699	0.848	0.722
RadGraph	0.584	0.205	0.673	0.251	0.748	0.284	0.628	0.207	0.703	0.249	0.747	0.265	0.804	0.665	0.822	0.690	0.840	0.715
MedTri	0.628	0.229	0.716	0.279	0.792	0.314	0.672	0.233	0.739	0.281	0.779	0.304	0.817	0.676	0.836	0.701	0.850	0.723
MedTri-K	0.645	0.241	0.732	0.286	0.784	0.305	0.689	0.242	0.746	0.284	0.771	0.292	0.823	0.681	0.840	0.703	0.851	0.722

Table 4: Downstream classification performance of Swin Transformer (SwinT) and ViT on different X-ray datasets under different text preprocessing strategies. Best performance is marked in bold.

	SwinT						ViT
	1%		10%		100%		1%		10%		100%
	ACC	F1	ACC	F1	ACC	F1	ACC	F1	ACC	F1	ACC	F1
Raw report	0.719	0.493	0.741	0.518	0.775	0.525	0.696	0.473	0.733	0.511	0.745	0.518
RadGraph	0.714	0.480	0.735	0.513	0.744	0.523	0.694	0.471	0.722	0.508	0.716	0.509
MedTri	0.723	0.499	0.753	0.520	0.785	0.538	0.704	0.488	0.745	0.518	0.775	0.537
MedTri-K	0.732	0.510	0.757	0.519	0.789	0.536	0.718	0.498	0.743	0.519	0.775	0.530

Table 5: Downstream classification performance of Swin Transformer (SwinT) and ViT on CT-RATE datasets under different text preprocessing strategies. Best performance is marked in bold.

Medical VLMs

Medical vision-language models connect clinical images with text for diagnosis, reporting, question answering, and image grounding.

See all 475 papers

Clinical Trajectory Alignment for Medical Vision-Language Pre-training
Sep 28, 2026Huimin Yan, Xian Yang, Zhi Wang +1Multimodal Pretraining Longitudinal Medical Image Analysis
Institute of Intelligent Information Processing, Shanxi University, Taiyuan, China · Alliance Manchester Business School, The University of Manchester, Manchester, UK
VIVID-Med: LLM-Supervised Structured Pretraining for Deployable Medical ViTs
Mar 10, 2026Xiyao Wang, Xiaoyu Tan, Yang Dai +3Medical Image Representation Learning Vision-Language Pretraining
Shanghai University of Engineering Science, China · National University of Singapore, Singapore · Case Western Reserve University, United States
3D Modality-Aware Pre-training for Vision-Language Model in MRI Multi-organ Abnormality Detection
Feb 27, 2026Haowen Zhu, Ning Yin, Xiaogen ZhouMedical VLMs Vision-Language Pretraining
School of Computer Science and Engineering, Southeast University, China · Department of Medical Imaging, Suzhou Traditional Chinese Medicine Hospital, China · School of Electronic, Electrical Engineering and Physics, Fujian University of Technology

Sep 28, 2026J/K move · Enter open · S save

Clinical Trajectory Alignment for Medical Vision-Language Pre-training

Huimin Yan, Xian Yang, Zhi Wang, Liang Bai

Institute of Intelligent Information Processing, Shanxi University, Taiyuan, China · Alliance Manchester Business School, The University of Manchester, Manchester, UK

Multimodal Pretraining Longitudinal Medical Image Analysis Medical VLMs Vision-Language Pretraining

Open paper AskPDF arXiv

Medical vision-language pre-training largely follows a visit-level image-report matching paradigm, aligning paired images and reports at individual visits. While effective for static cross-modal correspondence, this paradigm provides limited supervision for longitudinal clinical change, such as whether abnormalities improve, remain stable, or worsen over time. Learning such change is challenging because temporal semantics are implicit in free-text reports, and different abnormalities within the same patient may evolve asynchronously or even in opposite directions. We propose MedCTA, which reframes medical vision-language pre-training from visit-level cross-modal matching to learning clinical change. Rather than compressing a patient history into a single temporal representation, MedCTA models clinical change at two complementary scopes. At the abnormality scope, clinically grounded queries construct abnormality-conditioned visual and textual trajectories to capture heterogeneous abnormality evolution. At the patient-course scope, global image and report sequences are modeled to capture overall clinical progression beyond any individual abnormality. Structured trend supervision is extracted from longitudinal reports by an offline LLM parser, removing the need for manual temporal annotations. Combined with static image-report alignment, MedCTA learns representations that preserve visit-level cross-modal correspondence while encoding longitudinal change semantics. Experiments on temporal image classification, image-text retrieval, and zero-shot classification show consistent gains over strong medical vision-language baselines.

Abstract

Medical VLMs

Explore similar work

Clinical Trajectory Alignment for Medical Vision-Language Pre-training

VIVID-Med: LLM-Supervised Structured Pretraining for Deployable Medical ViTs

3D Modality-Aware Pre-training for Vision-Language Model in MRI Multi-organ Abnormality Detection