FedPref: Federated Preference Learning for Structured Radiology Report Extraction

Flint Xiaofeng Fan; Cheston Tan; Yew-Soon Ong; Roger Wattenhofer

cs.AIAug 17, 2026

FedPref: Federated Preference Learning for Structured Radiology Report Extraction

Authors: Flint Xiaofeng Fan, Cheston Tan, Yew-Soon Ong, Roger Wattenhofer

Organizations: ETH Zurich, Zurich, Switzerland · Agency for Science, Technology and Research (A*STAR), Singapore, Singapore

Abstract

Radiology reports describe findings and locations in free text, but downstream search and analysis require these relations in a fixed schema. Learning this extraction requires labels that are unevenly distributed across institutions: smaller hospitals have less local evidence, and pooling data may be infeasible. We introduce FedPref: frozen public language models propose alternative JSON extractions, local annotations rank them, and sites collaboratively train compact Qwen3-8B adapters while sharing only model updates. A heterogeneous teacher pool provides cross-model contrast when repeated single-model samples collapse. On development data from six simulated hospitals with unequal data volume and disease prevalence, FedPref improves client-mean F1 by 2.49 points and worst-site F1 by 9.10 points compared with training each site in isolation, with the largest gains at the sites holding the least data. Central training on the pooled preference-pair union is 2.66 points higher on client-mean F1. On a locked, 400-report manually validated gold test set, FedPref reaches 68.68 F1 and pooled training 71.67, preserving that same ordering. FedPref thus lets institutions with unequal, unpooled data benefit from collaboration without ever sharing reports or annotations.

Figures & tables

Figure 1: FedPref intuition: pooled versus federated preference learning. At each site, frozen teachers propose alternative structured extractions and the local structured annotation ranks eligible candidates into chosen

c^{+}

and rejected

c^{-}

. (a) Central pools the resulting preference pairs across sites and trains one shared adapter. (b) FedPref keeps reports, annotations, candidates, and preference pairs at their sites; each site performs local preference training, and only LoRA states are exchanged for pair-count-weighted FedAvg to form the global adapter. Local preference training denotes the chosen-response SFT initialization followed by fixed-reference DPO, detailed in Sec. 3 .

System	Global $F_{1}^{\mathrm{loc+dis}}$	Mean $F_{1}^{\mathrm{loc+dis}}$	Worst $F_{1}^{\mathrm{loc+dis}}$	Pref. acc.
Central	$71.77\pm 0.67$	$71.77\pm 0.65$	$70.03\pm 1.15$	$94.95\pm 0.27$
FedPref	$69.04\pm 0.22$	$69.11\pm 0.17$	$68.12\pm 0.22$	$91.49\pm 0.27$
Local	$67.32\pm 0.28$	$66.62\pm 0.32$	$59.02\pm 1.45$	$91.67\pm 0.85$

Table 1: Three-seed development performance at selected round 3. Values are mean

\pm

sample standard deviation; paired effects are reported in text with patient-cluster confidence intervals.

Panel of Figure 2 — Figure 2: Performance across communication rounds under matched preference-pair exposure. Error bars show sample standard deviation over three seeds. FedPref overtakes home-routed Local on client mean at round 2 and retains a nine-point worst-site advantage at the selected third round.

Site	S0	S1	S2	S3	S4	S5
Support	1,344/717	1,056/591	864/556	672/486	480/297	384/314
$\Delta F_{1}^{\mathrm{loc+dis}}$ (pt)	$-1.87$	$+0.87$	$-1.72$	$+3.73$	$+9.46$	$+4.47$

Table 2: Per-site support (training reports / retained pairs) and selected-round FedPref–Local effect.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Diseases	Locations
lung opacity; pleural effusion; atelectasis; enlarged cardiac silhouette; pulmonary edema/hazy opacity; pneumothorax; consolidation; fluid overload/heart failure; pneumonia	right lung; right apical zone; right upper lung zone; right mid lung zone; right lower lung zone; right hilar structures; right costophrenic angle; left lung; left apical zone; left upper lung zone; left mid lung zone; left lower lung zone; left hilar structures; left costophrenic angle; mediastinum; upper mediastinum; cardiac silhouette; trachea

Appendix

Table 3: Frozen output vocabularies in canonical order.

Location	Training ( $n=4{,}800$ )	Dev. ( $n=600$ )	Diagnostic ( $n=100$ )	Test ( $n=400$ )
Right lung	2,162	289	53	203
Right apical zone	173	26	3	15
Right upper lung zone	163	26	3	19
Right mid lung zone	416	57	8	29
Right lower lung zone	1,149	148	35	108
Right hilar structures	875	120	23	102

Appendix

Table 4: Report-level positive support by location. Training and development use automatically derived labels; diagnostic and test use gold-standard labels.

	Training ( $4{,}800$ outputs per teacher)					Development ( $600$ per teacher)
Teacher	Strict	Eligible	Discarded	Chosen	Rejected	Strict	Eligible	Discarded
Llama-3.1-8B	3,650	4,771	29	815	1,219	440	598	2
Mistral-7B	3,375	4,672	128	606	1,350	414	583	17
Qwen2.5-14B	4,715	4,799	1	1,061	244	593	600	0
Qwen3-14B	4,697	4,800	0	479	148	590	600	0

Appendix

Table 5: Teacher-output projection audit. “Strict” denotes a valid raw JSON object before projection; “eligible” denotes successful canonical projection.

Cohort	$N$	Observed $D$	Null 95th	Ratio	$p$
Training reports	4,800	9.53	1.49	6.41	0.0005
Retained training pairs	2,961	8.83	1.89	4.67	0.0005
Development reports	600	10.14	4.14	2.45	0.0005
Development retained pairs	376	9.53	5.31	1.80	0.0005

Appendix

Table 6: Realized case-mix divergence against capacity-matched IID allocations. Values of

D

and the null 95th percentile are percentage points.

Label	S0	S1	S2	S3	S4	S5	Global
Normal	34.3	43.1	23.6	7.2	24.2	6.7	25.7
Atelectasis	24.5	35.5	41.2	48.8	26.6	43.0	36.0
Consolidation	4.5	4.1	8.5	10.7	11.1	6.4	7.0
Enlarged cardiac silhouette	13.0	18.3	23.0	35.8	41.4	62.7	27.8
Fluid overload/heart failure	0.8	2.7	2.5	3.1	3.0	5.4	2.6
Lung opacity	61.5	50.8	73.4	89.5	62.6	72.3	67.4

Appendix

Table 7: Disease prevalence (%) by site among retained training pairs. A report may contain multiple diseases, so columns need not sum to 100.

Round	System	Client-mean $F_{1}^{\mathrm{loc+dis}}$	Worst-site $F_{1}^{\mathrm{loc+dis}}$	Preference accuracy
0	Central	$57.85\pm 0.14$	$55.72\pm 0.11$	$86.70\pm 0.27$
	FedPref	$54.93\pm 0.47$	$53.67\pm 0.18$	$82.00\pm 0.41$
	Local	$55.73\pm 0.37$	$53.32\pm 1.30$	$83.95\pm 0.15$
1	Central	$69.34\pm 0.07$	$67.45\pm 0.37$	$90.51\pm 0.15$
	FedPref	$61.24\pm 0.30$	$59.18\pm 0.47$	$86.44\pm 0.27$
	Local	$62.29\pm 0.13$	$53.80\pm 0.06$	$87.85\pm 0.31$

Appendix

Table 8: Development trajectories. Values are mean

\pm

sample standard deviation over three seeds; all metrics are percentages.

Clinical Information Extraction

Clinical Information Extraction turns clinical notes, reports, and patient narratives into structured, searchable facts and relations.

See all 106 papers

FedDRAW: Federated Dual Reputation Annealing Weighting for Heterogeneous Multi-Institutional Chest Radiograph Classification
Sep 4, 2026Maryam Moradpour, Anne-Christin HauschildFederated Learning Aggregation Heterogeneous Federated Learning
Institute for Predictive Deep Learning in Medicine and Healthcare, Justus-Liebig University, Gießen, Germany · Institute for Medical Informatics, University Medical Center Göttingen, Germany
ChestPheNoT: Deployable, Auditable Label-Status-Evidence Extraction from Radiology Reports
Sep 9, 2026Kai Yu, Chenyu Zhu, Zaifu Zhan +5Clinical Information Extraction Medical LLMs
University of California Davis Davis, USA
Personalized Federated Direct Preference Optimization via LoRA Adapters
Oct 9, 2026Zhanhong Jiang, Soumik SarkarPersonalized Language Model Alignment Direct Preference Optimization
Translational AI Center, Iowa State University, Ames, IA 50011 USA · Department of Mechanical Engineering, Iowa State University, Ames, IA 50011 USA

Sep 4, 2026J/K move · Enter open · S save

FedDRAW: Federated Dual Reputation Annealing Weighting for Heterogeneous Multi-Institutional Chest Radiograph Classification

Maryam Moradpour, Anne-Christin Hauschild

Institute for Predictive Deep Learning in Medicine and Healthcare, Justus-Liebig University, Gießen, Germany · Institute for Medical Informatics, University Medical Center Göttingen, Germany

Federated Learning Aggregation Heterogeneous Federated Learning Chest X-Ray Classification

Open paper AskPDF arXiv

Artificial intelligence models are promising for medical diagnosis, but they require large numbers of unbiased data, which in medicine are distributed across hospitals and cannot be centralized to protect patient privacy. Federated Learning (FL) addresses this, since hospitals train one shared diagnostic model while patient data remain local. Training proceeds in communication rounds, in which each hospital trains the shared model locally and returns it to the server for merging by weighted average. This aggregation weight determines whose institutional knowledge shapes the result. Federated averaging (FedAvg) sets it in proportion to local sample count, so a small but informative hospital is permanently assigned a small influence, andl argest clients could dominate the global model even when they are less informative. We propose Federated Dual Reputation Annealing Weighting (FedDRAW), a server-side aggregation method that combines a data-size prior with the cosine similarity between client and global parameters under two coupled annealing schedules. An inner schedule shifts client reputation from the size prior towards similarity. An outer, deferred annealing schedule on the softmax inverse temperature keeps the weighting selective in the early and middle rounds and relaxes it to uniformity at convergence. We evaluate FedDRAW on 12 simulated client-partition scenarios of two chest radiograph datasets (CheXpert and ChestMNIST), against seven federated baselines under identical local training settings. FedDRAW achieved the highest average rank among all eight methods under both AUC and the geometric mean (GM) of sensitivity and specificity, which a Friedman test with Nemenyi post-hoc analysis confirmed to be a statistically significant difference between the methods. Scheduling two signals, rather than fixing the weights by sample count alone, could enable less biased diagnostic models.

Abstract

Clinical Information Extraction

Explore similar work

FedDRAW: Federated Dual Reputation Annealing Weighting for Heterogeneous Multi-Institutional Chest Radiograph Classification

ChestPheNoT: Deployable, Auditable Label-Status-Evidence Extraction from Radiology Reports

Personalized Federated Direct Preference Optimization via LoRA Adapters