HyperCT: Low-Rank Hypernet for Unified Chest CT Analysis

Fengbei Liu; Sunwoo Kwak; Hao Phung; Nusrat Binta Nizam; Ilan Richter; Nir Uriel; Hadar Averbuch-Elor; Daborah Estrin

eess.IVApr 3, 2026

HyperCT: Low-Rank Hypernet for Unified Chest CT Analysis

Authors: Fengbei Liu, Sunwoo Kwak, Hao Phung, Nusrat Binta Nizam, Ilan Richter, Nir Uriel, Hadar Averbuch-Elor, Daborah Estrin, +1 more

Organizations: Cornell Tech and Cornell University · Columbia University Irving Medical Center · Weill Cornell Medicine

Abstract

Non-contrast chest CTs offer a rich opportunity for both conventional pulmonary and opportunistic extra-pulmonary screening. While Multi-Task Learning (MTL) can unify these diverse tasks, standard hard-parameter sharing approaches are often suboptimal for modeling distinct pathologies. We propose HyperCT, a framework that dynamically adapts a Vision Transformer backbone via a Hypernetwork. To ensure computational efficiency, we integrate Low-Rank Adaptation (LoRA), allowing the model to regress task-specific low-rank weight updates rather than full parameters. Validated on a large-scale dataset of radiological and cardiological tasks, \method{} outperforms various strong baselines, offering a unified, parameter-efficient solution for holistic patient assessment. Our code is available at https://github.com/lfb-1/HyperCT.

Figures & tables

Figure 1: Overview of HyperCT. Given a set of learnable task embeddings, e.g., $\{\mathbf{e}^{1},\mathbf{e}^{2},\mathbf{e}^{3}\}$ , a hypernet $h$ produces task-specific weight adjustments $\Delta\mathbf{W}^{1},\Delta\mathbf{W}^{2},\Delta\mathbf{W}^{3}$ , which modulate the weights of the base model. The base model, produces task-specific predictions $\{\hat{\mathbf{y}}^{1},\hat{\mathbf{y}}^{2},\hat{\mathbf{y}}^{3}\}$ . These outputs are compared with ground-truth task labels $\{{\mathbf{y}}^{1},{\mathbf{y}}^{2},{\mathbf{y}}^{3}\}$ via Binary Cross-Entropy Loss.

		CU Test								WCM Test
	Task	STL	EW	UW	RLW	DWA	GLS	MGDA	HyperCT	STL	EW	UW	RLW	DWA	GLS	MGDA	HyperCT
Overall Average		79.3	74.3	74.4	74.0	74.3	71.6	76.0	78.1	77.6	72.9	72.9	72.8	72.9	69.5	74.9	76.5
Conventional	Med. Mat.	88.2	83.6	83.2	81.8	83.6	76.4	85.9	85.8	89.3	85.5	85.2	84.4	85.5	78.9	87.6	87.3
	Art. Calc.	82.0	79.5	79.4	79.0	79.5	76.8	80.5	81.9	75.3	73.9	73.9	73.2	73.9	71.2	74.4	76.0
	Cardiomeg.	86.9	84.1	84.1	83.4	84.1	81.5	85.3	87.0	87.1	85.0	85.0	84.3	85.0	82.8	86.0	87.0
	Peri. Eff.	70.2	68.0	68.0	67.5	68.0	63.9	69.4	68.5	73.3	67.4	67.4	67.3	67.4	62.6	69.9	71.1

Table 1: Comprehensive comparison of AUC scores (%) on retrospective study. Best results among MTL methods are bolded , second best are underlined .

		CU Prospective								WCM Prospective
	Task	STL	EW	UW	RLW	DWA	GLS	MGDA	HyperCT	STL	EW	UW	RLW	DWA	GLS	MGDA	HyperCT
Overall Average		75.4	69.7	70.9	71.6	70.9	66.8	74.4	77.8	75.2	73.0	72.9	73.2	73.0	68.1	76.1	78.6
Opportunistic	Red. RV	77.0	74.8	73.8	73.6	73.9	68.7	75.1	77.2	80.6	79.7	79.5	78.5	79.7	76.2	79.9	79.2
	Red. LV	74.1	70.5	71.1	70.8	71.1	65.4	74.1	76.6	72.5	70.0	69.7	69.5	70.0	66.0	73.2	77.2
	Pulm. HTN	71.7	70.1	70.9	71.8	71.1	69.9	72.7	73.6	71.0	72.9	72.5	73.2	73.0	68.7	75.7	75.5
	Atrial Enlg.	80.2	74.1	72.0	72.3	72.0	68.6	74.3	80.0	83.5	77.1	77.1	77.7	77.1	69.8	80.2	80.8

Table 2: Comprehensive comparison of AUC scores (%) on prospective study. Best results among MTL methods are bolded , and second-best results are underlined .

Panel of Figure 2 — Figure 2: Decision Curve Analysis on CU prospective cohort for all 7 opportunistic cardiac tasks. HyperCT (blue) shows positive net benefit above “treat all” (orange) and “treat none” (gray) baselines across clinically relevant thresholds (5-80%).

Panel of Figure 3 — Figure 3: Decision Curve Analysis on WCM prospective cohort (external validation). HyperCT demonstrates consistent clinical utility across institutions, with positive net benefit maintained for all 7 opportunistic tasks.

	Method	Params	Med. Mat.	Art. Calc.	Cardiomeg.	Peri. Eff.	Cor. Art. Calc.	Hiatal Hernia	Lymphadenop.	Emphysema	Atelectasis	Lung Nodule	Lung Opacity	Pulm. Fibrosis	Pleural Eff.	Mosaic Attn.	Peribronchial	Consolidation	Bronchiectasis	Septal Thick.	Avg
CU	Attn Only	35.9M	85.3	81.8	86.7	68.8	88.3	67.1	67.4	77.7	77.0	70.4	77.4	84.2	95.3	70.3	65.6	85.1	80.1	75.4	77.9
	MLP only	49.3M	85.0	81.6	87.1	67.0	87.9	66.7	66.9	78.0	76.9	69.8	77.7	85.0	95.2	70.0	65.5	85.4	80.6	75.1	77.8
	HyperCT	85.0M	85.8	81.9	87.0	68.5	88.2	67.6	67.1	79.1	77.7	70.0	78.2	85.2	95.6	71.6	66.3	86.3	80.6	75.8	78.5
WCM	Attn Only	35.9M	87.1	76.0	86.4	71.2	82.1	67.0	69.4	73.5	78.4	64.8	76.9	80.7	95.6	65.7	65.3	80.7	76.5	79.0	76.5
	MLP only	49.3M	86.8	75.7	87.2	70.2	82.0	65.4	69.5	73.9	77.2	64.4	77.2	81.3	95.6	66.3	65.1	80.4	76.6	78.8	76.3
	HyperCT	85.0M	87.3	76.0	87.0	71.1	82.3	68.8	69.4	74.9	78.2	64.4	78.0	80.6	95.9	68.1	66.5	82.0	76.8	79.3	77.0

Table 3: Comparison of AUC scores (%) for LoRA module variants on retrospective study. Best results are bolded , and second best are underlined . Full table is in Appendix Sec. A

	Rank	Med. Mat.	Art. Calc.	Cardiomeg.	Peri. Eff.	Cor. Art. Calc.	Hiatal Hernia	Lymphadenop.	Emphysema	Atelectasis	Lung Nodule	Lung Opacity	Pulm. Fibrosis	Pleural Eff.	Mosaic Attn.	Peribronchial	Consolidation	Bronchiectasis	Septal Thick.	Avg
CU	$r=1$	76.0	80.2	85.4	66.1	84.8	65.8	65.8	73.9	74.5	69.8	75.7	82.3	94.5	66.6	64.0	81.1	78.9	74.2	75.2
	$r=2$	83.6	80.7	85.6	66.8	86.2	66.6	67.5	75.5	76.6	69.8	77.1	84.3	94.9	69.6	65.0	83.3	79.9	75.3	76.7
	$r=4$	84.8	81.1	86.7	68.5	87.1	65.4	67.4	76.8	76.6	70.2	77.4	84.1	95.2	70.5	65.6	84.9	80.3	75.1	77.4
	$r=8$	85.6	81.5	86.9	68.4	87.7	66.0	67.3	77.0	76.9	70.4	77.6	84.5	95.3	70.3	65.5	85.2	80.3	75.4	77.7
	$r=16$	85.8	81.9	87.0	68.5	88.2	67.6	67.1	79.1	77.7	70.0	78.2	85.2	95.6	71.6	66.3	86.3	80.6	75.8	78.5
WCM	$r=1$	76.6	74.5	84.8	69.1	79.1	62.8	68.2	70.1	75.7	63.9	74.6	80.3	94.6	62.2	63.5	76.3	75.0	77.8	73.5

Table 4: Ablation of LoRA Rank (

r

) dimensions on retrospective study (Conversional labels). Best results are bolded , and second-best results are underlined . Full table is in Appendix. Sec. D

	Method	Med. Mat.	Art. Calc.	Cardiomeg.	Peri. Eff.	Cor. Art. Calc.	Hiatal Hernia	Lymphadenop.	Emphysema	Atelectasis	Lung Nodule	Lung Opacity	Pulm. Fibrosis	Pleural Eff.	Mosaic Attn.	Peribronchial	Consolidation	Bronchiectasis	Septal Thick.	Avg
CU	CTViT	66.7	68.3	79.4	67.0	71.7	65.8	65.5	71.3	71.5	69.0	71.8	76.8	91.0	67.5	61.6	77.9	74.4	68.6	71.1
CU	DINOv3	85.8	81.9	87.0	68.5	88.2	67.6	67.1	79.1	77.7	70.0	78.2	85.2	95.6	71.6	66.3	86.3	80.6	75.8	78.5
WCM	CTViT	65.8	64.3	79.1	69.1	67.9	65.3	67.5	64.4	72.3	62.8	69.9	69.9	89.9	63.4	59.4	72.0	67.1	70.3	68.3
WCM	DINOv3	87.3	76.0	87.0	71.1	82.3	68.8	69.4	74.9	78.2	64.4	78.0	80.6	95.9	68.1	66.5	82.0	76.8	79.3	77.0

Table 5: Comparison of 3D (CTViT) and 2D (DINOv3) backbones on retrospective study (Conventional labels). Best results are bolded . Full table is in Appendix. C

Figure 4: Principle Component Analysis (PCA) of task-specific LoRA. Blue is opportunistic labels and orange is conventional labels. Number is the index of labels.

Figure 5: Saliency maps generated using Grad-CAM for different tasks. First row is opportunistic screening tasks and second row is part of the conventional screening tasks.

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

		CU Test			WCM Test
	Task	Attn Only	MLP only	HyperCT	Attn Only	MLP only	HyperCT
Overall Average		77.8	77.7	78.1	76.1	76.1	76.5
Conventional	Medical Material	85.3	85.0	85.8	87.1	86.8	87.3
	Arterial Wall Calcification	81.8	81.6	81.9	76.0	75.7	76.0
	Cardiomegaly	86.7	87.1	87.0	86.4	87.2	87.0
	Pericardial Effusion	68.8	67.0	68.5	71.2	70.2	71.1

Appendix

Table 6: Ablation Study: Comparison of AUC scores (%) for LoRA Module variants on Retrospective study. Best results are bolded.

		CU Prospective			WCM Prospective
	Task	Attn Only	MLP only	HyperCT	Attn Only	MLP only	HyperCT
Overall Average		77.7	77.5	77.8	78.9	79.6	78.6
Opportunistic	Reduced RV Systolic Function	76.6	76.6	77.2	78.7	80.5	79.2
	Reduced LV Systolic Function	76.9	76.8	76.6	79.0	80.7	77.2
	Pulmonary Hypertension	71.7	73.4	73.6	75.0	76.2	75.5
	Atrial Chamber Enlargement	80.1	79.5	80.0	81.6	82.1	80.8

Appendix

Table 7: Ablation Study: Comparison of AUC scores (%) for LoRA Module variants on Prospective Datasets. Best results are bolded.

Figure 6: Sample valid fraction heatmaps for opportunistic screening labels

Figure 7: Sample valid fraction heatmaps for conventional screening labels

	Method	Reduced RV	Reduced LV	Pulmonary Hypertension	Atrial Enlargement	Ventricular Enlargement	LA Pressure	RA Pressure	Avg
CU (Retro)	CTViT-Encoder	72.4	70.6	69.2	75.9	73.4	71.5	70.0	71.9
CU (Retro)	DINOv3	77.5	77.0	72.7	83.0	80.4	77.1	71.4	77.0
WCM (Retro)	CTViT-Encoder	72.5	66.3	70.0	72.1	66.9	71.4	70.0	69.9
WCM (Retro)	DINOv3	77.9	74.6	72.0	79.9	73.1	77.1	72.4	75.3
CU (Prosp)	CTViT-Encoder	70.8	68.6	69.7	75.6	79.2	74.8	68.1	72.4
CU (Prosp)	DINOv3	77.2	76.6	73.6	80.0	86.6	78.5	72.3	77.8

Appendix

Table 8: Comparison of 3D (CTViT) and 2D (DINOv3) backbones on Opportunistic Tasks across all datasets. Best results are bolded.

		Retrospective								Prospective
	Rank	Red. RV Sys.	Red. LV Sys.	Pulm. HTN	Atrial Enl.	Vent. Enl.	LA Pressure	RA Pressure	Avg	Red. RV Sys.	Red. LV Sys.	Pulm. HTN	Atrial Enl.	Vent. Enl.	LA Pressure	RA Pressure	Avg
CU	$r=1$	75.4	75.4	72.0	81.3	67.9	75.6	71.8	74.2	75.2	73.8	70.9	77.4	71.2	76.1	68.6	73.3
	$r=2$	76.2	75.6	71.9	81.1	77.2	75.2	72.4	75.6	75.4	74.7	71.9	78.0	79.0	76.1	73.6	75.5
	$r=4$	76.7	76.8	72.4	82.2	80.7	76.6	72.9	76.9	76.1	76.6	71.8	79.2	85.6	77.3	73.3	77.1
	$r=8$	76.6	77.1	72.4	82.6	80.3	76.9	73.7	77.1	76.2	76.3	72.1	79.0	85.8	77.5	73.6	77.2
WCM	$r=1$	76.0	72.4	70.9	78.5	64.6	76.0	71.0	72.8	78.4	77.6	73.2	79.9	75.0	77.8	73.2	76.4

Appendix

Table 9: Ablation of LoRA Rank (

r

) dimensions on Opportunistic Tasks (Retrospective vs. Prospective Test Sets). Best results are bolded per institution.

Sampling Strategy	CU Retro	WCM Retro	CU Prosp	WCM Prosp
Uniform Random (ours)	78.1	76.5	77.8	78.6
Inverse-Prevalence Weighted	77.9	76.5	77.5	78.3

Appendix

Table 10: Ablation Study: Comparison of task sampling strategies. Average AUC (%) across all 25 tasks. Results show minimal difference between strategies, indicating robustness to sampling choice.

Opportunistic (Cardiology)
	AUC (95% CI)		Sample Size
Task	CU Retro	WCM Retro	CU	WCM
Reduced RV Systolic Function	76.7 (75.0–78.3)	77.6 (76.2–79.1)	4,930	7,899
Reduced LV Systolic Function	77.0 (75.3–78.7)	74.6 (73.1–76.1)	5,174	8,110
Pulmonary Hypertension	72.0 (70.2–73.5)	71.5 (70.0–73.0)	3,498	5,935
Atrial Chamber Enlargement	82.7 (81.1–84.1)	80.0 (78.9–81.0)	2,748	6,372

Appendix

Table 11: Bootstrap 95% CIs for HyperCT on retrospective evaluation. AUC (%) with 95% CI over 1000 iterations.

	AUC (95% CI)		Sample Size
Task	CU	WCM	CU	WCM
Reduced RV Systolic Function	76.5 (73.2–79.6)	78.5 (72.9–83.3)	1,337	723
Reduced LV Systolic Function	76.7 (73.2–80.1)	78.1 (73.6–82.3)	1,411	817
Pulmonary Hypertension	73.5 (70.2–76.8)	76.1 (71.6–80.8)	842	449
Atrial Chamber Enlargement	80.2 (77.4–83.0)	80.8 (77.4–84.3)	921	628
Ventricular Enlargement	86.2 (82.4–90.1)	81.4 (75.9–86.4)	1,331	664

Appendix

Table 12: Bootstrap 95% confidence intervals for HyperCT on prospective evaluation. AUC (%) with 95% CI computed over 1000 bootstrap iterations. CI width scales inversely with sample size.

Component	Layer	Per Block	Total (12 blocks)
Attention	Query (Q) projection	1	12
	Key (K) projection	1	12
	Value (V) projection	1	12
	Output projection	1	12
MLP	fc1 (up-projection)	1	12
MLP	fc2 (down-projection)	1	12

Appendix

Table 13: LoRA target modules in HyperCT. All linear layers in attention and MLP blocks are adapted.

Component	Architecture	Output Dim
Mixer	Linear $\rightarrow$ SiLU $\rightarrow$ Linear $\rightarrow$ SiLU	latent
MLP blocks ( $\times$ 2)	Residual(LayerNorm $\rightarrow$ Linear $\rightarrow$ SiLU $\rightarrow$ Linear $\rightarrow$ SiLU)	latent
Output projection	LayerNorm $\rightarrow$ Linear $\rightarrow$ SiLU $\rightarrow$ Linear $\rightarrow$ SiLU	head_in_size
Per-module heads	Linear	$r\times(d_{\text{in}}+d_{\text{out}})$

Appendix

Table 14: Hypernetwork architecture in HyperCT.

Figure 8: Hierarchical clustering of task-specific LoRA weights. MDS Torgerson (1952) projection of 25 tasks colored by cluster assignment (cosine distance, complete linkage, k=4, silhouette=0.30). Tasks naturally group into clinically interpretable categories without any clinical priors.

Figure 9: Dendrogram of hierarchical clustering (complete linkage, cosine distance) showing the tree structure of task relationships. Colors indicate the four identified clusters at the optimal cut point (k=4).

Panel of Figure 10 — Figure 10: Decision Curve Analysis on CU retrospective cohort. HyperCT (blue) shows positive net benefit above “treat all” (orange) and “treat none” (gray) baselines.

Panel of Figure 11 — Figure 11: Decision Curve Analysis on WCM retrospective cohort.

Condition	Clinical Criteria for Presence
Left Atrial Filling Pressure (Elevated)	Considered present if any of the following echocardiographic findings are true: • Mitral inflow E/A ratio $\geq 2$ • OR Indexed Left Atrial Volume (LAVI) $>$ 34 mL/m 2 • OR Peak Tricuspid Regurgitation (TR) velocity $>$ 2.8 m/s
Right Atrial Filling Pressure (Elevated)	Peak Tricuspid Regurgitation (TR) velocity $>$ 3.4 m/s
Reduced Right Ventricular (RV) Systolic Function	Qualitative assessment of RV systolic function is anything other than ’normal’ (e.g., ’mildly’, ’moderately’, or ’severely’ reduced).
Reduced Left Ventricular (LV) Systolic Function	Left Ventricular Ejection Fraction (LVEF) $<$ 50%
Pulmonary Hypertension	Estimated Pulmonary Artery Systolic Pressure (PASP) $>$ 35 mmHg
Atrial Chamber Enlargement	Indexed Left Atrial Volume (LAVI) $>$ 34 mL/m 2

Appendix

Table 15: Clinical Definitions for Cardiovascular Condition Curation

Explore similar work

ARC-CT: Anatomy-Routed Contrastive Vision-Language Learning for 3D Chest CT
Aug 28, 2026Huseyin Umut Isik, Mehmet Alp Ozaydin, Sila Kurugol +1Radiology VLMs Medical Image Classification
Department of Computer Engineering, METU, Ankara, Turkey · Quantitative Intelligent Imaging Lab, Boston Children’s Hospital and Harvard Medical School, Boston, MA, USA · METU-DTX Digital Transformation and Innovation Center, Ankara, Turkey
MultiMedVision: Multi-Modal Medical Vision Framework
May 9, 2026Frank Li, Bardia Khosravi, Mohammadreza Chavoshi +5Medical Imaging Foundation Models Vision Transformer
Emory University · Yale University
Specializing Foundation Models via Mixture of Low-Rank Experts for Comprehensive Head CT Analysis
Feb 28, 2026Youngjin Yoo, Han Liu, Bogdan Georgescu +14Medical Imaging Foundation Models Low-Rank Adaptation
Digital Technology and Innovation, Siemens Healthineers, Princeton, NJ USA · Department of Computed Tomography, Siemens Healthineers, Forchheim, Germany · Department of Radiology, University of Maryland Medical Center, Baltimore, MD USA +4

Aug 28, 2026J/K move · Enter open · S save

ARC-CT: Anatomy-Routed Contrastive Vision-Language Learning for 3D Chest CT

Huseyin Umut Isik, Mehmet Alp Ozaydin, Sila Kurugol, Şeyda Ertekin

Department of Computer Engineering, METU, Ankara, Turkey · Quantitative Intelligent Imaging Lab, Boston Children’s Hospital and Harvard Medical School, Boston, MA, USA · METU-DTX Digital Transformation and Innovation Center, Ankara, Turkey

Radiology VLMs Medical Image Classification Contrastive Learning Multimodal Contrastive Learning Computed Tomography

Open paper AskPDF arXiv

Contrastive vision-language learning uses paired chest CT volumes and radiology reports to learn abnormality classifiers without manually annotated labels. However, two characteristics of chest CT challenge conventional global contrastive learning. First, many critical abnormalities are small or anatomically localized, and pooling an en- tire volume into a single embedding may dilute their visual evidence. Second, the standard contrastive objective treats every other scan in a batch as a negative. Because many chest CTs share abnormalities, this objective incorrectly pushes co-positive pairs apart. We propose Anatomy-Routed Contrastive Learning for 3D Chest CT (ARC-CT), a region-aware framework that addresses these limitations using only la- bels extracted from reports by an LLM, with no manual annotations or bounding boxes. ARC-CT combines three components: (1) an Anato- myQFormer localizing evidence via queries constrained by automatically generated organ masks; (2) a label-Jaccard soft InfoNCE objective in- tegrating the standard one-hot target with the label-set overlap of each pair, which reduces false-negative penalties between studies that share clinical findings; and (3) an organ-level alignment loss connecting mask- pooled visual features to organ-specific report text extracted offline with a large language model. ARC-CT achieves a 0.86 mask-free macro AUC across 18 abnormalities using a compact 3D ResNet-18 backbone. Over- all, ARC-CT outperforms both comparable efficient baselines and sev- eral larger transformer models. Our code and weights are available at https://github.com/arc-ct/arc-ct.

Abstract

Explore similar work

ARC-CT: Anatomy-Routed Contrastive Vision-Language Learning for 3D Chest CT

MultiMedVision: Multi-Modal Medical Vision Framework

Specializing Foundation Models via Mixture of Low-Rank Experts for Comprehensive Head CT Analysis