AURORA: Adaptive Unified Representation for Robust Ultrasound Analysis

Ufaq Khan; L. D. M. S. Sai Teja; Ayuba Shakiru; Mai A. Shaaban; Yutong Xie; Muhammad Bilal; Muhammad Haris Khan

cs.CVMar 19, 2026

AURORA: Adaptive Unified Representation for Robust Ultrasound Analysis

Authors: Ufaq Khan, L. D. M. S. Sai Teja, Ayuba Shakiru, Mai A. Shaaban, Yutong Xie, Muhammad Bilal, Muhammad Haris Khan

Organizations: MBZUAI, Abu Dhabi, UAE · NIT Silchar, India · Birmingham City University, UK

Abstract

Ultrasound images vary widely across scanners, operators, and anatomical targets, which often causes models trained in one setting to generalize poorly to new hospitals and clinical conditions. The Foundation Model Challenge for Ultrasound Image Analysis (FMC-UIA) reflects this difficulty by requiring a single model to handle multiple tasks, including segmentation, detection, classification, and landmark regression across diverse organs and datasets. We propose a unified multi-task framework based on a transformer visual encoder from the Qwen3-VL family. Intermediate token features are projected into spatial feature maps and fused using a lightweight multi-scale feature pyramid, enabling both pixel-level predictions and global reasoning within a shared representation. Each task is handled by a small task-specific prediction head, while training uses task-aware sampling and selective loss balancing to manage heterogeneous supervision and reduce task imbalance. Our method is designed to be simple to optimize and adaptable across a wide range of ultrasound analysis tasks. The performance improved from 67% to 85% on the validation set and achieved an average score of 81.84% on the official test set across all tasks. The code is publicly available at: https://github.com/saitejalekkala33/FMCUIA-ISBI.git

Figures & tables

Figure 1: Overview of the proposed unified multi-task framework. Shared Qwen3-VL encoder extracts multi-depth tokens, reshaped and fused into a four-level pyramid. Dense heads use pyramid features; global heads use pooled multi-scale embeddings.

Task	Category	IoU	MRE	Segmentation		Classification			Task Score
		IoU ${}_{\text{Raw}}$	MRE ${}_{\text{Raw}}$	DSC ${}_{\text{Raw}}$	HD ${}_{\text{Raw}}$	AUC ${}_{\text{Raw}}$	F1 ${}_{\text{Raw}}$	MCC ${}_{\text{Raw}}$
Detection	thyroid_nodule_det	0.717	–	–	–	–	–	–	0.920
Regression	IUGC	–	47.915	–	–	–	–	–	0.685
	FUGC	–	17.911	–	–	–	–	–	0.858
	fetal_femur	–	71.954	–	–	–	–	–	0.960
Segmentation	cardiac_multi	–	–	0.525	177.195	–	–	–	0.385

Table 1: Unified results table (raw metrics only). Dashes indicate metrics not applicable to that task.

Figure 2: Ablation across model variants, reported the overall average score for the validation split with backbones Qwen2.5-VL-7B-Instruct and Qwen3-VL-8B-Instruct.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Section	Hyperparameter	Value
Task sampling / loss	Temperature	$0.7$
	Loss EMA $\beta$	$0.98$
	Dynamic weighting $\gamma$	$0.5$
	Weight clamp	$[0.25,\ 4.0]$
	Sampler type	Temperature-based (frequency + EMA loss)
Input / preprocessing	Image resize	$448\times 448$

Appendix

Table 2: Training and inference hyperparameters.

Component	Setting
Operating System	Ubuntu 24.04.4 LTS
Programming Language	Python
GPU	NVIDIA L40S
VRAM	48GiB
Processor	AMD EPYC 7R13 Processor
Training Framework	Hugging Face Transformers, Accelerate

Appendix

Table 3: Implementation and environment details.

Task	Features	Head details
Segmentation	fpn_v1	SegmentationHeadV1 : $P_{5}$ projection, three bilinear up-fuse blocks, dropout, and a $1{\times}1$ classifier. Logits are sliced to the task-specific class count and upsampled to the input image size.
Classification	global	ClassificationHead : dynamic linear projection to 1024 dimensions, layer normalization, GELU, two gated-MLP residual blocks, and a final linear classifier.
Regression	global_v1	RegressionHeadV1 : dynamic linear projection to 1024 dimensions, layer normalization, GELU, dropout, and a linear regressor. The first $2M$ outputs are retained and passed through a sigmoid.
Detection	fpn	DetectionGridHead : uses $P_{2}$ together with upsampled $P_{3}$ and $P_{4}$ , plus normalized coordinate channels. A residual convolutional tower predicts four box channels and one objectness channel per grid cell.

Appendix

Table 4: Task routing and prediction heads.

Task	Category	IoU	MRE	Segmentation		Classification			Task Score
		IoU ${}_{\text{Norm}}$	MRE ${}_{\text{Norm}}$	DSC ${}_{\text{Norm}}$	HD ${}_{\text{Norm}}$	AUC ${}_{\text{Norm}}$	F1 ${}_{\text{Norm}}$	MCC ${}_{\text{Norm}}$
Detection	thyroid_nodule_det	0.920	–	–	–	–	–	–	0.920
Regression	IUGC	–	0.685	–	–	–	–	–	0.685
	FUGC	–	0.858	–	–	–	–	–	0.858
	fetal_femur	–	0.960	–	–	–	–	–	0.960
Segmentation	cardiac_multi	–	–	0.295	0.474	–	–	–	0.385

Appendix

Table 5: Unified results table (normalized metrics only). Dashes indicate metrics not applicable to that task.

Explore similar work

FreqDINO++: A Frequency-Guided Multi-Task Routing Vision Foundation Model for Universal Ultrasound Analysis
Sep 17, 2026Qing Xu, Yixuan Zhang, Yue Li +7Ultrasound Image Segmentation Medical Imaging Foundation Models
School of Computer Science, University of Nottingham Ningbo China, Ningbo, Zhejiang, China · School of Mathematical Sciences, University of Nottingham Ningbo China, Ningbo, Zhejiang, China · School of Computer Science, University of Nottingham, Nottingham NG72RD, UK +3
TAP-SLF: Parameter-Efficient Adaptation of Vision Foundation Models for Multi-Task Ultrasound Image Analysis
Feb 28, 2026Hui Wan, Libin LanVision Foundation Model Adaptation Medical Image Analysis
College of Computer Science and Engineering, Chongqing University of Technology, Chongqing, China
Unified Ultrasound Intelligence Toward an End-to-End Agentic System
Apr 18, 2026Chen Ma, Yunshu Li, Junhu Fu +3Medical Imaging Foundation Models Multi-Task Learning
College of Biomedical Engineering, Fudan University, Shanghai, China

Sep 17, 2026J/K move · Enter open · S save

FreqDINO++: A Frequency-Guided Multi-Task Routing Vision Foundation Model for Universal Ultrasound Analysis

Qing Xu, Yixuan Zhang, Yue Li, Xiangjian He+6

School of Computer Science, University of Nottingham Ningbo China, Ningbo, Zhejiang, China · School of Mathematical Sciences, University of Nottingham Ningbo China, Ningbo, Zhejiang, China · School of Computer Science, University of Nottingham, Nottingham NG72RD, UK+3

Ultrasound Image Segmentation Medical Imaging Foundation Models Multi-Task Learning Medical Image Classification Ultrasound Imaging Parameter-Efficient Fine-Tuning

Open paper AskPDF arXiv

Ultrasound image analysis plays a crucial role in cancer screening and prenatal diagnosis, yet comprehensive assessment requires jointly addressing tasks such as lesion segmentation and benign-malignant classification. While recent vision foundation models have shown remarkable universal representations, unlocking their potential for ultrasound is bottlenecked by the considerable domain gap from natural images. Existing methods typically fine-tune heavy vision encoders for isolated tasks, incurring substantial computational overhead while overlooking the underlying commonalities across heterogeneous tasks. In this work, we propose FreqDINO++, a frequency-guided multi-task routing vision foundation model for universal ultrasound analysis. We first introduce a Multi-task Routing Adapter (MR-Adapter) to support parameter-efficient integration of task-common and task-specific knowledge, a Frequency-aware Feature Enhancer (F

^2

-Enhancer) is then designed to capture the rich multi-scale frequency characteristics of ultrasound images, and a Task-aligned Collaborative Decoder (TC-Decoder) is devised to promote collaboration between dense and global prediction tasks through global-local token interaction. Extensive experiments on large-scale multi-task and external single-task ultrasound benchmarks demonstrate that FreqDINO++ consistently outperforms strong baselines and recent foundation models across 27 diverse clinical task scenarios, while also showing promising generalization to unseen data. The code is at https://github.com/MingLang-FD/FreqDINO-Plus.

Abstract

Explore similar work

FreqDINO++: A Frequency-Guided Multi-Task Routing Vision Foundation Model for Universal Ultrasound Analysis

TAP-SLF: Parameter-Efficient Adaptation of Vision Foundation Models for Multi-Task Ultrasound Image Analysis

Unified Ultrasound Intelligence Toward an End-to-End Agentic System