Ultrasound images vary widely across scanners, operators, and anatomical targets, which often causes models trained in one setting to generalize poorly to new hospitals and clinical conditions. The Foundation Model Challenge for Ultrasound Image Analysis (FMC-UIA) reflects this difficulty by requiring a single model to handle multiple tasks, including segmentation, detection, classification, and landmark regression across diverse organs and datasets. We propose a unified multi-task framework based on a transformer visual encoder from the Qwen3-VL family. Intermediate token features are projected into spatial feature maps and fused using a lightweight multi-scale feature pyramid, enabling both pixel-level predictions and global reasoning within a shared representation. Each task is handled by a small task-specific prediction head, while training uses task-aware sampling and selective loss balancing to manage heterogeneous supervision and reduce task imbalance. Our method is designed to be simple to optimize and adaptable across a wide range of ultrasound analysis tasks. The performance improved from 67% to 85% on the validation set and achieved an average score of 81.84% on the official test set across all tasks. The code is publicly available at: https://github.com/saitejalekkala33/FMCUIA-ISBI.git
Figures & tables
Figure 1: Overview of the proposed unified multi-task framework. Shared Qwen3-VL encoder extracts multi-depth tokens, reshaped and fused into a four-level pyramid. Dense heads use pyramid features; global heads use pooled multi-scale embeddings.
Task
Category
IoU
MRE
Segmentation
Classification
Task Score
IoU Raw
MRE Raw
DSC Raw
HD Raw
AUC Raw
F1 Raw
MCC Raw
Detection
thyroid_nodule_det
0.717
–
–
–
–
–
–
0.920
Regression
IUGC
–
47.915
–
–
–
–
–
0.685
FUGC
–
17.911
–
–
–
–
–
0.858
fetal_femur
–
71.954
–
–
–
–
–
0.960
Segmentation
cardiac_multi
–
–
0.525
177.195
–
–
–
0.385
Table 1: Unified results table (raw metrics only). Dashes indicate metrics not applicable to that task.
Figure 2: Ablation across model variants, reported the overall average score for the validation split with backbones Qwen2.5-VL-7B-Instruct and Qwen3-VL-8B-Instruct.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Section
Hyperparameter
Value
Task sampling / loss
Temperature
0.7
Loss EMA β
0.98
Dynamic weighting γ
0.5
Weight clamp
[0.25,4.0]
Sampler type
Temperature-based (frequency + EMA loss)
Input / preprocessing
Image resize
448×448
Appendix
Table 2: Training and inference hyperparameters.
Component
Setting
Operating System
Ubuntu 24.04.4 LTS
Programming Language
Python
GPU
NVIDIA L40S
VRAM
48GiB
Processor
AMD EPYC 7R13 Processor
Training Framework
Hugging Face Transformers, Accelerate
Appendix
Table 3: Implementation and environment details.
Task
Features
Head details
Segmentation
fpn_v1
SegmentationHeadV1 : P5 projection, three bilinear up-fuse blocks, dropout, and a 1×1 classifier. Logits are sliced to the task-specific class count and upsampled to the input image size.
Classification
global
ClassificationHead : dynamic linear projection to 1024 dimensions, layer normalization, GELU, two gated-MLP residual blocks, and a final linear classifier.
Regression
global_v1
RegressionHeadV1 : dynamic linear projection to 1024 dimensions, layer normalization, GELU, dropout, and a linear regressor. The first 2M outputs are retained and passed through a sigmoid.
Detection
fpn
DetectionGridHead : uses P2 together with upsampled P3 and P4 , plus normalized coordinate channels. A residual convolutional tower predicts four box channels and one objectness channel per grid cell.
Appendix
Table 4: Task routing and prediction heads.
Task
Category
IoU
MRE
Segmentation
Classification
Task Score
IoU Norm
MRE Norm
DSC Norm
HD Norm
AUC Norm
F1 Norm
MCC Norm
Detection
thyroid_nodule_det
0.920
–
–
–
–
–
–
0.920
Regression
IUGC
–
0.685
–
–
–
–
–
0.685
FUGC
–
0.858
–
–
–
–
–
0.858
fetal_femur
–
0.960
–
–
–
–
–
0.960
Segmentation
cardiac_multi
–
–
0.295
0.474
–
–
–
0.385
Appendix
Table 5: Unified results table (normalized metrics only). Dashes indicate metrics not applicable to that task.
Ultrasound image analysis plays a crucial role in cancer screening and prenatal diagnosis, yet comprehensive assessment requires jointly addressing tasks such as lesion segmentation and benign-malignant classification. While recent vision foundation models have shown remarkable universal representations, unlocking their potential for ultrasound is bottlenecked by the considerable domain gap from natural images. Existing methods typically fine-tune heavy vision encoders for isolated tasks, incurring substantial computational overhead while overlooking the underlying commonalities across heterogeneous tasks. In this work, we propose FreqDINO++, a frequency-guided multi-task routing vision foundation model for universal ultrasound analysis. We first introduce a Multi-task Routing Adapter (MR-Adapter) to support parameter-efficient integration of task-common and task-specific knowledge, a Frequency-aware Feature Enhancer (F2-Enhancer) is then designed to capture the rich multi-scale frequency characteristics of ultrasound images, and a Task-aligned Collaborative Decoder (TC-Decoder) is devised to promote collaboration between dense and global prediction tasks through global-local token interaction. Extensive experiments on large-scale multi-task and external single-task ultrasound benchmarks demonstrate that FreqDINO++ consistently outperforms strong baselines and recent foundation models across 27 diverse clinical task scenarios, while also showing promising generalization to unseen data. The code is at https://github.com/MingLang-FD/FreqDINO-Plus.
Qing Xu, Yixuan Zhang, Yue Li +7
School of Computer Science, University of Nottingham Ningbo China, Ningbo, Zhejiang, China · School of Mathematical Sciences, University of Nottingham Ningbo China, Ningbo, Zhejiang, China · School of Computer Science, University of Nottingham, Nottingham NG72RD, UK +3
Executing multiple tasks simultaneously in medical image analysis, including segmentation, classification, detection, and regression, often introduces significant challenges regarding model generalizability and the optimization of shared feature representations. While Vision Foundation Models (VFMs) provide powerful general representations, full fine-tuning on limited medical data is prone to overfitting and incurs high computational costs. Moreover, existing parameter-efficient fine-tuning approaches typically adopt task-agnostic adaptation protocols, overlooking both task-specific mechanisms and the varying sensitivity of model layers during fine-tuning. In this work, we propose Task-Aware Prompting and Selective Layer Fine-Tuning (TAP-SLF), a unified framework for multi-task ultrasound image analysis. TAP-SLF incorporates task-aware soft prompts to encode task-specific priors into the input token sequence and applies LoRA to selected specific top layers of the encoder. This strategy updates only a small fraction of the VFM parameters while keeping the pre-trained backbone frozen. By combining task-aware prompts with selective high-layer fine-tuning, TAP-SLF enables efficient VFM adaptation to diverse medical tasks within a shared backbone. Results on the FMC_UIA 2026 Challenge test set, where TAP-SLF wins fifth place, combined with evaluations on the officially released training dataset using an 8:2 train-test split, demonstrate that task-aware prompting and selective layer tuning are effective strategies for efficient VFM adaptation.
Hui Wan, Libin Lan
College of Computer Science and Engineering, Chongqing University of Technology, Chongqing, China
Clinical ultrasound analysis demands models that generalize across heterogeneous organs, views, and devices, while supporting interpretable workflow-level analysis. Existing methods often rely on task-wise adaptation, and joint learning may be unstable due to cross-task interference, making it hard to deliver workflow-level outputs in practice. To address these challenges, we present USTri, a tri-stage ultrasound intelligence pipeline for unified multi-organ, multi-task analysis. Stage I trains a universal generalist USGen on different domains to learn broad, transferable priors that are robust to device and protocol variability. To better handle domain shifts and reach task-aligned performance while preserving ultrasound shared knowledge, Stage II builds USpec by keeping USGen frozen and finetuning dataset-specific heads. Stage III introduces USAgent, which mimics clinician workflows by orchestrating USpec specialists for multi-step inference and deterministic structured reports. On the FMC_UIA validation set, our model achieves the best overall performance across 4 task types and 27 datasets, outperforming state-of-the-art methods. Moreover, qualitative results show that USAgent produces clinically structured reports with high accuracy and interpretability. Our study suggests a scalable path to ultrasound intelligence that generalizes across heterogeneous ultrasound tasks and supports consistent end-to-end clinical workflows. The code is publicly available at: https://github.com/MacDunno/USTri.
Chen Ma, Yunshu Li, Junhu Fu +3
College of Biomedical Engineering, Fudan University, Shanghai, China