Purpose: Ultrasound biometry is widely used across various clinical applications, including prenatal and cardiac examinations, to obtain quantitative measurements by placing calipers at anatomical landmarks. However, these measurements remain subject to interoperator variability as they rely largely on manual caliper placement. Conventional methods often use specialized prediction heads for different ultrasound tasks, whereas vision-language models offer flexibility through text instructions. However, direct coordinate prediction does not explicitly model spatial likelihood distributions, limiting the representation of localization ambiguity. Methods: We propose Kev-Bio, a vision-language model that uses clinical measurement questions to guide ultrasound landmark detection. Inspired by System One decision models, Kev-Bio uses decision representations of the language model to generate queries for each caliper endpoint. These queries are matched against shared dense ultrasound features to produce spatial probability maps, from which landmark coordinates and clinical measurements are derived. The model is trained end-to-end using both landmark localization and clinical measurement losses. Results: On the nine-view FU/GU Biometry 2026 dataset, Kev-Bio achieves a mean radial error (MRE) of 22.50 pixels and a clinical measurement mean absolute error (MAE) of 15.38, outperforming the compared baselines in overall performance. Conclusion: These results demonstrate the effectiveness of Kev-Bio for ultrasound landmark detection and clinical measurement across different anatomical views. Our code and models are available at https://github.com/moeinheidari7829/Kev-Bio.
Figures & tables
Figure 1: Overview of Kev-Bio. Qwen3.5 and USFM provide visual tokens to the language model, while DINOv2 contributes additional localization features. Clinical questions are encoded into decision states at ⟨decide⟩ tokens, which generate queries for each landmark endpoint. A convolutional-based Pixel Neck combines features from the three visual encoders into a shared dense feature map. Endpoint queries are matched against this map to produce spatial probability maps, from which landmark coordinates and clinical measurements are derived.
Task
View
Samples
LM
Meas.
Cardiac
A4C
Apical four-chamber
108
16
8 D
PLAX
Parasternal long-axis
87
22
11 D
PSAX
Parasternal short-axis
49
4
2 D
IVC
Subcostal inferior vena cava
38
2
1 D
Prenatal
Table 1: FU/GU Biometry 2026 labelled dataset. Samples: number of labelled images; LM: landmarks per image; Meas.: clinical measurements per image (D: distance, C: ellipse circumference, A: angle).
Method
Params (M)
Cardiac
Prenatal
Intrapartum
Avg.
A4C
PLAX
PSAX
IVC
HC
FA
Femur
FUGC
AoP
MRE (px) ↓
General Baselines
nnLandmark Ertl et al. (2026)
140.1
44.23
26.71
58.87
45.92
25.45
19.36
22.28
6.99
8.79
28.73 ± 4.12
MedSapiens Elbatel et al. (2026)
24.0
118.85
76.49
108.41
130.09
35.32
24.34
24.75
11.56
9.61
59.94 ± 4.15
Challenge Baselines
Table 2: Comparison on 5-fold cross-validation of the FU/GU Biometry 2026 training set (lower is better). Top: mean radial error of landmarks (MRE, px). Bottom: mean absolute error of clinical measurements (MAE; px, degrees for AoP). Per-task values are averaged over five folds; Avg. is the macro mean over nine tasks ± s.d. over folds. Params: total trainable parameters in millions. Best and second best per column among reported values.
Figure 2: Qualitative comparison across nine ultrasound views. Landmark predictions from Kev-Bio (red) and US-Foundation (blue), with ground truth shown in green. Values indicate mean landmark error in pixels.
Figure 3: Spatial probability maps and error detection. (a) Landmark probability maps showing confident and ambiguous predictions. (b) Landmarks are ranked by decreasing map entropy, and curves show the fraction of localization errors captured in each clinical domain. Errors are defined using ground truth as deviations exceeding 5% of the image diagonal. The black marker indicates the overall result, and the dashed line represents random selection.
Figure 4: Question-conditioned landmark localization. Different clinical measurement questions produce distinct landmark probability maps on the same ultrasound image. Green circles denote ground truth, and yellow regions indicate predicted spatial distributions.
Variant
DINOv2
LLM
Pointer
MRE (px) ↓
MAE ↓
w/o DINOv2
✗
✓
✓
24.85 ± 2.66
16.94 ± 0.94
w/o LLM
✓
✗
✓
23.66 ± 2.63
16.77 ± 1.23
w/o pixel pointer
✓
✓
✗
35.50 ± 3.12
23.64 ± 1.60
Kev-Bio (full)
✓
✓
✓
22.50 ± 2.10
15.38 ± 1.24
Table 3: Component ablation of Kev-Bio. We assess the impact of DINOv2, LLM-guided queries, and pointer-based localization. Results are the mean ± s.d. over five folds. Lower is better.
Vision-Language Models (VLMs) have significantly advanced medical visual question answering, yet their performance in ultrasound remains suboptimal. In clinical practice, sonographers explicitly focus on lesion regions to formulate reports, though diagnostic interpretations sometimes vary due to inherent subjectivity. However, existing VLMs are not explicitly structured to interactively zoom into lesions prior to diagnosis; moreover, they typically treat annotations as unbiased ground truths, failing to account for their inherent subjectivity and ambiguity. In this paper, we propose a framework specifically designed to consider the sonographer's cognitive workflow. We first introduce a structured Zoom-then-Diagnose paradigm, which replicates the interactive search process to enable lesion-focused reasoning. Furthermore, within the Group Relative Policy Optimization (GRPO) framework, we introduce an uncertainty-aware reward derived from stochastic group-wise rollouts to estimate prediction consistency as a proxy for model confidence. Together, these two components encourage the model to reinforce accurate predictions on clear cases while remaining cautious under ambiguity. Experiments across liver, breast, and thyroid datasets show that our framework improves lesion localization by 39.3%, demonstrating that our model has learned the ability to actively look closer and diagnose.
Yue Zhou, Erxuan Wu, Yikang Sun +5
Computer Aided Medical Procedures (CAMP), TU Munich, Germany · Munich Center for Machine Learning (MCML), Munich, Germany · Zhongshan Hospital, Fudan University, China +1
Ultrasound is one of the most widely used medical imaging modalities, and recent large vision-language models(VLMs) have shown increasing capabilities in ultrasound image understanding. However, these models fail to provide pixel-level visual evidence aligned with their semantic predictions, and their fine-grained grounding capability in ultrasound remains largely unclear. We introduce UltraG-Bench, a large-scale multi-task benchmark for evaluating pixel-level evidence grounding in ultrasound. UltraG-Bench is built by annotating 40 public ultrasound segmentation datasets spanning 13 anatomical categories, and comprises three progressive tasks: instruction-guided segmentation, evidence-grounded VQA, and evidence-grounded report generation, with 331125, 666779, and 138832 annotations, respectively. Comprehensive evaluation of 14 state-of-the-art models reveals a substantial gap between semantic understanding and fine-grained pixel-level localization. We further propose UltraG-Agent, which combines the semantic reasoning capabilities of a VLM with the ultrasound-specific segmentation capability of UltraSAM3. Experiments show that UltraG-Agent substantially improves both semantic prediction and pixel-level visual grounding. Our dataset and code are available at https://github.com/zhuqh19/UltraG-Bench.
Large vision-language models (LVLMs) have achieved strong performance across many medical imaging tasks, yet their application to ultrasound remains limited due to its inherent complexity and variability. In this work, we revisit what is truly needed to enable real-world ultrasound understanding. Instead of introducing complex architectures or elaborate training strategies, we show that data scale and clinically faithful data alignment are the key factors. We construct a large-scale dataset of 1.5M real-world ultrasound examinations, containing 17.7M images, multi-organ coverage, and paired uncurated clinical reports. Crucially, we organize the data at the examination level, aligning multiple images with their corresponding reports to reflect real clinical workflows. We then fine-tune a standard LVLM using low-rank adaptation (LoRA) on this dataset without task-specific modifications. Surprisingly, this simple recipe already leads to strong performance across diverse ultrasound understanding tasks, outperforming prior methods designed with more complex pipelines. Beyond these results, we present model and data scaling analyses that provide insights into the role of scale in ultrasound LVLMs.
Bingcong Yan, Chunlei Li, Jingliang Hu +3
1MedAI Technology (Wuxi) Co. Ltd. · 2Technical University of Munich