Risk Control Certifies Rates, Not Magnitudes: An Audit of Selective Fetal Biometry

Zhengshuai Yang; Linna Jiang; Changyun Li; Guihu Zhao

cs.AINov 4, 2026

Risk Control Certifies Rates, Not Magnitudes: An Audit of Selective Fetal Biometry

Authors: Zhengshuai Yang, Linna Jiang, Changyun Li, Guihu Zhao

Organizations: College of Information and Intelligent Science and Technology, Hunan Agricultural University, Changsha 410128, China · National Clinical Research Center for Geriatric Disorders, Xiangya Hospital, Central South University, Changsha 410008, China

Abstract

Prenatal ultrasound biometry must return a head circumference (HC) and refuse when the image cannot support one. Selective prediction trades coverage against error, but its finite-sample guarantees bound the rate of gross errors, not their magnitude. We audit a tool-grounded biometry pipeline on two separately pre-registered four-way splits of the 999 labelled HC18 images, retraining the segmenter three times on each. Calibrating the contour-completeness threshold by exact Clopper-Pearson risk control returns a certificate indistinguishable from a hand-set floor, yet the pipeline still emits a 271.3 mm error: a small, geometrically complete ring fitted to an image whose true HC is 324.6 mm. The accepted candidate scores 0.94 completeness, so the score cannot see the failure, and one outlier among $\approx 150$ answers leaves the certified risk bound ( $α= 0.10$ ) intact. A second, untouched split reproduces the mechanism on a different image (233.3 mm). Constraining the admissibility region---the anatomical range in which the measurand is defined---removes the family instead: raising the lower bound from 50 to 55 mm costs 0.4 coverage points and cuts the worst-case error on both splits to 11.4 and 11.1 mm, whereas suppressing it through the completeness threshold costs 19.0 and 12.1 points. At an anatomically defined 70 mm the frozen tests give 1.91 and 1.81 mm MAE.

Figures & tables

Figure 1: Measurement runs before VLM generation. The original end-to-end runs use

L=25

mm; offline audits and the second-family run also examine

L=50

mm. The VLM may accept the observation or abstain.

Strategy	Plane QA ( $n{=}97$ )		HC Biometry ( $n{=}96$ )				Tok. (k)	Lat. (s)
Strategy	Ans.	Acc.	Ans.	MAE	Med.	$>$ 7 mm	Tok. (k)	Lat. (s)
Paired benchmark: one interleaved four-arm run ( 89d79bf1 )
Direct	88	.841	0 1	41.0	41.0	1	290	0 9.3
Retrieval	92	.880	0 9	12.7	13.3	6	432	17.9
Staged	77	.766	0 0	—	—	0	1,424	43.0
Tool VQA	93	.871	71	6.50	2.30	16	373	14.8

Table 1: Plane accuracy and HC metrics are conditional on answered questions; tokens are run totals in thousands and latency the run-record mean. Only the first four rows share one run, so only those support the paired tests; the delivered run holds the tool arm alone and its plane accuracy is unpaired. ⋆ HC biometry only. Offline replay of the same candidates with the

L{=}50

mm floor gives 66 answers at 1.78 mm, so the language model adds no measurement.

Split 1, frozen test ( $n{=}156$ )
Split / floor $L$	Cov.	$n$	MAE (mm)	Gross ( $\geq$ 7 mm)	Worst case (mm)
$L{=}50$ mm	.722	112.7	0 3.51 $\pm$ 1.42	2.39%	271.3
$L{=}55$ mm	.718	112.0	1.93 $\pm$ 0.03	1.82%	0 11.4
Split 2, frozen test ( $n{=}152$ )
$L{=}50$ mm	.750	114.0	0 2.53 $\pm$ 1.13	1.41%	233.3
$L{=}55$ mm	.748	113.7	1.81 $\pm$ 0.17	1.10%	0 11.1

Table 2: Frozen-test evaluation on two pre-registered splits,

\tau{=}0.70

throughout, mean

\pm

sd over three retrained seeds; worst case is the largest absolute error in the split. The second split was registered after the first failure was identified, so it is a confirmatory replication.

Figure 2: The two failures that passed the completeness gate. Green: physician annotation; red dashed: the accepted conic fit, magnified in the inset. Left (split 1): 977_HC.png , gold 324.6 mm, fit 53.3 mm at $c{=}0.75$ , error 271.3 mm. Right (split 2): 945_HC.png , gold 284.9 mm, fit 51.7 mm at $c{=}0.88$ , error 233.3 mm. Both fits sit on the bright specular arc at the skull boundary, which is smooth and therefore scores high completeness.

Selective Prediction

Selective prediction lets models answer only when sufficiently reliable, trading coverage for lower error through abstention, referral, or extra evidence gathering.

See all 303 papers

Region-aware error triage for medical image segmentation with groupwise conformal risk control
Nov 6, 2026Nishi Kanta Paul, Md Shihabul Islam Shovo, Hoileong LeeMedical Image Segmentation Conformal Risk Control
NIMISHES Lab, Dhaka, Bangladesh · Faculty of Electronic Engineering and Technology, Universiti Malaysia Perlis, 02600 Arau, Perlis, Malaysia
Automated Fetal Brain MRI Biometry in Healthy and Pathological Cases
Aug 16, 2026Ema Masterl, Tina Vipotnik Vesnaver, Nejc Šubič +1Anatomical Landmark Detection Fetal MRI
Faculty of Medicine, University of Ljubljana, Ljubljana, Slovenia · University Medical Centre Ljubljana, Ljubljana, Slovenia · Faculty of Electrical Engineering, University of Ljubljana, Ljubljana, Slovenia
Certify or Refuse: A Cross-Model Map for Selective Risk Control with Coverage Floors under Covariate Shift
Aug 11, 2026Jiamiao Liu, Dewen Qiao, Yu Zhang +1Selective Prediction Covariate Shift
Department of Information, Xinqiao Hospital Army Medical University (Third Military Medical University) Chongqing 400037, China

Nov 6, 2026J/K move · Enter open · S save

Region-aware error triage for medical image segmentation with groupwise conformal risk control

Nishi Kanta Paul, Md Shihabul Islam Shovo, Hoileong Lee

NIMISHES Lab, Dhaka, Bangladesh · Faculty of Electronic Engineering and Technology, Universiti Malaysia Perlis, 02600 Arau, Perlis, Malaysia

Medical Image Segmentation Conformal Risk Control

Open paper AskPDF arXiv

Segmentation masks are increasingly reviewed rather than redrawn, which requires knowing where a mask is likely wrong. Learned error maps rank pixels by risk, but the number of pixels to warn is chosen ad hoc, and calibrating one warning threshold over all error pixels may leave errors far from the predicted contour under-flagged. We propose RACE-Triage, a post-hoc error triage framework that leaves the segmentation mask unchanged. A risk network is trained on out-of-fold predictions from backbone-independent inputs. The predicted contour partitions each image into a boundary band and a non-boundary region, and conformal risk control is applied separately in each region to bound the expected fraction of missed errors among cases with errors there. On ISIC 2018 dermoscopy and BUSI breast ultrasound (where patient identifiers are unavailable, so independence across images is assumed), pooled control met its overall target but missed on average 31.8% and 36.4% of non-boundary errors. Regional control reduced these fractions to 19.3% and 20.9% and raised the recall of size-filtered false-negative components from 0.722 to 0.818 and from 0.611 to 0.763. The mean warned area grew by 1.6 and 4.9 percentage points, and its 95th percentile reached 23.5% and 38.5%. The effect held across three seeds and a second backbone, and area-matched heuristics did not reproduce it. On the external PH

^2

set, frozen thresholds exceeded both targets, whereas recalibration with 100 local images brought the mean over repeated splits below them. Regional calibration improves non-boundary error coverage at a measured increase in warned area.

Abstract

Selective Prediction

Explore similar work

Region-aware error triage for medical image segmentation with groupwise conformal risk control

Automated Fetal Brain MRI Biometry in Healthy and Pathological Cases

Certify or Refuse: A Cross-Model Map for Selective Risk Control with Coverage Floors under Covariate Shift