Taurus: LLM-in-the-Loop CT Annotation Matches Coding Agents at Half the Cost

Bing Hu; Rishabh Sharma; Jatin Mehta

cs.AIOct 30, 2026

Taurus: LLM-in-the-Loop CT Annotation Matches Coding Agents at Half the Cost

Authors: Bing Hu, Rishabh Sharma, Jatin Mehta

Organizations: David R. Cheriton School of Computer Science, University of Waterloo, Waterloo, Ontario, Canada

Abstract

Dense labels of the aorta in CT are slow to draw, and the public aortic CT datasets that could stand in for them each label different structures under their own conventions. Large language model (LLM) agents can now write code and read images. We asked whether such a model labels these scans better as a free-form agent that writes its own segmentation code, or inside a harness, where an automated pass labels the scan and the model reviews the result. We built a harness, Taurus, and compared it with free-form agents by blind label recovery: we withheld the human labels of six public datasets (939 volumes) and scored every approach against each dataset's own classes. In the first version of Taurus, the model's review raised the primary metric on only two of six datasets, by less than the noise between two reviews of the same case. A free-form agent running Claude Opus 5.5 scored higher on most datasets. Moving what the free-form agents did well into the automated pass then carried most of the gain on 38 held-out cases. The final version smooths the label surfaces with a fixed rule and checks for the large errors the automated pass tends to make, such as an aneurysm drawn to half its size. Each check that fires puts a multiple-choice question to the model, with candidate corrections drawn on the scan. It matched the surface smoothness of the human references, and in the primary metric we found no difference from the free-form Opus agent, at under half its cost, $1.22 against$ 2.71 per case. Most of what made Taurus competitive came from the harness and its automation rather than from the language model itself. Taurus shows that this kind of annotation can be automated, so that expert time can move from drawing labels to checking them.

Figures & tables

Panel of Figure 2 — Figure 2: Two ways to let a language model label a CT scan. (a) The free-form agent loops through looking, writing code, running it and rendering until it writes the labels and a verdict. (b) The Taurus reviewer (Taurus-hybrid shown) starts from the automated label and works through harness calls, each re-rendering the case, until it records a verdict. (c) What the reviewer sees; the question card offers three outlines of the aneurysm sac (A 121, B 114, C 113 mL), and the reviewer chose B.

Reference	Dataset	Human label (type)	Field of view	Contrast	Volumes / reviewed / free-form / held-out	Primary metric
	AortaSeg24 [ 1 ]	23 vessel ids (zones 0–9, branches) (voxel label map)	whole body; type-B dissections	CTA	100 / 100 / 5 / 4	abdominal vessel macro Dice
	Siriapisith AAA/EVAR [ 3 ]	envelope a (voxel label map)	80 mm abdominal crops, 256 $\times$ 256 $\times$ 64; incl. EVAR	CTA; 10 NC	520 / 160 / 6 / 12	outer Dice
	AVT [ 2 ]	one binary arterial-tree lumen (voxel segmentation)	thorax–pelvis arterial tree	CTA; 1 NC	55 / 24 / 5 / 4	tree Dice
	VMR [ 6 ]	lumen per named vessel (lofted surface models, voxelized)	abdomen ( $\pm$ chest); 16 AAA, 1 healthy	CTA	17 / 17 / 4 / 4	per-vessel macro Dice
	Jamshidian 4D-CTA [ 5 ]	thrombus, sac and wall meshes (finite-element surface meshes, voxelized)	aneurysm-sac crops	CTA	149 b / 39 / 5 / 6	thrombus Dice
not shown: ND licence c	MosMedData AAA [ 4 , 12 ]	lumen, calcium (thrombus on 4) (voxel label map)	abdominal aortic segment (104–209 mm)	CTA	98 / 98 / 5 / 8	lumen + calcium mean Dice

Table 1: The six datasets label different structures, from 23 vessel ids to a single envelope mask, and each is scored against its own classes. The reference image shows the human label on one design-set case, and contrast is given over the reviewed cases.

Tool	What it does	Taurus-base	Taurus-tuned	Taurus-hybrid
Automated pass
Proposal	AI segmentation models (vessel model, whole-body organ model) or a bright core propose the aorta and branches	$\checkmark$	$\checkmark$	$\checkmark$
Trunk and sac growth	grows the trunk and sac; Taurus-tuned and Taurus-hybrid add a trunk tracker and a contour out to the aneurysm’s outer surface, where its wall meets the surrounding fat; Taurus-hybrid re-seeds the trunk when the labeled lumen is implausibly small or a brighter tube is left unlabeled, and checks the sac against a polar edge finder	$\checkmark$	$\checkmark$	$\checkmark$
Tissue classes	lumen, calcium, thrombus and wall from thresholds, which are fixed in Taurus-base and measured on the case in Taurus-tuned and Taurus-hybrid; Taurus-hybrid drops calcium that lies on the vertebral cortex	$\checkmark$	$\checkmark$	$\checkmark$
Branches, zones, flags	traces branches from the wall, names them and the aortic zones, raises quality warnings; Taurus-hybrid adds warnings for a rebuilt trunk, an under-sized sac, calcium specks and a header orientation that contradicts the anatomy	$\checkmark$	$\checkmark$	$\checkmark$
What the reviewer sees

Table 2: The harness toolkit and the Taurus versions that have each tool (

\checkmark

: yes; (

\checkmark

): in limited form; –: no).

	Primary metric per dataset						All	Per case
Approach	AortaSeg24 (4)	Siriapisith AAA/EVAR (12)	MosMedData AAA (8)	AVT (4)	VMR (4)	Jamshidian 4D-CTA (6)	all (38)	$	min
Taurus-base (auto)	0.572	0.496	0.021	0.027	0.490	0.002	0.321	–	0.7
Taurus-base	0.579	0.496	0.024	0.432	0.492	0.004	0.410	0.20	1.5
Taurus-tuned (auto)	0.732	0.783	0.462	0.437	0.464	0.534	0.620	–	1.0
Taurus-tuned	0.732	0.871	0.462	0.437	0.466	0.572	0.704	0.75	6.4
Taurus-hybrid (auto)	0.734	0.791	0.468	0.454	0.468	0.587	0.647	–	1.1

Table 3: On the held-out cases, Taurus-hybrid was ahead of both free-form agents on five of six datasets, and Free-form Sonnet led on AVT, where the automated pass found no usable trunk. Median primary metric per dataset (each in its own metric, so the pooled column is descriptive) and over all 38 cases, and model cost and minutes per case; for Taurus the minutes include the vessel proposal and the automated pass ( Appendix E ). Bold marks the highest median of the column, and asterisks mark medians that rest on two near-empty labels and on the calcium scoring convention ( Section 4.2 ). Spreads, the hard and typical cases and turns are in Appendix B .

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Comparison	Cases	Metric	Median $\Delta$	95 % CI	Better/ worse	$p$
Main experiment
base $-$ base (auto)	Siriapisith AAA/EVAR, 160	outer Dice	$+0.013$	0.000 to 0.025	92/15	$1.5\times 10^{-14}$
base $-$ base (auto)	AVT, 24	tree Dice	$+0.022$	0.001 to 0.049	21/1	0.0024
hybrid $-$ Free-form Opus	free-form, 30	primary metric	$+0.027$	–	21/6	0.001
		NSD at 1 mm	$+0.095$	–	23/6	0.0003
		MSD (mm)	$-0.53$	–	24/5	0.001

Appendix

Table 4: Paired comparisons behind the main-text Results: median difference of the first approach minus the second, bootstrap 95 % CI where computed, cases better and worse by more than 0.005 for the first approach, and Wilcoxon

p

. The two base-review rows of the main experiment are Holm-adjusted, all others unadjusted. Taurus approaches are named Taurus-base, Taurus-tuned and Taurus-hybrid, with (auto) marking the automated pass alone. For MSD, HD95 and terracing, lower is better.

Comparison	Dataset	$n$	median $\Delta$	95 % CI	$p$	$p$ (Holm)	better/worse
Taurus-base $-$ Taurus-base (auto)	AortaSeg24	100	+0.000	[+0.000, +0.000]	4.83e-05	0.00246	45/15
Taurus-base $-$ Taurus-base (auto)	Siriapisith AAA/EVAR	160	+0.013	[+0.000, +0.025]	2.62e-16	1.49e-14	92/15
Taurus-base $-$ Taurus-base (auto)	MosMedData AAA	98	+0.000	[+0.000, +0.000]	0.817	1	18/16
Taurus-base $-$ Taurus-base (auto)	AVT	24	+0.022	[+0.001, +0.049]	4.61e-05	0.0024	21/1
Taurus-base $-$ Taurus-base (auto)	VMR	17	+0.000	[+0.000, +0.002]	0.0754	1	8/3
Taurus-base $-$ Taurus-base (auto)	Jamshidian 4D-CTA	39	+0.000	[+0.000, +0.000]	0.291	1	10/12

Appendix

Table 5: Paired comparisons of the primary metric in the main experiment: median difference, bootstrap 95 % interval, Wilcoxon

p

, Holm-adjusted

p

and the number of cases better/worse for the first approach. Source: results/tables/T2_pairs.md .

Primary metric per dataset

All held-out cases / stratum

Cost per case

Approach

AortaSeg24 (4)

Siriapisith AAA/EVAR (12)

MosMedData AAA (8)

AVT (4)

VMR (4)

Jamshidian 4D-CTA (6)

all (38)

hard (26)

typical (12)

min

turns

Taurus- base (auto)

0.572

\pm

0.072

0.496

\pm

0.096

0.021

\pm

0.071

0.027

\pm

0.108

0.490

\pm

0.043

0.002

\pm

0.005

0.321

\pm

0.259

0.109

\pm

0.225

0.573

\pm

0.151

–

0.7

\pm

1.1

–

Taurus- base

0.579

\pm

0.076

0.496

\pm

0.096

0.024

\pm

0.076

0.432

\pm

0.405

0.492

\pm

0.037

0.004

\pm

0.004

0.410

\pm

0.260

0.144

\pm

0.222

0.640

\pm

0.146

0.20

\pm

0.44

1.5

\pm

4.2

\pm

Taurus- tuned (auto)

0.732

\pm

0.031

0.783

\pm

0.130

0.462

\pm

0.089

0.437

\pm

0.406

0.464

\pm

0.048

0.534

\pm

0.186

0.620

\pm

0.159

0.598

\pm

0.145

0.716

\pm

0.195

–

1.0

\pm

1.1

–

Taurus- tuned

0.732

\pm

0.032

0.871

\pm

0.037

0.462

\pm

0.104

0.437

\pm

0.406

0.466

\pm

0.064

0.572

\pm

0.222

0.704

\pm

0.202

0.698

\pm

0.199

0.716

\pm

0.196

0.75

\pm

0.23

6.4

\pm

4.7

\pm

Taurus- hybrid (auto)

0.734

\pm

0.031

0.791

\pm

0.135

0.468

\pm

0.105

0.454

\pm

0.428

0.468

\pm

0.047

0.587

\pm

0.180

0.647

\pm

0.154

0.619

\pm

0.137

0.725

\pm

0.202

–

1.1

\pm

1.3

–

Appendix

Table 6: Full held-out results on the 38 evaluation cases (26 hard, 12 typical), the long form of Section 3 : median primary metric per dataset and over all, hard and typical cases (each dataset in its own metric, so the pooled columns are descriptive), and LLM cost, minutes from scan to label (vessel proposal and automated pass, plus the review for the reviewed approaches) and turns per case;

\pm

half the interquartile range beneath each median. Bold marks the highest median of the column, and asterisks mark medians that rest on two cases where an approach left an almost empty label and on the calcium scoring convention. Figure 8 draws the paired differences, and Table 4 in the main text gives those quoted there. Taurus-base (auto) and Taurus-base form the paired control, re-run and re-reviewed on the same cases; Taurus-tuned starts from the frozen Taurus-tuned (auto) labels and Taurus-hybrid from its own automated pass, regenerated from the same proposals; Free-form Opus ran on all 38 cases. Taurus-hybrid was designed after the results of Taurus-tuned and of its own first build were seen, so its comparison is not blind. Source: results/tables/T8_A5.md , T11_v31_eval.md and T11_v31_eval_pairs.md .

Approach (cases)	Dice	NSD 1 mm	NSD 2 mm	MSD (mm)	HD95 (mm)
Taurus- base (auto) (38)	0.321 [0.018, 0.536]	0.118 [0.014, 0.366]	0.233 [0.040, 0.490]	12.73 [6.79, 18.68] (37)	29.9 [22.0, 54.5] (37)
Taurus- base (38)	0.410 [0.021, 0.542]	0.153 [0.026, 0.432]	0.290 [0.044, 0.575]	11.43 [5.84, 18.51] (37)	29.2 [19.8, 44.5] (37)
Taurus- tuned (auto) (38)	0.620 [0.451, 0.769]	0.393 [0.254, 0.560]	0.588 [0.458, 0.747]	2.59 [1.67, 6.51] (37)	14.4 [6.3, 23.5] (37)
Taurus- tuned (38)	0.704 [0.450, 0.854]	0.431 [0.276, 0.566]	0.662 [0.477, 0.768]	2.00 [1.43, 5.61] (37)	9.8 [5.5, 20.2] (37)
Taurus- hybrid (auto) (38)	0.647 [0.466, 0.773]	0.402 [0.255, 0.577]	0.582 [0.457, 0.756]	2.42 [1.64, 6.26] (37)	14.1 [7.1, 24.1] (37)
Taurus- hybrid (38)	0.805 [0.670, 0.899]	0.494 [0.371, 0.702]	0.761 [0.629, 0.860]	1.58 [1.07, 3.20]	8.0 [5.0, 14.0]

Appendix

Table 7: Boundary and surface quality on the 38 held-out cases against the human reference. Top and middle: median [IQR] per approach of Dice, NSD at 1 and 2 mm, mean surface distance (MSD), HD95, signed edge offset, lumen edge level, terracing and roughness, with the reference’s own edge level, terracing and roughness (

n

in brackets where a metric is undefined on some cases). Bottom: per-dataset medians of NSD at 1 mm and MSD. Per case each value is the mean over the dataset’s primary-metric classes; pooled medians are descriptive. Paired tests: Table 4 . Source: results/tables/T9_surface.md (block A) and T11_v31_eval.md .

#	Found in	Problem	Status
66	pilot, Siriapisith AAA/EVAR and Jamshidian 4D-CTA	blood value contaminated by calcium on a 4D phase; end slices refused by interp-outer ; card coordinates on the down-sampled grid; broad outer tools overshoot where no fat plane separates the aorta; false branches in crops	fixed (robust blood, end slices, coordinates)
67	pilot, whole-body	card coordinates relative to the working box; outer-trimming tools strip the wall ring; false arch-branch specks; working box too narrow; coverage warning on by-design lumen-only iliacs	fixed (wall ring, branch cleanup, box report)
68	pilot, chest scans	root recipes did nothing; card options computed before the regularization; branch-lumen pieces not removable; edits on the largest volume over 10 min	fixed (root tracker, options as differences, branch-remove ); latency on the largest volume open
69	held-out, Siriapisith AAA/EVAR and Jamshidian 4D-CTA	missing or collapsed sac runs; growth tools cap giant sacs, key-slice polygons needed; coverage check reported a gap as complete; vertebral branches survive the cleanup; a very bright phase left the lumen empty; guard rails refused calcium hysteresis and the regularization with an endoleak	open
70	held-out, Siriapisith AAA/EVAR and Jamshidian 4D-CTA	flat-cut sac ends and truncated trunks; growth over 30 % refused without forcing; new trunk voxels without a vessel id; tracker recipe crashed on its slice range; giant sac or hematoma undecidable	open; id and crash fixed during the run
71	held-out, MosMedData AAA	collar needs more than 10 % outer removal; side names reversed on header-flipped volumes; lumen on the vertebral body; branch tools refuse clicks inside the trunk; stent graft in a case listed without one	open

Appendix

Table 8: Issues found with Taurus-hybrid: 66–68 in the pilot on non-evaluation cases (fixed before the held-out run), 69–75 in the held-out reviews (kept out of the harness design; two generic code faults fixed during the run). Source: results/harness_issues.md .

Model	Input	Output	Cache read	Cache write (5 min)	Cache write (1 h)
Claude Opus 5.5 ( claude-opus-5-5 )	4.00	20.00	0.20	5.00	8.00
Claude Sonnet 5.5 ( claude-sonnet-5-5 )	2.00	10.00	0.20	2.50	4.00
Claude Sonnet 5 ( claude-sonnet-5 )	2.00	10.00	0.20	2.50	4.00

Appendix

Table 9: Anthropic’s listed token prices for usage (October 2026), in US dollars per million tokens.

Dataset (held-out cases)	L0 human	L1 Taurus-base (auto)	L2 Taurus-base	L2-noedit	L3 reference-seeded
MosMedData AAA (20)	0.804 $\pm$ 0.039	0.507 $\pm$ 0.006	0.522 $\pm$ 0.005	0.679 $\pm$ 0.025	0.723 $\pm$ 0.004
Siriapisith AAA/EVAR (40)	0.975 $\pm$ 0.005	0.583 $\pm$ 0.008	0.699 $\pm$ 0.014	0.555 $\pm$ 0.020	0.977 $\pm$ 0.000 a

Appendix

Table 10: Models trained on Taurus-base labels fell well short of models trained on human labels. Mean over the held-out cases of the dataset’s primary metric (MosMedData AAA: mean of lumen and calcium Dice over the classes present; Siriapisith AAA/EVAR: outer Dice), mean

\pm

s.d. over three training seeds, per training label source. The AortaSeg24 runs are still in progress. a Two seeds; the third is still training.

Automated Data Annotation

See all 23 papers

Oct 22, 2026

nnInteractive Agent: General-Purpose AI Agents as 3D Image Annotators

Interactive 3D segmentation models such as nnInteractive turn a few spatial prompts (points, scribbles, boxes) into accurate volumetric masks, but a human still does the annotation work: finding each structure, placing prompts, checking the result and correcting it. General-purpose AI agents can now interpret images, call tools and write code. We show that such agents can take over this work without additional training: the user makes a request in plain language, and the agent works out what has to be segmented, locates the targets, prompts nnInteractive, inspects the 3D result, corrects it and responds to feedback. Asked for 'the muscles used for chewing', an agent identifies and segments the masseters and pterygoids; asked to count liver lesions, it writes its own code to find candidates and splits a merged lesion on request. Our open-source nninteractive-agent provides nnInteractive as tools over the Model Context Protocol (MCP), together with a skill that encodes how experienced users prompt it. On a benchmark spanning CT, MRI, PET and micro-CT of humans, a mouse and an ant (14 images; 51 targets from organs and tumours to muscles, foot bones, vessel trees and an insect brain), Claude Code and Codex reach a mean Dice of 0.81 and 0.84 from the request alone; when told where each target is, both reach 0.85 and prompt nearly as effectively as an oracle that places prompts using the reference segmentation. A text-prompted 3D model (VoxTell) is more accurate on most standard radiological targets but less robust: it returns an empty mask for one of them and fails on the leg muscles, the foot bones, the mouse and the ant, where the agents remain accurate. Code: https://github.com/MIC-DKFZ/nninteractive-agent

Fabian Isensee, Constantin Ulrich Harsy, Maximilian Rokuss +3

Division of Medical Image Computing (MIC), German Cancer Research Center (DKFZ), Heidelberg, Germany · National Center for Tumor Diseases (NCT) Heidelberg, a partnership between the German Cancer Research Center (DKFZ) and Heidelberg University Hospital (UKHD), Heidelberg, Germany · Helmholtz Imaging, German Cancer Research Center (DKFZ), Heidelberg, Germany +4

Medical AI Agents Tool-Using Agents Interactive Medical Image Segmentation 3D Medical Image Segmentation Human-in-the-Loop Annotation

Ask PDF

Jun 21, 2026

Large Language Model-Assisted Cleaning of Report-Derived Labels in a Large-Scale Chest CT Dataset

Purpose: To evaluate whether large language model (LLM)-assisted label cleaning can identify label-report discordance in CT-RATE, a large-scale public chest CT dataset. Materials and Methods: After report-level deduplication, 24,446 unique radiology reports were identified. Twelve reports were excluded from the primary GPT-5.4 analysis because of Microsoft Azure AI Foundry content-safety filtering, leaving 24,434 reports and 439,812 label instances across 18 abnormality categories. GPT-5.4-derived binary labels were generated from report text using structured JSON output and compared with existing CT-RATE labels. Discordant instances were adjudicated by radiologists. In addition, 100 randomly sampled reports were manually annotated to compare CT-RATE labels, individual LLM-derived labels, and multi-LLM majority-vote labels against radiologist-annotated reference labels. Results: Overall agreement between GPT-5.4-derived and CT-RATE labels was 96.4%, with Cohen's kappa of 0.884. Lymphadenopathy showed the lowest agreement and kappa. In discordance review, radiologist adjudication supported GPT-5.4-derived labels in 72 of 97 (74.2%) general discordant instances and 91 of 99 (91.9%) targeted lymphadenopathy discordant instances. Against radiologist-annotated reference labels, multi-LLM majority-vote labels achieved the highest label-macro-averaged F1 score and Cohen's kappa. Conclusion: LLM-assisted label cleaning identified clinically meaningful label-report discordance in CT-RATE and may support scalable quality improvement of public imaging datasets. The cleaned dataset will be made publicly available to support future research.

Yosuke Yamagishi, Atsushi Takamatsu, Mototsugu Sato +4

Division of Radiology and Biomedical Engineering, Graduate School of Medicine, The University of Tokyo, Tokyo, Japan · Department of Computational Diagnostic Radiology and Preventive Medicine, The University of Tokyo Hospital, Tokyo, Japan · Department of Radiology, Kanazawa University Hospital, Ishikawa, Japan +3

Training Data Curation LLM-Assisted Annotation Computed Tomography Medical Image Benchmarks

Ask PDF

May 27, 2025

Good Enough? An Investigation on the Impact of Label Quality in Large-Scale Medical Datasets

Manually refining radiological segmentation masks is highly resource-intensive. To determine when this expert commitment is truly justified for the training of segmentation models, we investigate the relationship between label quality and model performance. Expanding beyond models trained directly for inference, we conduct the first study isolating the impact of label quality in pre-training datasets. While high-quality labels remain essential for models proceeding directly to deployment, we find no evidence that strict label quality is crucial for pre-training efficacy. These results question the necessity of exhaustive human-in-the-loop refinement for massive corpora intended for pretraining and suggest that expert effort is more effectively invested in well-curated downstream target datasets.

Alexander Jaus, Zdravko Marinov, Constantin Seibold +4

Karlsruhe Institute of Technology, Germany · HIDSS4Health - Helmholtz Information and Data Science School for Health, Karlsruhe/Heidelberg, Germany · Diagnostic and Interventional Radiology, Heidelberg University Hospital +1

Image Segmentation Medical Imaging Foundation Models Healthcare Medical Image Segmentation Learning with Noisy Labels Medical Image Representation Learning

Ask PDF

Abstract

Automated Data Annotation

Explore similar work

nnInteractive Agent: General-Purpose AI Agents as 3D Image Annotators

Large Language Model-Assisted Cleaning of Report-Derived Labels in a Large-Scale Chest CT Dataset

Good Enough? An Investigation on the Impact of Label Quality in Large-Scale Medical Datasets