Grounding Radiology Reports in 3D Chest CT: Results of the ReXGrounding Challenge
Authors: Mohammed Baharoon, Mahmoud Hussain Alabbad, Sung Eun Kim, Xiaoman Zhang, Lucas Bijnens, Zirong Chen, Jiayu Wang, Xi Ouyang, Mingyang Yu, Weiqiu Jin, Xiaodan Ye, Dijia Wu, Kwang-Hyun Uhm, Inhwa Son, Rachel van 't Hull, Tim Veenboer, Cees Snoek, Kevin Groot Lipman, Daesung Kim, Junha Im, Youngjun Jun, Sangwon Lee, Katelyn Morrison, Adam Perer, Mijin Yun, Ali Piracha, Reem Alwasiah, Talal Yahyan, Abdulrahman Alamoudi, Mashael Alshehri, Rayan Alsager, Hannah Hancock, Anna M. Hunkemöller, Peter Youssef, Mona A. Alhammad, Afnan Aljumayan, Brecht Van Berkel, Eric Bijnens, Saud Alhussain, Maarten Steyvers, Rolf Symons, Bryan Eelen, Ayed Alaklubi, Maha Buraykan Alshammari, Amirah Fahad Alshammeri, Tomas Gilvydis, Ibrahim AlTurki, Monerah Almohideb, Ali Alburkani, Shahad Alhomaiani, Fatima Jabali, Maximilian Rokuss, Moritz Langenberg, Yannick Kirchhoff, Benjamin Hamm, Klaus Maier-Hein, Ibrahim Ethem Hamamci, Sezgin Er, Michael Moritz, Hassan R. Alomaish, Luyang Luo, Pranav Rajpurkar
Segmenting each reported finding in a radiology scan would allow findings to be measured and tracked over time and let AI systems generate reports that are verifiable, but so far AI models have performed poorly at this task. To advance this task, we organized the ReXGrounding Challenge at MICCAI 2026, the first challenge dedicated to it. Given a non-contrast chest CT scan and the set of findings from its report, participants trained models to produce the corresponding set of 3D segmentations. The challenge used 2,992 training and 200 validation scans from the ReXGroundingCT dataset, and evaluated methods on a held-out test set of 300 scans with 582 findings, exhaustively annotated by board-certified radiologists. In total, 48 teams from 15 countries made 760 submissions, and the best method reached a Dice of 0.375 and a hit rate of 0.77, matching agreement between two independent radiologist groups (Dice 0.32, hit rate 0.76, on a 50-scan subset). On the 100 scans of the pre-challenge benchmark, the winner reached a Dice of 0.390, against 0.285 for VoxTell, the best model before the challenge, which five of the six best teams fine-tuned. Non-focal findings remained far harder to segment than focal ones, instance F1 stayed below inter-reader agreement (0.34 vs. 0.47), and models sometimes segmented the right abnormality in the wrong location. Top methods thus match radiologist agreement in Dice, but non-focal findings, findings with multiple instances, and adherence to the anatomical location described in the text remain open problems.
Figures & tables
Figure 1: Overview of the ReXGrounding Challenge. A) The task: given a non-contrast chest CT volume and one free-text finding from its report, a model must segment that finding in 3D. Each sentence is a separate query against the same volume, and the shown masks are the radiologist reference annotations of a test case (scan from CT-RATE ( Hamamci et al., 2026b ) ). B) Challenge workflow. During the development phase, teams trained on the ReXGroundingCT training set (2,992 scans with partial-instance masks) and the validation set (200 scans with exhaustive radiologist masks), and submitted predictions for the test set (300 scans, all visible to participants, with reference masks withheld), and the leaderboard reported scores on a fixed half of 150 scans (filled half; the dashed half was scored only after the deadline). After the deadline, each team’s final submission was scored on all 300 scans and the final ranking was announced at MICCAI 2026. C) Participating teams by country (48 teams from 15 countries). Circle area is proportional to the number of teams. Map lines are from Natural Earth (naturalearthdata.com, version 5.1.1) and do not necessarily depict accepted national boundaries.
Figure 2: Challenge data. A) Finding categories as a share of all findings in the training set (left, 7,687 findings in 2,992 scans) and the test set (right, 582 findings in 300 scans), grouped into focal and non-focal categories. Numbers give the share in percent. B) Reference volume of every test-set finding by category (dots). For categories with at least five findings, the bar spans the interquartile range and the open circle marks the median. The dashed line marks 1 mL. C) Number of findings per scan. D) Length of the finding sentences. In C and D, dark bars show the test set and light bars the training set.
Figure 3: Mean Dice per finding of the ten best teams by finding category on the full test set. Numbers in cells give the Dice, and colour shows how far each team is above (blue) or below (orange) the mean of the ten teams for that category. The last column gives that mean. Numbers in brackets give the number of test findings. Pneumothorax (1 finding) and other focal findings (2) are included in the focal total but not shown.
Figure 4: Submissions over the development phase. A) Scored submissions per week, in seven-day windows ending at the deadline. B) Dice on the leaderboard half of the test set for every scored submission (dots) and the best score reached so far (line). Dashed lines mark the opening of the leaderboard and the submission deadline.
Team
Dice [95% CI]
Hit rate [95% CI]
Instance F1 [95% CI]
Team 1
0.3755 [0.352–0.399]
0.7680 [0.732–0.802]
0.3410 [0.312–0.369]
Team 2
0.3654 [0.344–0.387]
0.7629 [0.730–0.797]
0.3428 [0.316–0.371]
Team 3
0.3621 [0.342–0.383]
0.7646 [0.737–0.795]
0.3336 [0.306–0.364]
Team 4
0.3605 [0.339–0.382]
0.7766 [0.744–0.809]
0.3116 [0.285–0.341]
Team 5
0.3526 [0.330–0.375]
0.7457 [0.713–0.781]
0.1899 [0.166–0.216]
Team 6
0.3519 [0.332–0.373]
0.7646 [0.732–0.799]
0.2435 [0.217–0.271]
Table 1: Final ranking on the full 300-scan test set (top 10). Teams are numbered by their final rank. CI, 95% bootstrap confidence interval over the test scans (1,000 replicates, Section 3.4 ). The full leaderboard is in C .
Figure 5: Examples of successful grounding by the three winning teams. Masks are rendered in 3D inside the lungs and viewed from the front (patient’s right on the left). The reference is shown in green, solid in the first column and translucent in the others. Predicted surfaces are blue where they lie within 3 mm of the reference and orange elsewhere. Lesions smaller than 1 mL are circled. Dice is computed over the whole 3D mask of the finding.
Figure 6: Examples of grounding failures by the three winning teams, shown as in Figure 5 . In a, other reference findings of the same scan are shown in grey: two of the teams segmented a consolidation that the report describes as a separate finding in the right upper lobe.
Figure 7: Detection, finding size and prompts. A) Hit rate against mean Dice on localized findings (Dice ≥ 0.1) for the ten best teams. Numbers give the final rank. The three winning teams are shown in darker blue. B) Dice (dark) and hit rate (light) by reference volume of the finding, pooled over the ten best teams. C) Prompt sensitivity analysis. Dice on the findings changed by each edit, with the original sentence (open) and the edited sentence (filled), averaged over Teams 1 to 4. Numbers give the number of edited findings.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 1: Team 1: the pretrained CIRCLE image encoder combined with a VoxTell-based decoder. The image encoder and segmentation decoder are fine-tuned, and the text encoder is frozen. Inset F shows the text–image fusion within each decoder stage.
Figure 2: Team 3: twelve VoxTell models and five TAP-CT seeds, each conditioned on the finding sentence, produce probability maps that are averaged per branch and fused 0.8/0.2, followed by four post-processing steps.
Figure 3: Team 4: semi-supervised training on CT-RATE (stage 2). Voxel logits for the finding are pooled into a presence score trained against whether the finding is present, and thresholded predictions of an EMA teacher serve as pseudo-masks.
Instance
Team
Institution
Country
Dice
Hit rate
Precision
Recall
F1
Submissions
Team 1
United Imaging Intelligence, ShanghaiTech University, Zhongshan Hospital
China
0.3755 [0.352–0.399]
0.7680 [0.732–0.802]
0.2720 [0.242–0.300]
0.4570 [0.413–0.502]
0.3410 [0.312–0.369]
6
Team 2
Gachon University, MEDAI
South Korea
0.3654 [0.344–0.387]
0.7629 [0.730–0.797]
0.2797 [0.251–0.312]
0.4427 [0.405–0.482]
0.3428 [0.316–0.371]
19
Team 3
Netherlands Cancer Institute
Netherlands
0.3621 [0.342–0.383]
0.7646 [0.737–0.795]
0.2537 [0.226–0.287]
0.4869 [0.451–0.522]
0.3336 [0.306–0.364]
38
Team 4
Yonsei University, NewcureM
South Korea
0.3605 [0.339–0.382]
0.7766 [0.744–0.809]
0.2419 [0.215–0.273]
0.4377 [0.398–0.477]
0.3116 [0.285–0.341]
62
Team 5
Deepnoid
South Korea
0.3526 [0.330–0.375]
0.7457 [0.713–0.781]
0.1174 [0.100–0.137]
0.4969 [0.460–0.533]
0.1899 [0.166–0.216]
21
Appendix
Table 1: Full leaderboard on the 300-scan test set, ordered by mean Dice. Teams are numbered by their final rank. Brackets give 95% bootstrap confidence intervals (1,000 replicates, Section 3.4 ). Submissions counts all submissions a team made to the leaderboard. Institutions are those of all team members and countries as given at registration, and – marks information that was not provided.
Figure 1: Ranking stability, from 1,000 bootstrap replicates of the 300 test scans. A) Rank of the twenty best teams across replicates. Circle area is the share of replicates with that rank, the line spans the central 95% of ranks, and the dashed diagonal marks the observed rank; the three winning teams are shown in darker blue. B) Pairwise comparisons among the ten best teams, labelled by rank. Each number is the Holm-adjusted p -value of a one-sided paired Wilcoxon test that the higher-ranked team scored higher, and filled cells are significant ( p<0.05 ). Confidence intervals of the mean Dice are given in Table 1 .
Figure 1: Examples from the prompt sensitivity analysis (Team 2). Each row shows one test finding segmented with its original sentence (left) and with edited sentences. Masks are rendered in 3D inside the lungs and viewed from the front (patient’s right on the left). The reference is shown in translucent green, and predicted surfaces are blue where they lie within 3 mm of the reference and orange elsewhere. Circles mark masks smaller than 1 mL. Dice is computed against the unchanged reference, so a prompt for one side of a bilateral finding (c) can cover at most part of it.
Figure 2: The findings of Figure 1 segmented by Team 1.
Group
Scans
Findings
Dice
Hit rate
Female
127
243
0.359 [0.330–0.384]
0.771 [0.704–0.811]
Male
173
339
0.346 [0.330–0.369]
0.732 [0.678–0.767]
Under 40 years
142
249
0.376 [0.351–0.403]
0.783 [0.727–0.811]
40–59 years
113
218
0.328 [0.314–0.346]
0.736 [0.670–0.771]
60 years and over
45
115
0.332 [0.293–0.363]
0.697 [0.643–0.730]
Appendix
Table 1: Performance of the ten best teams by patient sex and age group on the 300 test scans. Values are the mean over the ten teams, with the range across teams in brackets.
Item
Description
Location
1
Title conveys the essential information on the challenge
Title
2
Abstract: purpose, design and main results
Abstract
3
Keywords
Keywords
4a
Biomedical motivation
Section 1
4b
Technical motivation
Section 1
4c
Purpose of the challenge and gap addressed
Section 1
Appendix
Table 1: Location of each BIAS checklist item in this paper.
Accurate 3D abnormality segmentation in chest CT requires dense spatial supervision, but obtaining expert voxel-level labels is costly. Radiology reports, however, are routinely generated during clinical interpretation and contain instance-specific descriptions that can provide additional guidance without new dense annotation. Existing vision-language grounding methods typically require report-derived findings at inference, making localization dependent on paired text and limiting each forward pass to a queried finding. We propose Instance-Guided Report Anchoring (IGRA), a model-agnostic module that preserves the correspondence between each annotated abnormality instance and the report finding that describes it. IGRA pools each instance representation and anchors it to the corresponding finding embedding during training; all text-related components are discarded at inference. We further reformulate free-text grounding on ReXGroundingCT as multi-label volumetric segmentation by merging same-category instances, allowing all abnormality categories to be predicted in one image-only forward pass. IGRA improves Dice by 22.5% over the strongest image-only baseline (30.93 vs. 25.25) and is comparable to VoxTell on the single-finding subset (30.29 vs. 30.43). Applied unchanged to four standard 3D segmentation backbones, IGRA improves Dice and hit rate across all architectures. Zero-shot evaluation on LIDC-IDRI, PleThora, and a private in-house dataset further shows consistent gains over image-only baselines.
Zhenyu Bu, Haoyan Ding, Chushu Shen +7
Siemens Medical Solutions USA, Inc., Malvern, PA, USA · The Ohio State University, Columbus, OH, USA · University of California, Los Angeles, Los Angeles, CA, USA +2
Automatic voxel-level grounding of free-text findings in 3D chest Computed Tomography (CT) is critical for clinical interpretability. However, this task remains highly challenging due to the intricate spatial complexity of large 3D volumes and the heterogeneity of free-text findings. Existing end-to-end approaches often struggle to simultaneously learn the localized feature representations required for accurate 3D segmentation and the complex semantic understanding needed for text alignment, leading to suboptimal grounding performance. To overcome this fundamental limitation, we propose a novel decoupled framework that disentangles the problem into two specialized stages: (1) class-agnostic lesion segmentation and (2) text-volume reasoning. This structural separation allows the model to first extract candidate sub-volumes by localizing potential abnormalities. Subsequently, intensive cross-modal reasoning is performed to align these localized sub-volumes with free-text medical findings. To resolve the spatial ambiguities inherent in local regions, the reasoning module is augmented with explicit anatomical guidance, utilizing relative spatial coordinates and lung lobe priors. Evaluated on the ReXGroundingCT benchmark, our method achieves state-of-the-art performance in overall grounding quality on the official leaderboard. These results demonstrate that decoupling detection from reasoning is a highly effective paradigm for handling the complexity of 3D medical visual grounding. Our code is publicly available at https://github.com/khuhm/DAGG.
Kwang-Hyun Uhm, Inhwa Son, Sung-Jea Ko
Department of Artificial Intelligence, Gachon University, Republic of Korea · MEDAI, Republic of Korea
Visual grounding models can enhance radiology workflows by linking report findings to image regions. This is particularly valuable for 3D CT, where findings often occupy a tiny fraction of the volume. Training 3D grounding models requires large sets of paired phrases and regions, and building such datasets is expensive, requiring radiologists to annotate images by hand. We posit that this supervision is already created implicitly during routine reporting, as radiologists frequently place 2D annotations (e.g., distance measurement, arrows) on key images to make measurements and to support report interpretation. We introduce an automated pipeline that converts these routine clinical annotations into large-scale phrase-region supervision for 3D visual grounding. The pipeline links each annotation to the corresponding finding in the report through metadata matching, then uses a promptable 3D segmentation model to convert the 2D annotation into a volumetric mask. This produces phrase-mask-volume datasets without requiring additional radiologist annotation. Applied to a single institution's clinical picture archiving and communication system (PACS), our approach generated 105K phrase-mask-volume triplets from 59K abdominal CT exams. We also introduce two abdominal CT grounding benchmarks, LocusBench-Onc and LocusBench-ED, which comprise 240 oncology and 260 emergency-department radiologist-reviewed phrase-mask-volume triplets, respectively, with the latter spanning 13 distinct categories such as appendicitis, hematoma, and hernia. We further introduce LocusCT, a 3D visual grounding model trained on this dataset, which achieves hit rates of 0.725 on LocusBench-Onc and 0.773 on LocusBench-ED, substantially outperforming comparator models. These results show that routine PACS annotations are a scalable, previously unused source of supervision for 3D visual grounding.
Sam Church, Danyal Maqbool, Joshua D. Warner +4
University of Wisconsin–Madison, Department of Computer Sciences, Madison, WI USA · University of Wisconsin–Madison, Department of Radiology, Madison, WI USA