AnyCXR: Human Anatomy Segmentation of Chest X-ray at Any Acquisition Position using Multi-stage Domain Randomized Synthetic Data with Imperfect Annotations and Conditional Joint Annotation Regularization Learning

Zifei Dong; Wenjie Wu; Jinkui Hao; Tianqi Chen; Ziqiao Weng; Bo Zhou

cs.CVDec 19, 2025

AnyCXR: Human Anatomy Segmentation of Chest X-ray at Any Acquisition Position using Multi-stage Domain Randomized Synthetic Data with Imperfect Annotations and Conditional Joint Annotation Regularization Learning

Authors: Zifei Dong, Wenjie Wu, Jinkui Hao, Tianqi Chen, Ziqiao Weng, Bo Zhou

Organizations: Department of Radiology, Northwestern University, Chicago, IL, USA · Data Science Institute, Vanderbilt University, Nashville, TN, USA · Department of Orthopedics, The Second Hospital of Shanxi Medical University, Taiyuan, Shanxi, P.R. China · Second Clinical Medical College, Shanxi Medical University, Taiyuan, Shanxi, P.R. China ·

Abstract

Robust anatomical segmentation of chest X-rays (CXRs) remains challenging due to the scarcity of comprehensive annotations and the substantial variability of real-world acquisition conditions. We propose AnyCXR, a unified framework that enables generalizable multi-organ segmentation across arbitrary CXR projection angles using only synthetic supervision. The method combines a Multi-stage Domain Randomization (MSDR) engine, which generates over 100,000 anatomically faithful and highly diverse synthetic radiographs from 3D CT volumes, with a Conditional Joint Annotation Regularization (CAR) learning strategy that leverages partial and imperfect labels by enforcing anatomical consistency in a latent space. Trained entirely on synthetic data, AnyCXR achieves strong zero-shot generalization on multiple real-world datasets, providing accurate delineation of 54 anatomical structures in PA, lateral, and oblique views. The resulting segmentation maps support downstream clinical tasks, including automated cardiothoracic ratio estimation, spine curvature assessment, and disease classification, where the incorporation of anatomical priors improves diagnostic performance. These results demonstrate that AnyCXR establishes a scalable and reliable foundation for anatomy-aware CXR analysis and offers a practical pathway toward reducing annotation burdens while improving robustness across diverse imaging conditions.

Figures & tables

Figure 1: AnyCXR Data Pipeline: This workflow projects 3D CT volumes into diverse 2D Digitally Reconstructed Radiographs (DRRs) and segmentation masks using Multi-stage Domain Randomization. The pipeline includes an automated filtering step that discards invalid masks (red). The feature plot (bottom left) demonstrates that this method generates a data distribution significantly broader than typical real-world datasets.

Figure 2: MSDR Pipeline: This flow chart details the Multi-stage Domain Randomization engine, which converts 3D CT data into 2D synthetic DRRs through a sequence of stochastic steps. The pipeline involves Pre-projection Randomizations acting on Hounsfield Units and image artifacts, In-projection Randomizations modifying geometry, and Post-projection Randomizations simulating detector response and inverting polarity. The red boxes denote randomization names, while the blue boxes denote different approaches.

Figure 4: DRR Augmentation Demonstration for Major Innovation: This figure visually illustrates the effect of the primary Multi-stage randomization on the resulting DRR, starting from a Default DRR (left panel). Each subsequent column shows a different major 3D randomization applied in isolation: Bone-to-soft-tissue, Inter-bone, Intra-bone, Soft-tissue, and Projection geometry randomization, demonstrating how these 3D perturbations introduce diverse image appearances and contrast variations.

Figure 5: CAR Framework Overview: Input X-rays are segmented by a U-Net, with predictions substituting missing ground truth labels. Conditioned on image $\mathbf{X}$ , these maps are encoded by $G_{\theta_{1}}$ to align their latent distributions via $\mathcal{L}_{\text{Dist}}$ (optimizing the U-Net), while the decoder $G_{\theta_{2}}$ ensures latent integrity via the reconstruction loss $\mathcal{L}_{\text{Recon}}$ .

Figure 6: Segmentation Comparison on Synthetic DRR: This figure presents a side-by-side comparison of ground truth and model Prediction for major anatomical groups on the TotalSegmentator dataset. White circles indicate specific regions where the prediction deviated from the ground truth. While overall organ localization is highly accurate, the primary challenge is focused on subtle boundary adherence and fine detail resolution. The segmentation errors, particularly the fine organ boundaries of lungs and hearts, stem from the inherent difficulty in precisely delineating edges in the 2D projection plane, where overlap is severe and the model must rely heavily on learned anatomical priors to infer invisible or faint structures.

Figure 7: Detailed Comparison of Segmentation on Real-World CXRs and Expert Refinement: This figure presents a comparison of the model’s prediction against the expert corrections on real PA and LA view radiographs, illustrating the editing required to obtain clinically acceptable masks. White circles indicate specific regions where the prediction deviated from the expert annotation. While AnyCXR provides a highly effective initial segmentation proposal, these differences reflect areas of necessary expert editing. Specific challenges arise from image contrast variations and overexposure in clinical data, making detailed editing difficult, particularly for fine structures like the ribs and lungs. Furthermore, a non-trivial portion of the divergence relates to the inherent clinical ambiguity of 2D projection, such as precise lung lobe placement and boundary definition in superimposed regions, where the expert’s final delineation corrects the prediction based on clinical consensus.

View	Method	CAR	Overall $\uparrow$	Heart & Vessels $\uparrow$	Lung $\uparrow$	Rib $\uparrow$	Vertebrae $\uparrow$	Other Bone $\uparrow$
PA	DenseUNet	Off	0.825	0.861	0.890	0.752	0.795	0.829
PA	DenseUNet	On	0.844	0.872	0.900	0.783	0.825	0.843
PA	SegFormer	Off	0.841	0.875	0.916	0.742	0.821	0.852
PA	SegFormer	On	0.854	0.882	0.921	0.768	0.836	0.865
PA	Rad-DINO	Off	0.857	0.897	0.930	0.742	0.862	0.854
PA	Rad-DINO	On	0.863	0.892	0.934	0.747	0.874	0.869

Table 4: DenseUNet, SegFormer, and Rad-DINO performance of DSC with and without CAR

Figure 8: Ablation Study of the CAR model: This chart compares the DSC performance of the Full Aug model (MSDR + PostHoc augmentations) against the final CAR + Aug framework. The CAR framework consistently improves performance across all groups, with the largest gain ( $+1.51\%$ ) seen in the challenging Lateral Ribs group, validating CAR’s efficacy in enforcing anatomical priors where augmentation alone is insufficient.

Figure 9: Real CXR Segmentation Performance Comparison: This chart compares the DSC performance of FluoroSAM against AnyCXR on expert-refined real CXR annotations across PA and LA views. AnyCXR achieves substantially higher DSC scores across all evaluated organ groups, demonstrating stronger generalization to real CXR segmentation.

Figure 10: Segmentation comparison of FluoroSAM and AnyCXR on CXR images in PA and LA views.

Figure 11: Scaling Synthetic Supervision: This figure demonstrates the DSC of Full MSDR with and without CAR across different configurations. The 500,000 and 1,000,000 DRR configurations are generated under relaxed QC thresholds.

Figure 12: Visualization for CTR disease detection: This figure demonstrates the utility of the segmentation model for automatic Cardiothoracic Ratio (CTR) assessment in the PA view. The left panel shows the delineated heart and the measurement lines used to calculate the CTR score, distinguishing between a normal case and a pathological case. The right panels illustrate the ROC curves and distribution of derived CTR scores, achieving an overall AUC of 0.93 with statistically significant separation between normal and cardiomegaly subjects across age and gender groups.

Figure 13: Visualization for SPCA disease detection: This figure demonstrates the clinical utility of instance-level vertebral segmentation for preliminary Spine Curvature Assessment (SPCA). The left panel illustrates the identification and tracking of thoracic vertebral centroids to measure deviation, distinguishing between a normal spine and a case with probable scoliosis. The center and right panels show the distribution of SPCA scores and the ROC curve, achieving an AUC of 0.87, which validates its effectiveness as a screening tool for spinal deformities.

Pathology	Raw CXR	CXR w/mask	$\Delta$ AUC	Improve %
Mean AUROC	$80.26\%$	$82.30\%$	$2.04\%$	$2.54\%$
Pleural Thickening	$76.83\%$	$80.13\%$	$3.30\%$	$4.30\%$
Atelectasis	$81.09\%$	$84.45\%$	$3.36\%$	$4.14\%$
Mass	$86.46\%$	$89.56\%$	$3.10\%$	$3.59\%$
Cardiomegaly	$90.14\%$	$93.33\%$	$3.19\%$	$3.54\%$
Effusion	$87.45\%$	$90.39\%$	$2.94\%$	$3.36\%$

Table 5: Segmentation-Guided Chest X-ray Disease Classification

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 14: Detailed distribution of each organ after filtering

Figure 15: Segmentation Comparison on Synthetic Oblique View DRRs

Figure 16: Segmentation comparison of FluoroSAM and AnyCXR on synthetic DRR images in PA and LA views.

Figure 17: Segmentation comparison of Rad-DINO and AnyCXR on real CXR and synthetic DRR images in PA and LA views.

Medical Image Segmentation

Medical image segmentation delineates organs, lesions, and other structures in scans to support measurement, diagnosis, treatment planning, and research.

See all 788 papers

CheXanatomy: Anatomy-Aware Vision-Language Modeling for Chest Radiographs
Jun 7, 2026Sergios Gatidis, Curtis Langlotz, Christian BluethgenRadiology VLMs Medical Image Segmentation
Stanford Center for Artificial Intelligence in Medicine and Imaging, Stanford University, Palo Alto, CA, USA · Department of Radiology, Stanford University, Stanford, CA, USA
Clinically Aware Synthetic Image Generation for Concept Coverage in Chest X-ray Models
Mar 16, 2026Amy Rafferty, Rishi Ramaesh, Ajitha RajanSynthetic Data Augmentation Medical Image Synthesis
University of Edinburgh, UK · NHS Lothian, UK
RadGenome-Anatomy: A Large-Scale Anatomy-Labeled Chest Radiograph Dataset via Physically Grounded Volumetric Projection
May 17, 2026Shuchang Ye, Mingyuan Meng, Hao Wang +2Medical Imaging Medical Image Segmentation
The University of Sydney · Zhongguancun Academy · Shanghai Jiao Tong University +1

Jun 7, 2026J/K move · Enter open · S save

CheXanatomy: Anatomy-Aware Vision-Language Modeling for Chest Radiographs

Sergios Gatidis, Curtis Langlotz, Christian Bluethgen

Stanford Center for Artificial Intelligence in Medicine and Imaging, Stanford University, Palo Alto, CA, USA · Department of Radiology, Stanford University, Stanford, CA, USA

Radiology VLMs Medical Image Segmentation Medical Image Representation Learning

Open paper AskPDF arXiv

Vision-language models (VLMs) pretrained on large-scale image-text pairs demonstrate strong image-level understanding, but are primarily optimized for global alignment and do not explicitly encode fine-grained anatomical structure, limiting their suitability for spatially precise tasks such as segmentation. We introduce CheXanatomy, a framework that integrates explicit anatomical knowledge into a pretrained VLM through autoregressive token-space supervision. Instead of adding task-specific decoder heads, the model is trained to generate anatomical segmentation masks via next-token prediction. To enable scalable supervision, we synthesize realistic chest radiographs from CT volumes and forward-project CT segmentation labels to obtain anatomically consistent 2D masks. We evaluate the approach on synthetic and real chest radiographs against a U-Net baseline, including ablations on model scale, input resolution, and vision encoder fine-tuning. Autoregressive anatomical supervision achieves performance comparable to specialized convolutional models in-distribution and demonstrates improved geometric robustness under domain shift to real CXR data. In addition, anatomy-pretrained models exhibit improved sample efficiency when adapting to novel localization tasks under limited supervision. Larger models and higher input image resolution improve performance, while vision encoder fine-tuning has limited effect. These results show that embedding anatomical structure directly into the generative objective promotes spatially grounded representations and supports anatomy-aware medical vision-language modeling.

Abstract

Medical Image Segmentation

Explore similar work

CheXanatomy: Anatomy-Aware Vision-Language Modeling for Chest Radiographs

Clinically Aware Synthetic Image Generation for Concept Coverage in Chest X-ray Models

RadGenome-Anatomy: A Large-Scale Anatomy-Labeled Chest Radiograph Dataset via Physically Grounded Volumetric Projection