Raysemble: Metric 3D Bone Reconstruction from a Single X-ray
Authors: Xiaoyang Li, Xinghao Yan, Yuan Chai
Organizations: ORBIT Lab, College of Medicine and Biomedical Information Engineering, Northeastern University, Shenyang 110016, China · The John Curtin School of Medical Research, College of Science and Medicine, Australian National University, Canberra 2601, Australia
Raysemble reconstructs eight lumbar and pelvic bones from a single X-ray by separating shape and placement. Each 3D query samples regional and global features at its image projection. A shared decoder predicts normalized bone shapes. Predicted depth and camera geometry determine 3D centers and scales for resampling these shapes onto a common grid in millimeters. Evaluation uses 23 held-out patients with 43 CT scans, each generating one simulated anteroposterior radiograph. Without rigid alignment to CT,Raysemble leads 15 evaluated methods, averaging 65.24% Dice, 9.09 mm HD95, and 3.32 mm ASSD over three random seeds. With identical inputs, it improves Dice over VolumeNeRF by 5.86 percentage points.
Figures & tables
Figure 1: From one AP radiograph to eight bones. (a) The reconstruction task. (b) Depth ambiguity, projection magnification, and anatomical overlap. (c) Raysemble estimates normalized bone shape and metric placement in two branches. The predicted center and scale transform local occupancies into the shared right–anterior–superior (RAS) frame through x=c+sq . Resampling and fusion assemble the eight-bone reconstruction.
Figure 2: Raysemble architecture. (a) Global and regional probability-map features initialize each bone’s query center and scale. (b) Each query is projected once: the same detector coordinate (u,v) samples the global and ROI-relative feature maps. A residual shape predictor and shared decoder recover local occupancy through 123 , 243 , 483 , and 963 grids. (c) The placement branch predicts depth, back-projects centers, and corrects scale for magnification. (d) Local occupancies and final centers and scales supply separate assembly inputs. Resampling onto the common RAS grid and voxel-wise fusion produce eight bone labels.
Method
Params. (M)
Dice (%) ↑
HD95 (mm) ↓
ASSD (mm) ↓
Broadcast U-Net
36.35
56.45±1.17
14.19±2.40
5.01±0.63
Henzler
8.08
52.46±0.53
19.85±4.19
9.28±4.27
PatRecon
592.52
42.39±6.25
27.33±8.77
12.50±5.44
XctNet
33.94
52.60±2.16
25.35±12.62
14.35±12.46
3DDX-Direct
142.54
56.08±0.95
14.35±0.95
4.73±0.11
X2V
153.39
41.21±1.23
25.03±0.80
10.15±0.20
Table 1: Eight-bone reconstruction before alignment on 23 test patients. Means are patient-weighted and the standard deviation is across three seeds. Total inference parameters include the frozen FleXray front end where used. Bold marks the best mean in each metric. Per-bone results are in Supplementary Table S2 .
Figure 3: Synthetic AP reconstruction. (a) CT, input-matched VolumeNeRF, and Raysemble. (b) Anterior views of the whole anatomy and right hip, with 25-degree anterior oblique views of L1–L5, for 15 configurations. Colors show distances from seed-0 predictions to the nearest CT vertex of the corresponding bone (0–15 mm). Bold marks the best three-seed means per case.
Figure 4: Reconstruction with identical inputs. (a) Synthetic AP cases before alignment. (b) Real pelvic X-rays with X-ray–CT intervals of 16 and 70 days, rigidly aligned to CT reference masks without scaling. Insets share CT-defined L4–L5 or iliac-wing windows. Colors show distances from seed-0 predictions to the nearest CT vertex of the corresponding bone (0–15 mm). Bold marks the best three-seed mean for each real-case metric.
Metric
Mean gain
95% CI
Improved
Dice (pp)
+5.86
[3.35, 8.94]
18/23
HD95 (mm)
+10.55
[5.34, 18.82]
23/23
ASSD (mm)
+2.44
[1.37, 4.36]
23/23
Table 2: Paired differences from VolumeNeRF with matched inputs on 23 patients. Positive values favor Raysemble. We compute confidence intervals from 10,000 patient-level bootstrap resamples and round only after computing the differences.
Method
Dice (%) ↑
HD95 (mm) ↓
ASSD (mm) ↓
Initial placement
63.55±0.72
9.63±0.26
3.50±0.10
Residual placement
63.55±0.72
9.63±0.26
3.50±0.10
Absolute placement
65.24±0.46
9.09±0.15
3.32±0.06
Shape stage
Raw, full frame
53.73±0.41
13.70±0.25
5.58±0.11
Raw, predicted ROI
62.94±0.51
10.32±0.10
3.69±0.05
Table 3: Ablations on the test set before alignment. Scores are means ± standard deviations across seeds. The first three rows share the shape predictor. Validation selected the residual model at epoch 0 in all seeds. The overlap loss compares masks after placement.
Test group
Dice (%)
HD95 (mm)
ASSD (mm)
Before alignment
CT-TRI (14)
57.39 / 63.68
24.03 / 9.53
6.63 / 3.50
AbdomenCT-1K (9)
62.49 / 67.66
12.80 / 8.40
4.42 / 3.05
Rigid alignment without scaling
CT-TRI (14)
60.63 / 72.85
19.87 / 7.02
5.73 / 2.54
AbdomenCT-1K (9)
66.39 / 74.71
11.12 / 6.85
3.92 / 2.42
Table 4: Reconstruction with identical inputs, by CT source and alignment. Each cell gives VolumeNeRF / Raysemble. Means weight patients equally across three seeds.
Recovering the 6-DoF pose of the knee bones from a plain radiograph, given the patient's segmented pre-operative CT, turns a routine low-dose image into a quantitative measurement of joint geometry, without the added dose of a repeat CT or a fixed biplanar rig. Classic solutions align a rendered bone silhouette to image edges; recent alternatives refine pose by backpropagating an image-similarity loss through a differentiable X-ray renderer. Both operate one patient at a time and are fragile under a single view. Silhouettes are depth-ambiguous, and differentiable-rendering refinement has a narrow capture range at substantial per-iteration cost. We instead learn an amortized, subject-agnostic dense 2D-3D correspondence, supervised solely by projection geometry. One shared-weight model per bone, trained across 758 patients, registers patients unseen during training. The pose then follows in closed form from a global, initialization-free, render-free PnP+RANSAC solve. Because X-ray formation is transmissive, our correspondence target is transmission-aware rather than tied to a single surface. Though trained only to register, the representation is anatomically semantic: a simple classifier reads a landmark's anatomical region from its embedding across held-out patients, and the same features separate the knee's bones into a 2D-3D-consistent identity learned without any bone label. On a large single-institution cohort the model generalizes well to held-out patients.
Measuring the angle between bone structures is a routine task in medical image analysis and provides a key quantitative parameter for diagnosis and treatment planning. Automated methods can reduce time and cost while improving reproducibility. In this work, we address automatic bone pose estimation using a learning-based point candidate proposal followed by a line model to extract axis parameters. Since conventional line models such as least squares are sensitive to outliers, we incorporate false-positive reduction strategies and robust fitting techniques, such as RANSAC and Hough transforms, to improve robustness. We evaluate our method on three clinically relevant paediatric angle estimation tasks: fracture fragment assessment in radiographs and ultrasound and developmental dysplasia of the hip evaluation in ultrasound using the Graf method. Our approach achieves mean errors of 4.1∘, 5.4∘, and 5.51∘, respectively, not only remaining within the expected clinical observer variability, but also significantly outperforming landmark-based methods. Our code and annotations for fracture angle assessment in radiographs are publicly available on GitHub.
Ron Keuth, Christoph Großbröhmer, Franziska Halm +5
Medical Informatics, University of Lübeck, Ratzeburger Allee 160, 23562 Lübeck, Germany. · EchoScout GmbH, Maria-Goeppert-Str. 3, 23562 Lübeck, Germany · Institut of Radiology and Nuclear Medicine, University Hospital Schleswig-Holstein +1
Three-dimensional femoral reconstruction from radiographs supports surgical planning, implant sizing, and post-operative follow-up, but remains ill-posed as X-ray projections discard depth information. Existing methods often incorporate a 3D statistical shape model (SSM) as a shape prior to guide reconstructions toward anatomically plausible shapes, relying on iterative 3D-to-2D projection matching. Yet, these approaches are computationally expensive and constrain their SSM to a single dimensionality, leaving the statistical relationship between 2D observations and 3D geometry largely unexploited and unexplored. We instead propose a joint 2D-3D SSM that explicitly captures the co-variation between 2D and 3D segmentations in a shared latent space. During training, 2D and 3D segmentations are registered to a common 3D template and its corresponding 2D projections, and the resulting stationary velocity fields are jointly decomposed using principal component analysis (PCA). This joint modeling allows the 2D-to-3D mapping to be learned directly from data rather than computing correspondences at inference time. For unseen subjects, the 3D shape is recovered directly by lifting the 2D latent coordinates to the 3D PCA subspace, thereby eliminating the need for iterative 3D-to-2D projection. Experiments on NMDID demonstrate that the proposed joint 2D-3D SSM outperforms a widely-used 3D-only SSM baseline while achieving inference approximately 4 times faster, at under 3 seconds per subject. The code is available at: https://github.com/florence-dellaniello-picard/joint2d3d-ssm.