MedBOE: Optimization-Based Estimation of Complete Patient Body Models from Whole-Body Medical Imaging

Hoijoon Jung; Yuhao Wei; Mingxiao Tu; Younhyun Jung; Michael Fulham; Jinman Kim

cs.CVOct 20, 2026

MedBOE: Optimization-Based Estimation of Complete Patient Body Models from Whole-Body Medical Imaging

Authors: Hoijoon Jung, Yuhao Wei, Mingxiao Tu, Younhyun Jung, Michael Fulham, Jinman Kim

Organizations: School of Computer Science, The University of Sydney, Camperdown, NSW, 2050, Australia. · School of Computing, Gachon University, Seongnam-Si, Gyeonggi-Do, 13120, Republic of Korea. · Department of Molecular Imaging, Royal Prince Alfred Hospital, Camperdown, NSW, 2050, Australia.

Abstract

True whole-body medical imaging captures a patient from head to toe and provides comprehensive anatomical and/or functional information. This coverage enables reconstruction of the external body surface, yielding patient body models. However, scanner geometry (e.g., bore diameter), patient characteristics, and positioning can leave body regions missing (e.g., lateral arms/torso, hands, forearms, elbows), producing incomplete patient body models. Such incompleteness limits downstream medical analyses, including derivation of body-habitus metrics (BHMs), such as body volume and surface area, for injected radiotracer activity regimens, cardiometabolic risk assessment, and computational anatomical modeling for medical imaging simulation and radiation dosimetry. We present the Medical Body model Optimization-based Estimator (MedBOE), an automated, training-free framework for estimating complete patient body models from whole-body images with missing body regions. Our innovation is to leverage the Skinned Multi-Person Linear (SMPL) template, a parametric body mesh representing a broad range of body shapes and poses, in medical imaging. MedBOE aligns the SMPL template to the incomplete patient body models via attribute-guided optimization and estimates missing body regions through geometrically and volumetrically constrained deformation. We evaluated MedBOE using two public datasets: AutoPET (91 whole-body PET-CT images) and Human Implicit Tissues (HIT; 40 whole-body segmentations from MRI images). MedBOE achieved the best overall volumetric and geometric performance among evaluated methods. BHMs derived from MedBOE were within 2% error, suggesting low percentage errors for downstream analyses. MedBOE offers an automated, training-free method for estimating complete patient body models from whole-body images with missing body regions.

Figures & tables

Figure 1: Overview of the proposed MedBOE. A whole-body image with missing forearms, elbows, and hands (a) is processed through MedBOE’s three modules: (i) Body model reconstruction ; (ii) Global parametric fitting ; (iii) Missing Body Region Estimation and Surface Deformation , yielding outputs (b)–(e). The patient’s face is blurred for anonymization

Iteration	$E_{\mathrm{Sparse}}$	$E_{\mathrm{Dist}}$	$E_{\mathrm{Prior}}$	$E_{\mathrm{Weight}}$	$E_{\mathrm{Contact}}$	$E_{\mathrm{Dense}}$
0	100	0.5	0.5	10	0	10
30	80	2	0.5	10	0	10
100	50	5	0.5	10	0	5
200	5	10	0.5	1	0	2
300	2	10	0.5	0.5	0	2
400	2	10	0.5	0	2	2

Table 1: Attribute weights used in the global parametric fitting module during optimization.

Figure 2: Complete body models estimated from the AutoPET dataset. (a) is GT body models, (b) is simulated incomplete models where darker grey indicates the internal surface exposed by the trimming procedure. Results of (c) MedBOE, (d) IF-Net, (e) ShapeFormer, and (f) PCN. Blue dotted boxes highlight differences in estimating the missing body regions (elbows) between MedBOE and IF-Net

Figure 3: Detailed comparison of the missing body regions estimated by (a) MedBOE and (b) IF-Net. Blue dotted boxes highlight the key difference in estimated regions. GT models are shown as red wireframes

Figure 4: Complete body models estimated from the HIT dataset. (a) is GT body models, (b) is simulated incomplete models where darker grey indicates the internal surface exposed by the trimming procedure. Results of (c) MedBOE, (d) IF-Net, (e) ShapeFormer, (f) ArtEq, and (g) PCN. Orange dashed lines mark the 10 cm above-head apex cutoff. Blue dotted boxes highlight differences in forearms and hands region estimation among MedBOE, ShapeFormer, and PCN

Metrics	Volume $\downarrow$	Surface area $\downarrow$	Chest cir. $\downarrow$	Waist cir. $\downarrow$	Hip cir. $\downarrow$
APE	$0.85\pm 0.46$ (0.13–2.18)	$2.04\pm 1.75$ (0.01–7.27)	$2.05\pm 1.61$ (0.08–8.02)	$1.64\pm 1.22$ (0.12–5.67)	$0.77\pm 0.42$ (0.08–1.75)
MAE	$0.001\pm 0.000$ (0.000–0.002)	$0.042\pm 0.031$ (0.004–0.138)	$0.021\pm 0.017$ (0.001–0.096)	$0.016\pm 0.012$ (0.001–0.056)	$0.008\pm 0.005$ (0.001–0.022)

Table 3: BHM errors derived from the estimated complete body models of the HIT dataset. 0 0 footnotetext: APE: absolute percentage error, MAE: mean absolute error, cir.: circumference.

Figure 5: Circumferential measurement locations (red dotted lines) for chest, waist, and hip, on (a) the GT and (b) the MedBOE-estimated complete body models

	Dataset	RBE $\downarrow$	RVE $\downarrow$	IoU $\uparrow$	CD $\downarrow$	HD $\downarrow$
With the module	AutoPET Male	$1.47\pm 1.61$ * (0.12–9.04)	$0.90\pm 0.80$ * (0.03–4.22)	$0.92\pm 0.03$ * (0.83–0.94)	$0.02\pm 0.02$ * (0.02–0.08)	$0.10\pm 0.10$ * (0.03–0.39)
	AutoPET Female	$2.06\pm 1.81$ * (0.05–9.62)	$0.93\pm 0.72$ * (0.03–3.41)	$0.92\pm 0.01$ * (0.85–0.94)	$0.02\pm 0.02$ * (0.01–0.12)	$0.09\pm 0.10$ * (0.02–0.55)
	HIT Male	$1.85\pm 1.87$ * (0.01–7.27)	$0.86\pm 0.40$ * (0.16–1.52)	$0.94\pm 0.01$ * (0.93–0.95)	$0.04\pm 0.01$ * (0.02–0.06)	$0.19\pm 0.09$ (0.08–0.38)
	HIT Female	$2.15\pm 1.70$ * (0.06–7.09)	$0.85\pm 0.50$ * (0.13–2.18)	$0.94\pm 0.01$ * (0.93–0.95)	$0.03\pm 0.01$ * (0.02–0.07)	$0.14\pm 0.06$ (0.06–0.30)
Without the module	AutoPET Male	$3.83\pm 2.39$ (0.65–13.87)	$5.93\pm 2.45$ (1.68–11.19)	$0.78\pm 0.03$ (0.70–0.83)	$0.03\pm 0.02$ (0.02–0.09)	$0.10\pm 0.09$ (0.04–0.39)
	AutoPET Female	$4.23\pm 2.93$ (0.07–13.81)	$7.32\pm 2.78$ (0.02–15.51)	$0.77\pm 0.03$ (0.68–0.82)	$0.03\pm 0.02$ (0.02–0.14)	$0.10\pm 0.09$ (0.04–0.55)

Table 4: Performance of MedBOE with and without the missing body region estimation and surface deformation module. The better results are shown in bold. 0 0 footnotetext: An asterisk (*) indicates significance from the results without the module (paired t-test,

p<0.05

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: Representative front, left, and right views (left-to-right) comparing (a) the simulated trimmed incomplete patient body model and (b) the corresponding CT-derived incomplete patient body model. External surfaces are rendered in blue, and internal cut surfaces exposed by truncation are rendered in red

Figure 7: Zoomed views of the AutoPET male complete body model estimates. The top row shows the region above the shoulders, and the bottom row shows the leg region. (a) GT body model, (b) MedBOE, (c) IF-Net, (d) ShapeFormer, and (e) PCN

Figure 8: Zoomed views of the AutoPET female complete body model estimates. The top row shows the region above the shoulders, and the bottom row shows the leg region. (a) GT body model, (b) MedBOE, (c) IF-Net, (d) ShapeFormer, and (e) PCN

Figure 9: Representative challenging case from the AutoPET dataset. (a) GT body model, (b) incomplete patient body model, (c) MedBOE, (d) IF-Net, (e) ShapeFormer, and (f) PCN

Gender	Metric	Comparison	Mean difference	$p$ -value	Cohen’s $d$	95% CI
Male	RBE	MedBOE - IF-Net	$-1.817$	$0.439$	$-0.400$	$[-5.334,1.699]$
		ShapeFormer - IF-Net	$11.474$	$2.02\times 10^{-11}$	$1.532$	$[7.957,14.991]$
		ShapeFormer - MedBOE	$13.291$	$<1.00\times 10^{-12}$	$2.154$	$[9.774,16.808]$
	RVE	MedBOE - IF-Net	$-2.920$	$0.338$	$-0.487$	$[-7.833,1.993]$
		ShapeFormer - IF-Net	$15.539$	$6.51\times 10^{-11}$	$1.470$	$[10.626,20.452]$
		ShapeFormer - MedBOE	$18.459$	$<1.00\times 10^{-12}$	$2.112$	$[13.546,23.372]$

Appendix

Table 5: Pairwise method-comparison statistics from Tukey HSD tests for the AutoPET dataset.

Method	Metric	Mean difference	$p$ -value	Cohen’s $d$	95% CI
MedBOE	RBE	$0.590$	$0.109$	$0.340$	$[-0.134,1.315]$
	RVE	$0.032$	$0.849$	$0.042$	$[-0.300,0.363]$
	IoU	$0.001$	$0.785$	$0.067$	$[-0.008,0.011]$
	CD	$-0.001$	$0.807$	$-0.051$	$[-0.008,0.006]$
	HD	$-0.006$	$0.783$	$-0.060$	$[-0.047,0.036]$
IF-Net	RBE	$-1.876$	$0.088$	$-0.468$	$[-4.041,0.290]$

Appendix

Table 6: Male-female comparison statistics from Welch’s independent two-sample

t

-tests for the AutoPET dataset.

Figure 10: Zoomed views of the HIT male complete patient body model estimates. The top row shows the hand and forearm regions, and the bottom row shows the leg region. (a) GT body model, (b) MedBOE, (c) IF-Net, (d) ShapeFormer, and (e) PCN

Figure 11: Zoomed views of the HIT female complete patient body model estimates. The top row shows the hand and forearm regions, and the bottom row shows the leg region. (a) GT body model, (b) MedBOE, (c) IF-Net, (d) ShapeFormer, and (e) PCN. PCN is shown as point-cloud output

Figure 12: Representative challenging case from the HIT dataset. (a) GT body model, (b) incomplete patient body model, (c) MedBOE, (d) IF-Net, (e) ShapeFormer, (f) ArtEq, and (g) PCN

Method	Metric	Mean difference	$p$ -value	Cohen’s $d$	95% CI
MedBOE	RBE	$0.294$	$0.622$	$0.167$	$[-0.915,1.504]$
	RVE	$-0.019$	$0.896$	$-0.041$	$[-0.310,0.272]$
	IoU	$0.004$	$0.123$	$0.476$	$[-0.001,0.008]$
	CD	$-0.006$	$0.112$	$-0.510$	$[-0.012,0.001]$
	HD	$-0.048$	$0.074$	$-0.666$	$[-0.100,0.005]$
IF-Net	RBE	$-3.780$	$1.08\times 10^{-5}$	$-2.013$	$[-5.169,-2.391]$

Appendix

Table 8: Male-female comparison statistics from Welch’s independent two-sample

t

-tests for the HIT dataset.

Method	Metric	Mean difference	$p$ -value	Cohen’s $d$	95% CI
MedBOE	RBE	$0.202$	$0.543$	$0.116$	$[-0.458,0.862]$
	RVE	$-0.063$	$0.557$	$-0.093$	$[-0.274,0.148]$
	IoU	$0.021$	$<1.00\times 10^{-12}$	$1.218$	$[0.016,0.025]$
	CD	$0.011$	$5.30\times 10^{-5}$	$0.680$	$[0.006,0.015]$
	HD	$0.067$	$3.45\times 10^{-5}$	$0.748$	$[0.037,0.098]$
IF-Net	RBE	$8.401$	$<1.00\times 10^{-12}$	$2.266$	$[7.220,9.583]$

Appendix

Table 9: Dataset-wise comparison statistics between AutoPET and HIT.

Dataset	Gender	Metric	Mean difference	$p$ -value	Cohen’s $d$	95% CI
AutoPET	Male	RBE	$-2.359$	$9.85\times 10^{-9}$	$-1.271$	$[-2.996,-1.721]$
		RVE	$-5.036$	$<1.00\times 10^{-12}$	$-2.112$	$[-5.855,-4.217]$
		IoU	$0.133$	$<1.00\times 10^{-12}$	$6.648$	$[0.126,0.140]$
		CD	$-0.009$	$<1.00\times 10^{-12}$	$-3.779$	$[-0.010,-0.009]$
		HD	$-0.009$	$5.31\times 10^{-6}$	$-0.912$	$[-0.012,-0.005]$
	Female	RBE	$-2.168$	$4.97\times 10^{-9}$	$-0.926$	$[-2.795,-1.541]$

Appendix

Table 10: Ablation study statistics between with and without the missing region estimation and surface deformation module.

	Methods	RBE $\downarrow$	RVE $\downarrow$	IoU $\uparrow$	CD $\downarrow$	HD $\downarrow$
Male (35)	MedBOE	$30.28\pm 42.69$ * (0.32–225.96)	$46.48\pm 78.26$ * (3.06–386.43)	$0.44\pm 0.16$ * (0.09–0.70)	$0.06\pm 0.06$ (0.01–0.24)	$0.17\pm 0.19$ (0.02–0.76)
	IF-Net ^	$91.38\pm 7.46$ (68.81–99.98)	$97.67\pm 3.52$ (82.99–100.00)	$0.02\pm 0.03$ (0.00–0.17)	$0.16\pm 0.15$ (0.04–0.69)	$0.33\pm 0.24$ (0.06–0.90)
	Shape Former ^	$77.74\pm 71.95$ (10.34–412.23)	$94.45\pm 116.24$ (5.26–676.19)	$0.05\pm 0.07$ (0.00–0.23)	$0.28\pm 0.22$ (0.06–0.89)	$0.57\pm 0.34$ (0.09–1.33)
	PCN	-	-	-	$0.62\pm 0.32$ (0.25–1.54)	$1.18\pm 0.43$ (0.45–1.96)
Female (56)	MedBOE	$39.29\pm 57.76$ * (0.28–322.29)	$56.36\pm 72.07$ * (0.57–363.75)	$0.39\pm 0.16$ * (0.00–0.66)	$0.06\pm 0.07$ (0.01–0.35)	$0.17\pm 0.21$ * (0.02–0.84)
	IF-Net ^	$95.06\pm 10.92$ (21.09–99.93)	$95.06\pm 10.92$ (21.09–99.93)	$0.04\pm 0.05$ (0.00–0.25)	$0.12\pm 0.12$ (0.01–0.85)	$0.31\pm 0.26$ (0.02–0.92)

Appendix

Table 11: Results of missing region estimation on the AutoPET dataset. 0 0 footnotetext: An Asterisk (*) indicates statistical differences from the second-best results. A circumflex (^) indicates methods with estimation failure cases excluded from the results.

	Methods	RBE $\downarrow$	RVE $\downarrow$	IoU $\uparrow$	CD $\downarrow$	HD $\downarrow$
Male (15)	MedBOE	$19.50\pm 17.96$ (1.82–68.95)	$15.50\pm 15.97$ * (1.73–51.69)	$0.29\pm 0.12$ * (0.10–0.51)	$0.12\pm 0.07$ (0.03–0.28)	$0.53\pm 0.59$ (0.08–1.84)
	IF-Net	$94.03\pm 2.40$ (88.87–97.98)	$97.78\pm 1.39$ (93.85–99.25)	$0.02\pm 0.01$ (0.01–0.05)	$0.20\pm 0.06$ (0.12–0.32)	$0.63\pm 0.56$ (0.29–2.03)
	Shape Former	$56.05\pm 20.33$ (10.61–82.13)	$61.50\pm 23.69$ (6.27–88.70)	$0.16\pm 0.09$ (0.05–0.32)	$0.14\pm 0.06$ (0.05–0.30)	$0.70\pm 0.55$ (0.17–1.75)
	PCN	-	-	-	$0.16\pm 0.13$ (0.03–0.44)	$0.82\pm 0.37$ (0.30–1.60)
	ArtEq	$148.41\pm 89.68$ (11.85–293.73)	-	-	$0.60\pm 0.25$ (0.29–1.22)	$1.07\pm 0.48$ (0.46–1.82)
Female (25)	MedBOE	$26.50\pm 17.05$ (0.17–55.70)	$24.81\pm 19.84$ * (1.44–80.44)	$0.26\pm 0.09$ (0.12–0.44)	$0.10\pm 0.05$ (0.04–0.24)	$0.28\pm 0.37$ (0.06–1.71)

Appendix

Table 12: Results of body region estimation on the HIT dataset. 0 0 footnotetext: An Asterisk (*) indicates statistical differences from the second-best results.

3D Medical Imaging

3D medical imaging develops methods to analyze, reconstruct, and visualize volumetric scans such as CT, MRI, PET, and ultrasound.

See all 173 papers

SAM 3D Body: Robust Full-Body Human Mesh Recovery
Feb 17, 2026Xitong Yang, Devansh Kukreja, Don Pinkus +11Human Mesh Recovery 3D Reconstruction
Meta Superintelligence Labs
OmniFit: Multi-modal 3D Body Fitting via Scale-agnostic Dense Landmark Prediction
Apr 23, 2026Zeyu Cai, Yuliang Xiu, Renke Wang +83D Reconstruction Human Mesh Recovery
Nanjing University · Westlake University · Nanjing University of Science and Technology +2
SOMA: Unifying Parametric Human Body Models
Mar 17, 2026Jun Saito, Jiefeng Li, Michael de Ruyter +123D Morphable Models 3D Reconstruction
NVIDIA

Feb 17, 2026J/K move · Enter open · S save

SAM 3D Body: Robust Full-Body Human Mesh Recovery

Xitong Yang, Devansh Kukreja, Don Pinkus, Anushka Sagar+10

Meta Superintelligence Labs

Human Mesh Recovery 3D Reconstruction 3D Human Pose Estimation

Open paper AskPDF arXiv

We introduce SAM 3D Body (3DB), a promptable model for single-image full-body 3D human mesh recovery (HMR) that demonstrates state-of-the-art performance, with strong generalization and consistent accuracy in diverse in-the-wild conditions. 3DB estimates the human pose of the body, feet, and hands. It is the first model to use a new parametric mesh representation, Momentum Human Rig (MHR), which decouples skeletal structure and surface shape. 3DB employs an encoder-decoder architecture and supports auxiliary prompts, including 2D keypoints and masks, enabling user-guided inference similar to the SAM family of models. We derive high-quality annotations from a multi-stage annotation pipeline that uses various combinations of manual keypoint annotation, differentiable optimization, multi-view geometry, and dense keypoint detection. Our data engine efficiently selects and processes data to ensure data diversity, collecting unusual poses and rare imaging conditions. We present a new evaluation dataset organized by pose and appearance categories, enabling nuanced analysis of model behavior. Our experiments demonstrate superior generalization and substantial improvements over prior methods in both qualitative user preference studies and traditional quantitative analysis. Both 3DB and MHR are open-source.

Iteration	$E_{\mathrm{Sparse}}$	$E_{\mathrm{Dist}}$	$E_{\mathrm{Prior}}$	$E_{\mathrm{Weight}}$	$E_{\mathrm{Contact}}$	$E_{\mathrm{Dense}}$
0	100	0.5	0.5	10	0	10
30	80	2	0.5	10	0	10
100	50	5	0.5	10	0	5
200	5	10	0.5	1	0	2
300	2	10	0.5	0.5	0	2
400	2	10	0.5	0	2	2

Iteration	$E_{\mathrm{Sparse}}$	$E_{\mathrm{Dist}}$	$E_{\mathrm{Prior}}$	$E_{\mathrm{Weight}}$	$E_{\mathrm{Contact}}$	$E_{\mathrm{Dense}}$
0	100	0.5	0.5	10	0	10
30	80	2	0.5	10	0	10
100	50	5	0.5	10	0	5
200	5	10	0.5	1	0	2
300	2	10	0.5	0.5	0	2
400	2	10	0.5	0	2	2

Abstract

3D Medical Imaging

Explore similar work

SAM 3D Body: Robust Full-Body Human Mesh Recovery

OmniFit: Multi-modal 3D Body Fitting via Scale-agnostic Dense Landmark Prediction

SOMA: Unifying Parametric Human Body Models

Iteration	$E_{\mathrm{Sparse}}$	$E_{\mathrm{Dist}}$	$E_{\mathrm{Prior}}$	$E_{\mathrm{Weight}}$	$E_{\mathrm{Contact}}$	$E_{\mathrm{Dense}}$
0	100	0.5	0.5	10	0	10
30	80	2	0.5	10	0	10
100	50	5	0.5	10	0	5
200	5	10	0.5	1	0	2
300	2	10	0.5	0.5	0	2
400	2	10	0.5	0	2	2