ARNAI: Artifact Removal Network based on Autoencoding and Inpainting for Robust Spinal Image Segmentation and Measurement
Authors: Sang-Jin Park, Jinyoung Choi, Seokwon Kim, Seungeon Song, Insu Park, Dougho Park, Taeyeon Kim, Youjin Lee, Donghoon Yang, Jaeman Cho, Joongwon Yang, Mansu Kim, Heumdai Kwon, Hong Gyu Baek, Dae Chul Cho, Injung Kim
Organizations: Pohang Stroke and Spine Hospital, Pohang, Republic of Korea · School of CSEE, Handong Global University, Pohang, Republic of Korea · School of Convergence Science and Technology, Pohang University of Science and Technology, Pohang, Republic of Korea · Department of Neurosurgery, Kyungpook National University Hospital, Kyungpook National University College of Medicine, Daegu, Republic of Korea
Purpose: This study aims to develop an AI framework applicable for postoperative imaging for automated measurement of spinopelvic parameters on radiographs with robustness to the presence of spinal implants. Materials and Methods: We retrospectively reviewed lateral lumbar spine radiographs from two institutions (Internal: January 2017--December 2024; External: October 2021--September 2025). We developed the Restore, Segment, and Measure (RSM) framework, incorporating a novel Artifact Removal Network based on Autoencoding and Inpainting (ARNAI) to mitigate implant-related artifacts in postoperative radiographs. Segmentation and spinopelvic parameter (PT, LL, SS, SCA) measurement performance were assessed using Wilcoxon signed-rank tests and intraclass correlation coefficients. Results: When ARNAI was added to a recent Transformer-based segmentation model, FCBFormer, the mean DSC increased to 0.870 from 0.814, with marked gains at L3--L5 and smaller improvements at L1--L2. On 91 radiographs with implants, the mean L4--L5 segmental Cobb angle error decreased to 4.7 ° from 15.6--16.2 °, an average error reduction of 70%. The ICC for L4--L5 segmental Cobb angle improved to 0.54 (Rater 1) and 0.59 (Rater 2) from 0.18, and ICCs for pelvic tilt, lumbar lordosis, and sacral slope all exceeded 0.70. The improvement in L4--L5 segmental Cobb angle error was statistically significant in the internal implant-containing cohort after correction for multiple comparisons. Conclusion: The proposed RSM framework improved automated spinopelvic parameter measurement in implant-containing postoperative radiographs. By mitigating implant-related artifacts, ARNAI improved segmentation and downstream measurement accuracy, with the greatest benefit observed for L4--L5 segmental Cobb angle estimation, where the mean error was reduced by approximately 70%.
Figures & tables
Model
L1
L2
L3
L4
L5
Sacrum
F1
F2
Mean
Internal test set
FCBFormer [ 8 ]
0.8395 (0.2070)
0.8673 (0.1966)
0.7852 (0.2406)
0.6620 (0.1890)
0.6807 (0.1950)
0.8737 (0.0515)
0.9069 (0.0681)
0.8932 (0.0806)
0.8136 (0.1087)
RSM (Ours)
0.8575 (0.1781)
0.8860 (0.1610)
0.8811 (0.1613)
0.8461 (0.1594)
0.8170 (0.1771)
0.8733 (0.0498)
0.9033 (0.0718)
0.8935 (0.0785)
0.8697 (0.0970)
External test set (Generalization)
FCBFormer [ 8 ]
0.7905 (0.2669)
0.8480 (0.2471)
0.8332 (0.2237)
0.6737 (0.1773)
0.6661 (0.1897)
0.8951 (0.0375)
0.8930 (0.0944)
0.8735 (0.1331)
0.8091 (0.1712)
RSM (Ours)
0.7992 (0.2433)
0.8642 (0.1996)
0.8789 (0.1599)
0.8414 (0.1349)
0.8109 (0.1441)
0.8771 (0.0495)
0.8803 (0.0887)
0.8650 (0.1345)
0.8521 (0.1443)
Table 1: Segmentation performance of RSM in terms of DSC, compared with FCBFormer. Models were trained solely on the implant-free internal trainset and evaluated on both the internal test set and an external test set to demonstrate generalization capability. Values are reported as mean (standard deviation).
Param.
Cohort / Model
Relative to Rater 1
Relative to Rater 2
Mean (SD)
Median (IQR)
R
ICC (95% CI)
Mean (SD)
Median (IQR)
R
ICC (95% CI)
Internal test set
PT
Implant-free
1.7 (3.1)
1.1 (1.2)
0.93
0.92 (0.88–0.94)
1.1 (1.4)
0.9 (1.1)
0.98
0.98 (0.97–0.99)
Implant (Baseline)
2.3 (3.9)
1.2 (1.9)
0.87
0.87 (0.81–0.91)
2.4 (4.2)
1.3 (1.8)
0.86
0.85 (0.78–0.90)
Implant (RSM)
2.0 (3.8)
0.9 (1.4)
0.88
0.88 (0.83–0.92)
2.3 (3.9)
1.2 (1.9)
0.87
0.87 (0.81–0.91)
LL
Implant-free
4.6 (3.9)
3.4 (5.3)
0.92
0.88 (0.72–0.94)
5.0 (3.6)
4.1 (4.6)
0.92
0.88 (0.64–0.94)
Table 2: Comprehensive statistical evaluation of spinopelvic parameter estimation on the internal test set (143 implant-free images and 91 implant-containing images) and the external test set (88 implant-containing images). Only radiographs accepted by both models were included in the analysis.
Figure 1: An overview of the RSM framework. (1) Mitigation of implant-related artifacts using ARNAI, and (2) Segmentation mask extraction and spinopelvic parameter estimation results.
Figure 2: The procedure of ARNAI. (a) Gamma-corrected original input image. (b) The output of VQ-GAN. (c) The intensity difference between (a) and (b). The implant regions exhibit higher values than other regions. (d) The binary mask of implant regions. (e) The reconstructed image in which the implant regions were removed and reconstructed by the inpainting algorithm.
Figure 3: Qualitative comparison of inpainting methods for implant removal in spine radiographs. The figure presents two different cases (rows). (a) Original radiographs showing implants in the spine. (b) Inpainting results using Stable Diffusion [ 7 ] . (c) Inpainting results using Telea inpainting algorithm. As discussed in the text, Telea inpainting algorithm (c) better preserves the original structure of the bones and introduces fewer artifacts compared to Stable Diffusion Inpainting (b).
Figure 4: Qualitative comparison of segmentation results with and without the proposed ARNAI module on two implant-containing spine radiographs (cases 1 and 2). From left to right: (a) original input image, (b) ground-truth annotation, (c) baseline FCBFormer output (DSC ≈ 0.7651/0.8093), and (d) RSM output (DSC ≈ 0.9265/0.9119). Color overlays denote vertebrae L1–L5, sacrum (S), and bilateral femoral heads F1 and F2. Incorporation of ARNAI mitigates implant-related artifacts and improves anatomical boundary delineation.
Fine-grained action segmentation during renorrhaphy in robot-assisted partial nephrectomy requires frame-level recognition of visually similar suturing gestures with variable duration and substantial class imbalance. The SIA-RAPN benchmark defines this problem on 50 clinical videos acquired with the da Vinci Xi system and annotated with 12 frame-level labels. The benchmark compares four temporal models built on I3D features: MS-TCN++, AsFormer, TUT, and DiffAct. Evaluation uses balanced accuracy, edit score, segmental F1 at overlap thresholds of 10, 25, and 50, frame-wise accuracy, and frame-wise mean average precision. In addition to the primary evaluation across five released split configurations on SIA-RAPN, the benchmark reports cross-domain results on a separate single-port RAPN dataset. Across the strongest reported values over those five runs on the primary dataset, DiffAct achieves the highest F1, frame-wise accuracy, edit score, and frame mAP, while MS-TCN++ attains the highest balanced accuracy.
Jiaheng Dai, Huanrong Liu, Tailai Zhou +7
Fudan University, Shanghai, China · University of Macau, Macau, China · The Chinese PLA General Hospital, Beijing, China +1
Automated assessment of degenerative pathology in the lumbar spine on magnetic resonance imaging (MRI) requires access to large-scale datasets of expert-annotated radiological gradings. In contrast, segmentation pseudo-labels can be generated by automated tools at negligible radiologist cost. We examine whether pre-training on segmentation can effectively replace a fraction of the manual grading annotations required for downstream supervision. We pre-train a 3D ResNet encoder to segment the vertebrae, intervertebral discs (IVDs), and the spinal canal, then fine-tune lightweight task-specific grading heads using different proportions of the available training data, ranging from 10% to 100%. On a multicentre dataset of ∼2,000 subjects across 11 pathologies, segmentation pre-training, achieving a Dice score of 0.94 against pseudo-labels, improved the task-averaged (macro) one-vs-rest ROC-AUC at all proportions. With only 20% of grading labels after pre-training, the method achieved near full-supervision performance, with the largest gains observed for either low-prevalence or spatially grounded pathologies.
Monzon Maria, Zisserman Andrew, Jutzeler Catherine R. +1
Biomedical Data Science Lab, Dept. D-HEST, ETH Zurich, Zurich, Switzerland · Swiss Institute of Bioinformatics (SIB), Lausanne, 1015, Switzerland · Visual Geometry Group, Dept. of Engineering Science, University of Oxford, UK
Predicting postoperative spinal morphology from preoperative radiographs could provide valuable support for scoliosis surgical planning, but remains challenging because surgical correction induces large spatial changes while anatomical structures must be faithfully retained. We formulate this problem as postoperative scoliosis radiograph synthesis and construct ScoliSurg, the first paired dataset for this task, comprising 632 preoperative--postoperative whole-spine radiograph pairs with structured morphology information. We further propose ViT3Flow, a single-NFE conditional MeanFlow framework for efficient postoperative radiograph synthesis. ViT3Flow models surgical correction as finite-interval generative transport and replaces conventional self-attention with test-time-training token mixers that perform sample-specific inner adaptation to the anatomy and deformity pattern of each case. In addition, a Spinal Morphology Extraction Agent extracts distributions of dominant-curve region and direction from the preoperative radiograph. These distributions guide Diagnosis-Routed Interval Cross-Attention (DRICA), which performs interval-dependent vertical, horizontal, joint, and global retrieval from a separate preoperative token stream. This design enables the evolving postoperative representation to incorporate spatially corresponding anatomical evidence throughout the transport process. Extensive experiments on ScoliSurg demonstrate that ViT3Flow achieves the best performance among the compared methods in perceptual image quality, anatomical fidelity, and clinically relevant geometric accuracy, while requiring only a single network evaluation. These results highlight the potential of ViT3Flow for efficient and anatomically faithful postoperative radiograph synthesis in scoliosis surgical planning.