A thorough review on Deep Learning approaches to 2D-to-3D Registration in Medical Imaging
Authors: Andrea Borghesi, Julia van Beurden, Maarten van Alphen, Michiel van den Brekel, Luc Karssemakers, Jan-Jakob Sonke, Baris Karakullukcu, Jonas Teuwen, George Yiasemis
2D-to-3D image registration is a key component for several medical imaging purposes such as intraoperative guidance, enabling the alignment between preoperative 3D volumes and real-time 2D images. While conventional optimisation-based methods often struggle with non-convex objectives, sensitivity to initialisation, and high computational demands, deep learning (DL) has emerged as a promising alternative, offering improved alignment robustness and the potential for real-time performance. This review provides a comprehensive analysis of DL frameworks developed to address the challenges inherent in 2D-to-3D registration. We first outline how core concepts of DL are adapted to the registration problem. We then examine recent methodological advancements from multiple perspectives, including pipeline design, architectural innovations, and domain adaptation strategies. Particular attention is given to quantitative metrics, highlighting how performance is defined and measured across studies. Across the 41 included studies, we find that direct regression is the most common pipeline (23 studies, against 18 iterative ones), that hybrid CNN-attention architectures are gaining ground, and that research remains heavily concentrated on X-ray-to-CT registration. We conclude by identifying the absence of per-case uncertainty estimation and the strong neglect of temporal or sequential modelling as the two methodological gaps most consequential for clinical readiness, and discuss state-space models as a way to add temporal memory within intraoperative latency budgets. Further opportunities lie in more principled domain adaptation and in the integration of foundation models for representation learning.
Figures & tables
Figure 1: Number of included studies per publication year ( n=41 ). Sixty-eight per cent were published from 2024 onwards. ∗ 2026 covers January to June only, the end of the search window.
Figure 2: PRISMA 2020 flow diagram of the study selection for articles published between January 2015 and June 2026, from 414 records retrieved to 41 included studies. Exclusion reasons at each stage are given in the right-hand boxes.
Article
Architecture
Reg. Task
2D Modality
3D Modality
Train. data
Eval. data
Anatomy
Cao et al. [7]
LSTM
Deformable
Optical
CT
S
S+R
Liver
Chi et al. [11]
CNN
Rigid
US
CT
S+R
R
Kidney
Dai et al. [14]
CNN/SWIN
Deformable
X-Ray
CT
S
S+P
Lung
Downs et al. [17]
CNN
Rigid
DSA
CTA
S
R
Brain
Dupuy et al. [18]
CNN
Rigid
US
3D US
S
S
Prostate
Facente et al. [19]
CNN
Rigid
X-Ray
CT
S
R+P
Pelvis, Cerebral
Table 1: Overview of included studies by architecture, registration task, imaging modalities, type of data used for training and evaluation, and anatomical target.
Article
Sup. Type
L1 / L2
Geo- desic
NCC
MI
Anat.
Reg.
Other
Cao et al. [7]
Sup.
✓
✓
✓
Chi et al. [11]
SSL
✓
✓
✓
Dai et al. [14]
Sup.
✓
✓
✓
Downs et al. [17]
Sup.
✓
✓
✓
✓
Dupuy et al. [18]
Sup.
✓
Facente et al. [19]
Sup.
✓
✓
Table 2: Categorization of Loss Functions and Supervision across the Reviewed Articles.
Figure 3: Direct regression. Top: supervised training as in Eq. ( 4 ), where the predicted transformation φ^ is compared with a ground truth φGT . Bottom: self-supervised training as in Eq. ( 5 ), where the deformed volume Vdef=W(Vm,φ^) is projected or sliced and compared with If . Colour code, used in all schematics of this review: blue, learnable network; green, regression head; purple, differentiable geometric operator; amber, similarity measure or loss; red, classical optimiser; white, image, volume or label map. Dashed amber arrows denote weight updates (training only), dashed purple arrows iteration.
Figure 4: Iterative registration at inference. Top: iterative refinement network (Section 6.1 ). At each iteration, the network predicts a residual correction Δφt+1 from If and the moving volume Vm,t warped with the current estimate, and the warp applies it to give Vm,t+1 , the input of the next iteration. Bottom: hybrid test-time optimisation, where a single network pass provides the initialisation φinit for a classical optimiser, which returns the estimate φ^ (Section 6.2 ).
Figure 5: Single-stream encoder. Vm is first reduced to a 2D image Im by a spatial transformer, then concatenated (“+”) with If and processed by one backbone; the task becomes 2D-to-2D registration at the cost of volumetric context.
Figure 6: Dual-branch encoder. If and Vm are embedded by separate backbones and fused (“+”) before the MLP, so each input keeps its native dimensionality until fusion. Ye et al. [122] use this design with a projection of Vm .
Figure 7: Inference with the diffusion policy of Kobayashi et al. [52] . A U-Net segments the laparoscopic frame If into a label map Lp , and a renderer produces the label map Ls of the liver mesh Vm at the initial parameters φs . Conditioned on both label maps, the policy network predicts the parameter update Δφ^ through its denoising steps; the registered parameters are φs+Δφ^ .
Figure 8: RegiNet [ 121 ] . An MLP maps each voxel coordinate of Vm to a rigid estimate φi ; average pooling over all voxels yields the global estimate φ^ . The MLP is optimised per case, without a training set (Section 7.3.2 ).
Article
mTRE (3D)
mTRE (2D)
Parameter error
DVF error
DSC
Surface distance
Image similarity
Capture range
Success rate
Time
Other
Cao et al. [7]
✓
✓
✓
–
Chi et al. [11]
✓
✓
–
Dai et al. [14]
✓
✓
✓
CE
Downs et al. [17]
✓
✓
✓
–
Dupuy et al. [18]
✓
✓
✓
drift
Facente et al. [19]
✓
✓
✓
✓
–
Table 3: Evaluation metrics reported by the reviewed articles.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Article
Model Scope
Data Part.
Data Prov.
Pre- pro.
Sel. Opt.
Perf.
Base. Comp.
Model Exam.
Repro- duc.
Overall
Cao et al. [7]
18
Chi et al. [11]
15
Dai et al. [14]
15
Downs et al. [17]
9
Dupuy et al. [18]
13
Facente et al. [19]
18
Appendix
Table 4: Quality Assessment Scores (MI-CLAIM) across the Reviewed Articles.
3D/2D registration serves as a cornerstone technique in surgical navigation. Traditional iterative optimization algorithms suffer from low efficiency and high failure rates in intraoperative settings. Deep learning-based methods reformulate registration from iterative optimization to a regression problem that maps image appearance features to spatial pose, typically achieving improved real-time performance and accuracy. However, such learnable methods are confined to memory-driven retrieval of specific pose features rather than understanding the task of image alignment itself, which limits their generalization in complex scenarios. We propose LayersReg, a pioneering regression paradigm that endows the model with 3D anatomical awareness and searches for the correct pose in a progressive, layer-by-layer manner. Inspired by the iterative pose-searching optimization criterion of classical registration, LayersReg searches for correlations between the moving and fixed images in feature space, capturing the trend of pixel flow and thereby converging iteratively toward the correct spatial pose transformation. We further design a coupling of node-wise regression with the progressive registration framework to enhance the model's perception of spatial pose changes. Experimental results demonstrate that under large offsets and multimodality conditions, LayersReg achieves high accuracy on both X-ray/CT registration (0.68°, 1.41 mm) and slice localization (0.73°, 1.55 mm) tasks, outperforming existing state-of-the-art methods while meeting the intraoperative demands for precision and real-time capability.
Xiyuan Wang, Zhenchao Wang, Xinran Chen +3
School of Biological Science and Medical Engineering Southeast University Nanjing · School of Engineering, College of Engineering and Physical Sciences University of Birmingham Birmingham · State Key Laboratory of Digital Medical Engineering Southeast University Nanjing
Medical image registration is a critical component of clinical imaging workflows, enabling accurate longitudinal assessment, multi-modal data fusion, and image-guided interventions. Intensity-based approaches often struggle with interscanner variability and complex anatomical deformations, whereas feature-based methods offer improved robustness by leveraging semantically informed representations. In this work, we investigate DINO-style self-supervised pretraining directly on 3D medical imaging data, aiming to learn dense volumetric features well suited for deformable registration. We assess the resulting representations on challenging interpatient abdominal registration task across both MRI and CT modalities. Our domain-specialized pretraining outperforms the DINOv2 model trained on a large-scale collection of natural images, while requiring substantially lower computational resources at inference time. Moreover, it surpasses established registration models under out-of-domain evaluation, demonstrating the value of task-agnostic yet medical imaging-focused pretraining for robust and efficient 3D image registration.
Eytan Kats, Mattias P. Heinrich
Institute of Medical Informatics, University of Lübeck, Germany
Deformable image registration is a fundamental challenge in medical image analysis. Recent deep learning-based methods typically adopt either one-pass architectures that complete registration in a single network iteration, or recursive architectures that refine registration through multiple passes. While recursive methods often improve accuracy, they incur substantial computational overhead and slower inference speed. In this paper, we challenge the necessity of recursion for high-performance deformable registration and propose M2CG, a one-pass registration network with Multi-level Multi-space Contextual Guidance. It performs multi-step progressive pyramid registration within a single iteration, guided at each step by contextual cues from three levels (previous, current, and subsequent steps) and four spaces (image space, image feature space, segmentation space, and segmentation feature space). Our M2CG supports both unsupervised (without any labels) and semi-supervised (with segmentation labels for partial training data) registration, and has achieved state-of-the-art performance across two well-benchmarked inter-/intra-patient medical registration tasks with eight public datasets. Our findings suggest that the current reliance on recursion may stem from the absence of sufficient contextual cues that comprehensively characterize the remaining distance to the target alignment.
Yue Lin, Shenghai Wu, Mingyuan Meng
Zhongguancun Academy, Beijing, China. · Research Institute, Newland Digital Technology, Fuzhou, China.