Polaffini: A feature-based approach for robust affine and polyaffine image registration
Authors: Antoine Legouhy, Cosimo Campo, Ross Callaghan, Hojjat Azadbakht, Hui Zhang
Organizations: Hawkes Institute & Department of Computer Science, University College London, London, UK · Institut Pasteur, Universit´e Paris Cit´e, Unit´e de Neuroanatomie Appliqu´ee et Th´eorique, France · AINOSTICS ltd., Manchester, UK
In this work we present Polaffini, a robust and versatile framework for anatomically grounded registration. Medical image registration is dominated by intensity-based registration methods that rely on surrogate measures of alignment quality. In contrast, feature-based approaches that operate by identifying explicit anatomical correspondences, while more desirable in theory, have largely fallen out of favor due to the challenges of reliably extracting features. However, such challenges are now significantly overcome thanks to recent advances in deep learning, which provide pre-trained segmentation models capable of instantly delivering reliable, fine-grained anatomical delineations. We aim to demonstrate that these advances can be leveraged to create new anatomically-grounded image registration algorithms. To this end, we propose Polaffini, which obtains, from these segmented regions, anatomically grounded feature points with 1-to-1 correspondence in a particularly simple way: extracting their centroids. These enable efficient global and local affine matching via closed-form solutions. Those are used to produce an overall transformation ranging from affine to polyaffine with tunable smoothness. Polyaffine transformations can have many more degrees of freedom than affine ones allowing for finer alignment, and their embedding in the log-Euclidean framework ensures diffeomorphic properties. Polaffini has applications both for standalone registration and as pre-alignment for subsequent non-linear registration, and we evaluate it against popular intensity-based registration techniques. Results demonstrate that Polaffini outperforms competing methods in terms of structural alignment and provides improved initialisation for downstream non-linear registration. Polaffini is fast, robust, and accurate, making it particularly well-suited for integration into medical image processing pipelines.
Figures & tables
Figure 1: Fusion of multiple affine transformations. For each quadrant: a. local affine transformations and their associated weight maps; b. the resulting transformation obtained by naïve fusion through Euclidean average; and c. the resulting transformation obtained using the log-Euclidean polyaffine framework of Arsigny et al., 2009 (illustration inspired by that paper).
Figure 2: Illustration of several steps of the polyaffine estimation. a) Two sets of paired points (plain disc: moving, circle: reference). b) Delaunay tetrahedralisation performed on the reference set (blue) and pattern reproduced on the moving one (red). c) Local affine regression between two homologous neighbourhoods. d) Set of estimated local affine transformations. e) Colour gradient representing all weight maps combined. f) Overall polyaffine transformation. Black rectangles represent the frame of the reference image.
Figure 3: For a given set of local affine transformations (a), illustration of the influence of the choice of the background transformation for the construction of the overall polyaffine transformation: b) no background transformation, c) identity background transformation. d) affine background transformation. Black rectangles represent the frame of the reference image.
Figure 4: Usual (top) and proposed (bottom) paths for non-linear registration.
Database
Training set
Validation set
Testing set
IXI
20
5
100
UK Biobank
20
5
100
ADNI (HC/MCI/AD)
60 (20/20/20)
15 (5/5/5)
150 (50/50/50)
Table 1: Distribution of the subjects for the training, validation and testing sets.
Figure 5: Average Dice scores after registration with Polaffini with various values of the smoothness parameter σ , for subject-to-template (top) and subject-to-subject (bottom).
Figure 6: Example of failure cases. Images are shown in the grid of the reference image.
Flirt
ANTs-aff
Anima-aff
Aladin
Polaffini-aff
Polaffini-polyaff
subject-to-template
4.86
0
0.29
0
0
0
subject-to-subject
11.71
0
0.29
0
0
0
Table 2: Failure rate (%) for each registration method.
Figure 7: Average Dice scores after registration with Polaffini and various affine registration methods, for subject-to-template (top) and subject-to-subject (bottom). colours are linked to the similarity metric types (reddish: image-wise, bluish: block-wise, greenish: Polaffini ).
regions
rank
Flirt
ANTs-aff
Anima-aff
Aladin
Polaffini-aff
Polaffini-polyaff
subject-to-template
sub-cortical
1 st
0.3
0.9
1.2
0.6
96.99
2 nd
1.51
3.61
34.04
15.66
43.37
1.81
cortical
1 st
0
0
0
100
2 nd
2.11
1.2
6.02
12.35
78.31
0
white matter
1 st
0
0
0
0.3
2.11
97.59
2 nd
0.9
1.81
22.59
16.27
56.93
1.51
Table 3: Frequency (%) at which each method performs 1 st and 2 nd in terms of Dice score, for each registration, averaged by label.
methods
subject-to-template
subject-to-subject
A
B
sub-cort
cort
WM
sub-cort
cort
WM
ANTs-aff
Flirt
1.0e-23
1.1e-18
4.0e-20
3.5e-10
6.0e-22
5.5e-17
Anima-aff
Flirt
6.5e-52
1.8e-31
1.1e-48
1.0e-42
3.8e-42
4.9e-44
Anima-aff
ANTs-aff
4.5e-46
6.0e-17
3.1e-36
6.6e-43
2.8e-33
6.8e-40
Aladin
Flirt
8.3e-35
6.8e-49
1.9e-46
8.4e-43
2.1e-40
1.1e-46
Aladin
ANTs-aff
1.3e-19
3.1e-45
4.4e-38
2.5e-41
2.6e-28
4.7e-44
Table 4: Pairwise statistical comparison of average Dice scores after image registration using Polaffini and various affine registration methods. Raw p-values for Wilcoxon signed-rank tests ( α=0.05 ). Significant comparisons after Bonferroni correction in favor of method A (resp. method B) are highlighted in blue (resp. red); non-significant ones in gray.
component
set
geometric distance
linear part
GL(d,R)
dist(Li,Lj)
=
log(Li−1Lj)F
rotation
SO(d)
dist(Ri,Rj)
=
log(RiTRj)F
stretch
Sym+(d,R)
dist(Si,Sj)
=
log(Si−1/2SjSi−1/2)F
translation
Rd
dist(ti,tj)
=
∥ti−tj∥2
Table 5: Geometric distances for the affine matrix components. log is the matrix logarithm and ∥.∥F is the Frobenius norm.
Figure 8: Geometric differences between the affine transformations estimated by Polaffini-aff and the ones from various affine registration methods, for subject-to-template (top) and subject-to-subject (bottom).
affine init.
polyaffine init.
p-value
Cohen’s d
sub-cortical regions
affine
polyaffine
<10−115
1.864
affine + ANTs
polyaffine + ANTs
<10−14
0.447
affine + deep
polyaffine + deep
<10−108
1.762
cortex
affine
polyaffine
<10−107
1.745
affine + ANTs
polyaffine + ANTs
<10−38
0.798
affine + deep
polyaffine + deep
<10−176
3.019
Table 6: Statistics about differences between overlap scores after registration with affine vs proposed polyaffine initialisation. Reported p-values are for paired t-tests, Cohen’s d for paired samples (with polyaffine − affine on numerator).
Figure 9: Dice scores after non-linear registration with traditional (ANTs) and deep-learning approaches, initialised with intensity-based affine (affine) and Polaffini-polyaff (Polaffini).
Figure 10: Evolution of image similarity (LCC) and segmentation (average Dice on all regions) losses during training of deep-learning models with affine (blue) and proposed polyaffine (red) initialisation for training (light) and validation (dark) samples.
Figure 11: Dice scores after deep-learning non-linear registration for different configurations of initialisation with Polaffini-aff and Polaffini-polyaff at training and testing.
A. A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital · Department of Radiology, Harvard Medical School · Computer Science and AI Lab, Massachusetts Institute of Technology
Medical & Imaging Informatics, Department of Radiological Sciences, David Geffen School of Medicine at UCLA, USA · Magnetic Resonance Research Labs, Department of Radiological Sciences, David Geffen School of Medicine at UCLA, USA · Department of Radiation Oncology, David Geffen School of Medicine at UCLA, Los Angeles, USA