Image-to-Image Translation Framework Embedded with Rotation Symmetry Priors
Authors: Feiyu Tan, Heran Yang, Qihong Duan, Kai Ye, Qi Xie, Deyu Meng
Organizations: School of Mathematics and Statistics, Xi’an Jiaotong University, Shaanxi, P.R.China · MOE Key Lab for Intelligent Networks and Networks Security, Faculty of Electronic and Information Engineering, Xi’an Jiaotong University, Shaanxi, P.R.China
Image-to-image translation (I2I) is a fundamental task in computer vision, focused on mapping an input image from a source domain to a corresponding image in a target domain while preserving domain-invariant features and adapting domain-specific attributes. Despite the remarkable success of deep learning-based I2I approaches, the lack of paired data and unsupervised learning framework still hinder their effectiveness. In this work, we address the challenge by incorporating transformation symmetry priors into image-to-image translation networks. Specifically, we introduce rotation group equivariant convolutions to achieve rotation equivariant I2I framework, a novel contribution, to the best of our knowledge, along this research direction. This design ensures the preservation of rotation symmetry, one of the most intrinsic and domain-invariant properties of natural and scientific images, throughout the network. Furthermore, we conduct a systematic study on image symmetry priors on real dataset and propose a novel transformation learnable equivariant convolutions (TL-Conv) that adaptively learns transformation groups, enhancing symmetry preservation across diverse datasets. We also provide a theoretical analysis of the equivariance error of TL-Conv, proving that it maintains exact equivariance in continuous domains and provide a bound for the error in discrete cases. Through extensive experiments across a range of I2I tasks, we validate the effectiveness and superior performance of our approach, highlighting the potential of equivariant networks in enhancing generation quality and its broad applicability. Our code is available at https://github.com/tanfy929/Equivariant-I2I
Figures & tables
Fig. 1: (a) An input image for I2I containing numerous rotationally symmetric structures (illustrated using the iphone2dslr example). (b–c) Intermediate feature maps and translation results from a standard CNN, showing noticeable symmetry loss and structural artifacts. (d–e) Intermediate feature maps and translation results from the rotation equivariant network, demonstrating superior preservation of feature symmetry and producing higher-fidelity outputs.
Fig. 2: An example of rotation equivariant network architecture, which consists of equivariant convolution, equivariant transposed convolution (DeConv.), Instance Normalization (IN), and activation functions (ReLU).
Fig. 3: The feature maps of the trained I2I CNN and EQ-CNN, respectively, along with their corresponding outputs. The two exploited networks are both well trained for MRI translation (details can be found in Sec. V -B) We display the feature maps from shallow to deep layers, where a single channel is arbitrarily selected for representing each layer.
Fig. 4: Illustration of the complex symmetry of local structures in real images, where obvious differences in the curvature and thickness of different parts of the cup rim are caused by the non-overhead shooting perspective. This non-strict rotation symmetry can be converted into strict symmetry by additionally adopting a learnable transformation.
Fig. 5: Illustration of example network constructed by the proposed transformation learnable equivariant convolutions, where we set transformation group as \nicefrac2πi3 rotations, i=1,2,3 . (a) The construction process of the proposed learnable filters. A selected set of basis functions undergo rotation and learnable affine transformations. After discretization and the linear combination with coefficients v , the filters used in practice are obtained. (b)-(d) Input layer, intermediate layer, and output layer, respectively.
Fig. 6: Illustration of data symmetry. (a) The process of feature extraction from images using classical differential kernels. (b) The mean local feature values of testing natural images at different rotation angles. (c)-(d) Illustration of histograms under strict rotation and learned transformations.
Method
winter → summer
old → young
Param. (M)
FID ↓
FID ↓
NEGCUT [ 36 ]
75.8
45.8
-
MoNCE [ 64 ]
78.2
42.8
-
QS-Attn [ 65 ]
77.2
45.2
-
SRC [ 66 ]
71.6
42.7
-
CycleGAN [ 10 ]
72.437
41.774
11.378
TABLE I: Quantiative results on summer2winter_yosemite and old2young datasets.
Method
day → night
Param. (M)
FID ↓
CycleGAN [ 10 ]
43.507
11.378
EQ-CycleGAN
38.715
2.845
TLEQ-CycleGAN
36.921
2.845
CUT [ 11 ]
35.015
11.378
EQ-CUT
34.335
3.541
TABLE II: Quantiative results on BDD100K dataset.
Fig. 7: The generated samples on four tasks: winter → summer, old → young, day → night and iphone → dslr. We show the results using CycleGAN, CUT, and santa, along with their equivariant versions (EQ- and TLEQ-). Traditional CNN-based methods exhibit several limitations, including insufficient style transfer (e.g., CycleGAN for winter → summer, CUT for day → night) and color distortion (santa for iPhone → dslr). Additionally, CNNs struggle to preserve intrinsic image structures, whereas equivariant networks are better at preserving critical domain-invariant features (see representative examples annotated by red arrows in the figures).
Fig. 8: Visual comparison of translation results for three randomly selected pairs from the four MRI modalities in the BraTS 2019 dataset. For easy observation, the demarcated regions are zoomed in three times.
Method
MAE ↓
PSNR ↑
SSIM ↑
Param.(M)
StarGAN [ 72 ]
0.0083
30.75
0.905
50.72
DGGAN [ 73 ]
0.0081
30.81
0.908
58.74
SUGAN [ 74 ]
0.0087
30.37
0.909
82.63
ComboGAN [ 75 ]
0.0081
30.55
0.913
71.02
HyperGAN [ 76 ]
0.0067
32.30
0.9320
12.90
EQ-HyperGAN
0.0064
32.54
0.9362
3.08
TABLE III: Accuracies of different methods for arbitrary cross-contrast MRI translation on BraTS 2019 dataset.
Method
Rain100L [ 77 ]
PSNR
SSIM
Input
26.90
0.8384
DSC [ 78 ]
27.34
0.8494
GMM [ 79 ]
29.05
0.8717
JCAS [ 80 ]
28.54
0.8524
Clear [ 81 ]
30.24
0.9344
TABLE IV: The rain removal results of different competing methods on Rain100L dataset.
Method
Urban100 [ 88 ]
BSD100 [ 89 ]
Set14 [ 90 ]
Set5 [ 91 ]
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
CNN
30.63
0.8864
29.56
0.8167
29.88
0.8106
31.86
0.8803
SCUNet [ 92 ]
30.50
0.8887
29.55
0.8176
29.96
0.8159
31.79
0.8805
G-CNN [ 17 ]
30.75
0.8914
29.60
0.8199
29.89
0.8111
31.91
0.8825
E2-CNN [ 51 ]
30.66
0.8891
29.58
0.8177
29.79
0.8086
31.90
0.8817
PDO-eConv [ 53 ]
29.55
0.8628
29.19
0.7998
29.50
0.8030
31.47
0.8703
TABLE V: The denoising results of different competing methods on synthesized testing datasets.
Method
Urban100 [ 88 ]
BSD100 [ 89 ]
Set14 [ 90 ]
Set5 [ 91 ]
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
CNN
32.17
0.9297
32.20
0.9029
33.62
0.9198
38.06
0.9621
Har-Net [ 92 ]
31.99
0.9281
32.16
0.9025
33.53
0.9192
37.92
0.9619
G-CNN [ 17 ]
32.27
0.9306
32.21
0.9027
33.61
0.9197
38.04
0.9621
E2-CNN [ 51 ]
31.99
0.9283
32.15
0.9024
33.51
0.9191
37.91
0.9618
PDO-eConv [ 53 ]
30.91
0.9159
31.87
0.8985
33.17
0.9161
37.63
0.9606
TABLE VI: The SR results of different competing methods on 4 exploited image datasets.