Employing Discrete Fourier Transform in Representation Learning

Image representation learning via input reconstruction is a common technique in machine learning for generating representations that can be effectively utilized by arbitrary downstream tasks. A well-established approach uses autoencoders to extract latent representations at the network's compression point. These representations are valuable because they retain essential information necessary for reconstructing the original input from the compressed latent space. In this paper, we propose an alternative learning objective. Instead of using the raw input as the reconstruction target, we employ the Discrete Fourier Transform (DFT) of the input. The DFT provides meaningful global information at each frequency level, making individual frequency components useful as separate learning targets. When dealing with multidimensional input data, the DFT offers remarkable flexibility by enabling selective transformation across specific dimensions while preserving others in the computation. Moreover, certain types of input exhibit distinct patterns in their frequency distributions, where specific frequency components consistently contain most of the magnitude, allowing us to focus on a subset of frequencies rather than the entire spectrum. These characteristics position the DFT as a viable learning objective for representation learning. We validate our approach on the CIFAR-10 and STL-10 datasets under constrained training regimes. We discovered that integrating data augmentation during pretraining enhances the learned representations. Furthermore, we demonstrate that training exclusively on the lower-frequency components, those containing the highest magnitudes, produces results comparable to using the full frequency spectrum with only minimal reductions in accuracy.