Pairwise distortion distribution for compression and quantization
Organizations: Nokia Bell Labs · Department of Electrical & Computer Engineering, University of Southern California
Abstract
Choosing the latent dimension is a critical challenge in data compression, where conventional hyperparameter optimization is costly and ad hoc. Intrinsic dimension provides a good alternative, which connects information-theoretic notions such as Renyi information dimension (RID) to rate-distortion dimension and random codebook compression. In this work, we introduce the pairwise distortion distribution (PDD) and its asymptotic limit, pairwise dimension (PD), which generalize the correlation integral to general distortion measures. We show that PD bounds RID, and PDD provides a lower bound for random codebook quantization performance, which can guide a practical use for compression, quantization, and dimensionality reduction applications.
Figures & tables
Intrinsic Dimensionality
Intrinsic dimensionality measures the effective degrees of freedom underlying data embedded in high-dimensional spaces.
See all 62 papers