stat.MLAug 26, 2026

Efficient Estimation of High Information Projections using Nearest Neighbours

Authors: David P. Hofmeyr

Organizations: School of Mathematical Sciences, Lancaster University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

An intuitive method for dimensionality reduction is proposed, which is highly effective for finding interesting projections of multivariate data. Following similar intuitive motivation to a number of existing techniques, the proposed method is based on enhancing the nearest neighbour relationships in the data. The proposed projection arises from the spectral decomposition of a matrix designed to encode the local covariance structure in the data, where the local covariance at a point is captured by pairs of its nearest neighbours. We show that under standard regularity conditions this matrix is a consistent estimator of the so-called ``Density Information Matrix'' (DIM); a non-parametric analogue of the Fisher Information Matrix. Spectral decompositions of DIMs have been shown to be connected with the important problems of Independent Components Analysis and, in the supervised context, Sufficient Dimension Reduction. However, existing estimators of the DIM are computationally expensive to compute and only target the DIM of a surrogate density, which is proportional to the square of the true underlying density. In addition, we go on to explore the practical utility of our method in aiding the downstream tasks of cluster analysis and outlier detection.

Figures & tables

Figure 1: Two dimensional projections of (a) the Multiple Features Database; and (b) the Election2005 data set with potential outliers highlighted.
Figure 2: Average estimation error for I(fX)I(f_{X}) when X∼N(0,τ−1)X\sim N(\mathbf{0},\tau^{-1}) using (i) ΣX−1\Sigma_{\mathcal{X}}^{-1} ( – △\scriptstyle\triangle – ); (ii) Σ~X−1\tilde{\Sigma}_{\mathcal{X}}^{-1} ( – □\scriptstyle\square – ); and (iii) I(X,k0,k1)∗I(\mathcal{X},k_{0},k_{1})^{*} (– ∘\circ –). The dashed lines indicate one standard error deviations from the mean.
Figure 3: Each plot shows, for each q∈{1,2,...,200}q\in\{1,2,...,200\} , the average increase in ARI due to dimension reduction, using only the subset of data sets with dimension at least qq . Dashed lines correspond with two standard error deviations. Note that there are six data sets with dimension at least 200, i.e. the rightmost value in each plot is an average of six ARI differences while the leftmost is an average of all 48.
Figure 4: Single inlier class data sets (type (i)): Average increases in AUPRC due to dimension reduction, after filtering out data sets of dimension below qq , for q∈{1,2,...,200}q\in\{1,2,...,200\} . Dashed lines correspond with two standard error deviations.
Figure 5: Two inlier class data sets (type (ii)): Average increases in AUPRC due to dimension reduction, after filtering out data sets of dimension below qq , for q∈{1,2,...,200}q\in\{1,2,...,200\} . Dashed lines correspond with two standard error deviations.
Figure 6: Single outlier class data sets (type (iii)): Average increases in AUPRC due to dimension reduction, after filtering out data sets of dimension below qq , for q∈{1,2,...,200}q\in\{1,2,...,200\} . Dashed lines correspond with two standard error deviations.
More in this topic

Dimensionality Reduction

Dimensionality reduction maps high-dimensional data to compact representations for visualization, prediction, compression, and scientific modeling.

See all 145 papers

Explore similar work

CardsList

  1. A Universal Nearest-Neighbor Estimator for Intrinsic Dimensionality

    Mar 11, 2026Eng-Jon Ong, Omer Bobrowski, Gesine Reinert +1Intrinsic Dimensionality

    School of Mathematical Sciences, Queen Mary University of London · Department of Statistics, Oxford University

  2. A Fixed-Radius Distance-Band Benchmark for Dimensionality-Reduction Fidelity

    Aug 22, 2026Yoshio TakaedaDimensionality ReductionData Visualization

    toor Inc.

  3. Regularized Optimization on Grassmann Manifold: Theory, Algorithm and Applications

    Jul 23, 2026Zhuan Liang, Zheng ZhaiSpectral RegularizationRiemannian Optimization

    Department of Statistics, Faculty of Arts and Sciences at Beijing Normal University, Zhuhai