Interpretable intrinsic dimension estimation through componentwise calibration of distance and angle

Chih-Hsuan Huang; Chih-Wei Chen; Szu-Chi Chung

cs.LGSep 29, 2026

Interpretable intrinsic dimension estimation through componentwise calibration of distance and angle

Authors: Chih-Hsuan Huang, Chih-Wei Chen, Szu-Chi Chung

Organizations: Department of Applied Mathematics, National Sun Yat-sen University, No. 70, Lienhai Rd., Kaohsiung, 80424, Taiwan

Abstract

DANCo (Dimensionality from Angle and Norm Concentration) jointly calibrates nearest-neighbor distance and angular statistics and consistently reaches state-of-the-art accuracy on clean intrinsic-dimension (ID) benchmarks. Practical data, however, introduce neighborhood-relative noise and sample-amplitude heterogeneity that can distort these geometric signals. We reformulate DANCo componentwise, retaining separate distance and angular discrepancy curves so that the source of an estimate can be identified and interpreted. For the distance component, we derive a closed-form Kullback-Leibler divergence for the generic-order ratios of the generalized ratios ID estimator (Gride); when both angular parameters are matched (Full), Gride reduces mean percentage error from $27.7\%$ to $17.6\%$ at noise equal to $40\%$ of typical neighbor spacing on 24 manifolds. For the angular component, two sampling regimes motivate aligning mean direction while retaining concentration matching (Profiled). On a Gaussian scale mixture with generating dimension 70 embedded in 100 dimensions, profiling raises the Minimum Neighbor Distance (MiND) estimate from $22.8$ to $66.7$ , while removing the known amplitudes restores MiND-Full to $71.9$ ; the control thus attributes the Full shortfall to amplitude heterogeneity. On CIFAR-10 and ImageNet, amplitude-reducing normalizations move angular location toward the references and narrow the Full-Profiled gap, an observational counterpart to the controlled mixture. Across four pretrained convolutional neural networks, Gride-Profiled, the two-nearest-neighbor estimator (TWO-NN), and the maximum-likelihood estimator (MLE) exhibit similar rise-and-fall profiles, while Full-Profiled differences identify the layers most sensitive to angular calibration.

Figures & tables

Figure 1: Componentwise calibration. Minimum Neighbor Distance (MiND) or the generalized ratios ID estimator (Gride) supplies the distance discrepancy; Full or Profiled matching supplies the angular discrepancy. Each curve can be minimized separately, and their pointwise sum forms the combined objective. Solid arrows feed the combined objective; dashed branches lead to the component-only minima. Comparing the three minima shows how each component affects the estimate.

Figure 2: Finite-sample checks of the two angular regimes (

N{=}2500

k{=}10

). Dashed lines mark the local-interior mean

\pi/2

and homogeneous-shell limit

\pi/3

. (a) Uniform

d

-balls as

d

varies at fixed sample size. (b) Gaussian scale mixtures at

d=70

as the log-amplitude spread

\sigma_{S}

varies over the grid of the Section 4.4.1 sweep, the range anchored by the image classes.

Estimator	MPE (%)	$>10\%$ error	median time (s)
MiND–Full (DANCo)	6.33	0.138	0.03
TWO-NN	11.12	0.442	0.01
MiND–ML	13.58	0.537	0.06
MLE	18.84	0.596	0.05
ESS	20.20	0.417	101.61
TLE	20.45	0.667	0.51

Table 1: Ten intrinsic-dimension estimators on 24 manifolds with 20 data replicates per estimator. MPE averages each manifold’s estimate over its successful replicates and then averages the relative errors over the 24 manifolds; failures count as errors in the error rate. The last column reports the median time in seconds to estimate one dataset on an Intel i7-10700 CPU, and MiND–Full excludes the shared reference construction.

Figure 3: MPE over 24 manifolds under Gaussian noise scaled by Eq. ( 11 ) (ten data replicates; bands show one standard deviation). MiND–Full has lower clean-data error; Gride–Full has lower error for

\eta\geq 0.1

. Color, line style, and marker jointly identify each method.

Figure 4: Gaussian scale-mixture sweep at

d=70

embedded in

D=100

over 30 data replicates. The log-amplitude standard deviation is

\sigma_{S}

;

S_{i}>0

is the simulated amplitude in

X_{i}=S_{i}Z_{i}

, and

Z_{i}

is a standard Gaussian vector before embedding. Curves show the mean estimated ID with bands of

\pm

one standard deviation; the shaded band approximately maps the centered-norm variation of the 17 surveyed image classes (Supplementary Table S9) onto the log-amplitude spread; the diamond is MiND–Full after the known-amplitude control

X_{i}/S_{i}=Z_{i}

\sigma_{S}=0.25

; the dashed line marks the generating dimension.

Dataset	MiND	Gride	MiND	Gride	TWO-NN	$\hat{\nu}$	$\hat{\tau}$
	Full		Profiled
MNIST digit 3	20.39	20.39	20.20	20.00	14.88	1.166	41.3
MNIST digit 7	14.40	14.40	14.80	14.20	12.19	1.157	27.7
CIFAR-10 bird	18.78	15.80	34.00	33.20	25.55	0.898	63.2
CIFAR-10 cat	21.21	17.60	32.80	32.20	26.11	0.946	56.8
ImageNet koala	24.86	20.71	45.00	38.80	31.32	0.934	90.1

Table 2: Estimated ID and angular statistics on six raw image classes. Calibrated estimates are means over five reference replicates;

\hat{\nu}

is the observed mean direction in radians and

\hat{\tau}

is the dimensionless observed von Mises concentration. Reference-replicate standard deviations reach

2.1

for the Profiled objectives and

4.6

for the Full objectives.

Figure 5: Layer-wise ID profiles on identical activations in four pretrained CNNs. Gride–Profiled uses

k=10

and precomputed smoothing-spline references through candidate 400. Lines show the median over seven ImageNet categories and three 500-image subsamples; bands show their interquartile range. Color, line style, and marker jointly identify each estimator. The ID axis is logarithmic.

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Figure S1: Normalized centered norm

a_{i}/\bar{a}

and per-center mean direction within a

d=70

\sigma_{S}=0.25

GSM sample, with

\bar{a}

the mean centered norm. Deciles contain equal numbers of centers; error bars show within-decile standard deviations, and the dashed horizontal line marks

\pi/3

Figure S2: Representative pooled neighbor-angle distributions and fitted von Mises (VM) laws for the uniform 70-ball and raw ImageNet koala. The koala fit has a modest descriptive CDF gap; its dominant data–reference discrepancy occurs in angular location.

Figure S3: Calibration-side Bessel overflow for fixed data statistics and the

N_{\mathrm{ref}}=500

calibration. Ordinary evaluation loses validity at candidate

m=278

and becomes negative at

m=284

. Scaled evaluation remains finite and positive through candidate 400; dashed squares and solid circles denote ordinary and scaled evaluation, respectively.

Figure S4: Wall time for the explicitly defined workloads, shown as the median of three fresh-process measurements after warm-up. (a) Observed-data statistics versus ambient dimension. (b) Complete evaluation with precomputed spline, newly generated Monte Carlo, or reused Monte Carlo references versus candidate bound; solid lines with circles are CPU, while dashed lines with squares are GPU.

$N$	Data	reference min	reference max	observed $\hat{\nu}$	gap
547	ImageNet koala	1.142	1.443	0.934	0.208
648	ImageNet butterfly	1.144	1.447	0.915	0.229
1010	MNIST 3	1.150	1.464	1.166	$-0.016$
1028	MNIST 7	1.150	1.467	1.157	$-0.006$
5000	CIFAR-10 bird	1.169	1.495	0.898	0.271
5000	CIFAR-10 cat	1.169	1.495	0.946	0.223

Appendix

Table S1: Sample-size-specific reference mean-direction ranges. Each image row uses the five reference replicates over candidates

m=6,\ldots,200

; the last row gives the Gride reference surface of the CNN study at its effective sample sizes over

m=6,\ldots,400

. The observed gap is the lower reference boundary minus the data mean direction. A negative gap places the data value above the lower boundary, and both MNIST values also lie below the upper boundary, hence inside the range.

Manifold	$d$	CorrInt	MiND–Full (DANCo)	ESS	FisherS	MADA	MLE	MiND–ML	TLE	TWO-NN	lPCA
M10a_Cubic	10	8.6	10.1	10.2	10.3	9.2	8.7	8.9	9.6	9.1	11.0
M10b_Cubic	17	12.7	17.1	17.3	17.0	13.9	13.2	13.7	14.2	14.2	18.0
M10c_Cubic	24	16.2	24.5	24.4	24.0	18.0	17.2	18.0	18.1	18.8	25.0
M10d_Cubic	70	31.8	70.6	70.0	57.1	36.8	35.7	38.0	35.0	40.2	71.0
M11_Moebius	2	2.0	2.0	2.5	2.0	2.1	2.0	2.0	2.2	2.0	3.0
M12_Norm	20	12.7	20.0	19.8	20.0	16.2	15.5	16.3	15.5	17.0	20.0

Appendix

Table S2: Estimated ID of ten estimators on all 24 benchmark manifolds (mean over the successful data replicates out of 20). FisherS on M10d_Cubic averages its single successful replicate; its other 19 replicates failed.

Figure S5: Signed relative error on the 24-manifold benchmark. Faint points are individual manifolds; lines are means by generating dimension. Color, line style, and marker jointly identify each estimator.

Objective	MPE (%)	Signed error (%)
MiND distance-only	$10.1\pm 0.2$	$-1.6\pm 0.3$
Gride distance-only	$11.9\pm 0.6$	$-1.8\pm 0.4$
Full angle-only	$15.9\pm 0.7$	$+10.7\pm 0.7$
Profiled angle-only	$12.2\pm 0.3$	$+6.2\pm 0.2$
MiND–Full	$8.6\pm 0.7$	$+0.9\pm 0.9$
MiND–Profiled	$9.7\pm 0.2$	$-0.6\pm 0.2$

Appendix

Table S3: Clean component comparison on 14 eligible manifolds over five shared reference replicates. Entries are mean

\pm

standard deviation across replicates. MPE gives error magnitude; signed error retains direction, with negative values denoting underestimation. All eight objectives minimize a calibrated KL objective with fractional refinement.

Neighborhood sweep ( $N=2500$ )
Setting	MiND–Full	Gride–Full	MiND–Profiled	Gride–Profiled	TWO-NN
5	6.04	7.37	5.84	6.45	11.91
10	6.24	9.50	6.73	9.39	11.91
15	7.18	13.46	7.60	13.53	11.91
20	8.09	16.78	8.49	16.91	11.91
Sample-size sweep ( $k=10$ )

Appendix

Table S4: MPE (%) across neighborhood and sample-size sweeps, each averaged over five data replicates. The first block varies

k

N=2500

; the second varies

N

k=10

Restricted subset: 17 manifolds (excluding $d>5$ with $D\leq d+1$ )
$\eta$	MiND–Full	Gride–Full	MiND–Profiled	Gride–Profiled
0	6.75 $\pm$ 0.41	9.39 $\pm$ 0.38	6.82 $\pm$ 0.38	9.41 $\pm$ 0.26
0.05	7.67 $\pm$ 0.49	9.37 $\pm$ 0.34	7.69 $\pm$ 0.28	9.32 $\pm$ 0.28
0.1	11.38 $\pm$ 0.93	9.16 $\pm$ 0.53	11.36 $\pm$ 0.69	9.05 $\pm$ 0.50
0.2	16.48 $\pm$ 0.51	10.96 $\pm$ 0.55	15.84 $\pm$ 0.32	10.33 $\pm$ 0.65
0.4	27.74 $\pm$ 0.77	17.64 $\pm$ 0.58	27.32 $\pm$ 0.94	16.96 $\pm$ 0.43

Appendix

Table S5: Neighborhood-relative Gaussian noise. Upper panel: MPE (%) of the four combined objectives over all 24 manifolds, with a final row at

\eta=0.4

restricted to the 17 manifolds that remain after excluding those with

d>5

and

D\leq d+1

. Lower panel: signed error (%) of all eight objectives on the 14-manifold complete-case set of Section S4.2 , with negative values denoting underestimation. Entries are mean

\pm

standard deviation over the ten data replicates of the noise design; its

\eta=0

column therefore differs slightly from the five-replicate clean values of Table S3 .

Objective	Estimated ID	MPE (%)
MiND distance-only	$71.20\pm 1.75$	$2.48\pm 1.72$
Gride distance-only	$64.63\pm 1.59$	$7.67\pm 2.27$
Full angle-only	$6.26\pm 0.03$	$91.06\pm 0.05$
Profiled angle-only	$56.07\pm 3.26$	$19.90\pm 4.65$
MiND–Full	$22.80\pm 1.76$	$67.43\pm 2.51$
MiND–Profiled	$66.67\pm 1.81$	$4.76\pm 2.58$

Appendix

Table S6: Eight calibrated objectives on amplitude-heterogeneous GSM samples at

d=70

D=100

\sigma_{S}=0.25

, over 30 replicates. Entries are mean

\pm

standard deviation.

$\sigma_{S}$	$\hat{\nu}$	Distance-only	Full	Profiled
$0$	$1.137\pm 0.003$	$65.07\pm 1.01$	$71.90\pm 3.30$	$64.37\pm 0.89$
$0.1$	$1.055\pm 0.008$	$64.17\pm 1.31$	$52.05\pm 3.23$	$62.83\pm 1.09$
$0.15$	$0.980\pm 0.010$	$64.60\pm 1.30$	$35.87\pm 2.66$	$62.47\pm 1.25$
$0.2$	$0.903\pm 0.012$	$66.93\pm 1.14$	$28.49\pm 2.50$	$63.83\pm 1.55$
$0.25$	$0.832\pm 0.014$	$71.20\pm 1.75$	$22.80\pm 1.76$	$66.67\pm 1.81$
$0.3$	$0.769\pm 0.014$	$76.97\pm 1.96$	$19.95\pm 1.55$	$70.20\pm 1.88$

Appendix

Table S7: Observed mean direction

\hat{\nu}

and MiND estimates across the GSM amplitude sweep (

d=70

D=100

; 30 replicates). Entries are mean

\pm

standard deviation. The known-amplitude control at

\sigma_{S}=0.25

reuses the latent draws of the

\sigma_{S}=0

level, so its values are those of the

\sigma_{S}=0

row.

Figure S6: Complete KL analysis for MNIST digit 3 and ImageNet koala using one fixed reference replicate and the MiND distance model. The legend distinguishes components by color, line style, and marker; enlarged symbols mark the Full and Profiled joint minima.

	raw			centered radial			contrast-norm.
Dataset	$\hat{\nu}$	Full	Profiled	$\hat{\nu}$	Full	Profiled	$\hat{\nu}$	Full	Profiled
MNIST 3	1.166	20.4	20.2	1.198	20.2	20.0	1.135	20.8	20.8
MNIST 7	1.157	14.4	14.8	1.201	14.8	14.8	1.143	14.8	14.8
CIFAR-10 bird	0.898	18.8	34.0	0.982	22.5	33.0	1.064	28.2	34.0
CIFAR-10 cat	0.946	21.2	32.8	1.030	24.4	32.8	1.087	30.6	35.4
ImageNet koala	0.934	24.9	45.0	1.028	40.6	53.4	1.080	72.2	70.0

Appendix

Table S8: Normalization controls. The observed-data mean direction and Full and Profiled estimates are reported for raw, centered-radial, and contrast-normalized data over five reference replicates. Full and Profiled denote MiND–Full and MiND–Profiled.

Figure S7: Per-center mean direction against normalized centered norm

a_{i}/\bar{a}

for raw and contrast-normalized ImageNet koala data. Normalization largely removes the broad norm tail and weakens the negative relation. Solid lines are least-squares trends; the dashed horizontal line marks

\pi/3

, and

r

denotes the Pearson correlation between centered norm and per-center mean direction.

Source	Class	$N$	$D$	$\hat{\nu}$	$\operatorname{CV}(a)$	$r(\hat{\nu}_{i},\\|x_{i}-\bar{x}\\|)$
CIFAR-10	airplane	5000	3072	0.939	0.291	-0.08
CIFAR-10	automobile	5000	3072	1.037	0.219	-0.04
CIFAR-10	bird	5000	3072	0.898	0.330	-0.10
CIFAR-10	cat	5000	3072	0.946	0.256	-0.08
CIFAR-10	deer	5000	3072	0.903	0.312	-0.13
CIFAR-10	dog	5000	3072	0.938	0.243	-0.10

Appendix

Table S9: Per-class angular statistics at

k=10

. Here

r

is the Pearson correlation between per-center mean direction and centered norm, and

\operatorname{CV}(a)

is the population standard deviation of

a_{i}=\|x_{i}-\bar{x}\|_{2}

divided by their mean, with

\bar{x}

the class mean. ImageNet class codes are WordNet synset identifiers.

Architecture	Peak layer	depth (%)	$\hat{\nu}$	Full	Profiled	Median paired Full $-$ Profiled
AlexNet	second max-pooling checkpoint	22.2	0.800	17	115	-100
VGG-16	second max-pooling checkpoint	23.5	0.802	18	197	-179
ResNet-18	second residual stage	47.4	0.916	37	248	-214
ResNet-34	second residual stage	42.9	0.919	37	245	-208

Appendix

Table S10: Peak of the median Gride–Profiled curve across seven categories and three subsamples. Full and Profiled denote Gride–Full and Gride–Profiled, and the largest absolute difference between them occurs at the same layer in each architecture. The last column is the median of the per-cell paired Full

-

Profiled differences across category–subsample cells, not the difference of the two reported medians. Layer names denote mathematical checkpoints along the sequence of network transformations. Lower panel: mean direction at the first pooling checkpoint and at the output layer, with the output medians of both objectives and their paired difference.

Intrinsic Dimensionality

Intrinsic dimensionality measures the effective degrees of freedom underlying data embedded in high-dimensional spaces.

See all 56 papers

A Universal Nearest-Neighbor Estimator for Intrinsic Dimensionality
Mar 11, 2026Eng-Jon Ong, Omer Bobrowski, Gesine Reinert +1Intrinsic Dimensionality
School of Mathematical Sciences, Queen Mary University of London · Department of Statistics, Oxford University
Rethinking Intrinsic Dimension Estimation in Neural Representations
Apr 22, 2026Rickmer Schulte, David RügamerIntrinsic Dimensionality Neural Representation Geometry
LMU Munich, MCML
Sample-Optimal Estimation of the Fréchet Inception Distance
Oct 5, 2026Ziyun Chen, Jerry Li, Kevin Tian +1Generative AI Evaluation
University of Washington · University of Texas at Austin

Mar 11, 2026J/K move · Enter open · S save

A Universal Nearest-Neighbor Estimator for Intrinsic Dimensionality

Eng-Jon Ong, Omer Bobrowski, Gesine Reinert, Primoz Skraba

School of Mathematical Sciences, Queen Mary University of London · Department of Statistics, Oxford University

Intrinsic Dimensionality

Open paper AskPDF arXiv

Estimating the intrinsic dimensionality (ID) of data is a fundamental problem in machine learning and computer vision, providing insight into the true degrees of freedom underlying high-dimensional observations. Existing methods often rely on geometric or distributional assumptions and can significantly fail when these assumptions are violated. In this paper, we introduce a novel ID estimator based on nearest-neighbor distance ratios that involves simple calculations and achieves state-of-the-art results. Most importantly, we provide a theoretical analysis proving that our estimator is universal, namely, it converges to the true ID independently of the distribution generating the data. We present experimental results on benchmark manifolds and real-world datasets to demonstrate the performance of our estimator.

Abstract

Intrinsic Dimensionality

Explore similar work

A Universal Nearest-Neighbor Estimator for Intrinsic Dimensionality

Rethinking Intrinsic Dimension Estimation in Neural Representations

Sample-Optimal Estimation of the Fréchet Inception Distance