Interpretable intrinsic dimension estimation through componentwise calibration of distance and angle

Chih-Hsuan Huang; Chih-Wei Chen; Szu-Chi Chung

cs.LGSep 29, 2026

Interpretable intrinsic dimension estimation through componentwise calibration of distance and angle

Authors: Chih-Hsuan Huang, Chih-Wei Chen, Szu-Chi Chung

Organizations: Department of Applied Mathematics, National Sun Yat-sen University, No. 70, Lienhai Rd., Kaohsiung, 80424, Taiwan

Abstract

DANCo (Dimensionality from Angle and Norm Concentration) jointly calibrates nearest-neighbor distance and angular statistics and consistently reaches state-of-the-art accuracy on clean intrinsic-dimension (ID) benchmarks. Practical data, however, introduce neighborhood-relative noise and sample-amplitude heterogeneity that can distort these geometric signals. We reformulate DANCo componentwise, retaining separate distance and angular discrepancy curves so that the source of an estimate can be identified and interpreted. For the distance component, we derive a closed-form Kullback-Leibler divergence for the generic-order ratios of the generalized ratios ID estimator (Gride); when both angular parameters are matched (Full), Gride reduces mean percentage error from $27.7\%$ to $17.6\%$ at noise equal to $40\%$ of typical neighbor spacing on 24 manifolds. For the angular component, two sampling regimes motivate aligning mean direction while retaining concentration matching (Profiled). On a Gaussian scale mixture with generating dimension 70 embedded in 100 dimensions, profiling raises the Minimum Neighbor Distance (MiND) estimate from $22.8$ to $66.7$ , while removing the known amplitudes restores MiND-Full to $71.9$ ; the control thus attributes the Full shortfall to amplitude heterogeneity. On CIFAR-10 and ImageNet, amplitude-reducing normalizations move angular location toward the references and narrow the Full-Profiled gap, an observational counterpart to the controlled mixture. Across four pretrained convolutional neural networks, Gride-Profiled, the two-nearest-neighbor estimator (TWO-NN), and the maximum-likelihood estimator (MLE) exhibit similar rise-and-fall profiles, while Full-Profiled differences identify the layers most sensitive to angular calibration.

Figures & tables

Figure 1: Componentwise calibration. Minimum Neighbor Distance (MiND) or the generalized ratios ID estimator (Gride) supplies the distance discrepancy; Full or Profiled matching supplies the angular discrepancy. Each curve can be minimized separately, and their pointwise sum forms the combined objective. Solid arrows feed the combined objective; dashed branches lead to the component-only minima. Comparing the three minima shows how each component affects the estimate.

Figure 2: Finite-sample checks of the two angular regimes (

N{=}2500

k{=}10

). Dashed lines mark the local-interior mean

\pi/2

and homogeneous-shell limit

\pi/3

. (a) Uniform

d

-balls as

d

varies at fixed sample size. (b) Gaussian scale mixtures at

d=70

as the log-amplitude spread

\sigma_{S}

varies over the grid of the Section 4.4.1 sweep, the range anchored by the image classes.

Estimator	MPE (%)	$>10\%$ error	median time (s)
MiND–Full (DANCo)	6.33	0.138	0.03
TWO-NN	11.12	0.442	0.01
MiND–ML	13.58	0.537	0.06
MLE	18.84	0.596	0.05
ESS	20.20	0.417	101.61
TLE	20.45	0.667	0.51

Table 1: Ten intrinsic-dimension estimators on 24 manifolds with 20 data replicates per estimator. MPE averages each manifold’s estimate over its successful replicates and then averages the relative errors over the 24 manifolds; failures count as errors in the error rate. The last column reports the median time in seconds to estimate one dataset on an Intel i7-10700 CPU, and MiND–Full excludes the shared reference construction.

Figure 3: MPE over 24 manifolds under Gaussian noise scaled by Eq. ( 11 ) (ten data replicates; bands show one standard deviation). MiND–Full has lower clean-data error; Gride–Full has lower error for

\eta\geq 0.1

. Color, line style, and marker jointly identify each method.

Figure 4: Gaussian scale-mixture sweep at

d=70

embedded in

D=100

over 30 data replicates. The log-amplitude standard deviation is

\sigma_{S}

;

S_{i}>0

is the simulated amplitude in

X_{i}=S_{i}Z_{i}

, and

Z_{i}

is a standard Gaussian vector before embedding. Curves show the mean estimated ID with bands of

\pm

one standard deviation; the shaded band approximately maps the centered-norm variation of the 17 surveyed image classes (Supplementary Table S9) onto the log-amplitude spread; the diamond is MiND–Full after the known-amplitude control

X_{i}/S_{i}=Z_{i}

\sigma_{S}=0.25

; the dashed line marks the generating dimension.

Dataset	MiND	Gride	MiND	Gride	TWO-NN	$\hat{\nu}$	$\hat{\tau}$
	Full		Profiled
MNIST digit 3	20.39	20.39	20.20	20.00	14.88	1.166	41.3
MNIST digit 7	14.40	14.40	14.80	14.20	12.19	1.157	27.7
CIFAR-10 bird	18.78	15.80	34.00	33.20	25.55	0.898	63.2
CIFAR-10 cat	21.21	17.60	32.80	32.20	26.11	0.946	56.8
ImageNet koala	24.86	20.71	45.00	38.80	31.32	0.934	90.1

Table 2: Estimated ID and angular statistics on six raw image classes. Calibrated estimates are means over five reference replicates;

\hat{\nu}

is the observed mean direction in radians and

\hat{\tau}

is the dimensionless observed von Mises concentration. Reference-replicate standard deviations reach

2.1

for the Profiled objectives and

4.6

for the Full objectives.

Figure 5: Layer-wise ID profiles on identical activations in four pretrained CNNs. Gride–Profiled uses

k=10

and precomputed smoothing-spline references through candidate 400. Lines show the median over seven ImageNet categories and three 500-image subsamples; bands show their interquartile range. Color, line style, and marker jointly identify each estimator. The ID axis is logarithmic.

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Figure S1: Normalized centered norm

a_{i}/\bar{a}

and per-center mean direction within a

d=70

\sigma_{S}=0.25

GSM sample, with

\bar{a}

the mean centered norm. Deciles contain equal numbers of centers; error bars show within-decile standard deviations, and the dashed horizontal line marks

\pi/3

Figure S2: Representative pooled neighbor-angle distributions and fitted von Mises (VM) laws for the uniform 70-ball and raw ImageNet koala. The koala fit has a modest descriptive CDF gap; its dominant data–reference discrepancy occurs in angular location.

Figure S3: Calibration-side Bessel overflow for fixed data statistics and the

N_{\mathrm{ref}}=500

calibration. Ordinary evaluation loses validity at candidate

m=278

and becomes negative at

m=284

. Scaled evaluation remains finite and positive through candidate 400; dashed squares and solid circles denote ordinary and scaled evaluation, respectively.

Figure S4: Wall time for the explicitly defined workloads, shown as the median of three fresh-process measurements after warm-up. (a) Observed-data statistics versus ambient dimension. (b) Complete evaluation with precomputed spline, newly generated Monte Carlo, or reused Monte Carlo references versus candidate bound; solid lines with circles are CPU, while dashed lines with squares are GPU.

$N$	Data	reference min	reference max	observed $\hat{\nu}$	gap
547	ImageNet koala	1.142	1.443	0.934	0.208
648	ImageNet butterfly	1.144	1.447	0.915	0.229
1010	MNIST 3	1.150	1.464	1.166	$-0.016$
1028	MNIST 7	1.150	1.467	1.157	$-0.006$
5000	CIFAR-10 bird	1.169	1.495	0.898	0.271
5000	CIFAR-10 cat	1.169	1.495	0.946	0.223

Appendix

Table S1: Sample-size-specific reference mean-direction ranges. Each image row uses the five reference replicates over candidates

m=6,\ldots,200

; the last row gives the Gride reference surface of the CNN study at its effective sample sizes over

m=6,\ldots,400

. The observed gap is the lower reference boundary minus the data mean direction. A negative gap places the data value above the lower boundary, and both MNIST values also lie below the upper boundary, hence inside the range.

Manifold	$d$	CorrInt	MiND–Full (DANCo)	ESS	FisherS	MADA	MLE	MiND–ML	TLE	TWO-NN	lPCA
M10a_Cubic	10	8.6	10.1	10.2	10.3	9.2	8.7	8.9	9.6	9.1	11.0
M10b_Cubic	17	12.7	17.1	17.3	17.0	13.9	13.2	13.7	14.2	14.2	18.0
M10c_Cubic	24	16.2	24.5	24.4	24.0	18.0	17.2	18.0	18.1	18.8	25.0
M10d_Cubic	70	31.8	70.6	70.0	57.1	36.8	35.7	38.0	35.0	40.2	71.0
M11_Moebius	2	2.0	2.0	2.5	2.0	2.1	2.0	2.0	2.2	2.0	3.0
M12_Norm	20	12.7	20.0	19.8	20.0	16.2	15.5	16.3	15.5	17.0	20.0

Appendix

Table S2: Estimated ID of ten estimators on all 24 benchmark manifolds (mean over the successful data replicates out of 20). FisherS on M10d_Cubic averages its single successful replicate; its other 19 replicates failed.

Figure S5: Signed relative error on the 24-manifold benchmark. Faint points are individual manifolds; lines are means by generating dimension. Color, line style, and marker jointly identify each estimator.

Objective	MPE (%)	Signed error (%)
MiND distance-only	$10.1\pm 0.2$	$-1.6\pm 0.3$
Gride distance-only	$11.9\pm 0.6$	$-1.8\pm 0.4$
Full angle-only	$15.9\pm 0.7$	$+10.7\pm 0.7$
Profiled angle-only	$12.2\pm 0.3$	$+6.2\pm 0.2$
MiND–Full	$8.6\pm 0.7$	$+0.9\pm 0.9$
MiND–Profiled	$9.7\pm 0.2$	$-0.6\pm 0.2$

Appendix

Table S3: Clean component comparison on 14 eligible manifolds over five shared reference replicates. Entries are mean

\pm

standard deviation across replicates. MPE gives error magnitude; signed error retains direction, with negative values denoting underestimation. All eight objectives minimize a calibrated KL objective with fractional refinement.

Neighborhood sweep ( $N=2500$ )
Setting	MiND–Full	Gride–Full	MiND–Profiled	Gride–Profiled	TWO-NN
5	6.04	7.37	5.84	6.45	11.91
10	6.24	9.50	6.73	9.39	11.91
15	7.18	13.46	7.60	13.53	11.91
20	8.09	16.78	8.49	16.91	11.91
Sample-size sweep ( $k=10$ )

Appendix

Table S4: MPE (%) across neighborhood and sample-size sweeps, each averaged over five data replicates. The first block varies

k

N=2500

; the second varies

N

k=10

Restricted subset: 17 manifolds (excluding $d>5$ with $D\leq d+1$ )
$\eta$	MiND–Full	Gride–Full	MiND–Profiled	Gride–Profiled
0	6.75 $\pm$ 0.41	9.39 $\pm$ 0.38	6.82 $\pm$ 0.38	9.41 $\pm$ 0.26
0.05	7.67 $\pm$ 0.49	9.37 $\pm$ 0.34	7.69 $\pm$ 0.28	9.32 $\pm$ 0.28
0.1	11.38 $\pm$ 0.93	9.16 $\pm$ 0.53	11.36 $\pm$ 0.69	9.05 $\pm$ 0.50
0.2	16.48 $\pm$ 0.51	10.96 $\pm$ 0.55	15.84 $\pm$ 0.32	10.33 $\pm$ 0.65
0.4	27.74 $\pm$ 0.77	17.64 $\pm$ 0.58	27.32 $\pm$ 0.94	16.96 $\pm$ 0.43

Appendix

Table S5: Neighborhood-relative Gaussian noise. Upper panel: MPE (%) of the four combined objectives over all 24 manifolds, with a final row at

\eta=0.4

restricted to the 17 manifolds that remain after excluding those with

d>5

and

D\leq d+1

. Lower panel: signed error (%) of all eight objectives on the 14-manifold complete-case set of Section S4.2 , with negative values denoting underestimation. Entries are mean

\pm

standard deviation over the ten data replicates of the noise design; its

\eta=0

column therefore differs slightly from the five-replicate clean values of Table S3 .

Objective	Estimated ID	MPE (%)
MiND distance-only	$71.20\pm 1.75$	$2.48\pm 1.72$
Gride distance-only	$64.63\pm 1.59$	$7.67\pm 2.27$
Full angle-only	$6.26\pm 0.03$	$91.06\pm 0.05$
Profiled angle-only	$56.07\pm 3.26$	$19.90\pm 4.65$
MiND–Full	$22.80\pm 1.76$	$67.43\pm 2.51$
MiND–Profiled	$66.67\pm 1.81$	$4.76\pm 2.58$

Appendix

Table S6: Eight calibrated objectives on amplitude-heterogeneous GSM samples at

d=70

D=100

\sigma_{S}=0.25

, over 30 replicates. Entries are mean

\pm

standard deviation.

$\sigma_{S}$	$\hat{\nu}$	Distance-only	Full	Profiled
$0$	$1.137\pm 0.003$	$65.07\pm 1.01$	$71.90\pm 3.30$	$64.37\pm 0.89$
$0.1$	$1.055\pm 0.008$	$64.17\pm 1.31$	$52.05\pm 3.23$	$62.83\pm 1.09$
$0.15$	$0.980\pm 0.010$	$64.60\pm 1.30$	$35.87\pm 2.66$	$62.47\pm 1.25$
$0.2$	$0.903\pm 0.012$	$66.93\pm 1.14$	$28.49\pm 2.50$	$63.83\pm 1.55$
$0.25$	$0.832\pm 0.014$	$71.20\pm 1.75$	$22.80\pm 1.76$	$66.67\pm 1.81$
$0.3$	$0.769\pm 0.014$	$76.97\pm 1.96$	$19.95\pm 1.55$	$70.20\pm 1.88$

Appendix

Table S7: Observed mean direction

\hat{\nu}

and MiND estimates across the GSM amplitude sweep (

d=70

D=100

; 30 replicates). Entries are mean

\pm

standard deviation. The known-amplitude control at

\sigma_{S}=0.25

reuses the latent draws of the

\sigma_{S}=0

level, so its values are those of the

\sigma_{S}=0

row.

Figure S6: Complete KL analysis for MNIST digit 3 and ImageNet koala using one fixed reference replicate and the MiND distance model. The legend distinguishes components by color, line style, and marker; enlarged symbols mark the Full and Profiled joint minima.

	raw			centered radial			contrast-norm.
Dataset	$\hat{\nu}$	Full	Profiled	$\hat{\nu}$	Full	Profiled	$\hat{\nu}$	Full	Profiled
MNIST 3	1.166	20.4	20.2	1.198	20.2	20.0	1.135	20.8	20.8
MNIST 7	1.157	14.4	14.8	1.201	14.8	14.8	1.143	14.8	14.8
CIFAR-10 bird	0.898	18.8	34.0	0.982	22.5	33.0	1.064	28.2	34.0
CIFAR-10 cat	0.946	21.2	32.8	1.030	24.4	32.8	1.087	30.6	35.4
ImageNet koala	0.934	24.9	45.0	1.028	40.6	53.4	1.080	72.2	70.0

Appendix

Table S8: Normalization controls. The observed-data mean direction and Full and Profiled estimates are reported for raw, centered-radial, and contrast-normalized data over five reference replicates. Full and Profiled denote MiND–Full and MiND–Profiled.

Figure S7: Per-center mean direction against normalized centered norm

a_{i}/\bar{a}

for raw and contrast-normalized ImageNet koala data. Normalization largely removes the broad norm tail and weakens the negative relation. Solid lines are least-squares trends; the dashed horizontal line marks

\pi/3

, and

r

denotes the Pearson correlation between centered norm and per-center mean direction.

Source	Class	$N$	$D$	$\hat{\nu}$	$\operatorname{CV}(a)$	$r(\hat{\nu}_{i},\\|x_{i}-\bar{x}\\|)$
CIFAR-10	airplane	5000	3072	0.939	0.291	-0.08
CIFAR-10	automobile	5000	3072	1.037	0.219	-0.04
CIFAR-10	bird	5000	3072	0.898	0.330	-0.10
CIFAR-10	cat	5000	3072	0.946	0.256	-0.08
CIFAR-10	deer	5000	3072	0.903	0.312	-0.13
CIFAR-10	dog	5000	3072	0.938	0.243	-0.10

Appendix

Table S9: Per-class angular statistics at

k=10

. Here

r

is the Pearson correlation between per-center mean direction and centered norm, and

\operatorname{CV}(a)

is the population standard deviation of

a_{i}=\|x_{i}-\bar{x}\|_{2}

divided by their mean, with

\bar{x}

the class mean. ImageNet class codes are WordNet synset identifiers.

Architecture	Peak layer	depth (%)	$\hat{\nu}$	Full	Profiled	Median paired Full $-$ Profiled
AlexNet	second max-pooling checkpoint	22.2	0.800	17	115	-100
VGG-16	second max-pooling checkpoint	23.5	0.802	18	197	-179
ResNet-18	second residual stage	47.4	0.916	37	248	-214
ResNet-34	second residual stage	42.9	0.919	37	245	-208

Appendix

Table S10: Peak of the median Gride–Profiled curve across seven categories and three subsamples. Full and Profiled denote Gride–Full and Gride–Profiled, and the largest absolute difference between them occurs at the same layer in each architecture. The last column is the median of the per-cell paired Full

-

Profiled differences across category–subsample cells, not the difference of the two reported medians. Layer names denote mathematical checkpoints along the sequence of network transformations. Lower panel: mean direction at the first pooling checkpoint and at the output layer, with the output medians of both objectives and their paired difference.

Intrinsic Dimensionality

Intrinsic dimensionality measures the effective degrees of freedom underlying data embedded in high-dimensional spaces.

See all 56 papers

Mar 11, 2026

A Universal Nearest-Neighbor Estimator for Intrinsic Dimensionality

Estimating the intrinsic dimensionality (ID) of data is a fundamental problem in machine learning and computer vision, providing insight into the true degrees of freedom underlying high-dimensional observations. Existing methods often rely on geometric or distributional assumptions and can significantly fail when these assumptions are violated. In this paper, we introduce a novel ID estimator based on nearest-neighbor distance ratios that involves simple calculations and achieves state-of-the-art results. Most importantly, we provide a theoretical analysis proving that our estimator is universal, namely, it converges to the true ID independently of the distribution generating the data. We present experimental results on benchmark manifolds and real-world datasets to demonstrate the performance of our estimator.

Eng-Jon Ong, Omer Bobrowski, Gesine Reinert +1

School of Mathematical Sciences, Queen Mary University of London · Department of Statistics, Oxford University

Intrinsic Dimensionality

Ask PDF

Apr 22, 2026

Rethinking Intrinsic Dimension Estimation in Neural Representations

The analysis of neural representation has become an integral part of research aiming to better understand the inner workings of neural networks. While there are many different approaches to investigate neural representations, an important line of research has focused on doing so through the lens of intrinsic dimensions (IDs). Although this perspective has provided valuable insights and stimulated substantial follow-up research, important limitations of this approach have remained largely unaddressed. In this paper, we highlight a crucial discrepancy between theory and practice of IDs in neural representations, theoretically and empirically showing that common ID estimators are, in fact, not tracking the true underlying ID of the representation. We contrast this negative result with an investigation of the underlying factors that may drive commonly reported ID-related results on neural representation in the literature. Building on these insights, we offer a new perspective on ID estimation in neural representations.

Rickmer Schulte, David Rügamer

LMU Munich, MCML

Intrinsic Dimensionality Neural Representation Geometry Neural Network Interpretability

Ask PDF

Oct 5, 2026

Sample-Optimal Estimation of the Fréchet Inception Distance

The Fréchet Inception Distance (FID) is widely used to evaluate generative models, but its empirical plug-in estimator suffers from finite-sample bias [BSAG18, CF20]. We study the sample complexity

n

of estimating FID to error

ε

between

d

-dimensional Gaussians with bounded mean distance and covariances, when one distribution is known. Our contributions are threefold. (1) We establish tight finite-sample

Θ(\frac{d^2}{n})

bias and

Θ(\frac{d}{n} + \frac {d^2} {n^2})

variance bounds for the empirical plug-in estimator, establishing a

\gtrsim d^2

sample complexity. (2) To debias the empirical plug-in estimator, we generalize the

{\rm FID}_\infty

estimator of [CF20] to extrapolation methods of arbitrary order

k

. We further prove tight bias and variance bounds of

Θ(\frac{d^{k + 2}}{n^{k + 1}})

and

Θ(\frac d n + \frac{d^2}{n^2})

for any order-

k

extrapolation under our framework. (3) We introduce Relative Taylor Debiasing (RTD), a new, computationally efficient FID estimation algorithm using debiasing techniques inspired by U-statistics. We show that RTD achieves an

O(\frac d {ε^2})

sample complexity, and prove that this is optimal. We provide a complementary empirical evaluation of our new estimators. Our experiments on synthetic Gaussians validate the predicted residual bias and support the tightness of our bounds. On ImageNet with Inception embeddings, RTD achieves the lowest mean estimation error at the standard 50K sample budget, while our second-order variance-aware extrapolation estimator (VALE

_2

) uses only 10K samples to achieve accuracy comparable to FID

_\infty

at 50K samples.

Ziyun Chen, Jerry Li, Kevin Tian +1

University of Washington · University of Texas at Austin

Generative AI Evaluation

Ask PDF

Abstract

Intrinsic Dimensionality

Explore similar work

A Universal Nearest-Neighbor Estimator for Intrinsic Dimensionality

Rethinking Intrinsic Dimension Estimation in Neural Representations

Sample-Optimal Estimation of the Fréchet Inception Distance