stat.APSep 4, 2025

DRtool: An Interactive Tool for Analyzing High-Dimensional Clusterings

Authors: Justin Lin, Julia Fukuyama

Organizations: Department of Mathematics, Indiana University · Department of Statistics, Indiana University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

When faced with new data, we often conduct a cluster analysis to obtain a better understanding of the data's structure and the archetypical samples present in the data. However, the increases in data complexity and dimensionality have made this step very tricky. The large proportion of noise in high-dimensional data blurs patterns and trends, making clusters difficult to distinguish. As such, cluster-discovery tools and cluster-verification tools must be adapted to address the difficulties of high-dimensional data. Nonlinear dimension reduction is a step in the right direction, but even these methods are known to produce false structures, especially when mishandled. A common phenomenon that often goes undetected by the untrained eye is over-clustering of the data. In continuation of these efforts, we developed new cluster verification techniques, including visual assessments and a hypothesis test, that help analysts distinguish false clusters and better interpret their high-dimensional clustering results. For ease of use, these new methods are provided in an interactive toolbox available via R package DRTool.

Figures & tables

Algorithm 1 Simplified Medoid Subtree
Figure 1: Example of Algorithm 1 . The first step is to compute the medoid subtree T′T^{\prime} . Then non-medoid degree-2 vertices are replaced with single edges.
Algorithm 2 Robinson-Foulds Distance
Algorithm 3 Simplify Subtree
Figure 2: Example of Algorithm 3 . First, degree-2 vertices belonging to neither group are replaced with single edges. Then adjacent vertices belonging to neither group are collapsed into a single vertex.
Figure 3: MST stability results on the MNIST data set.
Figure 4: Power analysis of the MST test.
Figure 5: UMAP embedding of the MNIST data set colored according to k-means clustering.
Figure 6: Path between class 1 and class 2 in UMAP embedding of MNIST data set.
Figure 7: Path projection plot of classes 1 and 2.
Figure 8: Path projection plot of class 4 clusters.
Figure 9: Path projection plot of class 9 and remainder of cluster.
Figure 10: UMAP embedding of the Wong data set colored according to k-means clustering.
Figure 11: Path projection plot of class 4 clusters.
Figure 12: Heatmap of cell counts within the two Class 4 sub-clusters.
Figure 13: Distribution of cell labels among cells in the two class 4 sub-clusters.
Figure 14: Path projection plot of class 8 clusters.

Explore similar work

CardsList

  1. Beyond Feature Importance: A Comparative Analysis of Pattern Detection Methods in Cluster Interpretation

    Aug 6, 2026Benjamin Connor, Anna Jurek-Loughrey, Lu Bai +1Feature ImportanceClustering

    School of EEECS Queen’s University Belfast 18 Malone Road, Belfast, BT9 5AF

  2. Cluster Analysis with Resampling for Validation and Exploration (CARVE)

    May 29, 2026Kai R. Wycik, Tiffany M. Tang, Tarek M. Zikry +1ClusteringCluster

    Department of Statistics, Columbia University, New York, NY, USA · Center for Theoretical Neuroscience, Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA · Department of Applied and Computational Mathematics and Statistics, University of Notre Dame, Notre Dame, IN, USA +2

  3. Class Angular Distortion Index for Dimensionality Reduction

    May 1, 2026Kaviru Gunaratne, Stephen Kobourov, Jacob MillerDimensionality ReductionDistortion

    Technical University of Munich