The Rashomon Effect for Visualizing High-Dimensional Data

Yiyang Sun; Haiyang Huang; Gaurav Rajesh Parikh; Cynthia Rudin

cs.LGApr 1, 2026

The Rashomon Effect for Visualizing High-Dimensional Data

Authors: Yiyang Sun, Haiyang Huang, Gaurav Rajesh Parikh, Cynthia Rudin

Organizations: Duke University

Abstract

Dimension reduction (DR) is inherently non-unique: multiple embeddings can preserve the structure of high-dimensional data equally well while differing in layout or geometry. In this paper, we formally define the Rashomon set for DR -- the collection of `good' embedding -- and show how embracing this multiplicity leads to more powerful and trustworthy representations. Specifically, we pursue three goals. First, we introduce PCA-informed alignment to steer embeddings toward principal components, making axes interpretable without distorting local neighborhoods. Second, we design concept-alignment regularization that aligns an embedding dimension with external knowledge, such as class labels or user-defined concepts. Third, we propose a method to extract common knowledge across the Rashomon set by identifying trustworthy and persistent nearest-neighbor relationships, which we use to construct refined embeddings with improved local structure while preserving global relationships. By moving beyond a single embedding and leveraging the Rashomon set, we provide a flexible framework for building interpretable, robust, and goal-aligned visualizations.

Figures & tables

Figure 1: Three goals for generating and exploring the Rashomon set for dimension reduction

Figure 2: $\text{PaCMAP}_{\textrm{param}\text{ }}$ embedding with and without PCA-Informed Alignment. The colored curves overlaid on the embeddings are generated by applying the learned parametric DR mapping to points sampled along the first two principal component directions in the original high-dimensional space, thereby visualizing how the DR mapping transforms the PCA axes.

Figure 3: (a) MNIST PaCMAP param embedding, (b) PCA embedding, (c) PCA-informed embedding with $\lambda_{\text{PCA}}$ set to be $0.1$ . It is nicely aligned with the first two principal components while capturing the detailed cluster structure. (d) Triplet PCA score has improved after PCA-informed alignment.

	$\text{UMAP}_{\textrm{param}}$			NCVis			InfoNCE			NegtSNE			$\text{PaCMAP}_{\textrm{param}\text{ }}$
Dataset	Diff	Before	After	Diff	Before	After	Diff	Before	After	Diff	Before	After	Diff	Before	After
AirPlane	0.07	0.67	0.73	0.23	0.71	0.93	0.28	0.65	0.93	0.19	0.73	0.92	0.12	0.77	0.89
COIL20	0.12	0.68	0.8	0.11	0.68	0.79	0.10	0.73	0.83	0.12	0.68	0.8	0.05	0.74	0.79
FMNIST	0.00	0.87	0.87	0.00	0.87	0.88	0.01	0.87	0.88	0.00	0.87	0.87	0.02	0.83	0.85
MNIST	0.04	0.75	0.79	0.03	0.76	0.79	0.02	0.78	0.8	0.02	0.76	0.79	0.02	0.74	0.76
Mammoth	0.02	0.91	0.93	0.02	0.92	0.93	0.01	0.95	0.96	-0.01	0.91	0.9	-0.03	0.92	0.89

Table 1: Triplet PCA Score for embeddings before and after PCA-alignment within different datasets, orange are the scores that slightly decrease and the teal are the scores that improve. The almost uniform improvement indicates better alignment with the principal components; the fact that the improvements are small is consistent with preservation of local structure, i.e., staying within the Rashomon set.

Figure 4: (a) Concept-informed aligned $\text{PaCMAP}_{\textrm{param}\text{ }}$ embedding. Alignment is along the horizontal axis from feet (left) to head (right). Footwear is labeled in shades of red to orange, trousers in yellow, dresses in light yellow, pullovers and coats in green, shirts and t-shirts in blue, handbags in purple. (b) Evaluation metrics and losses for FMNIST before and after concept alignment, which remain generally unchanged.

Figure 5: (a) Original $\text{PaCMAP}_{\textrm{param}\text{ }}$ embedding of USPS dataset. (b) Common knowledge embedding using only stable neighbor pairs within the Rashomon set. (c) Quantitative comparison of original vs. combined DR embeddings across three evaluation metrics for five methods.

Figure 6: (a) MNIST embedding before (left) and after (right) common NN pairs are selected, (b) Examples of rejected NN pairs when finding common knowledge through the Rashomon set. These pairs are close in pixel space (but not close along the manifold of any digit).

Appendix figures & tables29 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	# Samples	# features	Tasks Types	Labels
Airplane ( Wu et al., 2015 )	24,141	3	PCA	Airplane Structure
COIL20 ( Nene et al., 1996 )	1,440	16,384	Concept, PCA	20 Objects
FICO ( FICO, 2018 )	9,861	23	Concept	External Risk Score
FMNIST ( Xiao et al., 2017 )	70,000	784	Concept, PCA	10 Clothes Type
Human Cortex ( Zhu et al., 2023 )	43,349	100	Concept, PCA	9 Cell Types
Kang et al. ( Kang et al., 2018 )	13,999	100	Concept, PCA	13 Cell Types

Appendix

Table 2: Data Descriptions and Task Types each dataset corresponds to

Figure 7: Comparison of original COIL20 embedding (left) PCA embedding (middle) and PCA informed embeddings (right) across different methods. We see alignment to principal components across all methods while preserving structure. We show that soft Jaccard distance, triplet PCA score, random triplet PCA score, and $\mathcal{L}_{\textrm{DR}}$ (bottom) remain mostly unchanged and that aligned embeddings consistently maintain structure.

Figure 8: Comparison of original FMNIST embedding (left) PCA embedding (middle) and PCA informed embeddings (right) across different methods. We see alignment to principal components across all methods while preserving structure. We show that soft Jaccard distance and $\mathcal{L}_{\textrm{DR}}$ (bottom) remain mostly unchanged and that aligned embeddings consistently maintain structure. Random Triplet PCA score and Triplet PCA score have shown an improvement after the alignment.

Figure 9: Comparison of original Human Cortex embedding (left) PCA embedding (middle) and PCA informed embeddings (right) across different methods. We see alignment to principal components across all methods while preserving structure. We show that soft Jaccard distance and $\mathcal{L}_{\textrm{DR}}$ (bottom) remain mostly unchanged and that aligned embeddings consistently maintain structure. Random Triplet PCA score and Triplet PCA score have shown an improvement after the alignment.

Figure 10: Comparison of original Kang et al. embedding (left) PCA embedding (middle) and PCA informed embeddings (right) across different methods. We see alignment to principal components across all methods while preserving structure. We show that soft Jaccard distance and $\mathcal{L}_{\textrm{DR}}$ (bottom) remain mostly unchanged and that aligned embeddings consistently maintain structure. Random Triplet PCA score and Triplet PCA score have shown an improvement after the alignment.

Figure 11: Comparison of original Mammoth embedding (left) PCA embedding (middle) and PCA informed embeddings (right) across different methods. We see alignment to principal components across all methods while preserving structure. We show that soft Jaccard distance and $\mathcal{L}_{\textrm{DR}}$ (bottom) remain mostly unchanged and that aligned embeddings consistently maintain structure. Random Triplet PCA score and Triplet PCA score have shown an improvement after the alignment.

Figure 12: Comparison of original Airplane embedding (left) PCA embedding (middle) and PCA informed embeddings (right) across different methods. We see alignment to principal components across all methods while preserving structure. We show that soft Jaccard distance and $\mathcal{L}_{\textrm{DR}}$ (bottom) remain mostly unchanged and that aligned embeddings consistently maintain structure. Random Triplet PCA score and Triplet PCA score have shown an improvement after the alignment.

Figure 13: Comparison of original MNIST embedding (left) PCA embedding (middle) and PCA informed embeddings (right) across different methods. We see alignment to principal components across all methods while preserving structure. We show that soft Jaccard distance, and $\mathcal{L}_{\textrm{DR}}$ (bottom) remain mostly unchanged and that aligned embeddings consistently maintain structure.

Figure 14: Comparison of original Stuart et al . embedding (left) PCA embedding (middle) and PCA informed embeddings (right) across different methods. We see alignment to principal components across all methods while preserving structure. We show that soft Jaccard distance and $\mathcal{L}_{\textrm{DR}}$ (bottom) remain mostly unchanged and that aligned embeddings consistently maintain structure. Random Triplet PCA score and Triplet PCA score have shown an improvement after the alignment.

Figure 15: Comparison of original CBMC embedding (left) PCA embedding (middle) and PCA informed embeddings (right) across different methods. We see alignment to principal components across all methods while preserving structure. We show that soft Jaccard distance, triplet PCA score, random triplet PCA score, and $\mathcal{L}_{\textrm{DR}}$ (bottom) remain mostly unchanged and that aligned embeddings consistently maintain structure.

Figure 16: Comparison of original USPS embedding (left) PCA embedding (middle) and PCA informed embeddings (right) across different methods. We see alignment to principal components across all methods while preserving structure and $\mathcal{L}_{\textrm{DR}}$ $\mathcal{L}_{\textrm{DR}}$ (bottom) remaining mostly unchanged and that aligned embeddings consistently maintain structure. Random Triplet PCA score and Triplet PCA score have shown an improvement after the alignment.

Figure 17: Comparison of original and aligned embeddings for MNIST using a concept-aware regularizer. The embeddings’ horizontal axis is aligned with the digit index and embedding evaluation metrics haven’t changed significantly after alignment.

Figure 18: Comparison of original and aligned embeddings for FMNIST using a concept-aware regularizer. The embeddings’ horizontal axis is aligned with the label index (from head to toe as mentioned in Section 6.1 ) and embedding evaluation metrics haven’t changed significantly after alignment.

Figure 19: Comparison of original and aligned embeddings for COIL20 using a concept-aware regularizer. The embeddings’ horizontal axis is aligned with the label index and embedding evaluation metrics haven’t changed significantly after alignment.

Figure 20: Comparison of original and aligned embeddings for FICO using a concept-aware regularize. The embeddings’ horizontal axis is aligned with the external risk score. Since the label is a continuous feature, we are not displaying barplot here.

Figure 21: Comparison of original and aligned embeddings for Human Cortex Single Cell dataset using a concept-aware regularizer. The embeddings’ horizontal axis is aligned with the cell type index, and the embedding performance hasn’t been changed significantly.

Figure 22: Comparison of original and aligned embeddings for Kang et al . dataset using a concept-aware regularizer. The embeddings’ horizontal axis is aligned with the cell type index, and the embedding performance hasn’t been changed significantly.

Figure 23: Comparison of original and aligned embeddings for the Stuart dataset using a concept-aware regularizer. The embeddings’ horizontal axis is aligned with the cell type index, and the embedding performance hasn’t been changed significantly.

Figure 24: Comparison of original and aligned embeddings for the CMBC dataset using a concept-aware regularizer. The embeddings’ horizontal axis is aligned with the cell type index, and the embedding performance hasn’t been changed significantly.

Figure 25: Comparison of original and aligned embeddings for USPS dataset using a concept-aware regularizer. Clusters are shown to align with the digits index and structure preservation across multiple DR methods.

Figure 26: MNIST $\text{PaCMAP}_{\textrm{param}\text{ }}$ Embedding under different label missingness ratios (rows) and label weights (columns). High label weight with high missingness ratio breaks the original structure.

Figure 27: $\mathcal{L}_{\textrm{DR}}$ and Jaccard Distance under different label missingness ratios.

Figure 28: MNIST embeddings and improved embedding using common knowledge. Combined DR improves structure and boosts downstream metrics.

Figure 29: FMNIST embeddings and improved embedding using common knowledge. Combined DR improves structure and boosts downstream metrics. NCVis and Neg-t-SNE and UMAP are showing a bit lower silhouette score due to the mixed area at the bottom right corner

Figure 30: USPS embeddings and improved embedding using common knowledge. Combined DR improves structure and boosts downstream metrics.

Figure 31: Kang et al. embeddings and improved embedding using common knowledge. Combined DR improves structure and boosts downstream metrics.

Figure 32: Human Cortex embeddings and improved embedding using common knowledge. Combined DR improves structure and boosts downstream metrics. NCVis and Neg-t-SNE and UMAP are showing a bit lower silhouette score due to the mixed area at the bottom right corner

Figure 33: Stuart et. al. embeddings and improved embedding using common knowledge. Combined DR improves structure and boosts downstream metrics.

Figure 34: COIL20 embeddings and improved embedding using common knowledge. Combined DR improves structure and boosts downstream metrics.

Dimensionality Reduction

Dimensionality reduction maps high-dimensional data to compact representations for visualization, prediction, compression, and scientific modeling.

See all 145 papers

When One Point Is Not Enough: Addressing Ambiguous Instances in Dimensionality Reduction by Splitting
May 22, 2026Diede P. M. van der Hoorn, Alessio Arleo, Fernando V. PaulovichData Visualization Dimensionality Reduction
Eindhoven University of Technology
Visualizing High-Dimensional Graph Embeddings via Informed Multi-View Projections
Jun 30, 2026Ya Ji, Xuefeng Li, Timo Brand +4Scientific Visualization Data Visualization
Khoury College of Computer Sciences, Northeastern University, Seattle · School of Computation, Information and Technology, Technical University of Munich, Heilbronn, Germany
Dimensionality Reduction Meets Network Science: Sensemaking on UMAP's kNN Graph
Jul 9, 2026Duen Horng Chau, Donghao Ren, Fred Hohman +1Dimensionality Reduction
Apple

May 22, 2026J/K move · Enter open · S save

When One Point Is Not Enough: Addressing Ambiguous Instances in Dimensionality Reduction by Splitting

Diede P. M. van der Hoorn, Alessio Arleo, Fernando V. Paulovich

Eindhoven University of Technology

Data Visualization Dimensionality Reduction

Open paper AskPDF arXiv

Dimensionality Reduction (DR) methods are widely used to visualize high-dimensional data. One key task in DR-based analysis is discovering neighborhoods, which relies on analyzing the fine-grained local structure of a projection. However, DR is an inherently lossy process; no technique can perfectly preserve the high-dimensional relationships, and projections therefore contain visual artifacts. In this paper, we highlight a typically overlooked source of visual artifacts: ambiguous instances. These are instances that are highly similar to multiple mutually dissimilar neighborhoods in the high-dimensional space. Standard DR methods cannot faithfully project such instances, since each data instance is mapped to a single point in the visual space. As a result, such an instance is placed in only one of its neighborhoods (or in none at all), so only part of its neighborhood structure is represented. We call this distortion partial neighborhood embedding. In this paper, we introduce a graph-based approach that identifies ambiguous instances and replicates them as multiple points in the projection, placing each copy within its respective neighborhood. We use UMAP for our results, but our approach also generalizes to other local graph-based DR techniques, and we show that our approach reveals previously hidden neighborhood memberships in projections and reduces partial neighborhood embedding across multiple examples, and is further supported by quantitative analyses.

Abstract

Dimensionality Reduction

Explore similar work

When One Point Is Not Enough: Addressing Ambiguous Instances in Dimensionality Reduction by Splitting

Visualizing High-Dimensional Graph Embeddings via Informed Multi-View Projections

Dimensionality Reduction Meets Network Science: Sensemaking on UMAP's kNN Graph