Learning Unified Distance Metric for Heterogeneous Attribute Data Clustering

Yiqun Zhang; Mingjie Zhao; Yizhou Chen; Yang Lu; Yiu-ming Cheung

cs.LGMar 3, 2026

Learning Unified Distance Metric for Heterogeneous Attribute Data Clustering

Authors: Yiqun Zhang, Mingjie Zhao, Yizhou Chen, Yang Lu, Yiu-ming Cheung

Organizations: School of Computer Science and Technology, Guangdong University of Technology, Guangzhou, 510006, Guangdong, China · Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, Xiamen, 361102, Fujian, China · Department of Computer Science, Hong Kong Baptist University, Hong Kong SAR, China

Abstract

Datasets composed of numerical and categorical attributes (also called mixed data hereinafter) are common in real clustering tasks. Differing from numerical attributes that indicate tendencies between two concepts (e.g., high and low temperature) with their values in well-defined Euclidean distance space, categorical attribute values are different concepts (e.g., different occupations) embedded in an implicit space. Simultaneously exploiting these two very different types of information is an unavoidable but challenging problem, and most advanced attempts either encode the heterogeneous numerical and categorical attributes into one type, or define a unified metric for them for mixed data clustering, leaving their inherent connection unrevealed. This paper, therefore, studies the connection among any-type of attributes and proposes a novel Heterogeneous Attribute Reconstruction and Representation (HARR) learning paradigm accordingly for cluster analysis. The paradigm transforms heterogeneous attributes into a homogeneous status for distance metric learning, and integrates the learning with clustering to automatically adapt the metric to different clustering tasks. Differing from most existing works that directly adopt defined distance metrics or learn attribute weights to search clusters in a subspace. We propose to project the values of each attribute into unified learnable multiple spaces to more finely represent and learn the distance metric for categorical data. HARR is parameter-free, convergence-guaranteed, and can more effectively self-adapt to different sought number of clusters $k$ . Extensive experiments illustrate its superiority in terms of accuracy and efficiency.

Figures & tables

Figure 1: Conventional taxonomy of attributes, which shows the traditional classification of attributes into Numerical and Categorical types. Categorical Attributes are further divided into Ordinal Attributes, with inherent order (e.g., small, medium, large), and Nominal Attributes, without natural order (e.g., colors or labels). This taxonomy emphasizes the distinct characteristics of each attribute type.

Symbol	Explanation	Symbol	Explanation
$X$ , $A$ , $O$ , $M$ , and $C$	Dataset, attribute set, possible value set, cluster prototypes set, and cluster set	$\mathbf{x}_{i}$ and $x_{i}^{r}$	$i$ -th data object and $r$ -th value of $\mathbf{x}_{i}$
$a^{r}$ , $v^{r}$	$r$ -th attribute, possible value number of $a^{r}$	$A_{u}$ and $A_{c}$	Numerical and categorical attribute set
$A_{n}$ and $A_{o}$	Nominal and ordinal attribute set of categorical attributes, $A_{c}=A_{n}\cup A_{o}$	$d$ , $k$ and $n$	Number of attributes( $d=d_{u}+d_{c}$ ), clusters, and data objects
$d_{c}$ and $d_{u}$	Number of categorical attributes ( $d_{c}=d_{n}+d_{o}$ ), and numerical attributes	$o_{h}^{r}$	$h$ -th possible value of $a^{r}$
$\mathbf{Q}$ and $c_{l}$	Object-cluster affiliation matrix and $l$ -th cluster	$q_{il}$	A value indicating the affiliation between $\textbf{x}_{i}$ and $c_{l}$
$\textbf{m}_{l}$	A vector describing data objects of $c_{l}$	$\gamma^{r}$	Number of endogenous spaces corresponding to $a^{r}$

Table 1: Frequently used symbols. Note that we uniformly use lowercase, uppercase, bold lowercase, and bold uppercase to indicate value, set, vector, and matrix, respectively.

Figure 2: Comparison of the “concepts” of numerical attribute, nominal attribute, and ordinal attribute, which illustrates how “concepts" and “values" are represented across different attribute types. Numerical attributes, like “income," maintain a continuous order (e.g., high to low). Nominal attributes, such as “occupation," consist of distinct categories without inherent order (e.g., nurse, lawyer, driver). Ordinal attributes, like “recommendation," contain ordered categories (e.g., accept, weak accept, marginal, reject), highlighting their hierarchical structure.

Figure 3: Relationships among different types of attributes in base distances computation. “A $\rightarrow$ B” means that base distances of Type-B attributes are computed with the contribution of Type-A attributes.

Figure 4: Projection processes of a categorical attribute $a^{r}$ . An attribute $a^{r}$ with $4$ possible values is expanded into $6$ distinct attributes after the projection process. Each projected attribute represents unique information, highlighting the rich information content of the original attribute.

Figure 5: Comparison of Hamming distance, one-hot encoding, and categorical attribute representation obtained through our method. It is obvious that our representation is more informative, and provides a homogeneous basis for distance computation on heterogeneous attributes.

No.	Dataset	Abbrev.	$d_{u}$	$d_{c}$	$d_{n}$	$d_{o}$	$n$	$k^{*}$	No. of Possible Values of Categorical Attributes
1	Inflammations Diagnosis	DS	1	5	5	0	120	2	{2, 2, 2, 2, 2}
2	Heart Failure	HF	7	5	5	0	299	2	{2, 2, 2, 2, 2}
3	Autism-Adolescent	AA	2	7	7	0	104	2	{2, 9, 2, 2, 33, 2, 6}
4	Amphibians	AP	2	12	6	6	189	2	{8, 5, 8, 7, 8, 3, 5, 6, 6, 6, 3, 2}
5	Dermatology	DT	1	33	1	32	358	6	{2, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 3, 4, 4, 4, 4,
									4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4}

Table 2: Statistics of the 14 real public datasets, including the number of numerical attributes

d_{u}

, categorical attributes

d_{c}

, nominal and ordinal categorical attributes

d_{n}

and

d_{o}

, i.e.,

d^{c}=d^{n}+d^{o}

, data objects

n

, true number of clusters

k^{*}

, and the number of possible values of categorical attributes.

Approach	DS	HF	AA	AP	DT	AC
KMD/KPT	0.2881 $\pm$ 0.2718	0.0693 $\pm$ 0.0662	-0.0027 $\pm$ 0.0125	-0.0068 $\pm$ 0.0129	0.4839 $\pm$ 0.1044	0.3253 $\pm$ 0.1025
OHE+OC	0.2745 $\pm$ 0.3484	0.0014 $\pm$ 0.0082	-0.0042 $\pm$ 0.0087	-0.0162 $\pm$ 0.0080	0.6402 $\pm$ 0.1597	0.1412 $\pm$ 0.2065
SBC	0.1728 $\pm$ 0.1719	-0.0022 $\pm$ 0.0067	-0.0106 $\pm$ 0.0038	-0.0073 $\pm$ 0.0005	0.5837 $\pm$ 0.1508	0.3622 $\pm$ 0.0856
JDM	0.2243 $\pm$ 0.1923	0.0039 $\pm$ 0.0099	-0.0028 $\pm$ 0.0137	-0.0043 $\pm$ 0.0162	0.5926 $\pm$ 0.1446	0.1451 $\pm$ 0.1376
CMS	0.3815 $\pm$ 0.1064	0.0782 $\pm$ 0.0066	0.0018 $\pm$ 0.0142	-0.0034 $\pm$ 0.0125	0.5099 $\pm$ 0.0970	0.1530 $\pm$ 0.0165
UDM	0.3943 $\pm$ 0.2863	0.0788 $\pm$ 0.0061	0.0069 $\pm$ 0.0208	-0.0005 $\pm$ 0.0293	0.6207 $\pm$ 0.1285	0.3769 $\pm$ 0.0436

Table 3: ARI performance of 12 different clustering approaches on six mixed datasets. The interval of ARI is

[-1,1]

, and a larger value of ARI indicates a better clustering performance. The best and second-best results on each dataset are highlighted in bold and underlined , respectively. The experimental results of KMD and KPT are combined and reported as “KMD/KPT”.

Approach	SB	SF	T3	HR	LG	MR	LE	SW
KMD/KPT	0.3185 $\pm$ 0.0362	0.2083 $\pm$ 0.0649	0.0247 $\pm$ 0.0318	-0.0054 $\pm$ 0.0085	0.0683 $\pm$ 0.0986	0.3420 $\pm$ 0.2311	0.0353 $\pm$ 0.0180	0.0485 $\pm$ 0.0168
OHE+OC	0.3913 $\pm$ 0.0446	0.1961 $\pm$ 0.0509	0.0109 $\pm$ 0.0098	0.0099 $\pm$ 0.0252	0.0892 $\pm$ 0.0563	0.4629 $\pm$ 0.2502	0.0343 $\pm$ 0.0207	0.0004 $\pm$ 0.0017
SBC	0.3871 $\pm$ 0.0254	0.1483 $\pm$ 0.0333	0.0297 $\pm$ 0.0182	-0.0097 $\pm$ 0.0098	0.0065 $\pm$ 0.0029	0.4591 $\pm$ 0.2494	0.0295 $\pm$ 0.0127	0.0577 $\pm$ 0.0115
JDM	0.3826 $\pm$ 0.0366	0.1225 $\pm$ 0.0355	0.0244 $\pm$ 0.0267	-0.0011 $\pm$ 0.0086	0.0395 $\pm$ 0.0602	0.3191 $\pm$ 0.1839	0.0359 $\pm$ 0.0115	0.0506 $\pm$ 0.0191
CMS	0.3356 $\pm$ 0.0449	0.2972 $\pm$ 0.0660	0.0171 $\pm$ 0.0217	-0.0011 $\pm$ 0.0063	0.0824 $\pm$ 0.0759	0.4210 $\pm$ 0.2690	0.0384 $\pm$ 0.0193	0.0608 $\pm$ 0.0159
UDM	0.3673 $\pm$ 0.0411	0.2800 $\pm$ 0.0703	0.0178 $\pm$ 0.0250	0.0340 $\pm$ 0.0354	0.0860 $\pm$ 0.0788	0.3675 $\pm$ 0.2519	0.0425 $\pm$ 0.0185	0.0620 $\pm$ 0.0132

Table 4: ARI performance of 12 different clustering approaches on eight categorical datasets. The interval of ARI is

[-1,1]

Approach	DS	HF	AA	AP	DT	AC
KMD/KPT	0.7375 $\pm$ 0.1352	0.6224 $\pm$ 0.0668	0.5361 $\pm$ 0.0330	0.5299 $\pm$ 0.0198	0.5964 $\pm$ 0.0893	0.7784 $\pm$ 0.0658
OHE+OC	0.7196 $\pm$ 0.1527	0.5415 $\pm$ 0.0330	0.5322 $\pm$ 0.0283	0.5079 $\pm$ 0.0097	0.6753 $\pm$ 0.1240	0.6267 $\pm$ 0.1442
SBC	0.6846 $\pm$ 0.1085	0.5251 $\pm$ 0.0235	0.5173 $\pm$ 0.0067	0.5410 $\pm$ 0.0024	0.6264 $\pm$ 0.1276	0.7946 $\pm$ 0.0646
JDM	0.7154 $\pm$ 0.1105	0.5580 $\pm$ 0.0259	0.5385 $\pm$ 0.0391	0.5479 $\pm$ 0.0326	0.6536 $\pm$ 0.1137	0.6633 $\pm$ 0.1031
CMS	0.7686 $\pm$ 0.0479	0.6436 $\pm$ 0.0054	0.5673 $\pm$ 0.0099	0.5397 $\pm$ 0.0284	0.6096 $\pm$ 0.0768	0.6971 $\pm$ 0.0102
UDM	0.7900 $\pm$ 0.1302	0.6441 $\pm$ 0.0050	0.5745 $\pm$ 0.0202	0.5614 $\pm$ 0.0402	0.6679 $\pm$ 0.0995	0.8069 $\pm$ 0.0175

Table 5: CA performance of 12 different clustering approaches on six mixed datasets. The interval of CA is

[0,1]

, and a larger value of CA indicates a better clustering performance. The best and second-best results on each dataset are highlighted in bold and underlined , respectively. The experimental results of KMD and KPT are combined and reported as “KMD/KPT”.

Approach	SB	SF	T3	HR	LG	MR	LE	SW
KMD/KPT	0.5021 $\pm$ 0.0436	0.4707 $\pm$ 0.0523	0.5692 $\pm$ 0.0499	0.3826 $\pm$ 0.0240	0.6028 $\pm$ 0.0933	0.7645 $\pm$ 0.1282	0.3276 $\pm$ 0.0344	0.3777 $\pm$ 0.0348
OHE+OC	0.5348 $\pm$ 0.0433	0.4506 $\pm$ 0.0393	0.5548 $\pm$ 0.0266	0.3989 $\pm$ 0.0561	0.6444 $\pm$ 0.0568	0.8167 $\pm$ 0.1277	0.3455 $\pm$ 0.1500	0.0210 $\pm$ 0.0941
SBC	0.5368 $\pm$ 0.0300	0.4130 $\pm$ 0.0284	0.5825 $\pm$ 0.0378	0.3591 $\pm$ 0.0349	0.5588 $\pm$ 0.0062	0.8145 $\pm$ 0.1295	0.3172 $\pm$ 0.0263	0.3812 $\pm$ 0.0195
JDM	0.5417 $\pm$ 0.0412	0.4046 $\pm$ 0.0306	0.5664 $\pm$ 0.0455	0.3932 $\pm$ 0.0269	0.5842 $\pm$ 0.0707	0.7613 $\pm$ 0.1101	0.3142 $\pm$ 0.0240	0.3738 $\pm$ 0.0274
CMS	0.5177 $\pm$ 0.0476	0.5042 $\pm$ 0.0484	0.5551 $\pm$ 0.0399	0.3962 $\pm$ 0.0209	0.6282 $\pm$ 0.0783	0.7958 $\pm$ 0.1368	0.3274 $\pm$ 0.0379	0.3866 $\pm$ 0.0292
UDM	0.5376 $\pm$ 0.0391	0.5020 $\pm$ 0.0532	0.5544 $\pm$ 0.0433	0.4470 $\pm$ 0.0521	0.6289 $\pm$ 0.0828	0.7757 $\pm$ 0.1296	0.3324 $\pm$ 0.0391	0.3819 $\pm$ 0.0231

Table 6: CA performance of 12 different clustering approaches on eight categorical datasets. The interval of CA is

[0,1]

Panel of Figure 6 — Figure 6: Average performance and ranks of different approaches based on the results in Tables 3 - 6 . The methods KMD/KPT, OHE+OC, SBC, JDM, CMS, UDM, HOD, HARR-V, and HARR-M are denoted as A, B, C, D, E, F, G, H-V, and H-M, respectively.

Figure 7: Results of the two-tailed BD tests on ARI (a) and CA (b) performances across 12 methods on 14 datasets, based on the results from Tables 3 - 6 . Confidence intervals are set at 0.95 (

\alpha=0.05

) and 0.9 (

\alpha=0.1

), the corresponding Critical Difference (CD) intervals CD_95%=3.8158 and CD_90%=3.4069, respectively.

Data	KMD/KPT	BD	HAR	HARR-V	HARR-M
DS	0.2881 $\pm$ 0.2718	0.3374 $\pm$ 0.2798	0.3374 $\pm$ 0.2798	0.3849 $\pm$ 0.1897	0.4599 $\pm$ 0.2409
HF	0.0693 $\pm$ 0.0662	0.0793 $\pm$ 0.0181	0.0793 $\pm$ 0.0181	0.0882 $\pm$ 0.0000	0.0882 $\pm$ 0.0000
AA	-0.0027 $\pm$ 0.0125	-0.0030 $\pm$ 0.0106	0.0430 $\pm$ 0.0313	0.0278 $\pm$ 0.0283	0.0289 $\pm$ 0.0285
AP	-0.0068 $\pm$ 0.0129	-0.0020 $\pm$ 0.0288	-0.0002 $\pm$ 0.0300	-0.0016 $\pm$ 0.0456	0.0412 $\pm$ 0.0534
DT	0.4839 $\pm$ 0.1044	0.6322 $\pm$ 0.1343	0.6322 $\pm$ 0.1343	0.6403 $\pm$ 0.1694	0.6826 $\pm$ 0.1429
AC	0.3253 $\pm$ 0.1025	0.3773 $\pm$ 0.0455	0.3773 $\pm$ 0.0455	0.3453 $\pm$ 0.0000	0.4200 $\pm$ 0.0000

Table 7: ARI performance of five ablated HARR versions on 14 datasets. The interval of ARI is

[-1,1]

, and a larger value of ARI indicates a better clustering performance.

\overline{AR}

row reports the average ranks.

Data	KMD/KPT	BD	HAR	HARR-V	HARR-M
DS	0.7375 $\pm$ 0.1352	0.7650 $\pm$ 0.1289	0.7650 $\pm$ 0.1289	0.7942 $\pm$ 0.1084	0.8187 $\pm$ 0.1247
HF	0.6224 $\pm$ 0.0668	0.6436 $\pm$ 0.0145	0.6436 $\pm$ 0.0145	0.6522 $\pm$ 0.0000	0.6522 $\pm$ 0.0000
AA	0.5361 $\pm$ 0.0330	0.5404 $\pm$ 0.0302	0.6197 $\pm$ 0.0475	0.6062 $\pm$ 0.0305	0.6062 $\pm$ 0.0302
AP	0.5299 $\pm$ 0.0198	0.5561 $\pm$ 0.0381	0.5614 $\pm$ 0.0446	0.5550 $\pm$ 0.0561	0.6111 $\pm$ 0.0535
DT	0.5964 $\pm$ 0.0893	0.6811 $\pm$ 0.1071	0.6811 $\pm$ 0.1071	0.6793 $\pm$ 0.1411	0.7161 $\pm$ 0.1130
AC	0.7784 $\pm$ 0.0658	0.8070 $\pm$ 0.0182	0.8070 $\pm$ 0.0182	0.7942 $\pm$ 0.0000	0.8246 $\pm$ 0.0000

Table 8: CA performance of five ablated HARR versions on 14 datasets. The interval of CA is

[0,1]

, and a larger value of CA indicates a better clustering performance.

\overline{AR}

row reports the average ranks.

\overline{IP}

row reports the improved percentage achieved by the current one w.r.t. the previous one. In Table 7 , since ARI is in the interval

[-1,1]

\overline{IP}

results are not reported.

Panel of Figure 8 — Figure 8: ARI and CA performance of different approaches with and without distinguishing ordinal attributes on AP, DT, AC, HR, and LG datasets. The two numbers in parentheses in the legend report the value of $d_{n}$ + $d_{o}$ used to treat the dataset.

Panel of Figure 9 — Figure 9: Total object-cluster distance $z$ of HARR-V and HARR-M on different mixed data sets. The blue circles and red boxes indicate that the weights were updated by HARR-V and HARR-M, respectively.

Figure 10: Comparison of the execution time of CMS, UDM, HDM, GBD, FBD, HARR-V, and HARR-M on a large synthetic dataset (

n=100,000

) sampled at different sampling rates

\varphi=\{0.001,0.2,0.4,0.6,0.8,1\}

Panel of Figure 11 — Figure 11: t-SNE visualization of the MR dataset represented by OHE, GBD, FBD, HARR-V, and HARR-M. Data points are marked in different colors according to the ‘true’ labels. A higher concentration of data points with the same label indicates that the corresponding method’s distance metric better discriminates between clusters.

Clustering

Clustering groups data points by shared structure, developing methods to discover, validate, and interpret groups across diverse data and constraints.

See all 345 papers

Learnable Weighting of Intra-Attribute Distances for Categorical Data Clustering with Nominal and Ordinal Attributes
Jul 6, 2026Yiqun Zhang, Yiu-ming CheungClustering Metric Learning
GRACE: LLM-Grounded Semantic Metric Spaces for Scalable Mixed-Data Clustering
Aug 8, 2026Zihua Yang, Zhencheng Xie, Junyang Chen +4Clustering Unsupervised Clustering
Guangdong University of Technology · Tsinghua University · Peking University +2
Bridge the Gaps: Heterogeneous Attributed Graph Clustering via Quaternion Representation Learning
Jun 22, 2026Xinxi Chen, Junyang Chen, Yiqun Zhang +2Graph Structure Learning Graph Neural Networks
School of Computer Science and Technology, Guangdong University of Technology, Guangzhou 510006, China

Jul 6, 2026J/K move · Enter open · S save

Learnable Weighting of Intra-Attribute Distances for Categorical Data Clustering with Nominal and Ordinal Attributes

Yiqun Zhang, Yiu-ming Cheung

Clustering Metric Learning

Open paper AskPDF arXiv

The success of categorical data clustering generally much relies on the distance metric that measures the dissimilarity degree between two objects. However, most of the existing clustering methods treat the two categorical subtypes, i.e. nominal and ordinal attributes, in the same way when calculating the dissimilarity without considering the relative order information of the ordinal values. Moreover, there would exist interdependence among the nominal and ordinal attributes, which is worth exploring for indicating the dissimilarity. This paper will therefore study the intrinsic difference and connection of nominal and ordinal attribute values from a perspective akin to the graph. Accordingly, we propose a novel distance metric to measure the intra-attribute distances of nominal and ordinal attributes in a unified way, meanwhile preserving the order relationship among ordinal values. Subsequently, we propose a new clustering algorithm to make the learning of intra-attribute distance weights and partitions of data objects into a single learning paradigm rather than two separate steps, whereby circumventing a suboptimal solution. Experiments show the efficacy of the proposed algorithm in comparison with the existing counterparts.

Abstract

Clustering

Explore similar work

Learnable Weighting of Intra-Attribute Distances for Categorical Data Clustering with Nominal and Ordinal Attributes

GRACE: LLM-Grounded Semantic Metric Spaces for Scalable Mixed-Data Clustering

Bridge the Gaps: Heterogeneous Attributed Graph Clustering via Quaternion Representation Learning