Organizations: School of Computer Science and Technology, Guangdong University of Technology, Guangzhou, 510006, Guangdong, China · Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, Xiamen, 361102, Fujian, China · Department of Computer Science, Hong Kong Baptist University, Hong Kong SAR, China
Datasets composed of numerical and categorical attributes (also called mixed data hereinafter) are common in real clustering tasks. Differing from numerical attributes that indicate tendencies between two concepts (e.g., high and low temperature) with their values in well-defined Euclidean distance space, categorical attribute values are different concepts (e.g., different occupations) embedded in an implicit space. Simultaneously exploiting these two very different types of information is an unavoidable but challenging problem, and most advanced attempts either encode the heterogeneous numerical and categorical attributes into one type, or define a unified metric for them for mixed data clustering, leaving their inherent connection unrevealed. This paper, therefore, studies the connection among any-type of attributes and proposes a novel Heterogeneous Attribute Reconstruction and Representation (HARR) learning paradigm accordingly for cluster analysis. The paradigm transforms heterogeneous attributes into a homogeneous status for distance metric learning, and integrates the learning with clustering to automatically adapt the metric to different clustering tasks. Differing from most existing works that directly adopt defined distance metrics or learn attribute weights to search clusters in a subspace. We propose to project the values of each attribute into unified learnable multiple spaces to more finely represent and learn the distance metric for categorical data. HARR is parameter-free, convergence-guaranteed, and can more effectively self-adapt to different sought number of clusters k. Extensive experiments illustrate its superiority in terms of accuracy and efficiency.
Figures & tables
Figure 1: Conventional taxonomy of attributes, which shows the traditional classification of attributes into Numerical and Categorical types. Categorical Attributes are further divided into Ordinal Attributes, with inherent order (e.g., small, medium, large), and Nominal Attributes, without natural order (e.g., colors or labels). This taxonomy emphasizes the distinct characteristics of each attribute type.
Symbol
Explanation
Symbol
Explanation
X , A , O , M , and C
Dataset, attribute set, possible value set, cluster prototypes set, and cluster set
xi and xir
i -th data object and r -th value of xi
ar , vr
r -th attribute, possible value number of ar
Au and Ac
Numerical and categorical attribute set
An and Ao
Nominal and ordinal attribute set of categorical attributes, Ac=An∪Ao
d , k and n
Number of attributes( d=du+dc ), clusters, and data objects
dc and du
Number of categorical attributes ( dc=dn+do ), and numerical attributes
ohr
h -th possible value of ar
Q and cl
Object-cluster affiliation matrix and l -th cluster
qil
A value indicating the affiliation between xi and cl
ml
A vector describing data objects of cl
γr
Number of endogenous spaces corresponding to ar
Table 1: Frequently used symbols. Note that we uniformly use lowercase, uppercase, bold lowercase, and bold uppercase to indicate value, set, vector, and matrix, respectively.
Figure 2: Comparison of the “concepts” of numerical attribute, nominal attribute, and ordinal attribute, which illustrates how “concepts" and “values" are represented across different attribute types. Numerical attributes, like “income," maintain a continuous order (e.g., high to low). Nominal attributes, such as “occupation," consist of distinct categories without inherent order (e.g., nurse, lawyer, driver). Ordinal attributes, like “recommendation," contain ordered categories (e.g., accept, weak accept, marginal, reject), highlighting their hierarchical structure.
Figure 3: Relationships among different types of attributes in base distances computation. “A → B” means that base distances of Type-B attributes are computed with the contribution of Type-A attributes.
Figure 4: Projection processes of a categorical attribute ar . An attribute ar with 4 possible values is expanded into 6 distinct attributes after the projection process. Each projected attribute represents unique information, highlighting the rich information content of the original attribute.
Figure 5: Comparison of Hamming distance, one-hot encoding, and categorical attribute representation obtained through our method. It is obvious that our representation is more informative, and provides a homogeneous basis for distance computation on heterogeneous attributes.
Table 2: Statistics of the 14 real public datasets, including the number of numerical attributes du , categorical attributes dc , nominal and ordinal categorical attributes dn and do , i.e., dc=dn+do , data objects n , true number of clusters k∗ , and the number of possible values of categorical attributes.
Approach
DS
HF
AA
AP
DT
AC
KMD/KPT
0.2881 ± 0.2718
0.0693 ± 0.0662
-0.0027 ± 0.0125
-0.0068 ± 0.0129
0.4839 ± 0.1044
0.3253 ± 0.1025
OHE+OC
0.2745 ± 0.3484
0.0014 ± 0.0082
-0.0042 ± 0.0087
-0.0162 ± 0.0080
0.6402 ± 0.1597
0.1412 ± 0.2065
SBC
0.1728 ± 0.1719
-0.0022 ± 0.0067
-0.0106 ± 0.0038
-0.0073 ± 0.0005
0.5837 ± 0.1508
0.3622 ± 0.0856
JDM
0.2243 ± 0.1923
0.0039 ± 0.0099
-0.0028 ± 0.0137
-0.0043 ± 0.0162
0.5926 ± 0.1446
0.1451 ± 0.1376
CMS
0.3815 ± 0.1064
0.0782 ± 0.0066
0.0018 ± 0.0142
-0.0034 ± 0.0125
0.5099 ± 0.0970
0.1530 ± 0.0165
UDM
0.3943 ± 0.2863
0.0788 ± 0.0061
0.0069 ± 0.0208
-0.0005 ± 0.0293
0.6207 ± 0.1285
0.3769 ± 0.0436
Table 3: ARI performance of 12 different clustering approaches on six mixed datasets. The interval of ARI is [−1,1] , and a larger value of ARI indicates a better clustering performance. The best and second-best results on each dataset are highlighted in bold and underlined , respectively. The experimental results of KMD and KPT are combined and reported as “KMD/KPT”.
Approach
SB
SF
T3
HR
LG
MR
LE
SW
KMD/KPT
0.3185 ± 0.0362
0.2083 ± 0.0649
0.0247 ± 0.0318
-0.0054 ± 0.0085
0.0683 ± 0.0986
0.3420 ± 0.2311
0.0353 ± 0.0180
0.0485 ± 0.0168
OHE+OC
0.3913 ± 0.0446
0.1961 ± 0.0509
0.0109 ± 0.0098
0.0099 ± 0.0252
0.0892 ± 0.0563
0.4629 ± 0.2502
0.0343 ± 0.0207
0.0004 ± 0.0017
SBC
0.3871 ± 0.0254
0.1483 ± 0.0333
0.0297 ± 0.0182
-0.0097 ± 0.0098
0.0065 ± 0.0029
0.4591 ± 0.2494
0.0295 ± 0.0127
0.0577 ± 0.0115
JDM
0.3826 ± 0.0366
0.1225 ± 0.0355
0.0244 ± 0.0267
-0.0011 ± 0.0086
0.0395 ± 0.0602
0.3191 ± 0.1839
0.0359 ± 0.0115
0.0506 ± 0.0191
CMS
0.3356 ± 0.0449
0.2972 ± 0.0660
0.0171 ± 0.0217
-0.0011 ± 0.0063
0.0824 ± 0.0759
0.4210 ± 0.2690
0.0384 ± 0.0193
0.0608 ± 0.0159
UDM
0.3673 ± 0.0411
0.2800 ± 0.0703
0.0178 ± 0.0250
0.0340 ± 0.0354
0.0860 ± 0.0788
0.3675 ± 0.2519
0.0425 ± 0.0185
0.0620 ± 0.0132
Table 4: ARI performance of 12 different clustering approaches on eight categorical datasets. The interval of ARI is [−1,1] , and a larger value of ARI indicates a better clustering performance. The best and second-best results on each dataset are highlighted in bold and underlined , respectively. The experimental results of KMD and KPT are combined and reported as “KMD/KPT”.
Approach
DS
HF
AA
AP
DT
AC
KMD/KPT
0.7375 ± 0.1352
0.6224 ± 0.0668
0.5361 ± 0.0330
0.5299 ± 0.0198
0.5964 ± 0.0893
0.7784 ± 0.0658
OHE+OC
0.7196 ± 0.1527
0.5415 ± 0.0330
0.5322 ± 0.0283
0.5079 ± 0.0097
0.6753 ± 0.1240
0.6267 ± 0.1442
SBC
0.6846 ± 0.1085
0.5251 ± 0.0235
0.5173 ± 0.0067
0.5410 ± 0.0024
0.6264 ± 0.1276
0.7946 ± 0.0646
JDM
0.7154 ± 0.1105
0.5580 ± 0.0259
0.5385 ± 0.0391
0.5479 ± 0.0326
0.6536 ± 0.1137
0.6633 ± 0.1031
CMS
0.7686 ± 0.0479
0.6436 ± 0.0054
0.5673 ± 0.0099
0.5397 ± 0.0284
0.6096 ± 0.0768
0.6971 ± 0.0102
UDM
0.7900 ± 0.1302
0.6441 ± 0.0050
0.5745 ± 0.0202
0.5614 ± 0.0402
0.6679 ± 0.0995
0.8069 ± 0.0175
Table 5: CA performance of 12 different clustering approaches on six mixed datasets. The interval of CA is [0,1] , and a larger value of CA indicates a better clustering performance. The best and second-best results on each dataset are highlighted in bold and underlined , respectively. The experimental results of KMD and KPT are combined and reported as “KMD/KPT”.
Approach
SB
SF
T3
HR
LG
MR
LE
SW
KMD/KPT
0.5021 ± 0.0436
0.4707 ± 0.0523
0.5692 ± 0.0499
0.3826 ± 0.0240
0.6028 ± 0.0933
0.7645 ± 0.1282
0.3276 ± 0.0344
0.3777 ± 0.0348
OHE+OC
0.5348 ± 0.0433
0.4506 ± 0.0393
0.5548 ± 0.0266
0.3989 ± 0.0561
0.6444 ± 0.0568
0.8167 ± 0.1277
0.3455 ± 0.1500
0.0210 ± 0.0941
SBC
0.5368 ± 0.0300
0.4130 ± 0.0284
0.5825 ± 0.0378
0.3591 ± 0.0349
0.5588 ± 0.0062
0.8145 ± 0.1295
0.3172 ± 0.0263
0.3812 ± 0.0195
JDM
0.5417 ± 0.0412
0.4046 ± 0.0306
0.5664 ± 0.0455
0.3932 ± 0.0269
0.5842 ± 0.0707
0.7613 ± 0.1101
0.3142 ± 0.0240
0.3738 ± 0.0274
CMS
0.5177 ± 0.0476
0.5042 ± 0.0484
0.5551 ± 0.0399
0.3962 ± 0.0209
0.6282 ± 0.0783
0.7958 ± 0.1368
0.3274 ± 0.0379
0.3866 ± 0.0292
UDM
0.5376 ± 0.0391
0.5020 ± 0.0532
0.5544 ± 0.0433
0.4470 ± 0.0521
0.6289 ± 0.0828
0.7757 ± 0.1296
0.3324 ± 0.0391
0.3819 ± 0.0231
Table 6: CA performance of 12 different clustering approaches on eight categorical datasets. The interval of CA is [0,1] , and a larger value of CA indicates a better clustering performance. The best and second-best results on each dataset are highlighted in bold and underlined , respectively. The experimental results of KMD and KPT are combined and reported as “KMD/KPT”.
Figure 6: Average performance and ranks of different approaches based on the results in Tables 3 - 6 . The methods KMD/KPT, OHE+OC, SBC, JDM, CMS, UDM, HOD, HARR-V, and HARR-M are denoted as A, B, C, D, E, F, G, H-V, and H-M, respectively.
Figure 7: Results of the two-tailed BD tests on ARI (a) and CA (b) performances across 12 methods on 14 datasets, based on the results from Tables 3 - 6 . Confidence intervals are set at 0.95 ( α=0.05 ) and 0.9 ( α=0.1 ), the corresponding Critical Difference (CD) intervals CD_95%=3.8158 and CD_90%=3.4069, respectively.
Data
KMD/KPT
BD
HAR
HARR-V
HARR-M
DS
0.2881 ± 0.2718
0.3374 ± 0.2798
0.3374 ± 0.2798
0.3849 ± 0.1897
0.4599 ± 0.2409
HF
0.0693 ± 0.0662
0.0793 ± 0.0181
0.0793 ± 0.0181
0.0882 ± 0.0000
0.0882 ± 0.0000
AA
-0.0027 ± 0.0125
-0.0030 ± 0.0106
0.0430 ± 0.0313
0.0278 ± 0.0283
0.0289 ± 0.0285
AP
-0.0068 ± 0.0129
-0.0020 ± 0.0288
-0.0002 ± 0.0300
-0.0016 ± 0.0456
0.0412 ± 0.0534
DT
0.4839 ± 0.1044
0.6322 ± 0.1343
0.6322 ± 0.1343
0.6403 ± 0.1694
0.6826 ± 0.1429
AC
0.3253 ± 0.1025
0.3773 ± 0.0455
0.3773 ± 0.0455
0.3453 ± 0.0000
0.4200 ± 0.0000
Table 7: ARI performance of five ablated HARR versions on 14 datasets. The interval of ARI is [−1,1] , and a larger value of ARI indicates a better clustering performance. AR row reports the average ranks.
Data
KMD/KPT
BD
HAR
HARR-V
HARR-M
DS
0.7375 ± 0.1352
0.7650 ± 0.1289
0.7650 ± 0.1289
0.7942 ± 0.1084
0.8187 ± 0.1247
HF
0.6224 ± 0.0668
0.6436 ± 0.0145
0.6436 ± 0.0145
0.6522 ± 0.0000
0.6522 ± 0.0000
AA
0.5361 ± 0.0330
0.5404 ± 0.0302
0.6197 ± 0.0475
0.6062 ± 0.0305
0.6062 ± 0.0302
AP
0.5299 ± 0.0198
0.5561 ± 0.0381
0.5614 ± 0.0446
0.5550 ± 0.0561
0.6111 ± 0.0535
DT
0.5964 ± 0.0893
0.6811 ± 0.1071
0.6811 ± 0.1071
0.6793 ± 0.1411
0.7161 ± 0.1130
AC
0.7784 ± 0.0658
0.8070 ± 0.0182
0.8070 ± 0.0182
0.7942 ± 0.0000
0.8246 ± 0.0000
Table 8: CA performance of five ablated HARR versions on 14 datasets. The interval of CA is [0,1] , and a larger value of CA indicates a better clustering performance. AR row reports the average ranks. IP row reports the improved percentage achieved by the current one w.r.t. the previous one. In Table 7 , since ARI is in the interval [−1,1] , IP results are not reported.
Figure 8: ARI and CA performance of different approaches with and without distinguishing ordinal attributes on AP, DT, AC, HR, and LG datasets. The two numbers in parentheses in the legend report the value of dn + do used to treat the dataset.
Figure 9: Total object-cluster distance z of HARR-V and HARR-M on different mixed data sets. The blue circles and red boxes indicate that the weights were updated by HARR-V and HARR-M, respectively.
Figure 10: Comparison of the execution time of CMS, UDM, HDM, GBD, FBD, HARR-V, and HARR-M on a large synthetic dataset ( n=100,000 ) sampled at different sampling rates φ={0.001,0.2,0.4,0.6,0.8,1} .
Figure 11: t-SNE visualization of the MR dataset represented by OHE, GBD, FBD, HARR-V, and HARR-M. Data points are marked in different colors according to the ‘true’ labels. A higher concentration of data points with the same label indicates that the corresponding method’s distance metric better discriminates between clusters.
The success of categorical data clustering generally much relies on the distance metric that measures the dissimilarity degree between two objects. However, most of the existing clustering methods treat the two categorical subtypes, i.e. nominal and ordinal attributes, in the same way when calculating the dissimilarity without considering the relative order information of the ordinal values. Moreover, there would exist interdependence among the nominal and ordinal attributes, which is worth exploring for indicating the dissimilarity. This paper will therefore study the intrinsic difference and connection of nominal and ordinal attribute values from a perspective akin to the graph. Accordingly, we propose a novel distance metric to measure the intra-attribute distances of nominal and ordinal attributes in a unified way, meanwhile preserving the order relationship among ordinal values. Subsequently, we propose a new clustering algorithm to make the learning of intra-attribute distance weights and partitions of data objects into a single learning paradigm rather than two separate steps, whereby circumventing a suboptimal solution. Experiments show the efficacy of the proposed algorithm in comparison with the existing counterparts.
Clustering mixed tabular data requires a unified metric space to bridge the inherent heterogeneity between continuous numerical measurements and discrete categorical symbols. Traditionally, algorithms rely entirely on dataset-internal statistics to estimate categorical relationships, which confines the learned metric to empirical co-occurrences and ignores conceptually obvious yet statistically unobserved affinities. Although LLMs offer external world knowledge, applying their text-centric reasoning to highly abstract tabular concepts presents significant challenges. Bridging this modality gap to construct a semantically complete metric typically requires embedding LLMs into iterative metric learning loops to dynamically optimize cross-modality representations. This incurs intractable computational overhead, forcing a compromise between semantic enrichment and scalability. Therefore, we propose GRACE, an LLM-grounded framework for scalable mixed-data clustering. GRACE shifts semantic acquisition to the attribute-value level via a multi-perspective LLM querying strategy, mapping heterogeneous values into knowledge-informed descriptions. Crucially, this one-shot grounding extracts general-purpose semantic representations that embed heterogeneous attributes into a unified space, decoupling expensive LLM invocation from iterative optimization. Furthermore, GRACE cross-validates these external semantics against dataset-internal statistical evidence to ensure alignment with the dataset-specific cluster structure. Ultimately, GRACE matches the scalability of conventional statistics-driven baselines while achieving superior clustering accuracy and conceptual interpretability over 11 competing methods. The source code is available at https://github.com/develop-yang/GRACE-GRACE-A
Zihua Yang, Zhencheng Xie, Junyang Chen +4
Guangdong University of Technology · Tsinghua University · Peking University +2
Attributed graph clustering partitions nodes by jointly exploiting node attributes and graph topology. It remains challenging due to attribute heterogeneity and representation degradation during graph learning. Real-world datasets often contain heterogeneous attributes, i.e., numerical and categorical attributes, complicating unified representation learning. This challenge becomes more complex in attributed graphs, where constructing a clustering-friendly graph structure from attributes and topology remains difficult. Under deep graph architectures, repeated graph propagation causes node embeddings to become overly similar, leading to the over-smoothing (OS) effect. Meanwhile, graph representation learning amplifies topological influence, making discriminative attribute information harder to exploit for clustering, an effect we refer to as over-dominating (OD). To bridge these gaps, an end-to-end framework, Any-type attributed Graph REpresentation lEarning (AGREE), is proposed. It unifies attributed graphs and any-type attributed data through multi-level alignment and similarity-based graph construction. Quaternion-based graph convolution strengthens attribute interaction to alleviate OD, while shallow graph architectures help relieve OS. The learned embeddings are jointly optimized for graph reconstruction and clustering, without requiring a predefined number of clusters during training. Experiments on diverse benchmarks show that AGREE achieves strong overall performance in accuracy, robustness, and adaptability.
Xinxi Chen, Junyang Chen, Yiqun Zhang +2
School of Computer Science and Technology, Guangdong University of Technology, Guangzhou 510006, China