Learning to Select Features with Manifold Deep Unfolding PCA

Jingjing Liu; Pei Tai; Xianchao Xiu; Wanquan Liu

cs.CVOct 14, 2026

Learning to Select Features with Manifold Deep Unfolding PCA

Authors: Jingjing Liu, Pei Tai, Xianchao Xiu, Wanquan Liu

Organizations: Shanghai Key Laboratory of Automobile Intelligent Network Interaction Chip and System, School of Microelectronics, Shanghai University, Shanghai 200444, China · School of Mechatronic Engineering and Automation, Shanghai University, Shanghai 200444, China · School of Intelligent Systems Engineering, Sun Yat-sen University, Guangzhou 510275, China

Abstract

Unsupervised feature selection (UFS) is of great significance in image processing and pattern recognition. Nevertheless, existing model-based methods often rely on manually tuned hyperparameters, while deep learning-based methods generally require massive training data. To address these issues, this paper proposes an efficient manifold deep unfolding feature selection network (MDUFS-Net). Technically, we first incorporate the $\ell_{2,0}$ -norm into the classic principal component analysis (PCA) framework, then unfold the alternating direction method of multipliers (ADMM)-based iterative algorithm into an end-to-end trainable network, thereby enabling automatic parameter learning. Moreover, a manifold optimization strategy is developed to improve the numerical robustness. On this basis, we further extend MDUFS-Net into an enhanced variant called MDUFS-Net+ by introducing the $\ell_{0}$ -norm for more discriminative feature representations. Extensive experiments validate that our proposed MDUFS-Net and MDUFS-Net+ outperform state-of-the-art methods in the UFS task, with promising performance in clustering. Our code is available at https://github.com/xianchaoxiu/MDUFS.

Figures & tables

Fig. 1: Performance-efficiency comparison of different methods based on average accuracy (ACC) and runtime (s) across eight real-world datasets.

Fig. 2: Illustration of our proposed MDUFS-Net and MDUFS-Net+.

X^{k+1}=\texttt{MGradNet}(X^{k},Y^{k},\Lambda^{k},G,\alpha_{k},\eta_{k})

Algorithm 1 Deep unfolding for MDUFS-Net

\displaystyle X^{k+1}=\texttt{MGradNet+}(X^{k},Y^{k},Z^{k},\Lambda^{k},\Pi^{k},G,\alpha_{k},\beta_{k},\eta_{k})

Algorithm 2 Deep unfolding for MDUFS-Net+

Types	Datasets	Features	Samples	Classes
Synthetic	Dartboard1	9	1000	4
Synthetic	Diamond9	9	3000	9
Real-world	COIL20	1024	1440	20
	ISOLET	617	1560	26
	USPS	256	1000	10
	PIE	1024	1166	53

TABLE I: Statistical information of selected datasets.

Panel of Figure 6 — Fig. 3: Visual comparisons on the Dartboard1 dataset, where (a)–(j) are the feature selection results.

Panel of Figure 7 — Fig. 4: Visual comparisons on the Diamond9 dataset, where (a)–(j) are the feature selection results.

Panel of Figure 8 — Fig. 5: Visual comparisons of the ACC metric under different real-world datasets with different numbers of selected features.

Panel of Figure 9 — Fig. 6: Visual comparisons of the NMI metric under different real-world datasets with different numbers of selected features.

Methods	COIL20	ISOLET	USPS	PIE	LUNG	MSTAR	MNIST	HAR	Average
ALLfea	64.73 $\pm$ 7.63	59.87 $\pm$ 2.78	59.82 $\pm$ 1.29	27.43 $\pm$ 0.96	72.57 $\pm$ 7.63	90.29 $\pm$ 3.94	47.69 $\pm$ 2.78	53.79 $\pm$ 0.99	59.52 $\pm$ 3.50
ALLfea	(10)	(10)	(10)	(10)	(10)	(10)	(10)	(10)	59.52 $\pm$ 3.50
LapScore	55.36 $\pm$ 1.58	54.07 $\pm$ 1.80	52.31 $\pm$ 2.68	33.90 $\pm$ 0.97	58.40 $\pm$ 0.75	68.67 $\pm$ 3.32	48.62 $\pm$ 0.15	61.59 $\pm$ 0.09	54.12 $\pm$ 1.42
LapScore	(90)	(100)	(100)	(30)	(100)	(100)	(80)	(100)	54.12 $\pm$ 1.42
SOGFS	43.32 $\pm$ 2.96	43.43 $\pm$ 1.84	45.23 $\pm$ 1.31	24.73 $\pm$ 0.94	54.02 $\pm$ 3.19	64.65 $\pm$ 2.66	41.26 $\pm$ 1.15	46.05 $\pm$ 0.38	45.34 $\pm$ 1.80
SOGFS	(100)	(30)	(100)	(100)	(100)	(60)	(80)	(100)	45.34 $\pm$ 1.80

TABLE II: Average ACC (mean %

\pm

std %) and number of selected features (in brackets) of

k

-means clustering, where the best results are marked in bold.

Methods	COIL20	ISOLET	USPS	PIE	LUNG	MSTAR	MNIST	HAR	Average
ALLfea	77.43 $\pm$ 1.10	74.57 $\pm$ 1.40	56.73 $\pm$ 1.10	52.42 $\pm$ 0.92	60.04 $\pm$ 3.59	87.39 $\pm$ 1.10	50.65 $\pm$ 0.26	55.63 $\pm$ 1.67	64.36 $\pm$ 1.39
ALLfea	(10)	(10)	(10)	(10)	(10)	(10)	(10)	(10)	64.36 $\pm$ 1.39
LapScore	69.47 $\pm$ 0.85	69.92 $\pm$ 0.83	50.10 $\pm$ 1.54	58.74 $\pm$ 0.63	44.62 $\pm$ 0.99	71.60 $\pm$ 0.46	50.67 $\pm$ 0.22	64.95 $\pm$ 0.09	60.01 $\pm$ 0.70
LapScore	(80)	(100)	(100)	(30)	(100)	(100)	(80)	(100)	60.01 $\pm$ 0.70
SOGFS	60.42 $\pm$ 1.40	59.18 $\pm$ 1.04	39.67 $\pm$ 1.41	50.31 $\pm$ 0.88	43.50 $\pm$ 3.32	66.78 $\pm$ 0.65	40.49 $\pm$ 1.15	48.22 $\pm$ 0.01	51.07 $\pm$ 1.23
SOGFS	(100)	(30)	(100)	(100)	(100)	(100)	(100)	(30)	51.07 $\pm$ 1.23

TABLE III: Average NMI (mean %

\pm

std %) and number of selected features (in brackets) of

k

-means clustering, where best results are marked in bold.

$\ell_{2,0}$ -norm	$\ell_{0}$ -norm	Metrics	COIL20	ISOLET	USPS	PIE	LUNG	MSTAR	MNIST	HAR
$\times$	$\times$	ACC	44.42	40.20	61.62	45.46	63.09	81.46	42.22	45.16
$\times$	$\times$	NMI	63.16	52.00	53.52	66.37	58.51	79.76	41.99	54.13
$\times$	$\checkmark$	ACC	45.34	40.31	59.21	44.52	60.43	81.48	41.83	50.93
$\times$	$\checkmark$	NMI	62.88	51.79	52.91	65.87	57.64	79.66	42.06	54.14
$\checkmark$	$\times$	ACC	63.79	62.84	66.58	44.76	65.18	87.05	56.10	50.60
$\checkmark$	$\times$	NMI	77.27	75.38	57.29	66.52	59.40	81.26	49.66	62.24

TABLE IV: Ablation studies of sparsity regularization terms, where the best results are marked in bold.

Projection	Retraction	Metrics	COIL20	ISOLET	USPS	PIE	LUNG	MSTAR	MNIST	HAR
$\times$	$\times$	ACC	56.23	21.57	44.30	20.49	49.67	52.15	42.39	37.77
$\times$	$\times$	NMI	66.51	36.13	39.23	41.77	38.52	55.68	41.55	47.47
$\times$	$\checkmark$	ACC	54.02	49.91	49.85	32.77	50.81	72.55	43.54	53.53
$\times$	$\checkmark$	NMI	68.86	61.94	46.18	59.33	41.23	73.08	45.22	52.64
$\checkmark$	$\times$	ACC	59.48	49.71	60.04	32.31	58.39	78.83	49.80	49.99
$\checkmark$	$\times$	NMI	72.21	62.53	54.23	56.35	49.49	74.52	47.02	49.03

TABLE V: Ablation studies of manifold optimization, where the best results are marked in bold.

Network	Metrics	COIL20	ISOLET	PIE	LUNG	MNIST
$\times$	ACC	60.81	55.70	40.02	58.83	49.91
$\times$	NMI	74.77	70.27	64.81	49.78	48.96
$\checkmark$	ACC	65.23	64.16	46.82	69.46	56.75
$\checkmark$	NMI	78.96	75.36	68.00	60.84	50.86

TABLE VI: Ablation studies of the network, where the best results are marked in bold.

Strategy	Metrics	COIL20	ISOLET	PIE	LUNG	MNIST
$\times$	ACC	61.26	53.68	43.14	61.27	51.77
$\times$	NMI	74.56	68.35	66.08	55.14	47.44
$\checkmark$	ACC	65.23	64.16	46.82	69.46	56.75
$\checkmark$	NMI	78.96	75.36	68.00	60.84	50.86

TABLE VII: Ablation studies of parameter learning, where the best results are marked in bold.

Panel of Figure 16 — Fig. 7: Model stability comparisons of all compared methods on eight real-world datasets.

Panel of Figure 17 — Fig. 8: Unfolding sensitivity of stages $K$ on eight real-world datasets.

Panel of Figure 18 — Fig. 9: Runtime comparisons among different methods on eight real-world datasets.

Feature Selection

Feature selection identifies compact, informative variables or input dimensions to improve prediction, efficiency, interpretability, and statistical inference.

See all 196 papers

When Features Become Instances: Inverted Contrastive Learning for Unsupervised Feature Selection
Sep 1, 2026Utsab Ghosh, Roshni ChakrabortyContrastive Learning Feature Selection
ABV-Indian Institute of Information Technology and Management, Gwalior, India.
i-IF-Learn: Iterative Feature Selection and Unsupervised Learning for High-Dimensional Complex Data
Mar 25, 2026Chen Ma, Wanjie Wang, Shuhao FanUnsupervised Clustering Feature Selection
SUSTech · NUS
Towards Truly Unsupervised Evaluation of Feature Selection -- Extended Version
Aug 12, 2026Hafiz Saud Arshad, Muhammad Rajabinasab, Arthur ZimekFeature Selection
University of Southern Denmark, Odense, Denmark

Sep 1, 2026J/K move · Enter open · S save

When Features Become Instances: Inverted Contrastive Learning for Unsupervised Feature Selection

Utsab Ghosh, Roshni Chakraborty

ABV-Indian Institute of Information Technology and Management, Gwalior, India.

Contrastive Learning Feature Selection

Open paper AskPDF arXiv

Unsupervised feature selection seeks a compact subset of informative features without access to class labels, making feature utility difficult to define. Existing UFS methods therefore rely on indirect structural criteria, such as similarity preservation, locality, sparsity, cluster geometry, or reconstruction quality. In this paper, we instead study UFS through representation consistency and propose Inverted Contrastive Learning for Unsupervised Feature Selection (ICLFS), a feature-wise contrastive framework that reformulates UFS as a representation learning problem over features rather than samples. ICLFS first inverts the data matrix so that each feature is represented by its sample-profile vector, then constructs multiple masked positive views together with a shuffled negative view, and learns projector-space representations that remain consistent across these structured perturbations under an InfoNCE-based objective. Motivated by recent findings that cosine-based and InfoNCE-based training affect embedding norms, we use projector-space embedding magnitude as the saliency signal for ranking features. The resulting norm-based ranking is subsequently refined through Laplacian-Gated Ranking Correction, which suppresses locally redundant candidates while preserving salient ones. Extensive experiments on 12 benchmark datasets show that ICLFS achieves the best clustering accuracy on 10 datasets against both classical and neural baselines under the standard clustering-based UFS evaluation protocol, while remaining competitive on the other two. These results show that feature-wise contrastive representation consistency provides a strong and effective alternative to neighborhood, cluster, and reconstruction-based UFS formulations.

Abstract

Feature Selection

Explore similar work

When Features Become Instances: Inverted Contrastive Learning for Unsupervised Feature Selection

i-IF-Learn: Iterative Feature Selection and Unsupervised Learning for High-Dimensional Complex Data

Towards Truly Unsupervised Evaluation of Feature Selection -- Extended Version