Overview

Feature selection studies which variables, channels, bands, or learned dimensions carry useful information, aiming to simplify models and improve prediction, interpretation, or measurement efficiency. Common approaches include Lasso and elastic net regularization, filters such as mRMR and Boruta, Bayesian spike-and-slab models, and wrapper or evolutionary searches; newer methods use neural gates, attribution scores, and sequential policies. Unsupervised methods select features for clustering or representation learning, while applications span tabular data, omics, EEG, hyperspectral imagery, and time series. A central challenge is that selected features can change with correlated inputs, evaluation design, or distribution shifts, so predictive accuracy must be considered alongside stability, error control, computational cost, and strong baselines.

Latest papers

213 papers23 papers in the last four weeks, down 12% on the four weeks before. 0.2% of all new papers.

Dec 13, 2024

Feature Selection for Latent Factor Models

Feature selection is crucial for pinpointing relevant features in high-dimensional datasets, mitigating the 'curse of dimensionality,' and enhancing machine learning performance. Traditional feature selection methods for classification use data from all classes to select features for each class. This paper explores feature selection methods that select features for each class separately, using class models based on low-rank generative methods and introducing a signal-to-noise ratio (SNR) feature selection criterion. This novel approach offers a theoretically sound guarantee of feature recovery under specific assumptions and has been demonstrated to outperform some existing feature selection methods on standard classification datasets. Additionally, it can be easily applied in a class-incremental learning setup, yielding performance comparable to that of some current methods.
Aug 5, 2024

Heterogeneous Graph Attention Network With Multimodal Feature Selection for Cancer Multiomics Integration

The increase in high-dimensional multiomics data demands advanced integration models to capture the complexity of human diseases. Graph-based deep learning integration models, despite their promise, struggle with small patient cohorts and high-dimensional features, often applying independent feature selection without modeling relationships among omics. Furthermore, conventional graph-based omics models focus on homogeneous graphs, lacking multiple types of nodes and edges to capture diverse structures. We introduce a Heterogeneous Graph ATtention network for omics integration (HeteroGATomics) to improve cancer diagnosis. HeteroGATomics performs multimodal feature selection through a multi-agent system, creating dedicated networks of feature and patient similarity for each omics modality. These networks are then combined into one heterogeneous graph for learning holistic omics-specific representations and integrating predictions across modalities. Experiments on three cancer multiomics datasets show that HeteroGATomics outperforms baseline methods across most evaluation metrics in cancer diagnosis. External validation on an independent breast cancer cohort further demonstrates its generalizability, where HeteroGATomics outperforms graph-based multiomics integration methods across evaluation metrics. Moreover, HeteroGATomics enhances interpretability by identifying important biomarkers contributing to the diagnosis outcomes.
Jul 14, 2019

Regularized Estimation and Feature Selection in Mixtures of Generalized Linear Experts

Mixtures of experts (MoE) are conditional mixture models in which both the mixing proportions and the component densities depend on the predictors, and are widely used for regression, classification and model-based clustering of heterogeneous data. Fitting MoE by maximum likelihood becomes unstable, and sometimes infeasible, when the predictors are numerous or correlated. We propose a regularized maximum likelihood framework for simultaneous parameter estimation and feature selection in MoE whose experts belong to the generalized linear model family, covering Gaussian, Poisson and multinomial responses within a single formulation. Sparsity is induced in both the gating network and the experts through ℓ1\ell_1 penalties, and the penalized log-likelihood is maximized by a proximal Newton-EM algorithm whose M-step reduces to weighted Lasso problems with closed-form coordinate-ascent updates. Unlike existing penalized MoE procedures, the algorithm requires neither a local quadratic approximation of the penalty nor any matrix inversion, it returns exactly sparse estimates without thresholding, and a proximal Newton-type variant guarantees a monotone increase of the penalized objective at every iteration. On simulated data and five real data sets, the method recovers the actual sparsity support and delivers prediction and clustering accuracy that is competitive with, and often better than, state-of-the-art regularized MoE. The source codes of our developed algorithms and their documentation are publicly available on Github at https://github.com/nv-thin/GLM-RMoE.