MOELIGA: a multi-objective evolutionary approach for feature selection with local improvement
Organizations: Research Institute for Signals, Systems and Computational Intelligence (sınc(i)), FICH–UNL/CONICET, Ciudad Universitaria UNL, Santa Fe, Argentina.
Abstract
Selecting the most relevant or informative features is a key issue in actual machine learning problems. Since an exhaustive search is not feasible even for a moderate number of features, an intelligent search strategy must be employed for finding an optimal subset, which implies considering how features interact with each other in promoting class separability. Balancing feature subset size and classification accuracy constitutes a multi-objective optimization challenge. Here we propose MOELIGA, a multi-objective genetic algorithm incorporating an evolutionary local improvement strategy that evolves subordinate populations to refine feature subsets. MOELIGA employs a crowding-based fitness sharing mechanism and a sigmoid transformation to enhance diversity and guide compactness, alongside a geometry-based objective promoting classifier independence. Experimental evaluation on 14 diverse datasets demonstrates MOELIGA's ability to identify smaller feature subsets with superior or comparable classification performance relative to 11 state-of-the-art methods. These findings suggest MOELIGA effectively addresses the accuracy-dimensionality trade-off, offering a robust and adaptable approach for multi-objective feature selection in complex, high-dimensional scenarios.
Figures & tables
| Samp. | Feat. | Class. | Description | |
| madelon | 2400 | 500 | 2 | Classification of artificial data 1 1 1 https://archive.ics.uci.edu [ 38 ] . |
| dermatology | 358 2 2 2 366 total instances, 358 instances without missing values. | 34 | 6 | Classification of Eryhemato-Squamous Disease type from biopsy and clinical features [ 39 ] . |
| movement | 360 | 90 | 15 | Classification of gestures in sign language (LIBRA) based on hand movements [ 40 ] . |
| arrhythmia | 452 | 279 | 13 | Classification of arrhythmia events from ECG data [ 41 ] . |
| smartphone-activity | 10299 | 561 | 6 | Classification of smartphone activity from time and frequency domains [ 42 ] . |
| isolet | 7797 | 617 | 26 | Classification of spoken letter-name from speech features [ 43 ] . |
| No. of fitness evaluations | |
|---|---|
| No. of Objectives | |
| No. of Subordinate pops. | |
| Replacement strategy | |
| Objective-II Sigmoid | |
| None | 0.5 | 1.5 | 15 | |||||||||
| 0.0025 | 0.005 | 0.025 | 0.0025 | 0.005 | 0.025 | 0.0025 | 0.005 | 0.025 | 0.0025 | 0.005 | 0.025 | |
| dermatology | 0.850 | 0.850 | 0.849 | 0.820 | 0.850 | 0.849 | 0.849 | 0.847 | 0.849 | 0.760 | 0.791 | 0.791 |
| movement | 0.787 | 0.787 | 0.800 | 0.778 | 0.777 | 0.801 | 0.811 | 0.782 | 0.773 | 0.773 | 0.755 | 0.774 |
| arrhythmia | 0.661 | 0.662 | 0.636 | 0.663 | 0.637 | 0.642 | 0.613 | 0.646 | 0.632 | 0.638 | 0.649 | 0.646 |
| madelon | 0.778 | 0.765 | 0.790 | 0.795 | 0.779 | 0.771 | 0.779 | 0.766 | 0.779 | 0.780 | 0.794 | 0.782 |
| isolet | 0.868 | 0.872 | 0.867 | 0.862 | 0.861 | 0.870 | 0.863 | 0.858 | 0.865 | 0.862 | 0.861 | 0.849 |
| Features | Moeliga | ReliefF | Surf | Surf | MSurf | MSurf | MI | RFE | KBest | SFS | SFFS | Boruta | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| derma | 6/34 | 0.96 | 0.70 | 0.69 | 0.54 | 0.65 | 0.53 | 0.67 | 0.68 | 0.50 | 0 .89 | 0 .89 | 0.72 |
| optd | 10/64 | 0.91 | 0.79 | 0.81 | 0.66 | 0.83 | 0.52 | 0.81 | 0.61 | 0.81 | 0 .85 | 0 .85 | 0.48 |
| move | 9/90 | 0.80 | 0.38 | 0.48 | 0.28 | 0.38 | 0.28 | 0.45 | 0.44 | 0.48 | 0 .60 | 0 .60 | 0.32 |
| arrh | 11/279 | 0.67 | 0.28 | 0.28 | 0.23 | 0.33 | 0.26 | 0.28 | 0.28 | 0.26 | 0 .38 | 0.35 | 0.25 |
| mad | 22/500 | 0.80 | 0 .83 | 0.83 | 0.82 | 0.82 | 0.84 | 0.63 | 0.64 | 0.76 | 0.77 | 0.77 | 0.83 |
| smart | 8/561 | 0.88 | 0.66 | 0.60 | 0.60 | 0.60 | 0.60 | 0.53 | 0.64 | 0.67 | 0 .79 | 0 .79 | 0.56 |
Feature Selection
Feature selection identifies compact, informative variables or input dimensions to improve prediction, efficiency, interpretability, and statistical inference.
See all 196 papers