Feature Selection in High Dimensional Spaces

In this study, two novel filter feature selection approaches are proposed as alternatives to state-of-the-art. The first proposed approach is a greedy-based feature selection method where redundancy is replaced by diversity to quantify the complementarity of a candidate feature with respect to the already selected subset. Both relevance and diversity are computed in terms of the ranks of positive instances, which is analogous to the computation of the area under the receiver operating characteristic curve (AUC). In the second approach, a novel dissimilarity metric based on Feature-to-Feature (F2F) scatter frequencies is proposed for clustering-based filter feature selection. The proposed metric is computed by obtaining feature-dependent ranks of samples and identifying the features which assign close ranks to each sample. Samples are represented as a set of affinity sets containing features having rank differences within a predefined proximity window size. The F2F dissimilarity of a pair of features is computed using the frequency of their appearance in different affinity sets. Features are then clustered into distinct groups using F2F dissimilarity metric. From each cluster, the feature having the highest relevance score is selected. The experiments conducted on 10 UCI and microarray gene expression data sets have confirmed that the proposed feature selection approaches provide better performance scores when compared to other competing methods. The proposed method outperforms the widely-used mutual information-based schemes in terms of classification accuracy, AUC and stability. Keywords: feature selection, ranks of instances, relevance, diversity, dissimilarity, scatter frequency, representative feature.