An ensemble machine learning model based on multiple filtering and supervised attribute clustering algorithm for classifying cancer samples

Introduction

Cancer is one of the most fatal diseases around the globe (Tabares-Soto et al., 2020; Hambali, Oladele & Adewole, 2020). According to the World Health Organization report, Cancer is marked as the second most deadly disease and an estimated 9.7 million deaths around the world in 2018 have occurred due to this signature disease (Hambali, Oladele & Adewole, 2020). Generally, one in every six deaths all over the world, occurs due to cancer. So, within 2,030, the number of new cancer patients per year will increase approximately by 25 million (Hambali, Oladele & Adewole, 2020; NIH, 2019). Although several advanced techniques are already developed for the detection of cancer, the proper prognosis of cancer patients, till date, is very poor and the survival rate is also very low (Tabares-Soto et al., 2020; Hambali, Oladele & Adewole, 2020; Kourou et al., 2015). It has been already found that for very accurate cancer sample classification or prediction, adequate information is not available from the clinical, environmental, and behavioral characteristics of patients (Kourou et al., 2015; Hambali, Oladele & Adewole, 2020; Tabares-Soto et al., 2020). Recently, due to different types of bio-molecular data analysis, several genetic disorders with different biological characteristics have been revealed which are very helpful for early identification and prognosis of cancer and also to discern the responses for different types of treatment (Colozza et al., 2005; Greller & Tobin, 1999; Li, Xie & Liu, 2018; Liu et al., 2011; Pilling, Henderson & Gardner, 2017; Su et al., 2001; Swan et al., 2013).

With the rapid advancements in genomic, proteomic, and imaging high-throughput technologies (Colozza et al., 2005; Greller & Tobin, 1999; Li, Xie & Liu, 2018; Liu et al., 2011; Pilling, Henderson & Gardner, 2017; Su et al., 2001; Swan et al., 2013), now it is possible to accumulate huge amount (in the order of thousands) of different bio-molecular information of patients. Using this huge amount of information, researchers have been trying to develop more advanced techniques for early detection and proper prognosis of cancer, and also to improve cancer therapy for improvement of patients’ survival rate. To analyze this huge amount of information, lab-based approaches are not adequate as these methods are costly and time-consuming. So, computational or in-silico methods like statistical methods, machine learning, deep learning, etc. have been being used extensively in this field.

It is well-known fact that in cancer-causing cells, gene expression is either overexpressed or under expressed (Tabares-Soto et al., 2020). So, measurement of gene expression in cancer cells can give adequate information to improve cancer diagnostic procedures. Nowadays, different developing countries have been using this procedure for cancer sample detection. It is already known that using DNA microarray technology it is possible to measure the expression level of a numerous number of genes for a single experiment/sample simultaneously. The outcome of DNA microarray technology is a gene expression data matrix. This matrix carries information about the expression level of a huge number of genes for a limited number of samples (such as diseased patient samples and normal samples). The presence of the limited number of samples in this data matrix is due to the lack of availability of samples. So, based on information of gene expression data matrix, cancer sample classification is one of the essential tasks in the field of cancer research (Chin et al., 2016; Dashtban & Balafar, 2017; Ding & Peng, 2005; Elyasigomari et al., 2017; Furey et al., 2000; Golub et al., 1999; Nada & Alshamlan, 2019; Tabares-Soto et al., 2020).

Using computational or in-silico approaches, gene expression-based cancer sample classification task has been reviewed extensively in different papers (Chin et al., 2016; Dashtban & Balafar, 2017; Ding & Peng, 2005; Elyasigomari et al., 2017; Furey et al., 2000; Golub et al., 1999; Nada & Alshamlan, 2019; Tabares-Soto et al., 2020). However, the main difficulties in the sample classification task arise due to several factors. First, in these data sets, a substantially small number of samples is available (generally in the order of hundreds) compared to the availability of a huge number of genes (generally in the order of thousands) (Chin et al., 2016; Hambali, Oladele & Adewole, 2020; Nada & Alshamlan, 2019). For sample classification, genes are treated as features/attributes. So, the high-dimensional gene space is an overhead for most classification algorithms. Second, only a very few genes are informative (differentially expressed) and the rest of the section is non-informative (noisy) (Chin et al., 2016; Hambali, Oladele & Adewole, 2020; Nada & Alshamlan, 2019) for sample classification and responsible for degrading the classifier’s performance. Gene dimension reduction by identification of informative genes as biomarkers can improve the classification accuracy of classifiers. Apart from the improvement of classification accuracy, the identification of informative biomarkers (here, informative genes) has great prospects from a biomedical point of view. These are beneficial for finding the biological reason for a disorder, assessing disease risk, and developing therapeutic targets. The third problem arises due to the small sample size which creates an overfitting problem in classifier construction. Another problem that degrades classifier performance is the sample class imbalance problem. This problem occurs due to the presence of more instances/samples of one class (majority class) with respect to other class(es) (minority class) in a dataset.

A fairly large number of works have been already developed for sample classification. These works are divided into two categories. In the first category (Chin et al., 2016; Hambali, Oladele & Adewole, 2020; Nada & Alshamlan, 2019), the major emphasis is given to the selection of relevant genes for the reduction of feature space. Then based on this reduced feature space, predictive/classification accuracy of the samples is measured using different existing single classification models like naïve Bayes, support vector machine, relevance vector machine, K-nearest neighbor, decision tree, logistic regression, etc. As gene selection is a feature selection task, so based on feature selection techniques, these methods are divided into different categories. These are (1) filter methods (2) wrapper methods (3) embedded methods and (4) hybrid methods. Before we mention the second category of classification methods, let us first elaborate on the first category methods one by one.

Filter methods (Chin et al., 2016; Hambali, Oladele & Adewole, 2020; Nada & Alshamlan, 2019) select a subset of features without taking any information from any classification model. These methods select features that are differentially expressed with respect to sample class labels. The filter methods rank individual features according to their class discrimination power based on some statistical score function and then select a number of high-ranked features to form a reduced and relevant feature subset. The popular statistical score functions used in filter methods are Fisher’s score, Signal to Noise ratio (SNR), correlation coefficient, mutual information, Relief (Das et al., 2019), etc. Filter methods are computationally simple, fast, and unbiased in favor of any specific classifier as these methods do not consider any knowledge from any classifier at the feature selection phase. The drawback of filter methods is that the number of selected features is based solely on the trial-error method.

Wrapper methods (Chin et al., 2016; Hambali, Oladele & Adewole, 2020; Nada & Alshamlan, 2019), on the other hand, judge discrimination capability of a feature subset using classification error rate or prediction accuracy of a classifier as the feature evaluation function. It selects the most discriminative feature subset via minimizing the classification error rate or maximizing the classification accuracy of a classifier. The wrapper methods generally achieve better classification accuracy than the filter methods because the selection of feature subset is classifier-dependent. One drawback of these methods is that these are biased to used classifiers and another drawback is that these are computationally more expensive than the filter methods as generation of the best feature subset for the high-dimensional dataset is an NP-complete problem. Due to these reasons, these methods are not applicable for high-dimensional datasets.

In Embedded methods (Chin et al., 2016; Hambali, Oladele & Adewole, 2020; Nada & Alshamlan, 2019), the optimal feature subset is selected through the unique learning procedure of a specific classifier at the time of classifier construction. Actually, in these methods, the optimal feature subset selection part is embedded as part of classifier construction. These methods are faster than wrapper methods but are biased to the specific classifier. In embedded approaches, the feature selection process is specific for a particular classifier and is not applicable to other classifiers. These are also computationally expensive. Due to these reasons for high-dimensional datasets, these methods are not applicable. On the other hand, recently hybrid feature selection methods (Chin et al., 2016; Hambali, Oladele & Adewole, 2020; Nada & Alshamlan, 2019) are also developed. In hybrid methods, different category-based methods are combined to take advantage of all of these methods for improving classification accuracy.

Apart from these methods, clustering techniques (Chin et al., 2016; Hambali, Oladele & Adewole, 2020) are also used for feature selection purposes. Clustering techniques divide the data space in such a manner that objects in the same cluster are similar while in different clusters they are dissimilar. For the feature selection task, clustering methods (famous as attribute clustering in feature selection domain) (Au et al., 2005) divide the features into several distinct clusters and then reduce the feature dimension by selecting a small number of significant features from each cluster. A lot of unsupervised gene (attribute) clustering algorithms (Au et al., 2005; Chin et al., 2016; Hambali, Oladele & Adewole, 2020) are already developed for this task. However, these methods are unsuccessful to find informative functional groups of genes for sample classification as in clustering genes, no supervised information from sample classes is considered (Au et al., 2005; Chin et al., 2016; Hambali, Oladele & Adewole, 2020). So, scientists have developed a number of supervised gene (attribute) clustering algorithms (Dettling & Buhlmann, 2002; Hastie et al., 2000; Hastie et al., 2001; Maji & Das, 2012) in which genes are grouped using supervised information from sample classes and a reduced gene set is formed via selecting the most informative genes from each cluster.

All the above-mentioned variants deliver comparable feature selection and classification accuracy. Quite often this type of classification models with only a few genes and with a limited number of training samples can classify the majority of training samples correctly, but the generalization capability of such classification models cannot be guaranteed (Bolón-Canedo, Sánchez-Maroño & Alonso-Betanzos, 2012; Ghorai et al., 2011; Nagi & Bhattacharyya, 2013; Wang, 2006,Wang, Li & Fanget, 2012; Yang et al., 2010). So, the most important task for a medical diagnosis system is to improve the classification accuracy of unknown samples (generalization performance) which cannot be solved by this type of classification model.

Apart from this problem, the microarray data is related to several uncertainties due to fabrication, hybridization, and image processing procedure in microarray technology. These uncertainties introduce various types of noise in microarray data. Due to the presence of these uncertainties with a limited number of training samples, the conventional machine learning approaches face challenges to develop reliable classification models.

To overcome the above-mentioned problems, it is therefore essential to develop general approaches and robust methods. In this regard, researchers are motivated to develop the second category-based model. These are the different robust ensemble classification models (Bolón-Canedo, Sánchez-Maroño & Alonso-Betanzos, 2012; Ghorai et al., 2011; Nagi & Bhattacharyya, 2013; Osareh & Bita, 2013; Wang, 2006; Wang, Li & Fanget, 2012; Yang et al., 2010) which can overcome small sample size problems and are capable of removing uncertainties of gene expression data.

Ensemble methods (Dietterich, 2000) are a class of machine learning technique which combines multiple base learning algorithms to produce one optimal predictive model. Ensemble classification model refers to a group of individual/base classifiers that are trained individually on the trained dataset in a supervised classification system and finally, an aggregation method is used to combine the decisions produced by the base classifiers. These ensemble classification models have the potential to alleviate the small sample size problem by applying multiple classification models on the same training data or on bootstrapped samples (sampling with replacement) of the training data to decrease the chance of overfitting in the training data. In this way, the training dataset is utilized more efficiently, and as a consequence, the generalization ability is improved.

Although different category-based ensemble classification models exist in the literature but these ensemble models are not capable of addressing all the above-mentioned problems (small sample size, high dimensional feature space, and sample class imbalance problem) related to microarray data.

In this regard, here a new Multiple Filtering and Supervised Attribute Clustering algorithm-based ensemble classification model named MFSAC-EC is proposed. In this model, first, a number of bootstrapped versions of the original training dataset are created. At the time of the creation of bootstrapped versions, an oversampling technique (Błaszczyński, StefanowskiŁ & Idkowiak, 2013) is adopted to solve the class imbalance problem. For every bootstrapped dataset a number of sub-datasets (each with a subset of genes) are generated using the proposed MFSAC method. The MFSAC is a hybrid method combining multiple filters with a new supervised attribute clustering method. Then for every sub-dataset, a base classifier is constructed. Finally, based on the prediction accuracy of all these base classifiers of all sub-datasets for all bootstrapped datasets an ensemble classifier (EC) is formed using the majority voting technique.

The novelty of the proposed MFSAC-EC model is that here the emphasis is given simultaneously on the high dimensionality problem of gene expression data, small sample size problem as well as the class imbalance problem. All of these problems at the same time are not considered in any existing ensemble classification model. First of all, due to the use of bootstrapping method with a class balancing strategy, the proposed model can handle a small sample size and overfitting problem. Second, in MFSAC, different filter methods are used with their unique characteristics. So, different characteristics-based relevant gene subsets are selected via different filters to form different sub-datasets from every bootstrapped dataset. Finally, every gene subset is modified using a supervised attribute clustering algorithm. In this way, the high-dimensionality problem of gene expression data is handled here. Apart from this, from the MFSAC generated sub-datasets, the frequency of occurrence is counted for every gene and informative genes are ranked accordingly. The prediction capability of the proposed model is experimented with over different microarray datasets and compared with the existing well-known models. Experimental outputs demonstrate the superiority of the proposed model over existing models.

Materials & Methods

The proposed MFSAC-EC model is composed of different filter score functions, a new supervised attribute clustering method, and an ensemble classification method. In the following subsections, first, a brief overview is given on different filter score functions and then the proposed MFSAC-EC model is described.

Proposed MFSAC-EC model

In the proposed MFSAC-EC model, initially, bootstrapping (sampling with replacements) with a class balancing procedure of samples is applied on training dataset $K$ to create $D$ number of different bootstrapped versions from the training dataset. Here, every bootstrapped dataset with $U$ samples is formed by random sampling with replacements $U$ times from the original dataset $K$. After that oversampling procedure is applied to each minority class to achieve data balance. Oversampling consists of increasing the minority class instances by their random replication to exactly balance the cardinality of the minority and majority classes in each bootstrapped dataset. Due to oversampling each bootstrapped dataset will contain more instances than the original dataset.

The MFSAC method of the MFSAC-EC model, which is an integration of multiple filters and a new supervised attribute (gene) clustering method, is applied on every newly created bootstrapped ( $B{K}_l$) training dataset. The proposed MFSAC method first calculates the class relevance score of every gene present in the bootstrapped training dataset using each filter score function $\left(F{T}_x\right),x=1to7$ mentioned above. Then for each filter score function, a sub-dataset $S{D}_{lx}$ with a gene subset ( $G{S}_{lx}$) is created by selecting a predefined number (let $P$) of the most relevant genes from the full gene set $G$. So, $\left|G{S}_{lx}\right|=P.$ After that on every gene subset $\left(G{S}_{lx}\right)$ of every sub-dataset $S{D}_{lx}$, the SAC (Supervised Attribute Clustering) method is applied and a set of clusters $CG{S}_{lx}$ and corresponding cluster representatives (considered as modified features) are formed. Finally, $Q$ numbers of most relevant cluster representatives are selected as modified features and a reduced sub-dataset $RS{D}_{lx}$ of the sub-dataset $S{D}_{lx}$ is formed. How the SAC method works on $G{S}_{lx}$ of every sub-dataset $S{D}_{lx}$ is discussed below.

For any sub-dataset $S{D}_{lx}$, the SAC method starts by selecting the gene from the subset ( $G{S}_{lx}$) with the highest $F{T}_x$ value. Let gene $G_{li}\in G{S}_{lx}$ with the highest $F{T}_x$ value be selected as the first member (let $F{T}_x\left(G_{li},C\right)=$ $A$) and it also becomes the initial cluster representative $R\left(R=G_{li}\right)$) of the first cluster $C_{1}G{S}_{lx}$ and $G_{li}$ is deleted from $G{S}_{lx}.$ In effect, $G_{li}\in C_{1}G{S}_{lx}$, and $G{S}_{lx}=G{S}_{lx}-\left\{G_{li}\right\}\mathrm{a}\mathrm{n}\mathrm{d}\mathrm{s}\mathrm{o}F{T}_x\left(R,C\right)=A$. This cluster is then grown up in parallel with the cluster representative refinement process which is described next. In this process, the gene (let $G_{lm}$) with next highest $F{T}_x$ value is taken from $G{S}_{lx}$ subset and is merged with the current cluster representative $R$. The merging is done in two ways. First, the expression profile of $G_{lm}$ is directly added with $R$and a temporary augmented representative $T{R}^{+}$ is formed and its $F{T}_x$ value (let $B_1$) is calculated. The second one is that the sign-flipped value of the expression profile of $G_{lm}$ is added with $R$ and another temporary augmented representative $T{R}^{-}$ is formed and its $F{T}_x$ value ( $\mathrm{l}\mathrm{e}\mathrm{t}{B}_2$) is calculated. If $F{T}_x\left(T{R}^{+},C\right)\ge F{T}_x\left(T{R}^{-},C\right)$ that is $B_1\ge B_2$ then $T{R}^{+}$ is chosen else $T{R}^{-}$ is chosen. Let $T{R}^{+}$ is chosen. Now if $F{T}_x\left(T{R}^{+},C\right)>F{T}_x\left(R,C\right)$ then $R=T{R}^{+}$ otherwise, $R$ is unaltered. Similar way if $T{R}^{-}$ is chosen and if $F{T}_x\left(T{R}^{-},C\right)>F{T}_x\left(R,C\right)$ then $R=T{R}^{-}$ otherwise, $R$ remains unchanged. If $R$ is modified then the gene $G_{lm}$ is included in the cluster $\mathrm{a}\mathrm{n}\mathrm{d}{G}_{lm}$ is deleted from $G{S}_{lx}$. In effect, $G_{lm}\in C_{1}G{S}_{lx},$ and $G{S}_{lx}=G{S}_{lx}-\left\{G_{lm}\right\}$. So, the next chosen gene is included in the current cluster if it improves the class relevance value of the current cluster representative. The merging process is described in Fig. 1.

Here g0 represents the current cluster representative $\left(R\right)$ and its class relevance score (( $F{T}_x,R$), here Pearson score), is shown. Now among all the genes g1, g2, g3, g4, and g5, the Pearson score of g1 is the highest. So, g1 is chosen for the merging process. Then g1 is added with $R$ to create the temporary augmented representative ( $T{R}^{+}=R+g1$) and also its sign-flipped value is added with the $R$ to form the temporary augmented representative ( $T{R}^{-}=R-g1$). The Pearson score of $T{R}^{+}$ is greater than the Pearson score of $T{R}^{-}$, so $T{R}^{+}$ is chosen. Now the Pearson score of $T{R}^{+}$ is greater than the Pearson score of $R$, so $T{R}^{+}$ is considered as the current cluster representative and $R=T{R}^{+}.$ This process is continued for all other genes. Now, g3 is chosen as it is the gene with the next highest Pearson value. g3 and its sign-flipped value are added individually with current cluster representative $R$ to form $T{R}^{+}=R+g3$ and $T{R}^{-}=R-g3$ respectively. In this case, Pearson score of $T{R}^{-}$ is greater than the Pearson score of $T{R}^{+}$. So, $T{R}^{-}$ is chosen. Then Pearson score of $T{R}^{-}$ is Checked with the Pearson score of $R$ and here Pearson score of $T{R}^{-}$ is greater than the Pearson score of $R$. So, $T{R}^{-}$ is considered as current cluster representative and $R=T{R}^{-}$. In this way, cluster representative is refined. This process is repeated for every member of $G{S}_{lx}$ subset.

After the formation of the first cluster and its corresponding augmented representative, $R$ is assigned to $A{R}_{lx1}$ that means $A{R}_{lx1}=R,$ and the supervised clustering process is repeated to form the second cluster with the gene (let $G_{lz})$ with next highest $F{T}_x$ value from $G{S}_{lx}$ subset. In this way a set of clusters $CG{S}_{lx}=\left\{C_{1}G{S}_{lx},C_{2}G{S}_{lx},\dots ..,C_{k}G{S}_{lx},\dots .\right\}$ and their corresponding augmented cluster representatives $A{R}_{lx}=\left\{A{R}_{lx1},\dots \dots \dots ,A{R}_{lxk},\dots \dots \right\}$ are formed. After that $Q$ number of most powerful augmented cluster representatives are chosen (as modified features) according to their $F{T}_x$ value from the generated clusters and with these $Q$ number of modified features, a reduced sub-dataset $RS{D}_{lx}$ of sub-dataset $S{D}_{lx}$ is formed.

In this way, for every bootstrapped version ( $B{K}_l$) of the training dataset, seven number of $RS{D}_{lx}$ sub-datasets are created and for every $RS{D}_{lx}$ an individual classifier is constructed using any existing classifier and finally, an ensemble classifier (EC) is formed by combining all these classifiers of all bootstrapped versions using the majority voting technique. To classify every sample using this ensemble classifier, each classifier votes or classifies the sample for a particular class, and the class for which the highest number of votes is obtained is considered as the output class.

MFSAC method based informative attribute ranking

For every gene (feature/attribute), the frequency of occurrence that means the total number of times it appears in all sub-datasets generated by the MFSAC method for all bootstrapped versions is calculated. Then according to their frequency of occurrence, those genes are ordered or ranked. The top-ranked genes with the highest occurrence frequency are considered the most informative cancer-related genes.

Algorithm::

MFSAC-EC

Input: A ${\mathrm{K}}_{U\times V}$ data matrix (here, gene expression data matrix) containing $U$ number of data objects (here, cancer samples) and $V$ number of attributes (here, genes).

Output: An ensemble classifier MFSAC-EC is formed to classify test samples. From MFSAC generated sub-datasets, informative genes are selected according to their rank. Every gene is ranked according to its frequency of occurrence.

Definitions:

$\mathrm{E}=\left\{E_1,E_2,\dots ..,E_s,\dots E_U\right\}$ is the set of objects or samples of ${\mathrm{K}}_{U\times V}$ data matrix. Every sample $E_s$ is a $V$ dimensional vector.

$G=\left\{G_1,G_2,\dots \dots ,G_t,\dots .G_V\right\}$ is the set of features or genes of ${\mathrm{K}}_{U\times V}$ data matrix. Every gene $G_t$ is a $U$ dimensional vector.

$BK=\left\{B{K}_1,B{K}_2,\dots ,B{K}_l,\dots ..,B{K}_D\right\}$ is a set of the bootstrapped version of the original training dataset. In every bootstrapped dataset the number of samples varies from the original dataset but the number of features is the same as the original dataset.

$C_{U\times 1}$ is a class vector representing the associated class label for every sample. For a data matrix $N$ distinct class labels exist and class labels are taken from a set $DC=\left\{d_1,d_2,\dots ..,d_k,\dots d_N\right\}$.

$F{T}_x\left(G_t,C\right)$ is $x^{th}$ filter score function which returns the class relevance value of $G_t$ gene with respect to class vector $C$ using $F{T}_x$ score function, for $x=1to7$ as $7$ represents the total number of filtering score functions used here.

$G{S}_{lx}\left(G{S}_{lx}=P\right)$ is a set of top-ranked genes of $G$ selected using $F{T}_x$ score function and $S{D}_{lx}$ is corresponding sub-dataset of $B{K}_l$. Here $S{D}_{lx}$ is a data matrix containing $P$ number of genes.

$CG{S}_{lx}=\left\{C_{1}G{S}_{lx},C_{2}G{S}_{lx},\dots ..,C_{k}G{S}_{lx},\dots .\right\}$ and $A{R}_{lx}=\left\{A{R}_{lx1},\dots \dots \dots ,A{R}_{lxk},\dots \dots \right\}$are the set of clusters and corresponding cluster representatives respectively generated from the corresponding subset $G{S}_{lx}$ of $S{D}_{lx}$. Here every $A{R}_{lxk}$ is a vector.

TR+, TR−, R are vectors similar to a gene vector.

$RS{D}_l=\left\{RS{D}_{l1},RS{D}_{l2},\dots \dots ..,RS{D}_{lx},\dots \dots RS{D}_{l7}\right\}$ is a set of sub-datasets each containing $Q$ number of most relevant cluster representatives formed for every bootstrapped dataset $B{K}_l$.

$C{F}_l=\left\{I{C}_{l1},I{C}_{l2},\dots \dots ..,I{C}_{lx},\dots \dots I{C}_{l7}\right\}$ is a set of classifiers formed for every bootstrapped dataset.

1. Create $D$ number bootstrapped version of training dataset $K$.

2. For Every bootstrapped dataset $B{K}_l$ repeat step 3

3. Repeat for $x=1to7$

A. Repeat for $t=1toV$

a) Calculate class relevance score $F{T}_x\left(G_t,C\right)$ of $G_t$ gene, where $G_t\in G,$ with respect to class vector $C$

B. Select $P$ number of top-ranked genes from $G$ based on $F{T}_x$ score function and form $G{S}_{lx}$ gene subset with corresponding $S{D}_{lx}$ sub-dataset

C. Set $k=0$

D. Repeat until $G{S}_{lx}=\mathrm{\varnothing }$

a) Set $k=k+1$

b) Set $A{R}_{lxk}=0,R=0,$ and $i=0$

c) Select the gene (let $G_{li}$) whose $F{T}_x$ score value is maximum among all genes of $G{S}_{lx}$ and set $R=G_{li}$

d) Add $G_{li}$ to $C_{k}G{S}_{lx}$, and delete $G_{li}$ from $G{S}_{lx}$

e) Set count =1

f) Repeat for $j=1to|G{S}_{lx}|$

I. Compute first augmented representatives $T{R}^{+}$ by adding $G_{lj}\in$ $G{S}_{lx}$ with $R$ that means $T{R}^{+}=R+G_{lj}$

II. Compute second augmented representatives $T{R}^{-}$ by adding sign-flipped version of $G_{lj}\in$ $G{S}_{lx}$ with $R$ that means $T{R}^{-}=R-G_{lj}$

III. Compute class relevance value $F{T}_x\left(T{R}^{+},C\right)$and $F{T}_x\left(T{R}^{-},C\right)$ using $F{T}_x$ score function

IV. If $F{T}_x\left(T{R}^{+},C\right)\ge F{T}_x\left(T{R}^{-},C\right)$then

If $F{T}_x\left(T{R}^{+},C\right)>F{T}_x\left(R,C\right)$ then

• Set $R=R+G_{lj}$ and add $G_{lj}$ to $C_{k}G{S}_{lx}$ and delete $G_{lj}$ from $G{S}_{lx}$       • count = count +1

V. If $F{T}_x\left(T{R}^{-},C\right)>F{T}_x\left(T{R}^{+},C\right)$ then

If $F{T}_x\left(T{R}^{-},C\right)>F{T}_x\left(R,C\right)$ then

• Set $R=R-G_{lj}$ and add $G_{lj}$ to $C_{k}G{S}_{lx}$ and delete $G_{lj}$ from $G{S}_{lx}$       • count = count + 1

g) Set $R=R/count$

h) Set $A{R}_{lxk}=R$

E. Select $Q$ number of most relevant cluster representatives according to $F{T}_x$ score from $A{R}_{lx}$ set and form $RS{D}_{lx}$ sub-data set.

F. Construct a classifier $C_{lx}$ for $RS{D}_{lx}$ sub-data set

4. Apply a test sample over all the classifiers of all bootstrapped dataset and calculate the prediction accuracy of each classifier

5. Apply simple voting over all predictions to form an ensemble classifier $EC$ and get final prediction.

6. Calculate number of occurrences for every gene for all $RS{D}_{lx}$ sub-datasets across all bootstrapped versions and rank them according to their count.

7. Select a number of top-ranked genes as informative genes.

8. End

DOI: 10.7717/peerj-cs.671/table-13

The block diagram of the proposed MFSAC-EC model is shown in Fig. 2, while the block diagram of the MFSAC method is shown in Fig. 3. The algorithm of the proposed model is described below.

Results

To assess the performance of the proposed MFSAC-EC model, four well-known existing classifiers named K-Nearest Neighbor (Duda, Hart & Stork, 1999), Naive Bayes (Duda, Hart & Stork, 1999), Support vector machine (Vapnik, 1995), and Decision tree (c4.5) (Duda, Hart & Stork, 1999) are applied independently in this model and four different ensemble classification models are formed. To prove the superiority of the proposed model, it is compared with existing well-known filter methods (used here) and existing recognized gene selection methods (Ding & Peng, 2005; Au et al., 2005; Maji & Das, 2012) and also with different existing ensemble classifiers (Bolón-Canedo, Sánchez-Maroño & Alonso-Betanzos, 2012; Nagi & Bhattacharyya, 2013; Osareh & Bita, 2013; Wang, 2006; Wang, Li & Fanget, 2012). To analyze the performance, the methods are applied to different publicly available cancer and other disease-related gene expression datasets. The major metrics used here for evaluations of the performance of the proposed classifier are the cross-validation method (LOOCV, fivefold, and tenfold), ROC Curve, and Heat map.

Classification performance of the proposed MFSAC-EC classifier

In Table 4, using the LOOCV method, the classification accuracy of our proposed MFSAC-EC model is 100% for different datasets (Leukemia, Breast, RBreast, Lung, RAOA, and RAHC) for all cases. In the Prostate dataset, we did not get 100% accuracy using our model with respect to any type of existing classifier. In MLL, Colon, and SRBCT it also gives 100% accuracy using all types of ensemble classifiers.

Table 4:

Classification accuracy of the proposed MFSAC-EC model with respect to LOOCV.

Classification accuracy (%) of MFSAC-EC model has been shown in terms of LOOCV with respect to four ensemble classifiers MFSAC-EC + NB, MFSAC-EC+KNN, MFSAC-EC+DT, and MFSAC-EC+SVM. Every ensemble classifier is run 50 times using LOOCV for every dataset and the accuracy is shown which is obtained maximum number of times.

Dataset	Proposed model		Cluster representatives			Dataset	Proposed model		Cluster representatives
			1	2	3				1	2	3
COLON	MFSAC-EC	NB	100	98.39	98.39	MLL	MFSAC-EC	NB	100	100	100
		KNN	98.39	100	100			KNN	100	100	100
		DT	98.39	98.39	98.39			DT	100	100	100
		SVM	100	98.4	98.4			SVM	100	100	100
Prostate		NB	97.06	97.79	98.53	SRBCT		NB	96.83	100	100
		KNN	97.79	97.79	98.53			KNN	96.83	100	100
		DT	97.79	98.53	97.79			DT	96.83	98.41	100
		SVM	98.53	99.26	99.26			SVM	82.54	98.41	100
Leukemia		NB	100	100	100	Lung		NB	100	100	100
		KNN	100	100	100			KNN	100	100	100
		DT	100	100	100			DT	100	100	100
		SVM	100	100	100			SVM	100	100	100
RAOA		NB	100	100	100	RAHC		NB	100	100	100
		KNN	100	100	100			KNN	100	100	100
		DT	100	100	100			DT	100	100	100
		SVM	100	100	100			SVM	100	100	100
Breast		NB	100	100	100	RBreast		NB	100	100	100
		KNN	100	100	100			KNN	100	100	100
		DT	100	100	100			DT	100	100	100
		SVM	100	100	100			SVM	100	100	100

DOI: 10.7717/peerj-cs.671/table-4

In Tables 5 and 6, it has been shown that using fivefold and tenfold cross-validation, MFSAC-EC does not provide 100% accuracy only for Colon and Prostate cancer datasets. For other datasets, it provides 100% accuracy with respect to all types of ensemble classifiers.

Table 5:

Classification accuracy of the proposed MFSAC-EC model with respect to fivefold cross validation.

Classification accuracy (%) of MFSAC-EC model has been shown in terms of fivefold Cross Validation with respect to four ensemble classifiers MFSAC-EC + NB, MFSAC-EC+KNN, MFSAC-EC+DT, and MFSAC-EC+SVM. Every ensemble classifier is run 50 times using fivefold Cross Validation for every dataset and the accuracy is shown which is obtained maximum number of times.

Dataset	Proposed model		Cluster representatives			Dataset	Proposed model		Cluster representatives
			1	2	3				1	2	3
COLON	MFSAC-EC	NB	96.77	96.77	96.77	MLL	MFSAC-EC	NB	100	100	100
		KNN	98.39	96.77	96.77			KNN	98.61	100	100
		DT	98.39	96.77	98.39			DT	98.61	100	100
		SVM	98.39	96.77	96.77			SVM	100	100	100
Prostate		NB	97.06	97.79	98.53	SRBCT		NB	98.41	100	100
		KNN	97.79	97.79	99.26			KNN	96.83	100	100
		DT	97.06	97.79	94.85			DT	96.83	98.41	100
		SVM	97.79	98.53	99.26			SVM	96.83	100	100
Leukemia		NB	100	100	100	Lung		NB	100	100	100
		KNN	100	100	100			KNN	100	100	100
		DT	100	100	100			DT	100	99.44	100
		SVM	100	100	100			SVM	100	100	100
RAOA		NB	100	100	100	RAHC		NB	100	100	100
		KNN	100	100	100			KNN	100	100	100
		DT	100	100	100			DT	100	100	100
		SVM	100	100	100			SVM	100	100	100
Breast		NB	100	100	100	RBreast		NB	100	100	100
		KNN	100	100	100			KNN	100	100	100
		DT	100	100	100			DT	100	100	100
		SVM	100	100	100			SVM	100	100	100

DOI: 10.7717/peerj-cs.671/table-5

Table 6:

Classification accuracy of the proposed MFSAC-EC model with respect to tenfold cross validation.

Classification accuracy (%) of MFSAC-EC model has been shown in terms of tenfold cross validation with respect to four ensemble classifiers MFSAC-EC + NB, MFSAC-EC+KNN, MFSAC-EC+DT, and MFSAC-EC+SVM. Every ensemble classifier is run 50 times using tenfold cross validation for every dataset and the accuracy is shown which is obtained maximum number of times.

Dataset	Proposed model		Cluster representatives			Dataset	Proposed model		Cluster representatives
			1	2	3				1	2	3
COLON	MFSAC-EC	NB	98.39	98.39	98.39	MLL	MFSAC-EC	NB	100	100	100
		KNN	98.39	98.39	100			KNN	100	100	100
		DT	98.39	98.39	98.39			DT	100	100	100
		SVM	98.39	98.39	98.39			SVM	100	100	100
Prostate		NB	97.06	97.79	98.53	SRBCT		NB	96.83	96.83	100
		KNN	97.79	97.79	99.26			KNN	92.06	100	100
		DT	97.06	97.79	94.85			DT	95.24	96.83	100
		SVM	97.79	98.53	99.26			SVM	80.95	92.06	100
Leukemia		NB	100	100	100	Lung		NB	100	100	100
		KNN	100	100	100			KNN	100	100	100
		DT	100	100	100			DT	100	100	100
		SVM	100	100	100			SVM	100	100	100
Breast		NB	100	100	100	RBreast		NB	100	100	100
		KNN	100	100	100			KNN	100	100	100
		DT	100	100	100			DT	100	100	100
		SVM	100	100	100			SVM	100	100	100
RAOA		NB	100	100	100	RAHC		NB	100	100	100
		KNN	100	100	100			KNN	100	100	100
		DT	100	100	100			DT	100	100	100
		SVM	100	100	100			SVM	100	100	100

DOI: 10.7717/peerj-cs.671/table-6

To show the generalization property of the proposed ensemble classifiers, the classification accuracy of these classifiers is also measured repeatedly with respect to the random splitting of the dataset into a training set (2/3 data of original dataset) and test set (1/3 data of original dataset). Random splitting is done with care such that class proportion is alike in the training set and test set. In Table 7, the classification accuracy of the above mentioned four different types of ensemble classifiers for the different number of cluster representatives is shown in different datasets which are based on the best result of 50 random splitting of the dataset into a training set (2/3 data of original dataset) and test set (1/3 data of original dataset).

Table 7:

Classification accuracy of the proposed MFSAC-EC model with respect to random splitting of the datasets.

Classification accuracy (%) of MFSAC-EC model has been shown in terms of random splitting with respect to four ensemble classifiers MFSAC-EC + NB, MFSAC-EC+KNN, MFSAC-EC+DT, and MFSAC-EC+SVM. Every ensemble classifier is run 50 times using random splitting for every dataset and the accuracy is shown which is obtained maximum number of times. For random splitting the dataset is divided into training (2/3) and testing (1/3) part 50 times randomly.

Dataset	Proposed model		Cluster representatives			Dataset	Proposed model		Cluster representatives
			1	2	3				1	2	3
COLON	MFSAC-EC	NB	98.39	98.39	98.39	MLL	MFSAC-EC	NB	100	100	100
		KNN	98.39	98.39	98.39			KNN	100	100	100
		DT	98.39	98.39	98.39			DT	98.61	100	98.61
		SVM	98.39	100	98.39			SVM	100	100	100
Prostate		NB	94.68	95.74	93.62	SRBCT		NB	95	85	95
		KNN	97.87	96.81	92.55			KNN	95	100	90
		DT	94.68	94.68	94.68			DT	80	90	95
		SVM	94.68	96.81	94.68			SVM	65	75	95
Leukemia		NB	100	100	100	Lung		NB	100	100	100
		KNN	100	100	100			KNN	100	100	100
		DT	100	100	100			DT	100	100	100
		SVM	100	100	100			SVM	100	100	100
RAOA		NB	100	100	100	RAHC		NB	100	100	100
		KNN	100	100	100			KNN	100	100	100
		DT	100	100	100			DT	100	100	81.25
		SVM	100	100	100			SVM	100	100	81.25
Breast		NB	100	100	100	RBreast		NB	91.94	91.94	91.94
		KNN	100	100	100			KNN	85.48	87.10	83.87
		DT	100	100	100			DT	83.87	79.03	80.65
		SVM	100	100	100			SVM	93.55	91.94	91.94

DOI: 10.7717/peerj-cs.671/table-7

From the results of Tables 4 to 7, it has been observed that classification accuracy in the LOOCV method, fivefold cross-validation, and tenfold cross-validation methods is higher than the random splitting of the dataset, and the overall generalization performance of the proposed classification model is also good.

The performance of the proposed model for different two-class datasets with respect to different parameters like SN, SP, PPV, NPV, FPR is shown in Table 8. From this table, it is found that the performance of the proposed model is very good with respect to all these parameters for all two-class datasets.

Table 8:

Evaluation of MFSAC-EC classifier based on SN, SP, PPV, NPV, FPR for two class data sets with respect to LOOCV.

The performance of the MFSAC-EC model for two class datasets is represented using Receiver Operator Characteristic (ROC) analysis. SN represents Sensitivity, SP represents Specificity, PPV represents Positive Predicted Value, NPV represents Negative Predicted Value, and FPR represents False Positive Rate.

Dataset	Proposed model		SN	SP	PPV	NPV	FPR	Dataset	Proposed model		SN	SP	PPV	NPV	FPR
Leukemia	MFSGC-EC	NB	100	100	100	100	0	Breast	MFSGC-EC	NB	100	100	100	100	0
		KNN	100	100	100	100	0			KNN	100	100	100	100	0
		DT	100	100	100	100	0			DT	100	100	100	100	0
		SVM	100	100	100	100	0			SVM	100	100	100	100	0
Prostate		NB	98.7	98.3	98.7	98.3	1.7	Rbreast		NB	100	100	100	100	0
		KNN	98.7	98.3	98.7	98.3	1.7			KNN	100	100	100	100	0
		DT	100	96.61	97.46	100	3.4			DT	100	100	100	100	0
		SVM	100	98.3	98.7	100	1.7			SVM	100	100	100	100	0
Colon		NB	100	100	100	100	0	Lung		NB	100	100	100	100	0
		KNN	100	100	100	100	0			KNN	100	100	100	100	0
		DT	100	100	100	100	0			DT	100	100	100	100	0
		SVM	100	100	100	100	0			SVM	100	100	100	100	0
RAHC		NB	100	100	100	100	0	RAOA		NB	100	100	100	100	0
		KNN	100	100	100	100	0			KNN	100	100	100	100	0
		DT	100	100	100	100	0			DT	100	100	100	100	0
		SVM	100	100	100	100	0			SVM	100	100	100	100	0

DOI: 10.7717/peerj-cs.671/table-8

In Fig. 4, the ROC curve is shown for different two-class datasets. In Figs. 4A, 4B, and 4C, the ROC curves are shown for Breast cancer using LOOCV, for Colon cancer using fivefold cross validation, and for RAHC dataset using tenfold cross-validation respectively. The ROC curves for Leukemia Cancer, and Lung cancer datasets using LOOCV are given in Figs. S1A and S1B respectively. For Breast cancer, Leukemia cancer, and Lung cancer, the AUC value is equal to 1.0 in every case. The ROC curves are shown for RAOA, and RBreast cancer datasets using fivefold cross-validation in Figs. S2A, and S2B respectively. For these datasets also the prediction accuracy using fivefold cross validation is very high according to the AUC value. In Fig. S2C, the ROC curves are shown for Prostate cancer using tenfold cross-validation. From these curves of tenfold cross validation, it may be seen that except for Prostate cancer, for all other datasets the AUC value is 1 and for Prostate cancer, the AUC value is close to 1.

In Figs. 5A and 5B, heatmap representation of the confusion matrix are shown for multi-class datasets: SRBCT and MLL with respect to fivefold cross-validation, and tenfold cross-validation respectively. From these figures, it is clear that for the proposed model prediction accuracy is accurate in most cases.

Comparison of MFSAC-EC Model with Well-Known Existing Gene Selection Methods

In Fig. 6, the MFSAC-EC model with different existing classifiers as base classifiers are compared with existing well-known supervised gene selection methods named mRMR (minimum redundancy maximum relevance framework) (Ding & Peng, 2005), MSG (mutual information based supervised gene clustering algorithm) (Maji & Das, 2012), CFS (Correlation-based Feature Selection) (Ruiz, Riquelme & Aguilar-Ruiz, 2006), and FCBF (Fast Correlation-Based Filter) (Ruiz, Riquelme & Aguilar-Ruiz, 2006) with respect to different classifiers using tenfold cross-validation method. From these results, it has been found that the proposed model outperforms in most of the cases.

In Fig. 7, the MFSAC-EC model is compared with well-known existing unsupervised gene selection methods named MGSACO (Tabakhi et al., 2015), UFSACO (Tabakhi, Moradi & Akhlaghian, 2014), RSM (Lai, Reinders & Wessels, 2006), MC (Haindl et al., 2006), RRFS (Ferreira & Figueiredo, 2012), TV (Theodoridis & Koutroumbas, 2008), and LS (Liao et al., 2014) with respect to DT, SVM, NB classifiers using random splitting method. From these results, it can be said that the MFSAC-EC model outperforms in all cases.

Discussion

In this paper, a new Multiple Filtering and Supervised Attribute Clustering algorithm-based ensemble classification model named MFSAC-EC is proposed. The main motivation behind this work is to develop a machine learning-based ensemble classification model to overcome the over-fitting problem which arises due to the presence of sample class imbalance problem, small sample size problem, and also high dimensional feature set problem in the microarray gene expression dataset, to enhance the prediction capability of the proposed model.

Nowadays, in designing machine learning models, the use of ensemble methodology has been increasing day by day as it incorporates multiple learning algorithms and also training datasets in different efficient manners to improve the overall prediction accuracy of the model. Due to the inclusion of prediction accuracy of multiple learning models and also the use of different bootstrapping datasets, the chances of potential overfitting in training data is greatly reduced in the ensemble models, and as a consequence the prediction accuracy increases. One necessary condition of the superior performance of an ensemble classifier with respect to its individual member/base classifier is that every base classifier should be very accurate and diverse (Osareh & Bita, 2013). A classifier is considered accurate if its generalization capability is high and two classifiers satisfy diverse property if their prediction in classifying the same unknown samples vary from each other. The general principle of ensemble methods is to rearrange training datasets in different ways (either by resampling or reweighting) and build an ensemble of base classifiers by applying a base classifier on every rearranged training dataset (Osareh & Bita, 2013).

In our proposed ensemble model, at first, a number of bootstrapped datasets of the original training dataset is created. In every bootstrapped dataset, the class imbalance problem is solved using the oversampling method. Then for every bootstrapped dataset, a number of sub-datasets are created using the MFSAC method (which is a hybrid method combining multi-filters and a new supervised attribute/gene clustering method), and then for every generated sub-dataset, a base classifier is constructed using any existing classification model. After that, a new ensemble classifier (EC) is formed using the majority voting scheme by combining the prediction accuracy of all those base classifiers.

The prediction accuracy of the proposed model is verified by applying it to high-dimensional microarray gene expression data From Figs. 6, and 7 it has been found that the classification accuracy of the MFSAC-EC model is much better than the well-known existing gene selection methods. From Tables 9, 10, and 11, it has been also found that the proposed MFSAC-EC classification model is superior to the existing ensemble classification models in almost every case. The superior performance of the proposed model is due to the following reasons:

• The generation of the different bootstrapped versions of training data and also the use of the oversampling procedure to balance the cardinality of majority class and minority class in every bootstrapped dataset reduces the chances of the overfitting problem of a classifier.

• Different types of filter methods are used in the MFSAC method. It has been already observed that one filter gives better performance for one dataset while the same gives poor results for other datasets. This is because every filter uses separate metrics and so the choice for a filter for a specific dataset is a very complex task. As different filter methods are used in the MFSAC method, so different sub-datasets with different characteristics-based attributes/genes are formed from each dataset. This is shown using Venn diagram in Figs. S4A and S4B. Here for Leukemia and Prostate cancer datasets, the first twenty genes, selected by each filter are shown. In case of Leukemia dataset, Relief measure generates non-overlapping gene subset while using other filter metrics presence of a small number of overlapping genes in different gene subsets are observed. In Prostate cancer dataset, Relief generates non-overlapping gene subset and also maximum number of genes are non-overlapping in gene subsets formed by Fisher score, MI (mutual information). From these figures, it is clear that using different filter methods different subsets of genes are selected and different sub-datasets are formed. It shows diversity of those filter methods. As a consequence, the base classifiers prepared on these diverse datasets are become diverse. This diversity increases the power of ensemble classifier.

• Moreover, the genes selected by different filter methos are good biomarker also. In Table 12, the top ranked eight genes selected by MFSAC-EC model are shown for Leukemia and Colon cancer datasets. Among these genes, gene MPO (with column number 1,720), CST3 (with column number 1,823), ZYX (with column number 4,788), CTSD (with column number 2,062), CD79A/MB-1(with column number 2,583), LYZ (with column number 6,738) in Leukemia dataset are important biomarkers as these are selected by different filter methods mentioned in Fig. S4.

• In MFSAC, at first, a sub-dataset of the most relevant genes is selected by each filter method. Then on each sub-dataset, the proposed supervised gene clustering algorithm is applied and a reduced sub-dataset of modified attributes/features in the form of augmented cluster representatives is generated. In this method, at the time of cluster formation, genes are augmented based on their supervised information. In other words, such augmentation is considered where it increases the class discrimination power. Thus effectively, the class relevance of any augmented cluster representative is greater than that of any single gene involved in that process. So, this modified sub-dataset containing a reduced feature set in the form of augmented cluster representatives is more powerful according to class discrimination power than the sub-dataset containing a subset of the most relevant genes. Apart from this, it is well known fact in gene expression data that two genes are functionally similar if they are pattern-based similar (either positively co-expressed or negatively co-expressed) (Das et al., 2016). So, at the time of the augmentation procedure, two types of augmentations are considered here. One is that a gene is added with its original value with the current cluster representative and another one is that the gene is added with its sign-flipped value with the current cluster representative. This is because if the current cluster representative and a gene are positively co-expressed then normal addition is considered but if they are negatively co-expressed then normal addition will hamper the addition process and in that case, sign-flipping of that gene will give proper result. The effect of augmentation with respect to every filter method is shown in Fig. 8. In Fig. 8, for the Breast cancer dataset, at the time of supervised cluster formation from each filter generated subset, the original gene, and its corresponding class relevance value, and also augmented gene and its corresponding class relevance are shown. From Fig. 8, it is clear that for every filter method the class relevance score of every original gene is increased with respect to that filter after augmentation. In Fig. 8, different class labels are distinguished by different colors.

• Finally, for each sub-dataset with modified attributes in the form of augmented cluster representatives, a classifier is constructed using any existing classifier, and these classifiers are combined using the majority voting technique to form an ensemble classifier (EC). The use of different sub-datasets with optimal gene subsets in the form of augmented cluster representatives and the formation of a classifier for every sub-dataset can solve the overfitting problem of any single classifier. This is due to the reason that not all sub-datasets can consistently perform well on all types of cancer datasets (due to inherent characteristics of the datasets), but due to the use of majority voting in ensemble classifiers, this problem can be solved or reduced.

Another outcome of our proposed model is to rank informative genes for every cancer dataset. For this task, the frequency of occurrence of each gene present in the form of augmented cluster representatives in every sub-dataset is counted and these genes are ranked according to the counted value to measure the importance of those genes for any specific disease, here cancer. To establish the biological significance of those selected genes for every cancer dataset, their contribution has been confirmed by other existing studies where they are referred already. From these existing studies, it is clear that the selected genes are important for cancer class discrimination and also are important as cancer biomarkers for molecular treatment targets.

Conclusions

Many machine learning and statistical learning-based classifiers for sample classification already exist in the literature, but these methods are prone to suffer from overfitting due to small sample size problems, class imbalance problems, and the curse of the high dimensionality of microarray data. Although some of the existing methods can mitigate these issues to quite an extent, the problems have still not been satisfactorily overcome. Due to this reason, here a novel feature selection-based ensemble classification model named MFSAC-EC is proposed. It has been shown that the proposed model can handle the above-mentioned issues present in existing models. To check the performance of the proposed MFSAC-EC model, this classifier is applied to test sample classification accuracy in high dimensional microarray gene expression data, a domain that will be beneficial in the field of cancer research. From the experimental results, it has been found that the proposed model outperforms all other well-known existing classification models combined with the different recognized feature selection methods and also the newly developed ensemble classifiers for all types of cancer datasets mentioned here. Apart from this classification task, the proposed model can also rank informative attributes according to their importance. The efficiency of the proposed model in this task is vindicated by finding the most informative genes for the colon cancer and leukemia cancer datasets using this model. These genes are biologically validated based on other well-known existing studies. Consequently, it is clear that the selected genes are vital for sample class discrimination and are also important biomarkers for molecular treatment targets of deadly diseases.

Supplemental Information