Artificial neural network with Taguchi method for robust classification model to improve classification accuracy of breast cancer

Experimental dataset

This research used the Wisconsin Diagnostic Breast Cancer (WDBC) dataset from the UCI Machine Learning Repository (http://archive.ics.uci.edu/ml/datasets/Breast+Cancer+Wisconsin+(Diagnostic)) to differentiate malignant tumors from normal tumor samples. The dataset was used to compare normal tumors with cancerous (malignant) tumors (Pobiruchin et al., 2016). The dataset contains 32 features, namely ID, diagnosis, and 30 real-valued input features, followed by 569 samples, of which 357 are normal and 212 are cancerous, with zero missing attribute values. The dataset features were originally computed from a digitized image of a fine needle aspirate (FNA) of a breast mass by the first user and have become a reference dataset for many recent studies on breast cancer classification. The attribute information is as follows:

Attribute Information

For this study, our experimental dataset was divided into training, validation, and testing datasets using four different partitions based on the Taguchi method (Followed by Eq. (1) and Table 1). The partitions were chosen based on the experimental performance. The dataset partitions are as follows: The first partition contains 50, 25, and 25; the second partition contains 60, 20, and 20; the third partition contains 70, 15, and 15; and, finally, the fourth partition contains 80, 10, and 10, where the first number represents the training set, the second number represents the validation set, and the third number represents the testing set for each partition, respectively. The subsequent sections 3.2 and 4 present details of how the Taguchi method was utilized with it‘s performance on different dataset partitions. (Table 2).

Taguchi Method

The Taguchi method is a robust experimental design (Wu & Wu, 2000) process that can be analyzed and improved upon by altering the relative design factors. It is also called the statistical method and can be used to realize the highest product quality. The Taguchi method utilizes a three-stage method: system design, parameter design, and tolerance design. In the system design, suitable working levels of design factors are accounted for. For design and testing, a system needs to be based on the designers’ judgement of factors such as materials, parts, nominal products, processes, or parameters, based on the latest technology. The parameter design is used to determine parametric levels in order to enhance the accuracy of the process being considered. Tolerance design is used to fine-tune the results. As a commonly used robust design approach, the Taguchi method has two mechanisms: Orthogonal array and signal-to-noise ratio (SNR) (Chuang et al., 2010), for improvement and analysis. To minimize experimental efforts, Orthogonal array is mainly utilized, using N number of design parameters.

An Orthogonal array is very useful, as it gives an extensive investigation of associations among all design factors and reasonably adjusts the methodical correlations of various dimensions of each factor. An orthogonal array is a two-dimensional array. Each column represents a certain design parameter, while each row denotes an experimental test with an actual arrangement of various levels for all of the design factors. In this research, the two-level Orthogonal array for determining optimal neurons is shown in Eq. (1): (1)${\displaystyle L_M\left(2^N\right),}$

where N is the number of columns in the Orthogonal matrix. M = 2K (M >N, K > log₂(N); M is the number of expected experimental trials, and K is an integer. Base 2 is the number of levels of every design parameter. In this study, there are 20 numbers of columns in the Orthogonal matrix (where F1 to F20 indicate the design factor the number of neurons in the hidden layer), and a two-level Orthogonal array was used for selecting appropriate number of neurons. The two-level Orthogonal array was created using L₂₁(2²⁰), as shown in Table 1.

Only 21 experimental trials are required for evaluation, analysis, and improvement. Conversely, all possible combinations of 20 design factors (i.e., 2²⁰ = 1,048,576) should be accounted for in the full factorial experimental design, which is frequently inapplicable in practice (Yang et al., 2008) (for neuron selection: 1: selected, 0: not selected).

This research used an orthogonal array mechanism to analyze and enhance the ANN algorithm performance by determining the optimal number of neurons in hidden layer. If a particular target has N different design factors, 2^N possible experimental trials will be considered in the full factorial experimental design.

Artificial neural network

An artificial neural network (ANN) (Rahman & Muniyandi, 2018a) is a machine learning approach, which models the human brain with a number of artificial neurons and interconnected associations. The neurons in ANNs tend to have fewer connections relative to a biological neural system. Figure 1 illustrates the basics of the ANN architecture.

Neurons are highly interconnected computational units inspired by the mammalian brain. The ANN system consists of the smallest processing nodes called neurons, or processing elements. To obtain outputs, a function called Sigmoid activation works on inputs and connects the weights with the help of neurons. The connections between weight (w) values and single nodes are called biases (b). The iterative flow of training data determines weight values throughout the network. During the training phase, weight values are verified until the network is able to detect a particular cluster using criteria for typical input data. Also, these weights assign the link relating one layer of neurons to another. Hence, changes in the relationship between input and output take place repeatedly, along with changes in weight values. The method of balancing the links’ weight values by repeatedly exposing the network to the input–output dataset for learning is called training.

ANN has various types of architectures. Multi-layer feed-forward neural networks have been widely used in cancer classification (Alesawy & Muniyandi, 2016; Hopfield, 1988; Bebis & Georgiopoulos, 1994). The architecture of a Multi-layer feed-forward neural network system is shown in Fig. 2, consisting of one or more input layer and one or more hidden layer along with an output layer.

Experimental setup and evolution methods

In this research, the experiment was conducted by using MATLAB Tool, WDBC dataset, Taguchi Method and ANN Algorithm, implemented on a system with the following configuration: Core i7 GPU, Windows 10 with 8GB RAM and 1TB HDD. The proposed ANN model comprises a single hidden layer with 15 neurons carefully selected using Taguchi method. Accuracy of the proposed model was obtained directly from the confusion matrix using the formula:

Accuracy $=\frac{\mathrm{TP}+\mathrm{TN}}{\mathrm{TP}+\mathrm{TN}+\mathrm{FP}+\mathrm{FN}}\times 100$.

ANN parameter optimization based on the Taguchi Method

To build an appropriate classification model using ANN, parameter selection is one of the most significant steps. The performance and stability of an ANN is dictated by its selected parameters. The Taguchi method is mainly used for experimental parametric selection, owing to the concept of engineering and technology. For instance, this method can decrease the vast number of experiments and simultaneously analyze several parameters. For excellent performance and low-cost computations, this method performs well in the context of systematic and efficient designs (Huang et al., 2014). A different partition for a dataset can be created to simplify the selection of a suitable dataset.

As per ‘Experimental dataset’, the L₂₁ Orthogonal array was used to determine the appropriate number of neurons in the hidden layer of the proposed IANN model. The number of neurons for the experiments ranges from 10 (the default value) to 20, as shown in Table 2. Also, different partitions of the dataset were utilized in order to determine a suitable partition for the breast cancer dataset. For the different partitions of the dataset, Taguchi Orthogonal array mechanism was implemented in the same way, where N = 3 is the number of design factors of dataset partitions. Therefore, the L₄ Orthogonal array was used for data partition (for more details, see ‘Experimental dataset’, Eq. (1)). The dataset was randomly divided into three partitions: training, validation, and testing. These partitions were utilized in different groups: first, 50%, 25%, and 25%; second, 60%, 20%, and 20%; third, 70%, 15%, and 15%; and fourth, 80%, 10%, and 10%.

It can be observed in Table 2 that the hidden layer parameter is crucial to the performance of the ANN. As established earlier, the experiments were conducted in four phases using different dataset partitions for each phase. For the first experiment, we partitioned the entire dataset into 50% training, 25% validation, and the remaining 25% for testing. Based on the three partitions used, the highest accuracy for the validation dataset was realized when the number of neurons in a single hidden layer is 15. In this experiment, the accuracy was found to be 93.90% for the validation dataset and 93.50% for the training dataset. However, when the number of neurons increased, the performance diminished significantly. This seemingly implied that when the number of neurons in the hidden layer increases further, the performance of the ANN model is negatively affected.

In the second experiment, when the partitioning of the dataset was changed to 60%, 20%, and 20% for training, validation, and testing, respectively, the Taguchi method confirmed that the highest performance was maintained at 15 neurons in a single hidden layer. It is therefore clear from Table 2 that utilizing the aforementioned partitions significantly improves the performance of IANN model by ∼3.0%.

In the third experiment, a new partition for the dataset was introduced, with 70% data for training, 15% for validation, and 15% for testing. In this scenario, the highest accuracy for the validation dataset was determined to be 98.30%, and that of the training dataset was 98.50% when the number of neurons in a single hidden layer was 15 using the Taguchi method.

Finally, in the fourth experiment, the partitioning of the dataset was 80%, 10%, and 10% for training, validation, and testing, respectively. This study confirmed that the results declined from 98.30% for the validation dataset to 93.80% with 4.5% when compared with the third experiment. All of the experiments proved that, 15 neurons are sufficient for modeling a single hidden layer in the proposed IANN for breast cancer classification.

From the empirical results, this study posits that the optimum conditions for breast cancer classification are 15 neurons in a single hidden layer and the partitioning of the data into 70%, 15%, and 15% for training, validation, and testing sets, respectively. Thus, we built an IANN model with three layers: input, hidden, and output. In summary, there were 31 input neurons at the input layer and 2 output neurons at the output layer, while the number of neurons in a single hidden layer was 15.

Breast cancer classification results

As established in ‘The proposed artificial neural network model with 15 neurons in a single hidden layer for breast cancer classification’, the suitable number of neurons for a single hidden layer based on the Taguchi is 15. In addition, 70% of the dataset for training, 15% for validation, and 15% for testing is the optimal partitioning. The experiments were simulated on MATLAB (R2017b) software using Neural Network Toolbox for the implementation of the proposed model. The proposed IANN model has been tested using the WDBC dataset.

Table 3 shows the results from 30 different classification simulations using the WDBC dataset with their corresponding percentage errors. The average of the tests’ accuracy was 98.80%. This study confirmed that the results of the simulations were relatively similar, which could be due to the high stability resulting from the selection of suitable parameters for the IANN classifier.

This study used 569-sample datasets. Each data sample consists of 32 features, with a total of 18,208 data points distributed as follows: 11,424 data points for normal and 6784 data points for cancerous. These features were used to train and simulate the proposed IANN model. A hidden-layer simple feed-forward neural network architecture is considered because our aim is to enhance the classification of breast cancer through optimal neuron selection in the hidden layer. The proposed IANN was tested with training, validation, and testing sets from the dataset. The network was adjusted based on the reported error during testing. Validation was used to simplify the network and halt training. Testing is ineffective in training; therefore, the performance of the network provides independent measures during and after training. A breast cancer dataset consisting of 569 samples was divided randomly into two groups: 399 samples (70%) for training and 170 samples (30%) for testing. The dataset for training was randomly divided into three groups: out of 399 samples 279 (70%) samples for training, and 60 (15%) samples each for testing and validation respectively.

Figures 5–11 show the IANN training performance, training state performance, error histogram, performance confusion matrix, ROC curve and testing dataset results for breast cancer, respectively.

It can be seen in Fig. 5 that the IANN training performance plot at the beginning of the training of cross-entropy resulted in the maximum error. The proposed system reported the best performance at epoch 20 iterations and an exact cross-entropy of 0.031613.

Figure 6 shows the network training state performance at epoch 26, when the gradient is 0.032875. The network halts the training session because its generalization stops improving.

Error histograms for training, validation, and testing data are shown with 20 bins in Fig. 7. The experimental results shown in error histogram of Fig. 7 indicate that, the proposed system can handle the dataset used successfully, since the error is close to zero.

To evaluate the accuracy, a confusion matrix was used for all partitions of the dataset. A confusion matrix is a two-dimensional array, r × r (where r is the number of classes). Figure 8 shows the MATLAB output representation of training dataset confusion matrix performance after 10-fold cross-validation. In Fig. 8, the first two rows and first two columns represent the actual confusion matrix. The third row and third column are the summary of percentage accuracy, and sensitivity and specificity, respectively. Row one, column one is the true positives (TP); row one, column two is the false negatives (FN); row two, column one is the false positives (FP); and row two, column two is the true negatives, respectively. All false positives (FP) are Type I errors, while all false negative (FN) are Type II errors generated by different classifiers.

The first two diagonal cells of the confusion matrix demonstrate the correct classification number and percentage accuracy of the trained network. The normal biopsy results are 247, which were correctly classified to represent 61.90% of the total 399 biopsies. On the other hand, the cancerous (malignant) biopsy results show 146 correctly classified tumors, representing 36.60% of the total biopsies. The experiment also revealed that three of the cancerous biopsies were incorrectly classified as normal, representing 0.8% of the total biopsy dataset, while three of the normal biopsies were incorrectly classified as cancerous, also representing 0.8% of the total biopsy data. The total result of 250 normal revealed that 98.80% was correct, and 1.20% incorrect. The total result of 149 cancerous was 98.00% correct and 2.00% incorrect. The total result of 250 normal cases was 98.80% correctly classified as normal, and 1.2% classified as cancerous. Out of 149 cancerous cases, 98.00% were correctly classified as cancer, and 2.00% classified as normal. The confusion matrix plots with 98.50% accuracy show that this system performed well and had 1.40% misclassification during its training stage from the proposed IANN.

The neural network training performance with receiver operating characteristic (ROC) plot is shown in Fig. 9. The ROC plot represents the performance of the binary classification system when the discrimination threshold fluctuates. The graph is formed by plotting the true positive rate (TPR) against the false positive rate (FPR). From ROC plot, it can be seen that the NN’s performance increases in response to the number of iterations. A perfect classification result was evident at 26 iterations. This shows that every class achieved perfect classification accuracy. Iteration 26 is the optimal iteration for the proposed model where the IANN performed at its peak.

After completing the training session, we tested the network for accurate classification, using 30% of the test dataset of breast cancer. Figure 10 shows the testing accuracy, otherwise known as the classification accuracy from the proposed IANN, to be rather high.

Figure 10 shows the experimental results with a test dataset of breast cancer: normal versus malignant tumor is correctly classified at 98.8% accuracy for the test case.

This study computed the receiver operating curve (ROC) and the results are shown in Fig. 11. The ROC curve indicates TPR and FPR at different edge settings of the network, with better results arising from the proposed system. The IANN, after training, validation, and testing, achieved 98.8% correct classification for two classes: normal and cancerous (malignant). The area under the curve was large.

Table 4 shows the model performance obtained from the confusion matrix shown in Figs. 8 and 10. In Table 4, the F-Score for the training dataset is 98.4%, while the F-Score for the testing dataset is 98.8%.

Discussion of findings and implications

Table 5 shows the performance of multiple machine learning algorithms relative to the proposed IANN model. Performance comparison was realized via MATLAB Classification learner tool-staking for the breast cancer dataset. Although the results obtained with other machine learning algorithms are equally good, the result reported by our model is optimal. It can be surmised that the proposed IANN outperforms the existing machine learning models in classifying breast cancer.

Table 6 indicates the superiority of this study in the context of cancer classification based on the same dataset but different methods for classification. Among the previous studies tabulated in Table 6, a lower classification accuracy of 92.98% was achieved by Xue, Zhang & Browne (2014). These authors proposed the PSO technique with novel initialization and updated mechanisms hybridized with a KNN classifier and 10-fold cross-validation to maximize the classification performance of the WDBC datasets and dataset were partitioned for training 70% and 30% for testing. Quy et al. (2020) achieved the best classification accuracy of 98.24% using APSO-NN. For the NN used 20 neurons in a single hidden layer. The WDBC dataset was used and partitioned 70% for training and 30% for testing. Abdar et al. (2020) used the WDBC dataset for the experiment and achieved a classification accuracy of 98.07% using SV-NB-3-MetaClassifiers with a K-fold cross-validation technique. The performance is quite good. Nekkaa & Boughaci (2015) used a memetic algorithm (MA) with support vector machine (SVM) to address the classification problem. The authors used particular datasets to compare certain popular classifiers for the data classification task. The model achieved 97.85% accuracy using WDBC datasets. Beside, Salama & Abdelhalim (2012) achieved a classification accuracy of 97.70% using sequential minimal optimization (SMO) technique with a confusion matrix based on the 10-fold cross-validation method. Amrane et al. (2018) used the KNN and Naive Bayes (NB) classifier machine learning technique for breast cancer classification. The author partitioned the dataset 60% for training and 40% for tesing. From KNN classifier achieved the highest classification accuracy of 97.51% which is not so high. Aalaei et al. (2016) used a GA-based classifier with ANN and reported a classification accuracy of 96.50% by making use of a single hidden layer with 5-neurons and 2-neurons in output layer, and the dataset partitioned of 80% for training and 20% for test data.

From the analysis of the results presented in Tables 5 and 6, it is obvious that the proposed method outperformed the other methods due to it realizing a classification accuracy of 98.80% —an improvement of 0.56% compared to Quy et al. (2020), which has the best performance among the existing methods. This could be due to the use of the Taguchi method for selecting the optimal number of neurons in a single hidden layer of ANN and suitable percentage data partition. Our results also show that the proposed method is more stable and reliable than existing classification models. By implication, incorporating IANN into breast cancer prediction could enable timely and accurate prediction of breast cancer, thereby helping medical practitioners to make the most appropriate decisions on breast cancer treatment. Table 7 (where K = 10) shows the 10-fold cross validation experimental performance which was conducted to validate the results shown in Table 5 and Table 6.