Detecting spam and ham SMS messages using natural language processing and machine learning algorithms

Background information

With the rise in mobile device usage and the declining cost of short message service (SMS), the volume of transmitted messages has grown exponentially. SMS is one of the cheapest and most popular communication methods worldwide for exchanging messages between servers (Cormack, 2006). Mobile users send SMS messages for various purposes, including greetings, business communications, and advertisements. This growth has also led to an increase in attackers sending spam SMS messages for malicious purposes (Almeida, Gomez Hidalgo & Yamakami, 2011; Parandeh Motlagh & Khatibi Bardsiri, 2018). Spam SMS messages can be defined as unsolicited SMSs sent without the recipient’s permission (Tekerek, 2017; Sheikhi, Kheirabadi & Bazzazi, 2020; Ahmed et al., 2022; Rajput, Athavale & Mittal, 2019; Alghoul et al., 2018). These messages can contain diverse content, such as promotions, awards, phishing links, or malware, including viruses and Trojans, making them potentially harmful (Mohammadi & Hamidi, 2018; Martin et al., 2005; Rajput, Athavale & Mittal, 2019). For example, a spam SMS might include a phishing link that redirects users to a fraudulent website designed to steal their credentials and, consequently, their financial assets. Spammers often collect users’ contact information from websites, chat rooms, and similar platforms to sell to other spammers, who then use it to send additional spam SMS messages.

Spam filtering systems can be developed using data-driven approaches such as machine learning algorithms (Tekerek, 2017; Sheikhi, Kheirabadi & Bazzazi, 2020; Nyamathulla et al., 2022; Sharma & Arjun, 2023; Moutafis, Andreatos & Stefaneas, 2023; Fatima et al., 2023; Shrivas, Dewangan & Ghosh, 2021; Reddy et al., 2023; Madhavan et al., 2021). Data-driven solutions have demonstrated not only high detection rates but also cost-effectiveness and greater objectivity compared to traditional methods. For instance, Reddy et al. (2023) proposed a natural language processing (NLP) model combined with machine learning algorithms to automatically classify incoming emails as either spam or legitimate.

In Fatima et al. (2023), a machine learning algorithm was proposed to detect spam emails. This algorithm optimizes hyperparameter settings using various parameter optimization methods, including manual search, random search, grid search, and genetic algorithm (GA). Additionally, it incorporates two feature extraction modules: Term frequency-inverse document frequency (TF-IDF) vectorizer and CountVectorizer. The algorithm has been applied to three datasets, one of which is the UCI SMS Spam dataset. The algorithm follows these preprocessing steps: converting all text to lowercase or uppercase, removing hyperlinks, whitespaces, numbers, and punctuation, deleting stopwords, and applying tokenization, stemming, and lemmatization. To detect spam SMS messages, the optimized algorithm employs various machine learning methods. The classification results, such as accuracy, could be further improved by developing a model using machine learning algorithms like random forest (RF) (Breiman, 2001; Cutler, Cutler & Stevens, 2011), multi-layer perceptron (MLP) (Rosenblatt, 1958; Singh & Banerjee, 2019), and support vector machine (SVM) (Cortes & Vapnik, 1995). In Reddy et al. (2023), a project focused on preprocessing using machine learning algorithms and NLP was introduced. This project aimed to provide a system for filtering spam emails. The dataset used consisted of emails classified as either spam or ham. For NLP, preprocessing steps included tokenization, stopword removal, stemming, and feature extraction. The machine learning algorithms employed in this project included naïve Bayes (NB) (Zhang, 2004), SVM, and K-nearest neighbors (KNN) (Cover & Hart, 1967). The accuracy of SVM in this context could be improved by developing a model that combines SVM and NLP techniques, incorporating methods such as punctuation removal, stemming, lemmatization, and advanced techniques like stratified shuffle split cross-validation.

Several studies have proposed models based on machine learning algorithms to detect spam SMS messages, such as the works in Sani, Abdulrahman & Adamu (2025), de Luna et al. (2023), Hossain et al. (2021), Sjarif et al. (2019), GuangJun et al. (2020), Asirvatham & Meenakshi (2025).

Decision tree (DT) (Breiman et al., 1984; Quinlan, 1993; Hastie, Tibshirani & Friedman, 2009), J48 (Frank, Hall & Witten, 2016), and NB algorithms have been compared to detect ham or spam short messages (Sani, Abdulrahman & Adamu, 2025). Two datasets from the UCI repository and ExAIS_SMS were used; the models using k-fold cross-validation were evaluated. Accuracy and area under the receiver operating characteristic curve (AUC-ROC) metrics are used to evaluate the performance of the models. The results showed that the DT achieved the highest accuracy based on the two datasets. NB has accomplished greater ROC-AUC results. The performance of NB and J48 depends on the dataset; both NB and J48 achieve higher accuracy and ROC-AUC results using the UCI dataset. However, they have lower accuracy and ROC-AUC results using the ExAIS_SMS dataset. Moreover, the proposed model in Sani, Abdulrahman & Adamu (2025) and the baseline model have been compared using the accuracy. The results demonstrated that the accuracy of the compared algorithms is higher than that of the baseline model. Therefore, the accuracy results of the compared algorithms are improved. The Nemenyi test can be used to compare different models or classifiers (Hollander, Wolfe & Chicken, 2015). The Nemenyi test P-values are used to show the statistical differences in the rankings of the model (Sani, Abdulrahman & Adamu, 2025). The DT and NB showed no notable differences between them. However, DT and J48 showed observable differences between them.

The SMS spam dataset used in de Luna et al. (2023) was resulted by created three datasets, two of which are from Kaggle, and the researchers obtained the third one. Eleven machine learning algorithms have been compared using accuracy to discover which algorithm provides the highest accuracy in detecting spam or ham SMSs (de Luna et al., 2023). The algorithms used in the comparison are logistic regression (Bisong, 2019), KNN, DT, SVM, RF, Extra Trees (Geurts, Ernst & Wehenkel, 2006), gradient boosting, AdaBoost, XGBoost, Bernoulli naïve Bayes, and Bagging. The results showed that logistic regression provides the highest accuracy result, while KNN provides the smallest accuracy result. The remaining algorithms yield adequate accuracy results. The second, third, and fourth highest accuracy results are achieved by Bernoulli naïve Bayes, RF, and Extra Trees, respectively. The highest four algorithms regarding to accuracy were chosen for optimization. The aim of optimizing these algorithms is to note the influence of optimized parameters on their performance. These optimized algorithms are compared using accuracy, precision, recall, and F1-score to determine which optimized algorithm provides the highest performance results. The results showed that Bernoulli naïve Bayes provides the highest accuracy, recall, and F1-score. RF achieves the second highest accuracy, recall, and F1-score. Logistic regression produces an accuracy higher than Extra Trees. Extra Trees provides higher recall and F1-score than those of logistic regression. In terms of precision, logistic regression provides the highest precision. RF presents the second highest precision. Extra Trees outperforms Bernoulli naïve Bayes in terms of precision. Additionally, the optimized algorithms enhance their accuracy results.

Bernoulli naïve Bayes, Gaussian naïve Bayes, and Multinomial naïve Bayes are compared using TF-IDF and CountVectorizer to determine which algorithm provides the highest accuracy and precision results (de Luna et al., 2023). The results showed that Bernoulli naïve Bayes accomplishes the highest accuracy and precision results using TF-IDF and CountVectorizer. Additionally, the accuracy and precision results of Bernoulli naïve Bayes using TF-IDF are the same as those for Bernoulli naïve Bayes using CountVectorizer. Multinomial naïve Bayes outperforms Gaussian naïve Bayes in terms of accuracy and precision using TF-IDF and CountVectorizer.

The SMS dataset used in Hossain et al. (2021) was taken from Kaggle. SVM and multinomial naïve Bayes algorithms are compared based on three feature extraction processes to determine which algorithm provides the highest accuracy and F1-score results, as well as the lowest computational time in seconds. The three feature extraction processes are pre-processing and TF-IDF, pre-processing and TF-IDF with stemming, and pre-processing, stemming, TF-IDF, and length of the messages. This comparison comprises six experiments, which show the results of accuracy, F1-score, and the computational time based on the three feature extraction processes for the compared algorithms.

The SVM displays the same accuracy and F1-score when the feature extraction process is either pre-processing and TF-IDF or pre-processing and TF-IDF with stemming. However, the computational time for the SVM using pre-processing and TF-IDF with stemming is lower than that of the SVM using pre-processing and TF-IDF. Moreover, SVM produces the smallest accuracy and F1-score results, and the highest computational time when using pre-processing, stemming, TF-IDF, and the Length of the messages.

Multinomial naïve Bayes achieves the highest accuracy using pre-processing and TF-IDF with stemming. Additionally, multinomial naïve Bayes using pre-processing and TF-IDF achieves higher accuracy than multinomial naïve Bayes using pre-processing, stemming, TF-IDF, and Length of the messages. Multinomial naïve Bayes using pre-processing and TF-IDF has the lowest computational time. Moreover, multinomial naïve Bayes using pre-processing and TF-IDF with stemming has lower computational time than multinomial naïve Bayes using pre-processing, stemming, TF-IDF, and length of the messages. Multinomial naïve Bayes accomplishes the same F1-score using any given feature extraction process.

Multinomial naïve Bayes outperforms the SVM in terms of accuracy, regardless of the feature extraction process used. In addition, the computational times of multinomial naïve Bayes are less than those of SVM using any given feature extraction method. Both Multinomial naïve Bayes and SVM achieve the same F1-score when either pre-processing and TF-IDF or pre-processing and TF-IDF with stemming is used. Multinomial naïve Bayes achieves a higher F1-score than the SVM when pre-processing, stemming, TF-IDF, and length of the messages are applied.

Multinomial naïve Bayes outperforms SVM in terms of the micro average area under the ROC curve (Hossain et al., 2021). As a result, Multinomial naïve Bayes can detect spam SMSs based on stemming and TF-IDF better than SVM.

In Hossain et al. (2021), another comparison has been conducted among Bernoulli naïve Bayes and the works in Sjarif et al. (2019) and GuangJun et al. (2020). The work in Sjarif et al. (2019) employed the RF algorithm, while the work in GuangJun et al. (2020) utilized the logistic regression algorithm. The compared algorithms use TF-IDF feature extraction. This comparison is based on accuracy and F1-score. The results showed that Bernoulli naïve Bayes outperforms both RF and logistic regression in terms of accuracy and F1-score. Additionally, logistic regression provides higher accuracy and F1-score results than RF.

Different machine learning algorithms have been compared based on accuracy, precision, recall, and F1-score to determine which algorithm offers higher performance in detecting spam or ham SMSs (Asirvatham & Meenakshi, 2025). The compared algorithms are linear regression, logistic regression, KNN, NB, and SpamSMS. The results demonstrated that SpamSMS provides the highest accuracy. Both SpamSMS and logistic regression offer the same and the highest precision result. SpamSMS and NB produce the same and the highest recall result. NB provides the highest F1-score.

Motivations

The primary motivations for this research are as follows:

• The potential theft of personal credentials through spam SMS messages.

• The consumption of mobile device resources caused by spam SMS messages.

• The disruption caused by the receipt of unwanted SMS messages, particularly spam.

Problem statement

SMS messages are a popular way of communicating between people. Spam SMSs are undesired messages because they may contain advertisements, malicious links, and other content that can harm people. Spam SMSs can also affect the security and privacy of people. Cybercriminals can use malicious links in spam SMS messages to commit cybercrimes, such as downloading malware, stealing user information, or redirecting users to phishing websites. Cybercriminals use spam SMSs to commit deceptive operations, such as stealing users’ financial assets. The device’s resources can be largely consumed when many spam SMSs are received, such as the device’s storage. As a result, the performance of networks is influenced. The recipient of SMSs spends more time filtering SMSs to specify whether an SMS is spam or ham.

The problems of this research are as follows:

• Cybercriminals can send spam SMSs that convey undesirable content to recipients, such as advertisements.

• Spam SMSs can be used to commit fraud, such as illegally obtaining users’ credentials and money.

• Spam SMS can contain malware that can be used to attack users’ devices and data.

• Using memory space in storing spam SMSs.

• Spending more time dealing with spam SMSs by users.

Research questions

The research questions addressed in this work are as follows:

• How can spam and ham SMS messages be detected?

• How can users’ information be secured?

• How can mobile device resources be preserved?

• Which machine learning algorithm among those compared is most effective in detecting spam SMS messages?

• Which algorithm offers the best classification results among the compared models?

Aims

Detecting spam SMS messages is crucial for achieving several aims, including but not limited to:

• Securing user information

• Protecting mobile devices

• Saving user time and resources

• Preserving device resources

These aims can be accomplished through the use of spam filter systems.

Main contributions

The main contributions of this research are as follows:

• The novel contribution is developing a data-driven process, which contributes to proposing a model based on machine learning algorithms and NLP.

• Proposing a data-driven model based on NLP and machine learning algorithms to detect spam and ham SMS messages. This model classifies SMS messages as either spam or ham, functioning as a spam filter system. It can block spam SMS messages and allow legitimate ham SMS messages to reach their recipients, contributing to the protection of users’ information, finances, and devices.

• Evaluating the performance of the proposed model using several machine learning algorithms, balanced dataset, and different evaluation metrics, such as accuracy, precision, recall, and F1-score. This evaluation helps determine which machine learning model provides the best classification performance, making it suitable for use as a detection system for spam and ham SMS messages.

• Evaluating the performance of the proposed model using several machine learning algorithms, an imbalanced dataset, and the abovementioned evaluation metrics, the precision-recall curve, and the area under the precision-recall curve. This evaluation helps determine which machine learning model provides the best classification performance, making it suitable for use as a detection system for spam and ham SMS messages. Additionally, evaluating the performance of classification models of the compared algorithms based on the precision-recall curve and the area under the precision-recall curve.

• Comparing the results of the proposed model with those of previously published works to create a benchmark for future research in spam and ham SMS detection. This comparison also evaluates the performance of the proposed model against prior methods, helping to establish the superiority of the model based on classification metrics.

Organization of the article

The remainder of this article is organized as follows. ‘Related Works’ presents the related works, ‘The Proposed Model’ details the proposed model, ‘Discussions’ discusses the findings, and ‘Conclusions’ concludes the article.

Comparison of the proposed model and related works

Tables 1 and 2 can be used to compare the proposed model with the related works based on the datasets used, dataset sizes, data preparation methods, and evaluation methods.

An overview of the similarities and differences between the proposed model and the related works is presented below:

The model in Nyamathulla et al. (2022) employed methods not used in the proposed model, including NB, maximum entropy, logistic regression, and LSTM. Conversely, the proposed model uses GB and MLP, which are not part of the model in Nyamathulla et al. (2022).

The models in Nyamathulla et al. (2022), Moutafis, Andreatos & Stefaneas (2023) used 75% of the dataset for training and 25% for testing, while the proposed model uses stratified shuffle split cross-validation.

The technique in Sheikhi, Kheirabadi & Bazzazi (2020) incorporates the methods used in the proposed model, along with additional techniques such as RT, NB, J48, decision stump, Hoeffding tree, SVM, and H2O + RF. The technique in Shrivas, Dewangan & Ghosh (2021) uses NB, RF, J48, RT, and AdaBoosting. The technique in Sharma & Arjun (2023) includes NB-based multinomial and Bernoulli, KNN, DTs, logistic regression, RF, bagging, extra trees, and XGBoost. The proposed model in this study uses KNN, DT, RF, GB, MLP, and SVM.

DT, RF, GB, and MLP are used in the proposed model but are not part of the techniques in Reddy et al. (2023), Madhavan et al. (2021). In addition, the technique in Reddy et al. (2023) uses only accuracy as the performance measure, while the technique in Madhavan et al. (2021) uses accuracy, spam precision, and spam recall. The technique in Sharma & Arjun (2023) uses accuracy, confusion matrix, and precision, while the proposed model uses accuracy, precision, recall, and F1-score. Stemming and lemmatization are employed in the proposed model, while only stemming is used in the techniques in Shrivas, Dewangan & Ghosh (2021), Pawlak (1982)

The differences between the optimized algorithm (Fatima et al., 2023) and the proposed algorithm are as follows:

The dataset of the optimized algorithm is split into 80% for the training set and 20% for the testing set. In contrast, the proposed algorithm splits the dataset using stratified shuffle split cross-validation, which divides the dataset into ten randomized splits, maintaining the percentage of samples for each class target as in the entire dataset. Additionally, the optimized algorithm uses both TF-IDF vectorizer and CountVectorizer modules for feature extraction, while the proposed algorithm uses only the CountVectorizer module. The optimized algorithm employs parameter optimization techniques to set the parameters to their optimal values, whereas these approaches are not applied in the proposed algorithm. The optimized algorithm converts letters to both upper and lower case, whereas the proposed algorithm converts all letters to lower case. The optimized algorithm removes numbers, white spaces, and hyperlinks, while the proposed algorithm does not remove these elements. Moreover, the optimized algorithm does not address the imbalanced dataset issue, while the proposed algorithm handles it using the random oversampling method.

The differences between the technique in Tekerek (2017) and the proposed algorithm are as follows: the technique uses 4,827 ham SMS messages, while the proposed algorithm uses 4,825 SMS messages. The technique uses 10-fold cross-validation, while the proposed algorithm uses stratified shuffle split cross-validation, which provides randomized splits, ensuring that the percentage of samples for each class label remains the same as in the entire dataset. The technique does not include several NLP steps, such as removing punctuation, converting SMS messages to lowercase letters, deleting stopwords, or using stemming and/or lemmatization. In contrast, the proposed algorithm applies all of these NLP steps. In the technique, the features are automatically generated from the dataset, whereas the proposed algorithm uses CountVectorizer for feature extraction. Additionally, the technique does not address the issue of an imbalanced dataset, as there are more ham SMS messages than spam SMS messages. On the other hand, the proposed algorithm tackles this issue by balancing the dataset.

The dataset

The SMS Spam Collection dataset, available on Kaggle (2023) and also referenced in Dua & Graff (2017), is used in this study. The dataset contains two columns: the first column represents the labels, and the second contains the SMS messages. The labels are divided into two classes: spam and ham. “Spam” refers to unwanted or unsolicited electronic messages, while "ham" refers to non-spam messages. This dataset consists of 5,572 SMS messages, with 747 messages classified as spam and 4,825 messages classified as ham. This creates an imbalanced dataset, with a significant disparity between the number of spam and ham messages. The minority class is spam, and the majority class is ham. The dataset was selected based on the following criteria: it is relevant to the problem of detecting spam SMS messages, as it includes both spam and ham SMSs. It has a sufficient size, offering a variety of both types of messages. Additionally, the dataset is sourced from Kaggle, a reliable platform, ensuring its credibility for research purposes.

Preprocessing methods of the data

This subsection outlines the data preparation and cleaning steps applied in this study. NLP methods are utilized to prepare and clean the data. NLP, a subfield of artificial intelligence (AI), enables computers to understand human languages (Khurana et al., 2023). One of the key applications of NLP is text classification. In this article, the following NLP methods are applied to preprocess the data:

• Punctuation marks are removed from the SMS messages.

• All SMS messages are converted to lowercase letters.

• Tokenization is performed, where the SMS messages are split into tokens for easier processing.

• Common English stopwords, which are frequently occurring words with little meaningful value, are removed.

• Stemming is applied to reduce words to their root forms by removing suffixes. However, stemming may result in incorrect spellings and less meaningful words. The stemming algorithm used is Porter Stemming algorithm (Porter, 1980), which proposed by Porter in 1980.

• Lemmatization is used to reduce words to their base forms, known as lemmas. Lemmatization maintains correct spelling and meaning and generally provides more accurate results than stemming. However, it is slower than stemming because it involves using a dictionary to determine the base form of a word.

• The feature extraction method employed is CountVectorizer, which is available in Scikit-learn (Python). This method converts a set of text documents into a matrix where each column represents a unique token, and each row represents an SMS message. The values in the matrix correspond to the count of each token in the respective SMS message.

Stratified shuffle split cross-validation

Stratified shuffle split is a cross-validation method used to obtain the indices for train/test splits, with the goal of dividing the data into training and testing sets. The result of this method is a series of stratified randomized splits, meaning that each split maintains the same percentage of each class label as in the entire dataset. In other words, the splits preserve the same proportion of samples for every class label.

Random oversampling

As previously mentioned, the dataset is imbalanced, with the number of samples in the spam class (the minority class) being smaller than the number of samples in the ham class (the majority class). To address this imbalance, the random oversampling method is applied. This method increases the number of samples in the minority class by randomly duplicating existing samples. In this article, random oversampling ensures that the number of samples in the minority class matches that of the majority class.

Training and testing the models

Once the dataset has been preprocessed and is ready for training, machine learning models are trained and tested using various algorithms, including KNN, DT, RF, GB, MLP, and SVM. The classification results for each algorithm are evaluated using the following metrics: accuracy, precision, recall, and F1-score.

Hypotheses of the models

This subsection presents hypotheses that are related to the proposed model based on machine learning algorithms and NLP; these hypotheses are:

• All steps of the proposed data-driven process model, based on NLP and machine learning algorithms, are applied to offer satisfactory accuracy results.

• All steps of the proposed data-driven process model, based on NLP and machine learning algorithms, are applied to offer satisfactory precision results.

• All steps of the proposed data-driven process model, based on NLP and machine learning algorithms, are applied to offer satisfactory recall results.

• All steps of the proposed data-driven process model, based on NLP and machine learning algorithms, are applied to offer satisfactory F1-score results.

• All steps of the proposed data-driven process model, based on NLP and machine learning algorithms, are applied to offer satisfactory performance of classification models based on the precision-recall curves.

• All steps of the proposed data-driven process model, based on NLP and machine learning algorithms, are applied to offer satisfactory classification models’ performance based on the areas under the precision-recall curves.

• Certain studies could benefit from improvements in classification metrics, such as recall and F1-score.

Interpretation of the results

The results of the proposed model, based on NLP and various machine learning algorithms using the balanced or the imbalanced dataset, are presented in this section. The algorithms compared include KNN, DT, RF, GB, MLP, and SVM. The comparison is made using the following classification metrics: Accuracy, precision, recall, F1-score, precision-recall curve, and the area under the precision-recall curve. The goal of this comparison is to identify which algorithm yields the best classification results. All algorithms used stratified shuffle split cross-validation, where the dataset is split into ten train/test splits. In each split, the percentage of samples for each class label is maintained to be the same as in the entire dataset. The software used is PyCharm, the versions are 2022.3.3 and 2023.3.4. The library used is Scikit-learn (Pedregosa et al., 2011).

This subsection is organized as follows: ‘Parameter Settings’ provides the parameter settings of the compared algorithms. ‘Classification Measures’ presents the classification measures. The classification results for both stemming and lemmatization based on balanced dataset and imbalanced dataset are discussed in ‘Results Relying on Stemming and Lemmatization based on a Balanced Dataset’ and ‘Results Relying on Stemming and Lemmatization based on an Imbalanced Dataset’, respectively. The area under precision-recall curve results are presented in ‘The Area Under Precision-Recall Curve Results Relying on Stemming and Lemmatization based on an Imbalanced Dataset’. ‘Comparison between the Proposed Model and Other Models’ offers a comparison between the proposed model and other machine learning models.

The area under precision-recall curve results relying on stemming and lemmatization based on an imbalanced dataset

This sub-subsection presents the precision-recall curves and the areas under precision-recall curves for KNN, DT, RF, GB, MLP, and SVM algorithms. The imbalanced dataset is applied. A comparison is made among these algorithms based on the precision-recall curves and the area under precision-recall curves. Stemming and lemmatization are used. For stemming, the precision-recall curves for KNN, DT, RF, GB, MLP, and SVM are shown in Figs. 2, 3, 4, 5, 6, and 7, respectively. Precision-recall curves for KNN, DT, RF, GB, MLP, and SVM using lemmatization are shown in Figs. 8, 9, 10, 11, 12, and 13, respectively. Table 11 shows the area under precision-recall curves for the compared algorithms.

Table 11:

The areas under the precision-recall curves of the compared algorithms for stemming and lemmatization using the imbalanced dataset.

Algorithm	The area under precision-recall curve based on stemming	The area under precision-recall curve based on lemmatization
KNN	0.973065622	0.97260274
Decision Tree	0.984264471	0.986894638
Random Forest	0.999135405	0.999278528
Gradient Boosting	0.996588804	0.99682781
Multi-Layer Perceptron	0.996211744	0.995947991
SVM	0.997582607	0.997641867

DOI: 10.7717/peerj-cs.3232/table-11

Figures 2–13 illustrate the precision vs. recall results based on different classification thresholds. Precision-recall curves are employed to evaluate the performance of classification models (Precision-Recall Curve, 2024). This sub-subsection uses precision-recall curves to evaluate the performance of the compared algorithms when an imbalanced dataset is used. Figures 2–13 show high areas under the precision-recall curves. Therefore, both precision and recall results are high, which leads to the classification algorithm models performing well.

Table 11 shows the area under precision-recall curves for the compared algorithms using stemming and lemmatization. This table reveals that the results of the areas under precision-recall curves for the compared algorithms using stemming and lemmatization are high. Therefore, the performance of these algorithms is acceptable. In this table, the highest results are for RF, which means the model performance of RF is the highest, and RF has the highest ability to balance between precision and recall. SVM has the second highest model performance because SVM has the second highest results for the area under the precision-recall curve. SVM has the second highest ability to offer balancing between precision and recall. GB has the third highest model performance because GB has the third highest results for the area under the precision-recall curve. GB has the third highest ability to present balancing between precision and recall. MLP has the fourth highest model performance because MLP has the fourth highest results for the area under the precision-recall curve. MLP has the fourth highest ability to present balancing between precision and recall. The model performance of DT is better than the model performance of KNN because the results of the area under the precision-recall curve for DT are better than those of KNN. DT has a better ability to balance precision and recall than KNN.

Implications

This subsection is divided into two parts: ‘Theoretical Implications’ presents the theoretical implications, and ‘Practical Implications’ discusses the practical implications.

Limitations

The proposed model, based on NLP and machine learning algorithms, has the following limitations:

• Some legitimate (ham) SMS messages are misclassified as spam by the machine learning models. This results in recipients missing important messages, which could contain critical information.

• The DT and GB models misclassified some spam SMS messages as legitimate. This allows spam SMS messages to reach recipients, potentially containing phishing or malware links. This poses a risk to recipients by exposing them to data theft, financial loss, or malware that could compromise their devices and data.

Future work

The future directions for this research study are as follows:

• Applying the proposed machine learning model to datasets of varying sizes.

• Evaluating the performance of the machine learning models using TF-IDF vectorizer as the feature extraction method.

• Developing models based on other machine learning and deep learning algorithms (LeCun, Bengio & Hinton, 2015) to further enhance the classification performance.