Guiding the retraining of convolutional neural networks against adversarial inputs

Introduction

In recent years, deep learning (DL) systems, defined as those software systems with functionalities enabled by at least one DL model, have become widespread due to their outstanding capacity to solve complex problems. The use of DL systems ranges from applications in autonomous driving systems to applications in medical treatments, among others (Stallkamp et al., 2012; Castanyer, Martínez-Fernández & Franch, 2021a; Tran et al., 2021). However, given their statistical and black box nature, it is often the case that DL systems exhibit an unexpected behavior or produce anomalous results when inferring with inputs different to those used in the training stage. We refer as vulnerabilities to such unexpected behaviors or anomalous results, because they can lead to undesirable consequences in real-world DL systems already in production.

The existence of vulnerabilities is commonplace in any type of software system. In traditional systems, i.e., not embedding any DL model, we may find a vast amount of testing approaches to ensure a certain level of robustness (Könighofer & Bloem, 2012). When the tests detect a problem, there are specific solutions to fix the problem or at least mitigate the effects to a large extent (Khan, 2011). In contrast, testing DL systems is completely different, due to their non-deterministic nature: the “correct” answer or “expected” behavior in response to a certain input is often unknown at the time of training the DL model (Barr et al., 2014; Amershi et al., 2019).

One of the most critical vulnerabilities in DL systems is their exposure to adversarial input attacks. Adversarial inputs are inputs that can be misclassified by applying them a perturbation not distinguishable by human eyes. The perturbation is learned by an attacker, which optimizes the inputs to maximize the DL model’s prediction error (Szegedy et al., 2013). Therefore, comprehensive DL system testing approaches need to include techniques to manage adversarial inputs (Machado, Silva & Goldschmidt, 2021). One way to counteract this vulnerability is by retraining the DL models, with the idea that new inputs can be used for training a better model than the original. In this way, selecting the most informative inputs becomes key when retraining.

This work focuses on improving the testing and retraining of DL models in the presence of adversarial inputs. To this end, we apply two different approaches: (i) using the information provided by a number of guidance metrics, which try to identify the most useful inputs according to the criteria of each metric, and (ii) adding adversarial inputs to augment the dataset employing different training inputs from the original training set. Still, questions arise: What is the accuracy, resource utilization, and execution time of using such metrics? What is the accuracy and resource utilization of using a certain retraining configuration? (the execution time of retraining configuration is not included in this question as the number of used inputs determines the computation expense of the retraining). Thus, it is needed to determine specific scenarios in which a particular guidance metric should be employed, as well as optimal retraining configurations. In this context, this work aims at comparing both guidance metrics and configurations for retraining DL models with adversarial inputs based on their accuracy, to improve the DL models’ performance (Vogelsang & Borg, 2019), resource utilization and execution time, both to obtain computational benefits. The results of the research benefit data scientists to test their models’ accuracy against adversarial inputs and to adapt their models to be aware of adversarial inputs, while keeping the trade-off with the resource utilization of the retraining process. We evaluate our method in one typical DL application field, namely image classification. For this reason, we focus on one particular type of DL model, namely convolutional neural networks (CNN), which are widely used for image classification due to their high performance. Since LeCun et al. (1989) introduced them, there has been significant progress in the literature with respect to their accuracy even in challenging datasets, as well as the explanation of their behavior and abstraction (Krizhevsky, Sutskever & Hinton, 2012; Zeiler & Fergus, 2014).

The main contributions of this work are:

• We conduct a study based on five datasets, in which we evaluate and compare various retraining runs guided by six guidance metrics. We show that using uncertainty metrics for guiding the retraining of CNN models outperforms neuron coverage, surprise-based and random metrics w.r.t. accuracy (i.e., using an augmented test set with adversarial inputs from three different adversarial attacks), resource utilization and execution time of the metrics.

• In the same study, we conduct a comparison, in terms of accuracy (i.e., using also an augmented test set with adversarial inputs from three different adversarial attacks) and resource utilization, of three configurations of retraining CNN models. In particular, when doing adversarial retraining from original model weights can achieve results that are aligned with green AI goals.

• We provide guidelines to help data scientists to improve their CNN models against adversarial attacks from the perspective of green AI. We recommend retraining CNN models using an augmented training set with adversarial inputs, from original model weights, and the guidance of uncertainty metrics to improve efficiency without diminishing the models’ accuracy.

• We provide a replication package, available online (https://doi.org/10.5281/zenodo.7933484).

This document is structured as follows. Section 2 respectively introduces the field of study of our work, the DL architecture used for the models, the guidance metrics, and the adversarial attacks used to create the adversarial inputs. Section 3 describes related work. Section 4 presents the research questions and the study design. Section 5 presents the results. Section 6 presents the discussions. Section 7 describes the threats to validity, and Section 8 draws conclusions and future work.

Background

In this section, we introduce (i) the field of study of our work, namely deep learning (DL) system testing; (ii) the type of DL model we used, namely convolutional neural networks (CNN); (iii) the guidance metrics used in our study, namely: neuron coverage (NC), two surprise adequacy (SA) metrics: likelihood-based surprise adequacy (LSA) and distance-based surprise adequacy (DSA) and two uncertainty metrics: DeepGini (DG) and softmax entropy (SE); (iv) the adversarial attacks used to create adversarial inputs, namely Fast Gradient Sign Method (FGSM), Basic Iterative Method (BIM) and Carlini-Wagner (CW).

Related work

From the reviewed literature, there are many studies focusing on the creation of adversarial inputs in order to uncover debilities and vulnerabilities of DL systems or models (Goodfellow, Shlens & Szegedy, 2014; Kurakin, Goodfellow & Bengio, 2018; Carlini & Wagner, 2017). Goodfellow, Shlens & Szegedy (2014) were the first to formulate a concrete definition of the adversarial inputs and clarify their generalization in different architectures and training sets. Additionally, they defined FGSM which allowed generating adversarial inputs. Since then, several adversarial attacks have been proposed (Carlini & Wagner, 2017; Kurakin, Goodfellow & Bengio, 2018; Ren et al., 2020).

Other studies centered on being able to detect such adversarial inputs for different purposes. Feinman et al. (2017) aimed at distinguishing adversarial samples from their normal and noisy counterparts. Ma et al. (2018a) used their coverage criteria to quantify defect detection ability using adversarial inputs created with different adversarial techniques. Kim, Feldt & Yoo (2019) used their proposed SA criteria to show that they could detect adversarial inputs. Wang et al. (2021) considered augmenting the test set with adversarial inputs, to investigate the performance of their test input prioritization approach, PRIMA (PRioritizing test inputs via Intelligent Mutation Analysis), on this type of input.

Although there are several techniques to face the problem of adversarial examples in ML literature (Croce et al., 2020), there is a research challenge focused on using adversarial inputs together with guidance metrics in retraining with the purpose of improving the models (Pei et al., 2017; Ma et al., 2018b; Kim, Feldt & Yoo, 2019), see Table 1 for a summary. In these few works, we can identify in the majority of them, a configuration similar to the adversarial fine-tuning presented in Section 4 of this work, in which the authors retrain with only a few adversarial inputs and do not clearly show the steps for the retraining. Kim, Feldt & Yoo (2019) showed that sampling inputs using SA for retraining can result in higher accuracy, Pei et al. (2017) showed that error-inducing inputs generated by DeepXplore can be used for retraining to improve accuracy. Feng et al. (2020) demonstrated that retraining a model with the prioritization of DG is more effective in improving the DL model accuracy than coverage-based metrics.

Most recently, Ma et al. (2021) proposed to use testing metrics as a retraining guide, looking for an answer on how to select additional training inputs to improve the accuracy of the model. They used the original training set starting from a previous trained model but ordering these inputs following the metrics’ guidance. Similarly, Weiss & Tonella (2022) used these metrics (Test Input Prioritizers, in their study) to identify the inputs which expose model faults to integrate them to the training set of future retraining runs. They replicated the findings of Feng et al. (2020), however, they found that DG does not in general perform better than other uncertainty metrics and they did not utilize adversarial inputs, but out-of-distribution inputs & inputs from a test set. Compared to these works, we aim to conduct the retraining using the information of an augmented dataset with adversarial inputs and inputs from the original training set.

As the use of adversarial inputs has been proved to provide many improvements, we identified a research gap there to take advantage of them during a retraining process. Moreover, we want to understand how to order an augmented dataset with adversarial inputs guided by metrics so that the retraining is efficient (with highest accuracy and using fewest inputs) for data scientists. To the best of our knowledge, our study is the first one that applies metrics to guide a retraining using an augmented dataset with adversarial inputs in order to improve the model accuracy against adversarial inputs and keeping the trade-off with resource utilization, in order to obtain computational benefits, which is key as the retraining phase is time consuming and the creation of adversarial images has a high computational cost (Shafahi et al., 2019).

In our study, we consider an augmented dataset with adversarial inputs obtained from the original training set using the FGSM, BIM and CW attacks. Therefore our study provides the following novel contributions:

• A comparison between state-of-the-art guidance metrics for a guided retraining.

• A comparison of three different configurations for doing a retraining against adversarial inputs.

• Guidelines to help data scientists to improve their CNN models against adversarial attacks from the perspective of green AI.

Empirical study

Study design

Figure 1 shows the study design. First, the data collection and preprocessing which consists of the collection and preprocessing of raw data. Second, the model training which consists of the traditional training and proposed retraining phase. Finally, we provide answers to the RQs by analyzing our results w.r.t. DL testing properties.

Data collection and preprocessing: datasets.

We evaluate the guidance metrics on CNNs using five multi-class, single-image classification datasets.

First, the German Traffic Sign Recognition Benchmark (GTSRB) dataset, with 43 classes containing 39,208 unique images of real traffic signs in Germany. It has been widely used in DL research (Castanyer, Martínez-Fernández & Franch, 2021a; Stallkamp et al., 2012; Loukmane, Graña & Mestari, 2020). This type of images are characterized by large changes of visual appearance for different causes (e.g., weather conditions). While humans can easily classify them, for DL systems (e.g., autonomous driving systems) still is a challenge.

Second, the Intel Image Classification (Intel) dataset with six classes containing 17,034 labeled images of natural scenes around the world, used in many studies as well (Wu et al., 2020; Rahimzadeh et al., 2021; Ren & Li, 2022). It was provided by Intel corporation to create another benchmark in image classification tasks such as scene recognition (Bansal, 2019). This scene recognition task is a daily task for humans and widely used application in industries such as tourism or robotics, still an emerging field for computer vision (Matei, Glavan & Talavera, 2020; Sobti et al., 2021).

Third, the Canadian Institute for Advanced Research (CIFAR-10) dataset, with 10 classes containing 60,000 labeled images of animals and transports, such as dogs, cats, airplanes and ships (Krizhevsky, Nair & Hinton, 2009). This dataset is commonly used to train machine learning and DL models. Compared to MNIST, it is more difficult to train a model due its size and complexity.

Fourth, the Modified National Institute of Standards and Technology (MNIST) dataset, with 10 classes containing 70,000 labeled images of handwritten digits from 0 to 9 which are size-normalized and centered (LeCun & Cortes, 2010). This dataset is also commonly in the training of machine learning and DL models, known for its simplicity in terms of computation.

Fifth, the Fashion-MNIST dataset (Fashion), which considers 10 classes and also contains 70,000 labeled images of clothing, such as T-shirts, pullovers and sneakers (Xiao, Rasul & Vollgraf, 2017). This dataset was provided to be a more challenging alternative to the well-known MNIST dataset.

The adversarial inputs are obtained using the FGSM, BIM and CW attacks from the foolbox library (Rauber, Brendel & Bethge, 2017). Table 3 describes the models we use in our study and the total number of inputs of the augmented test set with adversarial inputs generated by the adversarial attacks.

Table 3:

CNN models.

Dataset	# Layers of the CNN (using convolution and dense layers with max-pooling and dropout)	# Neurons	# Original test set inputs	# Augmented test set inputs
GTSRB	9	1,259	3,923	7,846
Intel	7	970	3,000	6,000
CIFAR-10	8	634	6,001	12,002
MNIST	4	234	7,001	14,002
Fashion	5	330	7,001	14,002

DOI: 10.7717/peerj-cs.1454/table-3

Model training

The traditional training of a model consists of using an available original training set identified as “Train” in Fig. 1 and then evaluating the model against an original test set identified by “Test” in Fig. 1. The traditional training is represented in the upper part of Fig. 1 (ii) Model training phase. We plan to add in this phase a retraining process as it is represented in the lower part of Fig. 1 (ii) Model training phase. This retraining uses adversarial inputs guided by the metrics and three different configurations, which are inspired by ML literature (Szegedy et al., 2013; Chen, Wang & Chen, 2020; Jeddi, Shafiee & Wong, 2020), to improve the original model, M, against adversarial inputs. After obtaining a retrained model, $M^{\ast }$, we evaluate the DL testing properties of this retrained model. In our study, we consider and refer to the following weights:

• Weights from scratch: Random initialization of the model’s weights.

• Original model weights (M): Obtained after traditional training, using only inputs from the original training set “Train”.

• Retrained model weights ( $M^{\ast }$): Obtained after the retraining process (according to the retraining configuration). These weights are from the 20 data points by selecting the model that achieves the highest accuracy.

The retraining process is shown more extensively in Fig. 2. It comprises the following steps:

• 1. Step 1. Create adversarial inputs for training and testing: First, we obtain the adversarial training set, “Adv. Train”, by applying the adversarial attacks to a subset of the original training set, “Train”. The number of the adversarial inputs for this case is a small proportion of the entire “Train” set, thus we do not increase the artificial inputs used in retraining and the original input distribution is not diminished either (represented in the upper part of Fig. 3). Putting these two sets together, we obtain the augmented training set, “Train*” (represented in the lower part of Fig. 3). In our experimental setup, each of the three adversarial attacks is applied to only one third of the involved set.

  Second, we obtain the adversarial test, “Adv. Test”, set by applying in the same way the adversarial attacks to the entire original test set “Test”. Putting these “Test” and “Adv. Test” sets together, we obtain the adversarial test set “Test*”. Figure 3 provides a comprehensive depiction of this step.

• 2. Step 2. Compute guidance metrics: Based on the original trained model, M, and the “Train” set, we compute the different metrics for the augmented training set, “Train*” (see Fig. 2 Step 2).

• 3. Step 3. Order inputs w.r.t. the guidance metrics: According to each metric, we order the inputs for the retraining. With this, we expect that the retrained model, $M^{\ast }$, will be trained first with the images that are more difficult to classify and more informative according to the metrics’ value (see Fig. 2 Step 3).

• 4. Step 4. Retraining according to the configuration: We implement the retraining in three different ways using ordered inputs according to the following configurations (see Fig. 2 Step 4 and Table 4):

Table 4:

Retraining configurations.

Retraining configuration	Initial weights	Final weights	Trained with (inputs are sorted by each guidance metric)	Tested with
OAR	From scratch	$M^{\ast }$	Train*	Test*
AR	M	$M^{\ast }$	Train*	Test*
AF	M	$M^{\ast }$	Adv. Train	Test*

DOI: 10.7717/peerj-cs.1454/table-4

• (a) One-step Adversarial Retraining (OAR): Starting from scratch using the new adversarial inputs and original training set. The model $M^{\ast }$ is retrained with the “Train*” set ordered by highest score of LSA, DSA, DG, SE, NC and Random, respectively, starting from scratch.

• (b) Adversarial retraining (AR): Starting from the original model M using the new adversarial inputs and original training set. The model $M^{\ast }$ is also retrained with the “Train*” set as in OAR with the difference that the model $M^{\ast }$ is retrained from the original model M weights.

• (c) Adversarial fine-tuning (AF): Starting from the original model M using only the new adversarial inputs. The model $M^{\ast }$ is retrained with the “Adv. Train” set, also ordered by highest score of LSA, DSA, DG, SE, NC and Random, respectively, starting from the original model M weights.

For each configuration, we execute a retraining of the models guided by the six metrics and for each metric we obtain 20 data points as shown in Figs. 4–8. Regarding the hyperparameters configuration, the same configuration is used for the original model and retrained model. Each retraining run for each data point is computed from their respective initial weights. Also, we address randomness of resulted values by executing the retraining randomly, and not through incremental training, which means each data point is obtained by starting the retraining from its initial model weights instead of incremental training (starting from its previous data point).

Data analysis

Figures 4–8 correspond to the accuracy of the models against the augmented test set, “Test*”. Table 5 shows the accuracy against the same “Test*” set and resource utilization of the experiments by dataset, configuration and metric used in each case. The “Original accuracy” column shows the accuracy of the original model M against the “Test*” set, which is low due to the adversarial inputs, the “Accuracy w.r.t. augmented test set” column shows the highest accuracy during its respective retraining and the “Resource utilization” column shows the input size to obtain that highest accuracy. Table 6 shows the execution time to compute the considered metrics for every input of the corresponding dataset.

Table 5:

Accuracy against augmented test set (original test set and adversarial test set) and resource utilization when the model reach the maximum accuracy for OAR, AR and AF applied to each dataset.

			Accuracy w.r.t. augmented test set						Resource utilization
Dataset	Config.1	Original accuracy	LSA	DSA	DG	SE	NC	Random	LSA	DSA	DG	SE	NC	Random
GTSRB	OAR	0.848	0.971	0.968	0.971	0.975	0.968	0.973	35,000/35,287	35,287/35,287	35,000/3,5287	15,750/35,287	35,287/35,287	35,287/35,287
Intel	OAR	0.687	0.736	0.753	0.756	0.750	0.748	0.749	12,600/14,224	14,224/14,224	12,600/14,224	11,900/14,224	12,600/14,224	13,300/1,4224
CIFAR-10	OAR	0.577	0.669	0.668	0.664	0.664	0.666	0.659	47,999/47,999	38,400/47,999	38,400/47,999	43,200/47,999	43,200/47,999	38,400/47,999
MNIST	OAR	0.977	0.987	0.989	0.992	0.990	0.986	0.988	53,200/55,998	25,200/55,998	11,200/55,998	11,200/55,998	55,998/55,998	53,200/55,998
Fashion	OAR	0.834	0.901	0.907	0.904	0.906	0.893	0.902	55,998/55,998	33,600/55,998	44,800/55,998	44,800/55,998	47,600/55,998	44,800/55,998
GTSRB	AR	0.848	0.945	0.957	0.958	0.961	0.943	0.940	17,500/35,287	3,500/35,287	8,750/35,287	12,250/35,287	35,287/35,287	29,750/35,287
Intel	AR	0.687	0.735	0.724	0.730	0.740	0.728	0.738	14,000/14,224	13,300/14,224	13,300/14,224	10,500/14,224	12,600/14,224	14,000/14,224
CIFAR-10	AR	0.577	0.653	0.660	0.652	0.643	0.636	0.641	47,999/47,999	14,400/47,999	33,600/47,999	26,400/47,999	45,600/47,999	28,800/47,999
MNIST	AR	0.977	0.987	0.988	0.992	0.991	0.986	0.987	55,998/55,998	22,400/55,998	14,000/55,998	14,000/55,998	55,998/55,998	55,998/55,998
Fashion	AR	0.834	0.901	0.909	0.906	0.909	0.894	0.892	55,998/55,998	22,400/55,998	28,000/55,998	42,000/55,998	55,998/55,998	36,400/55,998
GTSRB	AF	0.848	0.958	0.962	0.961	0.960	0.952	0.957	3,610/3,921	3,921/3,921	2,850/3,921	2,850/3,921	3,921/3,921	3,230/3,921
Intel	AF	0.687	0.721	0.701	0.724	0.720	0.719	0.718	2,850/3,000	2,100/3,000	2,250/3,000	3,000/3,000	2,850/3,000	2,400/3,000
CIFAR-10	AF	0.577	0.634	0.643	0.640	0.636	0.639	0.653	6,000/6,000	5,700/6,000	6,000/6,000	5,100/6,000	5,400/6,000	5,400/6,000
MNIST	AF	0.977	0.988	0.988	0.989	0.990	0.986	0.988	6,999/6,999	5,440/6,999	3,060/6,999	3,060/6,999	5,100/6,999	5,780/6,999
Fashion	AF	0.834	0.904	0.909	0.904	0.907	0.905	0.906	6,460/6,999	6,999/6,999	5,100/6,999	6,120/6,999	6,999/6,999	6,999/6,999

DOI: 10.7717/peerj-cs.1454/table-5

Notes:

¹Configuration of retraining.

Bold numbers indicate the accuracy (at most 0.001% worse than the highest value) and resource utilization for the model with the highest accuracy during retraining.

Table 6:

Execution time in hours (hh:mm:ss) to obtain the values of the metrics.

Dataset	LSA	DSA	DG	SE	NC	Random
GTSRB	00:03:32	00:36:19	00:00:48	00:00:49	05:28:57	00:00:00
Intel	00:01:11	00:01:47	00:00:09	00:00:09	01:11:02	00:00:00
CIFAR-10	00:05:54	00:19:47	00:00:25	00:00:27	06:57:19	00:00:00
MNIST	00:08:51	00:39:02	00:00:22	00:00:23	02:32:26	00:00:00
Fashion	00:04:38	00:25:10	00:00:12	00:00:12	01:20:43	00:00:00

DOI: 10.7717/peerj-cs.1454/table-6

In order to compare the impact that guidance metrics and retraining configurations have on the accuracy and resource utilization of the retrained models, we select the best model, according to the accuracy, during the retraining of the 20 data points for each combination of configuration and guidance metric (see marked points on Figs. 4–8 in the Results). We obtain the accuracy and resource utilization of these points and report these values in Table 5 to determine if they can be improved by guiding the retraining with the studied metrics and using the studied retraining configurations. In addition to that, we compare the execution time it takes to compute the guidance metrics in Table 6.

We answer the RQs from two different angles. First, we compare the accuracy changes against the augmented test set in the 20 data points of the retrained models according to each configuration and guidance metric. The augmented test set is composed of the original test set and the adversarial test set created with the same adversarial attacks but using inputs of the original test set; this way we ensure that the test data is never used during retraining. Figure 9 shows samples from the adversarial test set of the used datasets, these are first misclassified and after the retraining process are well classified. Second, we compare the input size required to obtain the model with highest accuracy between those data points according to each configuration and guidance metric.

Results

In this section, we report the results of our empirical study, answering RQ1 and RQ2, and highlighting key takeaways.

To better describe the results, we use Figs. 4–8 and Tables 5 and 6, as explained in the Data analysis section.

Discussion

Our experimental results focused on the retraining of models against adversarial inputs, taking into consideration the most used and practical state-of-the-art guidance metrics: LSA, DSA, DG, SE, NC, and Random. On top of that, we considered the manner (i.e., retraining configuration) in which the retraining is done.

As observed in the results, we have verified that NC is not a consistent metric to take into consideration to select inputs when retraining. Our results concur with previous studies, mainly Harel-Canada et al. (2020), which have already stated that NC should not be trusted as a guidance metric for DL testing. On the one hand, NC measures the proportion of neurons activated in a model and it assumes that increasing this proportion improves the quality of a test suit. On the other hand, Harel-Canada et al. (2020) showed that increasing NC value leads to fewer defects detected, fewer natural inputs, and more biased prediction preferences, which is contrary to what we would expect when selecting inputs for a retraining process to have a better model and reduce the input size for that. In our study, the results confirm that NC should not be trusted as a guidance metric. Furthermore, Table 6 shows an evident disadvantage on execution time when computing NC. Overall, different metrics rather than NC should be used when doing guided retraining.

Ma et al. (2021) found that uncertainty-based (variance score and Kullback-Leibler score) and surprise-based metrics (SA metrics, namely DSA and LSA) are the best at selecting retraining inputs and lead to improvements in the number of used inputs, up to twice faster than random selection, in order to find an answer on how to select additional training inputs to improve classification accuracy. Regarding SA metrics, as we did not consider the same uncertainty metrics, we confirm the results obtained in that study: SA metrics, especially DSA, compared to the baseline of random selection and also to NC, are a better option for retraining a DL model w.r.t. accuracy and number of used inputs. For their part, Weiss & Tonella (2022) claimed that SA metrics and NC-based metrics are often outperformed by other uncertainty approaches such as DG or simpler uncertainty metrics including SE, which is not considered in Feng et al. (2020), when retraining a DL model. Regarding uncertainty metrics, we also confirm their results: Uncertainty metrics compared to SA metrics, NC, and random selection, are better and lead to faster improvements. We consider this work as a complement to the results of the RQ about the selection for retraining of Ma et al. (2021), and the same for the similar RQ of Weiss & Tonella (2022), as we focused our work on increment the model’s accuracy against adversarial inputs (see Table 7).

Although our results show that these metrics improve the model faster (in terms of the input size used for retraining), we have found that the size of the dataset greatly affects the results, because when using small datasets it is not so fast. Ma et al. (2021) and Weiss & Tonella (2022) only considered datasets with more than 50,000 available inputs to use in the retraining (the entire training set), but in real-world applications is not always possible to have such large datasets. Unlike these works, we have experimented with a smaller dataset in which the improvements are minor (Intel dataset).

Additionally, an important finding is a variable that we also considered: the configuration of retraining. All the related works have only experimented with their own way to execute the retraining and even sometimes they are not explicit with how they did the retraining process. As we observe in the results, the configuration can change the performance of the model considerably. These results confirm the importance on finding efficient configurations to retrain deep learning models. Considering the studied configurations, using the best configuration studied in this work (AR) can give data scientists a fast and efficient method of retraining.

Regarding the computational complexity of the retraining process, we acknowledge that it increases the complexity of the entire training process. Indeed, a defense strategy against adversarial attacks is needed and a retraining process is one of the most effective strategies (Serban, Poll & Visser, 2020). In that way, according to our results, the least computationally complex process will depend on the application of the model and by following our guidelines it is possible to do a retraining against adversarial inputs without increasing by much the complexity of the retraining. The differences between configurations are the input set used for the retraining and the initial weights, as the same model is used for the retraining process, without increasing the model complexity. In that sense, a lower value of resource utilization with the help of guidance metrics translates into fewer operations.

When using the combination of uncertainty metrics, especially SE metric, and AR retraining configuration, we observe that it can lead to results that are aligned with green AI research, which refers to research that takes into account computational costs to reduce the computational resources spent (Schwartz et al., 2020; Castanyer, Martínez-Fernández & Franch, 2021b), as fewer inputs are needed for retraining and also less time to compute the values of the metric w.r.t. the evaluated options for both independent variables. We observe greater benefits of using AR over the other approaches, as we give greater weight to the efficiency without diminishing the models’ accuracy, and encourage data scientists to build greener DL models.

Previous studies, our results, and new implementations of these metrics (Weiss, Chakraborty & Tonella, 2021; Ouyang et al., 2021; Kim et al., 2020) aim to be a baseline for test methods in DL testing: for data selection in retraining processes, data generation, data selection, etc. In this way, we offer guidelines for data scientists in the context of testing with adversarial inputs during the retraining of CNNs for image classification:

• Guidance metrics and accuracy. Retraining with the guidance of uncertainty metrics can achieve CNN models with higher accuracy compared with the rest of the studied metrics.

• Guidance metrics and resource utilization. When prioritizing the number of inputs used for retraining, uncertainty metrics need fewer inputs to reach higher accuracy.

• Guidance metrics and execution time. When considering the execution time to obtain the metrics, uncertainty metrics are by far the best choice w.r.t. to other metrics, with the exception of random.

• Retraining configuration and accuracy. When prioritizing accuracy, retraining with original inputs and adversarial inputs from scratch (OAR) is the best option. However, retraining with original inputs and adversarial inputs from original model weights (AR) can reach similar accuracies without considerable differences and this configuration uses fewer inputs.

• Retraining configuration and resource utilization. Retraining with original inputs and adversarial inputs from original model weights (AR) has a good resource utilization, in addition to high accuracies w.r.t. the other configurations. Nevertheless, retraining with only adversarial inputs (AF) is the best option when prioritizing only resource utilization.

• Green AI. When considering efficiency without diminishing the accuracy of the model, retraining with original inputs and adversarial inputs from original model weights (AR), and the guidance of uncertainty metrics, especially SE, is the best way to retrain a CNN model in the mentioned context.

Threats to validity

In this section, we report the limitations of our empirical study and some mitigation actions taken to reduce them as much as possible.

Regarding construct validity, we include five original models using CNN architecture trained in five different datasets, respectively, in order to mitigate mono-operation bias. Derived from the metrics selected, we use LSA, DSA, DG, SE, NC, and Random metrics to guide the retraining runs, which may have potential threats. However, these metrics have been used in previous work as shown in the Related Works section, lowering this threat. And according to threats related to the configurations, we have reviewed the relevant literature and searched for retraining configurations using adversarial inputs.

Concerning conclusion validity, the quality of the DL models and implementations depends on the experience of the developers. To mitigate this, we provide the implementations organized and following the “Cookiecutter Data Science” project structure (https://drivendata.github.io/cookiecutter-data-science), making them as simple as possible. To increase reliability in our study, we detail the procedure to reproduce our work: the process is shown in the Emperical study section, datasets and replication package are available online. Also, we address the randomness of our results by starting the retraining runs for each data point from their respective initial weights (from original model weights for AR and AF, and from scratch for OAR).

Two threats to internal validity are the implementation of the studied DL models, as well as the computation of the metrics. We used available replication packages from the authors of the metrics, using the same configurations they used for the experiments to minimize this risk. Also, different models are used with different datasets, mitigating that the results of our study of guidance metrics and configurations are caused accidentally.

Threats to external validity stem mainly from the number of datasets, models and adversarial generation algorithms considered. Our results depend also on the datasets, the type of architecture considered, and the device used for training. We believe our results are applicable to image classification datasets. Some of these threats are addressed by the use of five image datasets, several state-of-the-art metrics and three adversarial attacks widely used by the scientific community. Regarding the architecture type, as the adversarial inputs can be generalized across different of types of architectures (Goodfellow, Shlens & Szegedy, 2014) and we use them for the retraining, we also believe that results for other architecture types could be similar to the results for CNN architecture, only further experiments can reduce this threat. Furthermore, our research findings are focused on guiding retraining to address adversarial inputs. Thus, it is possible that our results may not be applicable to other related problems, such as test selection. Additionally, it should be noted that in certain models, metrics derived from softmax may not be an adequate proxy for uncertainty (Gal & Ghahramani, 2016).

Conclusions and future work

In this work, we have studied DL testing metrics for guiding the retraining of models, and we adapted the models’ weights to be aware of adversarial inputs, as the retraining process involves adjusting the weights to better handle them. We performed an empirical study with the metrics and also considered three different configurations of retraining against adversarial inputs and did a comparison of the metrics and the configurations. In summary, we observe that (i) the models are increasing their accuracy against the augmented test set with adversarial inputs as it was sought in the objectives of this work, and (ii) there are computational benefits of using certain metrics and configurations.

The empirical study showed that the uncertainty metrics (such as DG and SE) as guidance for a retraining phase are useful for data scientists when using the following configuration: adversarial retraining from original model weights.

With the previous configuration and metrics, we can improve the accuracy of the models against adversarial inputs by up to 13.3% on the GTSRB dataset, 7.7% on the Intel dataset, 14.4% on the CIFAR-10 dataset, 1.5% on the MNIST dataset and 9.0% on the Fashion-MNIST dataset without the need of using many inputs. Therefore, this can be done by using 34.7%, 73.8%, 30%, 25% and 75% (similar accuracy gain achieved since using 30% unlike Intel dataset) of the available inputs, on the GTSRB, Intel, CIFAR-10, MNIST and Fashion-MNIST datasets, respectively.

Concerning SA metrics, DSA presents competitive values for dependent variables except for the execution time variable. Using random can guide to similar levels of accuracy but when using the recommended configuration, the computational benefits of using uncertainty metrics are that fewer inputs are required as discussed above in percentage, which in turn, translates into fewer adversarial inputs created. Regarding the NC metric, we do not recommend the use of it as it presents the greatest disadvantages on the studied variables. In cases where one-step adversarial retraining from scratch is considered, it can be a time-consuming process as it requires using almost 100% of the inputs to achieve similar accuracy to the recommended configuration.

Additionally, we revealed that the size of the dataset is important when implementing the recommended metrics and configuration. Taking this into account, we need to assess whether it would be worth calculating the metrics when using only small datasets.

In the next step, the reproduction of our experiments on different datasets of varied sizes, unbalanced datasets, and also non-images datasets (Kim & Yoo, 2021) are required to generalize our findings. Moreover, future experiments should study the effect of the dataset size on the studied metrics. Particularly, experiments with other DL architectures are required to confirm or reject that our findings can be used across different architectures. Also, our results should be compared to guided retraining runs using different testing metrics such as new refinements of SA (Kim et al., 2020; Weiss, Chakraborty & Tonella, 2021; Ouyang et al., 2021) or other uncertainty-based metrics (Guo et al., 2022a).

While the results of our study provide meaningful outcomes, there are several dimensions to empirically explore in the future. Thus, it would be relevant to evaluate this type of retraining process against adversarial inputs in the context of continual learning (CL), as there is scarce research (Khan et al., 2022b; Bai et al., 2023; Chou, Huang & Lee, 2022; Khan, Bouaynaya & Rasool, 2022a) in addressing adversarial inputs for CL methods (De Lange et al., 2021). Additionally, it would be valuable to study the effects of adversarial attacks, which utilize distinct algorithms and have different computational complexity (Ren et al., 2020).