A comparison of impact and risk assessment methods based on the IMO Guidelines and EU invasive alien species risk assessment frameworks

Setting the scene: comparison of the IMO and EU risk assessment frameworks

The frameworks of both the IMO (2007) and European Union (2018) regulations were compared to provide a support for a common evaluation procedure (Fig. 1; see also Tables S1 and S2). Accordingly, we screened the RA frameworks as follows: (i) key principles of an assessment process, (ii) assessment components and, (iii) categories of bioinvasion impacts to be taken into account using the above RA frameworks.

The evaluation procedure

Based on the comparison of the IMO Guidelines and the EU Regulation, the evaluation procedure included eight key principles of the IMO Guidelines, twenty-nine RA components of the EU Regulation and four main bioinvasion impact types, compiled from both documents (Fig. 1). In addition, we incorporated impact categories as proposed in earlier risk assessment frameworks (Emerton & Howard, 2008; David & Gollasch, 2015; Olenin et al., 2016; Vilà & Hulme, 2017). In all, 41 categories were defined: human health (six categories), economy (11), environment (20), social-cultural aspects (four). Descriptions of the impact categories are provided in the supplementary table (Table S2).

We developed a scoring scheme in order to assess the compliance with each of the eight key principles (Table 1). The RA components and the categories of impact were considered to be either present or absent based on the original description of the selected. The overall ranking of selected methods is based on an accumulated score, and expressed as a percentage of compliance with our selected criteria. We discuss the advantages and limitations of this approach.

Selection and review of the bioinvasion impact and risk assessment methods

To select the bioinvasion impact and risk assessment methods for the analysis we used the list of the most relevant methods identified Roy et al. (2017) and the COST Action Alien Challenge TD1209. They performed a worldwide literature search for the methods of invasive species risk assessment (RA), and crosschecked the references for additional relevant publications to obtain twenty-nine original RA methods. We used these methods for an analysis based on the following criteria: (a) the method is applicable for the aquatic realm; (b) the assessment results are either in a quantitative or in qualitative form, and (c) it takes into account at least one of the four categories of bioinvasion impacts. From this preliminary analysis we selected nine methods out of the 29 reviewed by Roy et al. (2017) and we searched the literature to include any further methods which were not considered relevant in their review, yet met our criteria. We found fifteen methods suitable for our analysis (Table 2). The selected methods represent different regions and we recognize that there might be further methods worldwide which did not come to our attention. It should be noted that our main goal was to test the evaluation procedure on a sufficient number of methods.

We refer to each method by an acronym (Table 2), while some have changed their names with further development, for example, AS-ISK (Copp et al., 2016) was originally known as FISK “Fish Invasiveness Screening Kit” (Copp et al., 2009) and the Biopollution level (BPL) (Olenin, Minchin & Daunys, 2007) was later computerized and renamed as the Bioinvasion impact/ Biopollution assessment system, BINPAS (Narščius et al., 2012). Most of the methods (75%) were published in peer-reviewed journals, one as a book chapter, and three appeared in national or international environmental reports. The methods were divided into three groups, according to their assessment goals as: (1) the screening tools (AS-ISK, CMIST, HARMONIA+), (2) risk assessment tools (GB NNRA, TRAAIS, SBRA, WISC, RABW), (3) impact assessment indexes/schemes (CIMPAL, BINPAS, GISS, GABLIS, GEIAA, GISS IUCN, GLOTSS).

Similarities and differences between the two legislative documents

The screening revealed differences and similarities between two documents, which are summarized in Table 3.

Key principles of assessment process

The summary of the evaluation of compliance with the key principles is presented in Table 4 and detailed evaluation results are given in Table S3.

Risk assessment components

Based on the analysis we found that all methods incorporated at least some general information about non-indigenous species under consideration (Table 5; Table S1), i.e., taxonomic identity, scope of a RA, etc. (Art 5 (1) (a) (EU, 2018). The RA components concerning reproduction and spread (Art 5 (1) (b)), pathways (Art 5 (1) (c)), stages of invasion process (Art 5 (1) (d)), distribution (Art 5 (1) (e)) and impacts (Art 5 (1) (f)) were incorporated within most methods (Table 5). The least covered components were the estimated consequences of economic damage (Art 5(1)(g)) and any known uses and benefits (Art 5(1)(h)). This involved four and two methods, respectively.

The incorporation of a RA component into a method was considered as being complete, should all of the elements be covered. For example, all three elements for reproduction and spread were incorporated within eight methods (Table S1). Only one method (SBRA) incorporated in full or in part all the components, while four methods (GABLIS, GB NNRA, HARMONIA+, WISC) incorporated more than 75% of the RA components.

Types of impact

The impacts on human health were considered in 57% of the methods, however, this was mostly as a “general impact on human health”, without further clarification. While three methods (GB NNRA, GISS, HARMONIA+) included more detailed information on human health, accordingly: parasites, pathogens, toxic compounds, poisoning and venomous organisms. These results are summarized in Table 6, Fig. 2 and additional information in Table S4.

No method included all environmental impact categories. However, all had at least one environmental impact category: parasites and pathogens affecting native species, parasite vector, predation, competition, hybridization, habitat change, or population loss caused by an invasive species, etc. A single method (GISS) incorporated 90% of all of the environmental categories, followed by TRAAIS (80%) and SBRA (75%).

Sixty per-cent of the methods included an economic impact category with either general management costs (60%) or impacts to aquaculture (47%), fisheries (40%) or in relation to irrigation and abstraction (40%). The methods GABLIS, GISS, and SBRA covered more than a half of these economic categories: 55, 64 and 64%, respectively.

Social-cultural impacts were taken into account by 53% of methods, the most frequent category being consequences for recreation and tourism (53%). SBRA took into account seventy-five per-cent of the social-cultural impact categories, the highest coverage of any RA method.

Overall evaluation of the methods

Our general assessment of the methods by the key principles, RA components and categories of impacts are presented in Fig. 3, while the assessment of the methods according to criteria appears in Fig. S1. The method that met most of our criteria was SBRA. This complied with most of the key principles and RA components, and covered the broadest spectrum of the impact categories, followed by HARMONIA+ and AS-ISK. However, none of the methods complied with all our criteria.

A further method that generally complied well with key principles and RA components was RABW, a method developed for the BWMC. However, it has fewer impact categories, focusing only on those associated with aquatic environments. In contrast, GISS incorporates the highest number of impact categories, but has comparatively low compliance with RA components.

Key principles and quality of methods

The two legislative documents reviewed in this study were developed for different purposes: while the EU regulation (European Union) has a wide spectrum of application and addresses the invasive alien species of all taxa and within all habitats, the IMO Guidelines (IMO) focuses on harmful aquatic organisms and pathogens transferred by a single vector of introduction. However, the cross-comparison of both documents highlighted the common features that stem from their overall orientation on biosecurity. Such comparative analysis is especially needed nowadays when EU countries are to implement both legally binding instruments, the BWMC (IMO, 2004), which entered into force in 2017 (IMO, 2017) and the EU the Regulation on the prevention and management of the introduction and spread of invasive alien species (Union European, 2014).

The comparison of the related risk assessment frameworks (IMO, 2007 and European Union, 2018) helps to achieve a more comprehensive, integrative view on the risk assessment process. As the result, the approach developed in this study is based on three criteria. Of these, the key principles and assessment components form the basic criteria in risk assessment, while the categories of the bioinvasion impacts were complimentary and added to complete the full evaluation procedure. This is because such impacts were not specified in either document.

The screening of similar regional and international regulations and frameworks, e.g., Convention on Biological Diversity (CBD, 2011), North America Free Trade Agreement—(General Accounting Office , GAO), Asia-Pacific Economic Cooperation (Williamson et al., 2002), ICES Code of Practice on the Introductions and Transfer of marine organisms (2005) did not reveal different criteria to what we have examined. Barry et al. (2008), Dahlstrom, Hewitt & Campbell (2011) and David & Gollasch (2015) who analyzed biosecurity risk assessment regulatory documents and bioinvasion risk and impact assessment methods also did not reveal criteria other than what we have used. It would seem that the key principles and RA components are universal for evaluation of bioinvasion risk and methods. The categories of the bioinvasion impacts may vary depending on the scope of the assessment and should be used as complimentary criteria.

In our opinion, the compliance with the key principles shows the quality of a method. Our analysis showed that only three methods made their assessment tools and documentation available via an online database. This must be considered the highest “Transparency” level and an example for other methods to follow. This is because decision-makers should have access to the full information to be able to compare the usage of bioinvasion risks and impacts assessment methods in similar situations worldwide.

Online information sources for NIS already exist for specific areas, e.g., for prioritizing most impacting NIS (e.g., target species lists), defining their pathways and vectors and with recommendations for their management (Olenin et al., 2014). Lehtiniemi et al. (2015) have stated that there is little value in monitoring of NIS unless the knowledge obtained is timely and can be directly used. The importance of dissemination of information, the transparency principle, was stressed many times at international and national levels (e.g., Awad et al., 2014; Costello et al., 2014; Sing & Tan, 2018). In addition, the availability of the information is important to achieve “a uniform high level of performance, using a common process and methodology” (IMO, 2007).

Policy relevance of the methods

All bioinvasion risk and impact assessment methods reviewed here have been designed to support management decisions in a manner consistent with recommendations from multiple publications, e.g.: the method allows “a comparison and thus a prioritization of species” (Nentwig, Kühnel & Bacher, 2010), “enables an effective priorization of management efforts” (Sandvik et al., 2013), “identification of hotspots areas, and prioritization of sites, pathways and species for management actions” (Katsanevakis, Tempera & Teixeira, 2016), “inform management and policy decisions” (Drolet et al., 2016). Consequently, the methods should conform with the policy documents involved. In our study, the policy relevance may be defined as usefulness of a method for those who make decisions on biosecurity.

Our approach may help in choosing the most appropriate method, for example, to test the policy relevance of a method for the implementation of the BWMC. While only one method purposefully designed for the BWMC was analyzed in our study (RABW), several other methods may be used for the BWMC purposes with adjustment, should they follow the key principles and take into account RA components. For example, the GB NNRA method covers the components such as NIS spread, pathways, distribution and impacts, which are needed when considering a risk assessment of ballast water (Behrens, Leppäkoski & Olenin, 2005; Werschkun et al., 2014; Olenin et al., 2016). It is noteworthy that the method integrating most of the RA components (SBRA) was especially designed for one of the shipping vectors, i.e., for the species biofouling risk assessment (Hewitt et al., 2011). Ideally, all methods should comply with the key principles and RA components as far as possible, while the bioinvasion impact categories may vary and should be selected according to the purpose of the RA. For example, the risk to human health is an important issue (Conn, 2014); however, not all methods, even those purposefully designed for BWMC, take into account this impact category. In the earlier study by Barry et al. (2008) only two out of the eight reviewed “ballast water risk assessment systems” refer to the importance of human health categories without considering any details. Generally, our study has shown that more attention is paid to environmental impacts rather than to human health or economic impacts.