Usability and acceptance of crowd-based early warning of harmful algal blooms

Literature review

This subsection presents the theoretical basis, such as the crowdsourcing concept, the technology and acceptance model (TAM), the usability measurement, HABs, Alboom, and similar studies.

Alboom

Alboom can be a solution for detecting and mitigating HABs. The system’s workflow begins with the input of required information, which consists of images of water and the surrounding environment as well as automatic recording of location coordinates and time. Both automatic and manual recording can be used when the user’s smartphone device is on the Internet or offline. In offline conditions, the user’s data are stored in the device storage and then transmitted to the data server when connected to the Internet. After providing the images, the user (reporter) performs manual qualitative input of environmental conditions related to weather observations, water conditions, and a visual assessment of the situations. Prior to using Alboom for the first time, all users were trained via direct and online individual or group training sessions. The training was aimed at standardizing the user’s qualitative assessment of the environmental conditions to be inputted into the application. Since all input parameters are in the form of qualitative assessments using simple expressions, as shown by the workflow in Fig. 1, the users had no difficulty in understanding and performing the data input. The data input into the server are then analyzed and verified automatically and relayed to other Alboom users via the map viewer. This relay speed is relatively short so that the occurrence of HABs in one place will be immediately known (near real-time) by other users in different places.

Alboom collects data and information on weather and water conditions that could indicate the occurrence of algae bloom on pre-bloom, when the bloom starts to occur, and post-bloom conditions. Pre-bloom alert signs are detected through weather conditions on sunlight intensity and rainfall when the data is recorded and about two to three days before. Puspasari et al. (2018) found that algae bloom is triggered by heavy rainfall followed by high sunlight intensity. Therefore, reporting on the rainfall occurrence two or three days before the high intensity of sunlight could become an alert sign for algae bloom and precautions for users to increase their visual monitoring of the color of the water.

Alboom records the water condition data through the users’ observation of color changes from the expected condition, water density, the itchy and slimy feel if the water is touched, and the existence of bioluminescence at night to indicate the occurrence of blooming. Color changes in the water are the easiest way to detect the occurrence of algae bloom. The color turns reddish, yellowish, brownish, orange, or dark green, indicating increased algae biomass containing particular pigment colors. For example, Grate-Lizrraga et al. (2004) found increased biomass of Cochlodinium polykrikoides, which contains chlorophyll-a and c, peridinin, diadinoxantin, and beta-carotene pigments turned the color of the water from normal bluish into reddish. This phenomenon is usually called red tide. Meanwhile, Gopakumar, Sulochanan & Venkatesan (2009) showed that the increased biomass of Noctiluca scintilans changed the color of the water from normal bluish to dark green. Furthermore, Puspasari et al. (2018) found that the sensation of itchy and slimy waters was felt when Cochlodinium polykrikoides blooms in Lampung Bay due to the mucus production by the massive concentration of the cells. Another indication of algae blooms is the bioluminescence at night. Gopakumar, Sulochanan & Venkatesan (2009) also detected some species of algae, like Noctiluca scintilans, contain high fluor concentrations that cause luminescence phenomena in the water when there is limited light. Sometimes, when the increased concentration of algae biomass is not significant but higher than average, users cannot observe the blooms through a visual and touching sensor. However, the occurrence of the algae blooms could still be detected from the post-bloom conditions through the existence of massive dead algae in the form of clumpy and foamy mass floating on the surface water.

The adverse effects of HABs are caused by the drop in oxygen concentration in the water column or the toxic effect of some toxic species, according to Karlson et al. (2021). Barokah, Putri & Gunawan (2016) found that 12 species were identified as HABs causative species, and seven of them contain toxic in Lampung Bay. Blooming of toxic algae causes the death of organisms due to the increase of toxic concentration, even though blooming occurs with a low abundance of algae. In this case, the color of the water might not change, and the toxic species blooming could not be indicated visually. Alboom detects the occurrence of toxic species bloom through other data input, such as an itchy and slimy sensation caused by the toxin produced by algae. There is no specific data record on the toxicity conditions in Alboom. However, if the inputted data indicate an occurrence of blooming algae, other users are notified and then can verify it. In this case, the function of Alboom as sharing system in the community has dominated the role of the early warning system.

Research gaps, objectives, and contributions

Nevertheless, acceptance studies for crowd-based technologies using TAM are still rare. Only three relevant studies were found, showing that all proposed TAM hypotheses were accepted. The ease of use and usefulness were essential for stakeholders to use crowdfunding in a study by Djimesah et al. (2022). Perceived ease of use and utility significantly influenced users’ intention to use RISCOVID for tracing contacts of persons infected with Covid-19 in a study by Cruz et al. (2020). Moreover, according to a study by Minkman, Rutten & van der Sanden (2017), usefulness, the relevance of the task, and the demonstrability of benefits significantly influenced acceptance of mobile technology for citizen science in water resource management.

Data collection

Data collection was performed using online survey. Before collecting questionnaire data, two general lecture and training workshop webinars (http://alboom.mict.id) were conducted to increase participants’ awareness about the role of technology and community participation in detecting and mitigating HAB incidence. Participants were part of the EWS stakeholders for HABs consisting of fish farmers, fishers, governments, fishery instructors, researchers, and students.

In the first webinar, Alboom was introduced to the public for the first time. Then, the attendees were asked to install the mobile application on their smartphone devices. The users must input their profile data when registering themselves in the application. Informed consent was obtained before creating the user account. The new users were required to agree to the terms and conditions, as shown in Fig. 3, such as:

• Any information users submit will be kept confidential.

• Users’ name or any information that might identify their profile in the study report will not be used.

• Users’ participation is completely voluntary.

• Users have the right to withdraw their participation at any time, or refuse to answer some of the questions.

• The data that users upload will be analyzed only for academic purposes and will be published in the form of statistical aggregations in scientific journals or conferences.

After consenting and finishing registration, users were asked to collect data using Alboom regularly. Users who consistently contributed data six times between the first and the second webinar have been rewarded a certificate of appreciation. After the second webinar, the questionnaire was distributed, and all users were invited to respond through the mobile application.

Research instrument

The research instrument was divided into two parts. The first part consisted of standard SUS questionnaires, as shown in Table 2. The instrument has 10 questions, where odd items are positive statements, while even ones are negative ones. Meanwhile, the second part of the questionnaire consisted of the augmented TAM instruments is shown in Table 3.

Analysis method

The analysis method was performed using statistics tools available in SmartPLS 4 (https://smartpls.com/) to calculate all statistical computations. The model was analyzed using partial least squares structural equation modeling (PLS-SEM) algorithm. Before performing analyses, reliability and validity tests were conducted by measuring Cronbach’s alpha, factor loading, average variance extracted (AVE), and discriminant validity. Meanwhile, calculating SUS score simple, only requiring odd and even question numbers to be distinguished. If the number is odd, then the result of the respondent’s value is reduced by 1; if the number is even, then the value is five minus the value of the respondent. The scores of the ten questions are summed and then multiplied by 2.5 as shown by Eq. (1) where U_i refers to the rating of the ith item. (1)${\displaystyle SUSscore=2.5\times \left[\sum _{n=1}^5\left(U_{2n-1}-1\right)+\left(5-U_{2n}\right)\right].}$

Participant demographics

The first webinar was attended by 488 people. Meanwhile, only 138 participants installed Alboom (https://appho.st/d/JM80Ljzf). Alboom was introduced continuously to the stakeholders between the first and the second webinars, and afterward. Thus, the number of new users increased to 223. However, only 109 people have ever uploaded data using Alboom at least once. All users were offered to respond to the questionnaires, and 104 of 223 people provided responses.

Table 4 shows the demographic information of the respondents. The proportion of men and women in this survey was balanced. Most respondents were 17–25 years old (54%), which indicates that the majority were digital natives. Based on reported job distributions and education levels, most of the respondents were well educated. Also, 32% of respondents have installed and used Alboom but never uploaded data. Meanwhile, 42% of respondents reserved the reward of certification because they uploaded data using Alboom six times or more.

System usability scale (SUS) measurement

In order to determine whether the SUS instruments had good reliability, it is necessary to calculate Cronbach’s alpha value, which is a test score reliability coefficient to measure how closely related a set of items are as a group. The results of the calculation, as shown in the Cronbach’s alpha (α) column in Table 5, have values above 0.7. Because reliability theory, according to Nunnally (1975), requires a Cronbachs’s alpha value of at least 0.7, the reliability of the variables and the level of internal consistency of the instrument are confirmed.

The SUS instruments, as shown in Table 2, have ten questions that contain both positive and negative meanings. For example, questions on numbers 1, 3, 5, 7, and 9 have positive connotations, while questions 2, 4, 6, 8, and 10 have negative meanings. This difference in positive and negative statements will result in a different grade. If a question is positive, selecting a higher value (e.g., strongly agree) will yield a true value. In contrast, if the question is negative, a lower value (e.g., strongly disagree) yields a higher score. Therefore, the values must be normalized to find the absolute highest value.

The normalized results shown in Table 5 were calculated by reducing the value input from the respondent by lifting one (1) for positive questions. In contrast, by reducing five (5) by the value of the respondent, the value will be between zero (0) and four (4). Table 5 also shows the percentage of respondent values one (1) to five (5). To calculate the optimal ratio, in the % two highest values column, the calculation adds one (1) and two (2) if the question is negative, and if the question is positive, it adds three (3) and four (4). The tenth item has an average normalization result below 1.47 with 22%. This item stated that users had to learn many things before they could use Alboom, which indicates that the respondents required adaptation to use Alboom.

Referring to Eq. (1), the normalized results were then multiplied by 2.5 to determine the level of usability perception in the Alboom application. The average score of all respondents, as calculated using Eq. (2), is the final SUS score, which ranges from 0 to 100, given the number of respondents, n, 104. Table 5 shows the SUS score for the Alboom application of 66.59. This value is sufficient, has a grade “D” scale, adjective ratings of “OK”, and a high-marginal acceptability range, according to Bangor, Kortum & Miller (2009). (2)${\displaystyle \overline{SUSscore}=\frac{1}{n}\sum _{i=1}^{n}SUSscor{e}_i.}$

Assessment of measurement model

The measurement model was assessed based on factor loading, construct reliability using Cronbach’s alpha, AVE parameter, and discriminant validity. As shown in Table 6, seven construction item indicators (AWA, PEU, PUF, SOC, ATT, REW, and INT) had loading values between 0.56−0.96. A factor loading is the correlation coefficient for the variable and factor. It describes the variance the variable explains on that particular factor. According to Hair Jr et al. (2014), the ideal allowable factor loading should exceed 0.7, which indicated that the factor removed sufficient variance from the variable. Thus, the construct indicators of awa1, awa2, att1, and soc3 were dropped because their loading values were below 0.7. Furthermore, all Cronbach’s alpha values were more significant than 0.7, indicating that all constructs were reliable.

Also, convergent validity was assessed using AVE. As shown in Table 6, all AVE values in the six construct parameters exceeded the minimum threshold value of 0.5, according to Fornell & Larcker (1981), which indicated that the variance captured by the construct was larger than the variance due to measurement error. Thus, all constructs were valid. Table 7 shows the discriminant validity of each indicator for all construct parameters. According to Monecke & Leisch (2012), the discriminant validity values of less than 0.2 are not shown in the output. The largest values for each indicator were in the construct parameter, and the built indicators were appropriate for measuring the construct parameters. Based on these results, the measurement model was satisfactory.

Assessment of structural model

Bootstrapping testing was performed to test the significance of the effect of one variable on another. This study accepted a hypothesis if the p-value is less than a significant level of 0.05. Therefore, all hypotheses in the proposed model were supported except H8, as shown in Table 8.

As expected, PEU was found to be significantly affected by USA (H1: β = 0.72; p < 0.001). Furthermore, PEU significantly affected PUF (H2: β = 0.55; p < 0.001) but PEU had small significant effect on ATT (H3: β = 0.22; p = 0.046). PUF and AWA positively predicted ATT (H4: β = 0.36; p < 0.001, H5: β = 0.30; p = 0.001). Moreover, INT was found to be significantly influenced by ATT and SOC. Based on the magnitude of the path coefficient value, which is significant in each construct, the attitude toward using CBEWS plays the most important role in determining a person’s desire to use CBEWS subsequently (H7: β = 0.59; p < 0.001). The path coefficient value was more than twice the coefficient of the path from social influence to intention (H6: β = 0.25; p = 0.001). Nevertheless, REW did not significantly positively predict INT (H8: β =  − 0.04; p = 0.638).

Research implications

Although many studies debate whether an HCI-based approach, like usability, could be combined with an IS-based approach, this study showed a successful integration of usability, particularly SUS score, into TAM. Nevertheless, studies by Pal & Vanijja (2020) and Albastaki (2022) showed that perceived ease of use had high similarity with usability, particularly SUS score. Therefore, as an implication of the research, usability could be optionally included in the future acceptance models if perceived ease of use or a similar variable is already incorporated. This study showed a strong relationship between the two variables, where usability significantly influenced perceived ease of use.

It is worth revisiting the reward factor in subsequent research. Most crowd-based technology acceptance studies found reward as a significant driver of use. It could be because those studies used monetary rewards. Meanwhile, in this study, a general term of reward was used. In fact, non-financial rewards, such as certificates, credits, et cetera, were given to participants. However, due to a limited budget and regulations, financial rewards were not provided. Therefore, further studies are needed to investigate the monetary reward and its relationship with intention to use, attitude, or even perceived usefulness.

Practice implications

Even though perceived ease of use and usability is considered a similar variable, in practice, it is suggested that organizations or practitioners conduct both analysis, IS-based approaches first, followed by HCI-based approaches. For example, suppose perceived ease of use or another similar factor is found to be significant. In that case, the degree to which users can use the application to achieve quantified objectives with efficiency, satisfaction, and effectiveness in a quantified context of use should be measured. Then, if the usability score is below average, the application should be improved. In this study case, the usability of Alboom should be revamped because the SUS score was only 66.59.

Reward was not a determinant of the continual use intention of CBEWS. Therefore, this result might imply that providing a reward is not a solution for organizations or practitioners to boost application use. A reward might not guarantee that users will continue to use CBEWS in the future. However, awareness positively influenced attitude toward using CBEWS. Based on this finding, it is suggested that organizations or practitioners should frequently increase citizens’ awareness regarding the hazards’ context. In this study case, webinars about HABs were organized for citizens to educate them regarding the indications and impacts of HABs. Also, a live on-field training could be conducted to demonstrate how the entire system works. Furthermore, social influence positively affected the continual use intention, which might indicate that the more users that use CBEWS, the more likely that other users are to be socially influenced to also use CBEWS. Therefore, organizations should encourage inspired people, such as managers and leaders or respectable persons, to embrace CBEWS and persuade others to use it on a long-term basis.

Limitations and threats to validity

This subsection identifies limitations and threats to the validity of this research and discusses how they can possibly be addressed. This study considers four validity threats: internal, external, construct, and conclusion validity.

To control for the internal validity threat of multiple submissions from the same participant, users were asked to log in to the mobile application before submitting the response to the questionnaire. Thus, it was ensured that participants who completed the questionnaires had installed and used Alboom. All users were encouraged to respond to the questionnaires. Reminders were sent to the users’ mobile applications and emails every day.

Although respondents consisted of 47% of the population of Alboom’s users, this does not mean the results can be generalized. The respondents were dominated by students (54%), researchers (20%), and university lecturers (9%), which indicates that current Alboom users are primarily scholars. The primary target users of Alboom in the future will be fish farmers, fishermen, and others that have primary related jobs in coastal waters because they spend most of their time in the field, which makes them available at any time to upload data. However, respondents from the most expected users, such as fishery instructors (8%) and fishers (5%), were limited in this study. Also, fishers currently using Alboom are not purely voluntary because they were facilitated with smartphone devices financed by this project budget. The variable in this study, the awareness factor, had a significant influence on the attitude toward using CBEWS and might be affected by the background of the highly educated respondents, which leads to a stronger understanding. These limitations are potential threats to external validity. However, in future work, this study will be repeated when the number of Alboom non-scholar users increases.

Threats to construct validity were manageable because Cronbach’s alpha and factor loading for each question in SUS and TAM questionnaires were beyond the standard value of 0.7. If their value was less than 0.7, the question items were dropped. These items were primarily adapted from highly cited studies on TAM and SUS. Furthermore, convergent validity was checked using AVE measures, and discriminant validity was also tested to ensure that the constructs that should have no relationship indeed do not have any relationships. Only reliable and valid items were considered in the SEM analysis.

For statistical conclusion validity, the SEM technique was used in this study to fit a theoretical model to the data. Model fit indicators indicate that the model is sound. SEM further improves conclusion validity by adjusting for multiple comparisons, measurement error (by inferring latent variables from observable variables), and testing the full model (rather than one hypothesis at a time). Alternative path modeling techniques, such as partial least squares path modeling like Bayesian approaches, are regarded as inferior to SEM, according to Rnkk & Evermann (2013).

Future works

More recent theories should be implemented in the future. For example, UTAUT or its second version should be used to investigate other factors that could positively influence attitude, such as facilitating conditions, performance, and effort expectancy, because the attitude toward using CBEWS was the most influential factor in this research. Moreover, latent variables could be added in subsequent studies, based on the second version of the TAM created by Venkatesh & Davis (2000) or the third version by Venkatesh & Bala (2008). Also, the actual use variable could be examined in the future by looking at actual usage data. Specific actions could be implemented to achieve particular targets by knowing specific factors to promote positive behavior.

Furthermore, only one CBEWS was examined in this study. Future works should investigate whether the same results are acquired in other CBEWSs. Also, usability addressed in this study was only measured with a single approach. Future research could check whether other usability measures, survey-based or even usability testing approaches, would generate the same results. Moreover, individuals’ perceptions of technology may evolve over time. As a result, the current findings could serve as a starting point for future longitudinal studies into the shifting roles of predictors in users’ acceptance and subsequent use of CBEWS.