Improving bioinformatics software quality through incorporation of software engineering practices

The challenge of requirements

According to Segal (2008), scientists usually lie with requirements because they are either unaware of their meaning or cannot properly define them. This is directly at odds with the practice in software engineering of gathering requirements to determine the intended purpose of the software to be developed. To be more complete, the definition process includes surveying the data, studying the current system, identifying business requirements, researching alternatives, and proposing a system (Bourgeois, 2014). Accurate definition of requirements ensures that the system to be developed will be unambiguous, testable, traceable, and measurable (Hay, 2003). Making the situation yet more challenging, in the context of bioinformatics, software requirements are naturally dynamic (Rother et al., 2012). With life science datasets continuing to yield new information and directions, the recording of novel occurrences requires a functional and flexible system (Hauth & Burger, 2008; Abdurakhmonov, 2016). Moreover, a study by Segal & Morris (2008) indicated that software requirements for scientific functions are different from those for business functions. Their study described a model of software developed by bioinformatics engineers wherein the bioinformatics engineers did not appreciate the process of gathering requirements for software engineers because they already knew the requirements of the scientific problem. The authors came to the conclusion that the bioinformatics engineers were often untruthful in discussions about their needs because they lacked awareness or clarity of definition of the requirements. In a twist, a different study by Rother et al. (2012) showed that bioinformatics engineers were often asked only for functional requirements, preventing associated software engineers from gathering adequate information about the system itself. This seems a bit ironic given that two other studies found that bioinformatics engineers working on projects of various sizes assigned software requirements the lowest rank in terms of project importance (Hannay et al., 2009; Garousi et al., 2019). Perhaps what is missing here is establishment of effective process more than rigorous enforcement of requirements.

Due to bioinformatics being an investigative discipline, the gathering of software requirements might benefit from the Agile approach, which can be summarized as “principles that emphasize building working software people can get their hands on quickly rather than spending a lot of time writing specifications up front” (Duka, 2013). This application of Agile is supported by Kane et al. (2006) who stated that the Agile method is balanced effectively due to the iterative and exploratory nature of the scientific inquiry. Similarly, Verma et al. (2013) proposed an approach to resolve the challenge of gathering bioinformatics software requirements based on the application of four methodologies: UP, domain engineering, SSADM, and Agile. O’Connor et al. (2008) recommended the sharing of customized analyses through the Generic Model Organism Database (GMODWeb), an integrated workflow system. A few examples of software packages that have been developed with best practices in mind include Tophat (Trapnell et al., 2012), BioPerl project (Leprevost et al., 2014), and Bowtie (Kishchuk et al., 2018). At a basic level, the usage of requirements tracking platforms (e.g., Jira: https://www.atlassian.com/software/jira) and development platforms (e.g., GitHub: https://github.com/) can help with definition of requirements, effective communications, and successful collaboration of bioinformatics engineers towards the development of high-quality software.

The challenge of documentation

Proper documentation is often essential for maintaining custom software, especially software encompassing the integral complexities of bioinformatics research. Unfortunately, bioinformatics engineers often do not focus on documentation (Karimzadeh & Hoffman, 2018). Without documentation, it can be difficult if not impossible to understand the impetus for certain software design decisions. Such documentation can often take the form of a technical manual or a user manual. A technical manual describes the underlying functionality of the software system. A user manual, as its name implies, helps end-users of a software system properly leverage the software (Chimalakonda & Venigalla, 2020; Venigalla & Chimalakonda, 2021). Additionally, a user manual can provide some indication of the purpose or input-output requirements of the software.

When it comes to bioinformatics software development, the documentation process is not only necessary for the increasing number of end-user developers (Letondal & Mackay, 2004) but also the growing number of research projects analyzing and interpreting massive life science datasets which require the development of bioinformatics software (Brandies & Hogg, 2021). When Umarji et al. (2009) questioned a group of bioinformatics engineers, 90% of them indicated that documentation was valuable. The work of Taschuk & Wilson (2017) corroborated the need for documentation. Katerbow et al. (2018) also highly recommended documentation when developing scientific software. Hoda, Salleh & Grundy (2018) described the importance of knowledge in documentation and how it aided software development when using the Agile method. This conclusion chains into the work of Kane et al. (2006) who found that the Agile method can be applied across different bioinformatics tools utilizing extensive quantities of data through the creation of data types and formats to create a functional workflow. Further, documentation helped to embed domain knowledge in source code and improve domain-specific language to promote word sense disambiguation in the software (Ecale Zhou et al., 2019). Documentation can also provide both developers and end-users with supplementary information such as benchmarking results (Ison et al., 2019). There have been other efforts made in improving documentation for software developed for scientific bioinformatics.

According to Ison et al. (2016), improved documentation can likely assist in sustaining stable sharing mechanisms for metadata, particularly for institutional collections (i.e., Institute of France Bioinformatics (IFB)) and specialized registries (i.e., BioContainers), as well as improve community standards through OpenAPI and many biological databases (i.e., Web APIs).

To aid in the creation and formatting of documentation to improve its effectiveness, templates can be constructed for each type of documentation (e.g., user guides and system requirements) to that will give guidance to bioinformatics engineers (Karimzadeh & Hoffman, 2018). Furthermore, the use of the Agile methodology for effective documentation and software functionality development helps meet the information needs of the user while simultaneously allowing the application of different tools, such as automated workflow composition (Harris et al., 2019). Automated workflow, which refers to process design, execution, and automation based on workflow rules, can make the curation of bioinformatics software easier by employing different curation tools to derive information from standardized environments (e.g., Galaxy) or other code repositories (e.g., GitHub) to help identify and correct inconsistent information. Proper documentation is particularly important for the bioinformatics community and the international infrastructure of life science research when it comes to explaining and evaluating discoveries and building upon the research of others (Orengo et al., 2020).

The challenge of testing

Software testing assesses the functionality of a software application to determine if predetermined requirements have been met and the resulting product performs to expectations (Patton, 2006). Each step of the software life-cycle is testable. Examples of types of software testing include validation testing, function testing, and unit testing. For scientific bioinformatics software, testing has often proven the most ambiguous and challenging task in the development process due to the software frequently being either data-driven or algorithm-driven, which can prove difficult to validate or verify. This is a type of oracle test problem where given a particular set of inputs for a system, there exists a “challenge of distinguishing the corresponding, desired, correct behavior from potentially incorrect behavior” (Barr et al., 2014).

This challenge can prove quite detrimental for bioinformatics engineers lacking software engineering skills who develop ad-hoc software for specific projects/tasks (Georgeson et al., 2019). The resulting ad-hoc used to determine software performance at an early stage, especially without proper documentation, planning, or processes, yields results that tend to be unstructured and inconsistent, frequently relying on the knowledge and memory of the expert doing the testing (Chhabra, 2012). When developing bioinformatics software, developers must have an in-depth understanding of the knowledge domain, whereas in other areas of software development (e.g., ERP systems), developers often do not need such extensive knowledge (Amershi et al., 2019). Furthermore, the interdisciplinary nature of bioinformatics requires its professionals to possess a wide range of abilities, resulting in bioinformatics engineers being more likely to be end-users (Allen & Mehler, 2019). To address testing issues (Soergel, 2014) argued that multiple sets of eyes are required in order to validate bioinformatics software and any research results that it may deliver. Hardware issues present a somewhat unique challenge for testing bioinformatics software. Limited hardware infrastructure can greatly restrict solution options for enormous life science datasets, affecting scalability, latency, performance of data infrastructures, and processing the data itself (Khan et al., 2014).

There have been some efforts to address testing of scientific bioinformatics software. Chen et al. (2009) proposed a metamorphic testing technique that tests properties from software outputs and end-users in parallel rather than using the traditional input-output testing approach. Kamali et al. (2015) surveyed current efforts in bioinformatics software testing and concluded that more research is needed. Lundgren & Kanewala (2016) experimented with subtle faults in testing bioinformatics software by applying pseudo-oracle and metamorphic tests on a genome alignment tool (BBMap). They found that metamorphic tests were effective in detecting subtle faults. Studies in scalability and validation have been performed and extensively reviewed with solutions such as cloud-based (Troup et al., 2016), divide-and-conquer, and multiple execution being examined for large bioinformatics datasets (Yang, Troup & Ho, 2017). Georgeson et al. (2019) created a command-line tool for developing scientific bioinformatics software that provides integrated unit and integration testing capabilities.

Version control systems can also aid in the testing of bioinformatics software through the use of certain tools, such as regression tests. Many free services exist, such as thefreecountry.com and Github.com, that provide version control solutions with tools that could prove helpful for bioinformatics software testing. Mulder et al. (2018) strongly encouraged bioinformatics engineers to utilize version control systems in order to maintain code quality and availability for the community. This also has the added benefit of exposure of the utilized software management processes to experienced software developers who can provide professional feedback and assistance. The Agile methodology can be applied when managing a project to divide the project into several different phases that involve constant collaboration with stakeholders and continuous enhancement at early stages. Two studies of software development using Agile methodology proved that scientists, in general, were well-satisfied with the resulting software testing (Umarji et al., 2009; Dingsøyr et al., 2018). A testing approach that could prove to be very effective involves scientists exposing all relevant source code during publication processes. This would not only allow for third-party validation, but also bolster testing results for reproducibility. Numerous efforts such as the Bioconductor (Gentleman et al., 2004) and OpenEBench (https://openebench.bsc.es) platforms have been undertaken for this purpose (Capella-Gutierrez et al., 2017).

The challenge of integration

Being able to replicate and reproduce scientific results has progressively grown in significance for most of academic disciplines (Stodden, Borwein & Bailey, 2013; Howison et al., 2015; Miyakawa, 2020). For bioinformatics software, integration plays an important role in reproducibility. Gulledge (2006) concluded that “integration implies that all relevant data for a particular bounded and closed set of business processes are processed in the same software application.” According to Barker & Thornton (2004), integration has been proven to be the most important challenge for bioinformatics software over the next decade. However, the vast diversity of life science datasets, including genome sequences, literature, and software, can make integration seemingly impossible (Hannay et al., 2009; Lapatas et al., 2015). Problems can be faced when trying to handle the challenges of integrating data of differing qualities while incorporating sources of disseminated information (Miyakawa, 2020). Platform and system peculiarities can pose particular problems for integration due to interoperability concerns between them (Karasavvas, Baldock & Burger, 2004; Koru et al., 2007; Chilana, Palmer & Ko, 2009; Mangul et al., 2019). Differences between libraries from one operating system or version to another can result in varying outcomes when making system calls with software. If bioinformatics software is developed on a UNIX platform, for example, a developer will incorporate some libraries in the code that will enable it to operate on the UNIX platform (Gentleman et al., 2004). Public sharing of developed bioinformatics software, especially porting efforts, can help to address issues with system and platform variations. (Steinberg, 2003) asked bioinformatics engineers to create an online community in which they could share not only data but also developed software and stressed the value of reusing software by encouraging bioinformatics software be made open-source. Following Steinberg’s research, great standardization efforts have been developed and deployed. Standardized API documentation using OpenAPI (Mulligan, 2008) and BrAPI (Selby et al., 2019), ontologies for annotating bio tools (e.g. EDAM (Ison et al., 2013)), workflow systems including a great number in life sciences ranging from analytic platforms like KNIME (Fillbrunn et al., 2017) to Jupyter Notebooks (Kluyver et al., 2016), command-line workflow tools like Nextflow (Di Tommaso et al., 2017), and dependency systems like SnakeMake (Köster & Rahmann, 2012) have all aided the bioinformatics community. Additionally, containers such as Bioboxes (Belmann et al., 2015) and BioContainers (da Veiga Leprevost et al., 2017), packagers such as Bioconda (Grüning et al., 2018), and distributions (https://wiki.debian.org/DebianScience/Biology) have proven very useful. The Common Workflow Language (http://www.commonwl.org) is an effort primarily driven by the life science community. All of these efforts have helped to tackle the challenge of integration for scientific bioinformatics software.

With integration and reproducibility of bioinformatics software being so closely tied when performing data-driven research, embracing open approaches to managing software development have proven highly beneficial. It enables bioinformatics engineers to build on the work of other works while also allowing others to shape their efforts and help solve their collective problems. This leads to the establishment the software with greater longevity, more effective problem solving, and longer sustainability (Ivie & Thain, 2018).