BioAssay templates for the semantic web

Data model

The semantic description of templates and annotations uses a small number of additional URIs, each of which has the root stem http://bioassayontology.org/bat, and is denoted using the Turtle-style⁵ abbreviated prefix “bat.”

The hierarchical model for describing a template is shown in Fig. 3. Parent:child relationships denoted by an arrow indicate one-to-many relationships, while the properties listed in the boxes underneath the nodes are one-to-one relationships. A template definition begins with the root, which is distinguished by being of type bat:BioAssayTemplate. The root is also of type bat:Group, and has some number of child nodes, which are themselves either assignments or subgroups.

An assignment node has several scalar properties, including label and description, and it also refers to a property resource. These are typically mapped to URI resources found within the BAO (e.g. http://www.bioassayontology.org/bao#BAO_0000205, label: “has assay format”). Each assignment has some number of values associated with it, and these make up the list of available options. Each value is primarily identified by the resource that it maps to, which is typically found in the BAO (e.g. http://www.bioassayontology.org/bao#BAO_0000219, label: “cell based format”). Besides the label and description, which are customizable within the template data model, the reference URI has its own implied class hierarchy (e.g. “cell based format” is a subclass of “assay format”), which is not encoded in the template data model, but is inferred once it is paired with the BAO and its linked ontologies.

The schema for annotation of assays is shown in Fig. 4. The assay is given a distinct URI, and is associated with several properties such as label and description. The template is recorded, as is an optional reference to the origin of the assay (which may be a semantic web resource, or a DOI link to a journal article). The free-text description of the assay can also be recorded using the hasParagraph predicate.

The assay is associated with some number of annotations, which are primarily linked to assignments within the corresponding template. For annotations that assert a URI link, the hasValue predicate typically corresponds to one of the available values that was prescribed for the assignment in the template definition, and generally refers to a term defined in the BAO, though custom references can be used–or the annotation may be specified using the hasLiteral predicate instead, which means that the user has entered data in a different form, typically text or a numeric value. The hasProperty predicate is generally copied from the corresponding assignment.

When annotating an assay, each assignment may be used any number of times, i.e. zero instances means that it has been left blank, while asserting two or more triples means that all of the values apply. The relationship between assays and annotations has no nesting: the intrinsic group/sub-group structure of any particular annotation can be inferred from the template, since the usesTemplate and isAssignment predicates refer to the origins in the template.

Software

The BioAssay Schema Editor is available from GitHub (https://github.com/cdd/bioassay-template) and may be used under the terms of the Gnu Public License 2.0.⁶ The code is written using Java 8, and the user interface is based on JavaFX. Semantic web functionality is implemented by incorporating the Apache Jena library.⁷ The project includes a snapshot of the BAO⁸ and some of the linked ontologies, as well as the latest version of the CAT schema. It should be assumed that the project will continue to evolve until well after the publication date of this article.

The application operates on a datafile referred to as a schema, which is represented as a collection of triples (in Turtle format, with the extension .ttl). A schema is expected to include a single template, for which the root node is of type bat:BioAssayTemplate, and may optionally contain any number of assays that have been (or will be) annotated using that same template. Triples are used as the serialization format in order that the editable files can be used as-is by a Triple store, and become a part of the semantic web with no further modification.

Figure 5 shows the main window for the application, which has loaded a contemporary version of the CAT, and has several accompanying assays awaiting annotation. The components that make up the template are shown as a hierarchy on the left hand side of the panel. Selecting any of the groups or assignments causes the detail view on the right to be filled in with the corresponding content.

Adding, deleting, renaming etc. of groups, assignments and values is fairly mundane, and follows standard desktop user interface design patterns. Selecting URI values for properties and values requires a more specific interface, and is composed by summarizing the BAO vocabulary, which is loaded into the application at the beginning. Resources can be selected using a dialog box that can present the list of options in a flat list, with an optional search box for restricting the list (Fig. 6A) or by using the hierarchy view that shows the position in the BAO ontology (Fig. 6B). The dialog box can also be used to add multiple values at once, which is particularly convenient when a branch of the BAO encompasses multiple terms that are all valid options. When a resource is selected, its label and description are imported from the BAO into the template: these values can be edited after the fact, but by default they are the same as in the underlying vocabulary.

The primary role of the schema editor is to provide a convenient way to edit templates, but in support of this goal, it also provides an interface to use the template to annotate assays. The interface can be used for generating training data (e.g. for model generation), but it is mainly intended as a way to ‘test drive’ the current template. Because the annotation process is directly derived from the template, having the two editing processes side by side is advantageous when the template is being designed. For example, the operator can begin annotating an assay, and if a value is missing from one of the assignments, or a new kind of assignment turns out to be necessary, this can be added to the template within the same editing session.

Figure 7A shows an example of an assay that has been annotated. The detail view has a placeholder for description text, which is particularly useful when the content has been imported from some external source, and the annotations are being made by converting the protocol text into semantic annotations. Clicking on any of the annotation buttons brings up a panel of options (Fig. 7B) that represent the prescribed values for the assignment. Each of the assignments can be left blank, annotated once, or given multiple values. The ideal use case is when the value (or values) occurs within the list of prescribed values, but since the data model allows any URI, the user interface also allows the user to insert a custom URI. In cases where no URI is listed in the template (e.g. a concept that does not have an established URI), it is possible to add plain text for any of the assignment annotations. While this has no meaning from a machine-learning point of view, it can serve as a convenient placeholder for terms that will be invented in the future.

Templates

We set out to create a CAT that includes the basic details essential to defining any bioassay: assay type, format, target and biology, results and pharmacology, and other details. The CAT was developed with the opposing goals of identifying assignments that (1) would be limited in number in order to be not overly burdensome vs. (2) comprehensively cover the majority of the information contained in written descriptions of bioassays. We also considered the type of information that would be utilized by an end user attempting to search, filter, and aggregate assays by their bioassay annotations. For example, details such as the assay footprint (plate type), assay kit, and detection instrument were included because they may be useful terms for identifying experimental artifacts. Biological process and other target-related information were included to enable aggregating results across similar drug discovery projects for model-building and other applications. Finally, we limited assignments to those where the BAO offered sufficient options for possible values. Since the goal of the project is to generate machine-readable assay annotations, we avoided assignments where BAO terms were not available, such as those characterizing in vivo assays, and especially assignments whose values would be very specific for each assay, such as negative and positive controls. These areas will be addressed in the future once the underlying vocabulary (BAO or otherwise) is available sufficient to expand the domain. Similarly, the CAT falls short of capturing detailed protocol steps. In its present incarnation, it cannot be considered as a complete replacement for the text that is typically used to describe an assay, though we do intend to pursue this level of detail in future work. For the present, we are primarily concerned with utilizing the rich vocabulary within the BAO to achieve maximum impact with minimum additional burden on the end user workflow.

To develop the CAT, we used the following process: first, biologists independently considered each of the terms available in the BAO and prioritized assignments for the CAT. Each assignment was associated with a number of possible values based on the BAO hierarchy. Then, quantitative and qualitative approaches were used to determine if the prioritized assignments included in the CAT were sufficient to fully describe most assays. For the quantitative approach, we assessed the set of 1066 PubChem bioassays (Wang et al., 2014) that were previously annotated by hand by BAO experts (Schürer et al., 2011). In that exercise, the BAO experts aimed to fully annotate each assay, capturing all applicable information for more than a hundred different categories or terms. If there was not an applicable value, the assignment or category was left blank. We analyzed the use of the BAO terms to assess the utility and comprehensiveness of the assignments included in the CAT compared to the remaining terms. We found that the 16 CAT assignments were annotated in 81% of the 1066 PubChem assays compared to 33% for the remaining terms. We also found that 95% of the values for CAT assignments were BAO terms rather than literal or non-URI based terms, compared to 63% in the remaining categories. These results suggested that the CAT includes assignments that are both relevant to the majority of assays as represented in PubChem and well covered by the BAO.

For an in-depth qualitative assessment of the CAT, biologists annotated a wide variety of assays, encompassing different assay types (e.g., cell viability, enzyme activity, binding, and ADMET), assay formats (e.g., cell-based, biochemical, microsome, organism, tissue, etc.), and assay design methods (e.g., ATP quantitation, cell number, immunoassays, gene expression, radioligand binding, etc.), as summarized in Table 1. We found that in many cases, both from assay descriptions available from PubChem and from in-house screening assay descriptions, the CAT captured much of the relevant information. For example, annotating an assay for cell viability (PubChem ID 427) shows that all but two of the 16 CAT assignments are readily annotated from the short descriptive information provided (Fig. 8). ‘Target’ is left blank, as it is not applicable (this assay aims solely to identify cytotoxic compounds); ‘Detection Instrument’ was not noted. Similarly, as shown in Fig. 9, all applicable CAT assignments (15 of the 16) are annotated from the description of a competitive binding assay (PubChem ID 440). Figure 9 also illustrates that multiple values can be annotated for a single assignment, enabling content from complex assays to be captured. Together, these two examples highlight that both cell-based and biochemical assays can be extremely well-suited to be annotated using the CAT.

However, there were some cases where the CAT was less effective in capturing important information. For example, 14 of the 16 CAT assignments could be annotated for PubChem ID 488847, some with multiple values; however, the ‘big picture’ view of this rather complex primary assay is not as readily apparent from its ‘CAT profile’ as from a single sentence in the description (Fig. 10). In addition, this PubChem record had extensive technical details such as reagent components, liquid handling volumes and instruments, times of incubation and plate processing steps, which could be important for identifying matching assays or interpreting the results. Another example of a poor fit for the CAT, as noted earlier, are in vivo assays. These are largely beyond the scope of this effort, which is currently constrained to terms defined by the BAO: key parameters such as route of administration, dose, dose units, type of model (e.g. xenograft, disease) are not well represented. These and other limitations will be addressed in the future by adding or extending the underlying ontologies.

Finally, as noted earlier, we designed the CAT to be a ‘one-size-fits-most’ template. A summary of assignments for the complete set of assays annotated in the course of developing the CAT shows we have achieved this (Table 1). One consequence of this ‘one-size-fits-most’ strategy is that certain attributes (such as those highlighted in green or red in Figs. 8 and 9) have been omitted. Depending on one’s perspective, these types of data (such as positive and negative controls, data processing/normalization steps, relevant disease indication, and specific protocol details such as pre-incubation of compounds with the target, time or temperature of an assay) could be viewed as essential. We decided to exclude this type of information from the CAT because of irregularity of appearance in bioassay descriptions, the lack of coverage by the BAO, or incompatibility with the current data model. Expanding into this area is an opportunity for future development, and it should be noted that the CAT may be used as a starting point for templates that provide a set of assignment options that are customized for subcategories of assays, or even specific projects. We believe the next immediate step should be to apply our CAT to a large (> 10,000) set of assays, both to facilitate new meta-analyses and to identify potential gaps in annotation revealed by such studies.

PubChem

Possibly the most voluminous source of openly accessible bioassay data can be found on PubChem, which hosts more than 1.1 million assay records at the time of publication, and is growing rapidly. These are individually associated with the chemical structures of the compounds for which the measurements were made. Each of the assays is decorated with several descriptive fields that are essentially plain text, and which are populated by contributors during the upload process, or in some cases by an import script transferring data from other sources. While many of the entries contain a significant amount of detail, the phrasing style and level of detail varies considerably, often erring on the side of too little or too much information about the assay protocol.

Nonetheless, the PubChem assay collection represents one of the best and most convenient sources of data for annotation purposes, and for this reason we have added a feature to the BAT editor that explicitly searches for PubChem records, as shown in Fig. 11.

The dialog box allows the user to type in a PubChem Assay ID number, or to hit the button labelled Random, which picks an arbitrary assay from the entire collection, and fills in the corresponding text and URI of origin. While a large proportion of assays loaded into PubChem contain only sparse tags about the data source, or the abstract of the corresponding publication, there are a significant number of records that contain lengthy descriptions of the assay. The dialog box provides an opportunity for the user to tidy up the text (e.g. removing irrelevant content) prior to importing it into the schema. The content is then added to the list of assays being annotated within the schema model, whereby the origin is recorded as a link to the assay, and the text is associated using the hasParagraph predicate. Once the text is augmented with annotations using the current template, it becomes a useful entry for training data. This is one of our main strategies for generating a corpus of data for machine-learning purposes, which will ultimately find its way into a user friendly ELN for bioassay annotation.

Analysis

Because the data model we describe is based on semantic web triples, and the file format that is used by the BioAssay Schema Editor is made up of triples (in Turtle format), it means that any templates and assay annotations can be loaded directly into a triple store database, and queried using SPARQL queries. Content can be hosted on private servers for local use, or it can be exposed to the greater web of connected data. Supplementary Information 1 describes a configuration script for the open source Apache Fuseki Jena server which can be used to load the BAO, its related ontologies, and some number of files saved with the BioAssay Schema Editor, which can then be served up as read-only content.

Once the content is available via a SPARQL endpoint, there are a number of boilerplate queries that can be used to extract summary and specific information. Fetching a list of all bioassay templates can be accomplished using the following query:

PREFIX bat: <http://www.bioassayontology.org/bat#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT ?template ?label ?descr WHERE
{
    ?template a bat:BioAssayTemplate ; rdfs:label ?label .
    OPTIONAL {?template bat:hasDescription ?descr .}
}

The above query identifies any resource that is tagged as having the BAT type. Obtaining information about the assignments that are associated with a template can be done by looking for resources of type Group that are associated with it. Obtaining a summary list of assignments that are attached to the top level (i.e. not within a subgroup) can be accomplished with a query similar to the following (using the same prefixes as above) which explicitly references the CAT:

SELECT ?assn ?label ?descr ?property ?numValues
{
    <http://www.bioassayontology.org/bas#CommonAssayTemplate>
                       bat:hasAssignment ?assn .
    ?assn a bat:Assignment ;
        rdfs:label ?label ;
        bat:hasProperty ?property .
    OPTIONAL {?assn bat:hasDescription ?descr .}
    {
        SELECT ?assn (COUNT(?value) as ?numValues) WHERE
        {
            ?assn bat:hasValue ?value .
        }
        GROUP BY ?assn
    }
}
ORDER BY ?label

Similarly, assignments with one level of nesting can be obtained with a slightly longer query, which explicitly inserts a subgroup in between the template and assignment:

SELECT ?group ?glabel ?assn ?label ?descr ?property ?numValues
{
    <http://www.bioassayontology.org/bas#CommonAssayTemplate>
                       bat:hasGroup ?group .
    ?group a bat:Group ;
        rdfs:label ?glabel ;
        bat:hasAssignment ?assn .
    ?assn a bat:Assignment ;
        rdfs:label ?label ;
        bat:hasProperty ?property .
    {
        SELECT ?assn (COUNT(?value) as ?numValues) WHERE
        {
            ?assn bat:hasValue ?value .
        }
        GROUP BY ?assn
    }
}
ORDER BY ?glabel ?label

To query for information about the prescribed values for assignment (in this case the bioassay assignment from the CAT), the following query can be used:

SELECT ?property ?value ?label
{
    <http://www.bioassayontology.org/bas#Bioassay>
        bat:hasProperty ?property ;
        bat:hasValue
        [
            bat:mapsTo ?value ;
            rdfs:label ?label
        ] .
}

The query specifically pulls out the property field, which is typically a link into the BAO property terms, and the value field, which is typically a link into the BAO classes. Pursuing either of these resources provides a wealth of implicit information, partly from the hierarchical nature of the BAO terms, and the unlimited opportunities for these terms to be linked to other semantic resources.

To obtain a list of assays that have been annotated using one of the templates, the following query can be used:

SELECT ?assay ?label ?descr ?template WHERE
{
    ?assay a bat:BioAssayDescription ;
        rdfs:label ?label ;
        bat:usesTemplate ?template .
    OPTIONAL {?assay bat:hasDescription ?descr .}
}

Obtaining all of the annotations for such an assay can be done with:

SELECT ?assn ?label ?property ?value ?literal ?group WHERE
{
    <http://www.bioassayontology.org/bas#ExampleAssay>
                       bat:hasAnnotation ?annot .
    ?annot bat:isAssignment ?assn ;
        rdfs:label ?label ;
        bat:hasProperty ?property .
    OPTIONAL {?annot bat:hasValue ?value}
    OPTIONAL {?annot bat:hasLiteral ?literal}
    ?group a bat:Group ; bat:hasAssignment ?assn .
}

Because annotations are directly attached to an assay description, hierarchical information about the nature of the assignment can be obtained by further investigating the template definition of the assignment (?assn) or either of the linked BAO terms (?property and ?value).