Publications in biology, especially those related to biodiversity, generally include images to portray habitus and morphological structures of the organisms that are the subject of those works. An increasing number of repositories are making available large collections of digital images of organisms and their parts, drawing on sources ranging from natural history collection digitization to citizen science platforms. These images are increasingly used as a source for many forms of research, including morphological measurements, identification of individual organisms, and automated taxonomic identification. Especially as repositories become larger and research more automated, usefulness of these images for research will increase greatly because of the ability to describe the depicted part of an organism and its orientation using a controlled vocabulary.

Morphbank :: Biological Imaging*1 is an online repository project that began in 1998 and expanded after receiving funding from the National Science Foundation in 2003 to eventually include over 216 000 images of biological specimens and living organisms*2. A key feature of Morphbank was the ability to define characteristics of the image according to the part of the organism depicted, the orientation of the part (or the organism) in the image, and the imaging technique. The set of characteristics of the image was called a "view". Image uploaders could select from an existing view, or define a new view if the characteristics of their image differed from available existing views. This provided great flexibility, but also resulted in a great proliferation of views, eventually reaching over 17 000 possible views*3. Recognizing this problem of proliferation of views, Baskauf and Kirchoff (2008) attempted to standardize views for live plant images, resulting in the creation of collections of recommended Morphbank views for four categories of live plants*4.

The Audubon Core Multimedia Resources Metadata Schema, often referred to simply as "Audubon Core" and abbreviated as AC, is a set of metadata vocabularies for describing biodiversity-related multimedia resources and collections (Morris et al. 2013a, Morris et al. 2013b). Morphbank was a key stakeholder in the development of Audubon Core, and when Audubon Core was originally ratified as a standard, it contained two terms that captured key aspects of Morphbank views: ac:subjectPart (http://rs.tdwg.org/ac/terms/subjectPart), defined as "The portion or product of organism morphology, behaviour, environment, etc. that is either predominantly shown or particularly well exemplified by the media resource." and ac:subjectOrientation (http://rs.tdwg.org/ac/terms/subjectOrientation), defined as "Specific orientation (= direction, view angle) of the subject represented in the media resource with respect to the acquisition device." When these terms were introduced, it was recognized that there was no formal controlled vocabulary for values of the terms, and it was assumed that the values would be text strings.

Soon after the formation of the Audubon Core Maintenance Group in 2018, it chartered a Views Task Group*5 whose task was to develop controlled vocabularies that conformed to the TDWG Standards Documentation Specification (SDS; Baskauf et al. 2017b) for the terms ac:subjectPart and ac:subjectOrientation. One consequence of following the guidelines of the SDS was distinguishing between controlled strings and the internationalized resource identifiers (IRIs) that denote the concepts associated with those strings. The convention established in Audubon Core was to have pairs of analogous terms where one term intended for use with IRI values had a simple local name (e.g., ac:reviewer) and an analogous term intended for use with controlled strings had the same local name with "Literal" appended (e.g. ac:reviewerLiteral). To conform to this convention, the 2022-02-23 version of Audubon Core revised the definitions of ac:subjectPart and ac:subjectOrientation to clarify that their values are denoted by IRIs and created new terms ac:subjectPartLiteral and ac:subjectOrientationLiteral to be used with controlled string values*6.

As a coordinated addition to Audubon Core, the Views Task Group has followed the process outlined in Section 4 of the TDWG Vocabulary Maintenance Specification (VMS; Baskauf et al. 2017a). Specifically, the Task Group generated a Feature Report based on input from the community and this paper comprises the Implementation Experience Report produced by the Task Group to satisfy the requirements of Section 4.2.2 of the VMS. The purpose of this report is to demonstrate that the desired features of the vocabularies generated from use cases submitted by the community were implementable by the contributing testers. The hope is that this report will assist the Audubon Core Maintenance Group and the TDWG community in the review process during ratification of the controlled vocabularies.

After acceptance of the Task Group charter, the group began to hold regular meetings. The work carried out at those meetings was documented in a series of meeting notes*7.

The initial task of the group was to solicit use cases from the community. Responses were received from image producers and consumers (including biodiversity researchers) and representatives of image aggregators. Contributors followed a template that was provided and the submitted use cases were organized by categories*8. The assembled use cases were discussed, then used by the Task Group to generate a candidate list of requirements for the controlled vocabularies*9. In this initial screening, some of the desired outcomes identified from the use cases were determined to be out of scope or not practical to satisfy within the time frame of the Task Group's work. This candidate list of requirements comprised a first version of the Feature Report described in Section 4.2.1 of the VMS.

During the course of the Task Group's work, it became apparent that subjectPart designations could apply to only part of an image if the image contained multiple parts or even multiple organisms. Standardizing the description of regions of interest (ROIs) was not in the scope of this Task Group, but the eventual addition of an ROI vocabulary to Audubon Core *10facilitated the Task Group's work by making it possible to apply subjectPart and subjectOrientation terms to specific ROIs within an image rather than only to the image as a whole.

Following the generation of candidate requirements, the Task Group reviewed existing approaches to describing subject parts and orientations, and identified ontologies that could be used to define parts and orientations unambiguously. Then work began on the actual construction of the two controlled vocabularies in the form of Simple Knowledge Organization System (SKOS) concept schemes (Isaac and Summers 2009), one describing parts and the other describing orientations.

The concept schemes were relatively unstructured, except for a few cases where concepts had a broader relationship to other concepts (for example, "right" orientation had the broader concept "lateral" orientation; Table 1). One of the challenges in developing the concept schemes was meeting two of the candidate requirements involving categorization of the concepts: 1.1 "Subject part values are grouped appropriately for broad categories of organisms (e.g., trees, quadrupeds)." and 5.2 "Subject orientations are grouped appropriately for subject parts." Categorization was established by creating SKOS collections (Isaac and Summers 2009), where one set of collections indicated which subject part concepts were suitable for various organism groups, and another set of collections indicated which orientations were suitable for various subject parts (Fig. 1).

Figure 1.

Relationships among components of the controlled vocabularies.

During the construction of the controlled vocabularies, it became apparent that although satisfying some of the candidate requirements would be desirable, doing so would not be practical. In some cases, meeting a requirement would have made the vocabularies too complex for most users. In other cases, it would have made some of the concepts so granular that few users would use them and the size of the vocabularies would be impractically large. As a result, a large number of the candidate requirements were dropped prior to the creation of the final list of requirements (i.e., the final version of the VMS Feature Report). There were two cases where the candidate requirements from the Task Group helped drive other developments within Audubon Core:

• The term sawsdlrdf:modelReference was borrowed from the W3C Semantic Annotations for WSDL and XML Schema Recommendation*11 as a means to link terms with external definitions. In the context of the Views Task Group, this term was used to link concepts to terms in the external ontologies. (Candidate requirement 6-ANATOMY-1)
• The creation of the ac:RegionsOfInterest class and the associated terms used to describe its instances made it possible to apply subjectPart and subjectOrientation properties to parts of images in cases where the image depicted multiple organisms or multiple parts. (Candidate requirement 2-FILTER-1)

The final requirement list is presented in Appendix A. Of the seventeen potential requirements derived from the submitted use cases, seven were included in the final requirements.

During the development process, ac:subjectPart and ac:subjectOrientation were added as custom metadata fields in Zenodo. Controlled string values appropriate for insects from the preliminary vocabularies were used to categorize images of fly specimens when they were submitted. Fig. 2 shows an example specimen*12 image*13 that includes these fields. Including values for these fields makes it possible to search images to retrieve all images of a particular subjectPart (e.g., https://zenodo.org/search?page=1&size=20&custom=%5Bac:subjectPart%5D:%5Bhead%5D) or a particular subject part in a particular orientation (e.g., https://zenodo.org/search?page=1&size=20&custom=%5Bac:subjectPart%5D:%5Bhead%5D&custom=%5Bac:subjectOrientation%5D:%5Bventral%5D). At that time, IRIs had not yet been assigned to candidate terms nor had the "Literal" term analogs been adopted. So this was a preliminary test that did not include all of the features of the final submitted draft vocabularies. Nevertheless, this demonstrated the practicality of using the controlled vocabulary terms to sort out images having a particular view from a larger set of specimen images.

Figure 2.

Specimen image https://zenodo.org/record/6084051 showing an anterior view of the head of a fly, Lachnocorynus stenocephalus. Image used under a CC BY license. Dikow, Torsten. (2022). Lachnocorynus stenocephalus Boschert and Dikow, 2021, head, anterior. Zenodo. https://doi.org/10.5281/zenodo.6084051

After the completion of the draft controlled vocabularies, the Task Group began planning for implementation testing by identifying potential testers and creating an implementation testing guide (Suppl. material 1). The group invited participation from researchers specialized in diverse groups of organisms that included non-vascular plants and non-arthropod invertebrate animals in addition to the vascular plants and arthropods that were the primary focus of the vocabulary development, with the intention of judging how extensible the terms were.

The group recognized that not every tester would use the vocabularies in the same way, so the guide described testing by manual entry, machine-guided entry, and machine processing. The guide also included the questions that would be asked on the feedback form at the conclusion of testing so that the participants would have a better idea of what to consider while carrying out the testing. The vocabularies were also made available to the testers as CSV tables*14, 15, text documents*16, 17, and machine-readable JSON-LD files*18, 19.

After potential testers were identified, the Task Group held an optional workshop that would give the testers an opportunity to try applying the terms to a small number of images while they could ask clarifying questions to the workshop presenters. Testers then carried out testing on a larger sample of images.

After completing the testing, implementers submitted feedback using a Google form (Suppl. material 2). The Task Group reviewed the feedback and in some cases considered modifications to the vocabularies to address the concerns raised by the test implementers. When the revisions were complete, the labels and definitions were translated into Spanish.

There were five institutions participating in the implementation testing (Table 2).

Manual entry

Machine processing

Unimplemented requirements

The testing included some organism groups that were not included in the original set: cnidarians and bryophytes.

The testing suggested that the existing terms were not adequate when applied to cnidarians. Although "lateral side" was an appropriate description, "dorsal side" and "ventral side" did not make sense when applied to non-bilaterally symmetric animals. Therefore, two additional orientations: "oral side" and "aboral side" were added to make the vocabularies useable with groups like cnidarians and echinoderms.

The existing terms developed for seed plants were not adequate for all bryophyte images. During the testing period, additional terms were proposed for bryophytes, ferns, and fungi. Since there was insufficient time for these new terms to be tested in order for them to be included in the initial submission, they were designated as candidate terms to be considered as future additions to the vocabularies*20.

One result of the discussion about using the vocabularies with these other groups was changing the subjectOrientation terms for "adaxial side" and "abaxial side" to "upper side" and "lower side". That made them more broadly usable in organism groups that did not have a clearly defined central axis. It also made the terms easier for human users to select without error, since the strings "adaxial" and "abaxial" are more difficult to distinguish due to their visual similarity.

Controlled vocabularies are necessary for standardizing the way we manage and process information. By developing these views vocabularies, we provide a means to describe the content of images by manual entry, machine-guided entry, and machine processing. Test implementers were able to use the terms proposed without major issues. Although the controlled vocabularies for subjectPart and subjectOrientation were not able to handle every scenario imagined in the original request for use cases, they fulfilled most of the final requirements established by the Task Group (the "Feature Report" required by the SDS).

The usefulness of these controlled vocabularies would be increased if they were incorporated within tools for manually demarcating or automatically detecting regions of interest that correspond to subjectParts depicted in an image. We hope that developers will take advantage of this opportunity to associate parts of images with machine-readable metadata for describing what is depicted in those parts.

Moving forward, when the vocabularies are adopted and broadly used, we expect that these will expand over time to include subjectParts that were not tested in this implementation. Through the TDWG term change process (Baskauf et al. 2017a), we expect suggestions from a diversity of users spanning across biological groups that have not been tested yet. Those additions should be submitted for testing, but we also acknowledge that finding test implementers can represent a challenge.

One reason for using SKOS to describe the controlled vocabularies is that it provides a mechanism for enriching them to make them more broadly usable. As we noted, SKOS collections have been used to group the concept terms in meaningful ways and skos:prefLabel has been used to add labels and definitions in Spanish. In the future, labels and definitions in other languages may be added as translations are completed and skos:altLabel may be used to document alternative labels for which users may search.

Overall, the results of implementation testing demonstrated that the vocabularies are ready for adoption and inclusion as part of the Audubon Core standard.

Note: source use cases*8 follow a requirement (in parentheses).

Subject part

1 Categorization

1.1 Subject part values are grouped appropriately for broad categories of organisms (e.g., woody angiosperms, insects). Selecting a SKOS Collection will allow a user to find a group of part concepts appropriate for a particular category of organisms. (1-CATEGORIZE-1)

1.2 Concepts are linked to well-known ontologies to clarify definitions and standardize labels. However, the actual concepts are TDWG-adopted terms, providing stability that might not exist in the source ontologies. (6-ANATOMY-1) Ontologies used were the Biological Spatial Ontology (BSPO)*21, Phenotype and Trait Ontology (PATO)*22, Common Anatomy Reference Ontology (CARO)*23, Plant Ontology (PO)*24, Uber-anatomy Ontology (UBERON)*25, Drosophila gross anatomy Ontology (FBBT)*26, and Ontology for the Anatomy of the Insect SkeletoMuscular system (AISM)*27.

1.3 Concepts allow for distinguishing between sexes (if multimorphic) by selecting narrower categories of subjectPart. (added during discussion)

1.4 Specify multiple parts in an image by applying subjectPart concepts to Regions of Interest within an image. (2-FILTER-1)

1.5 Distinguish between single and aggregate parts (e.g. one vs. several leaves) by applying multiple subjectPart concepts of the same type to Regions of Interest within a single image. (7-CLARITY-2)

2 Relationship between part and orientation

2.1 Determine what orientations are appropriate for subject parts other than whole organism. (3-MEASURE-4, 1-CATEGORIZE-2)

2.2 For some organism groups, filter orientations so that selection is only possible if the feature is visible for a particular subject part. (8-ORIENT-1)

Results from specific testers to questions on the feedback form are included in Table 3.

Specific comments about test images from the Bioimages test are included in Table 4.

Specific comments about test images from the California test are included in Table 5.

The Task Group thanks the Audubon Core Maintenance Group for their oversight and support during the vocabulary development process. Martin Stein was part of the original Task Group but was not able to continue working with the group. Torsten Dikow provided valuable feedback and examples during the vocabulary development process. Tomomi Suwa participated in the implementation testing on behalf of the Field Museum. John Oswald engaged the Task Group with interesting ideas from his work categorizing images that did not end up being incorporated as features of the vocabularies. David Fichtmueller, Sharif Islam, and Doug Palmer provided useful comments for improving the paper through their reviews.

SJB, JCGD, and MN wrote the manuscript and were core task group members. NSC and RS were core task group members. KCS, ZK, MP, DA, and AMLK were implementation testers. DA also participated in the task group during the development phase.