Much attention has been given to the “data deluge” (Hey et al. 2009) and the need to render these data computable (Li and Chen 2014) in order to successfully address pressing societal challenges (Guo et al. 2015). The challenges are especially acute because of the large amounts of “long tail” and “legacy” data that are not findable, accessible, integratable, or reusable (Thessen and Patterson 2011, Heidorn 2008). One of the critical steps to unifying data is the application of data and metadata standards so that information can be effectively discovered, indexed, organized, and made ready for analysis (Schriml et al. 2020). Some subdisciplines within biology have worked to address this problem by developing various types of data and metadata standards (e.g., Field et al. 2008, Wieczorek et al. 2012). Projects like the Monarch Initiative (Shefchek et al. 2020), Planteome (Cooper et al. 2018), Translator (Fecho et al. 2022), the Global Biotic Interactions database (Poelen et al. 2014), the Global Biodiversity Information Facility (Telenius 2011), and many more use a wide variety of community-developed data, metadata, formatting, and exchange standards to integrate and make computable incredibly heterogeneous biology knowledge. The application of these standards has seen a steady increase in some disciplines, but the backlog of non-computable data remains vast (Marshall et al. 2018, Petty et al. 2020). There is a great need for tools and services that can scale up to create and maintain metadata for the quantity and variety of data in the long tail. The absence of computable metadata has severely impaired data discoverability in biodiversity (see Thessen et al. (2012b) for a specific example; also Walls et al. (2014), Mounce (2015)) and slows progress toward data-driven biology.

One unique aspect of biodiversity data is that scientific names can be used as near universal metadata (Patterson et al. 2010). There are rules of nomenclature that govern the use, representation, and modification of scientific names. Despite this standardization, names make poor identifiers because they are not unique or persistent identifiers for taxonomic concepts (such as species) and the continual nature of scientific discovery prevents them from ever being so. Names are not represented consistently in publications and data sources (Patterson et al. 2016, Page 2011). Adequately managing scientific names as metadata requires tools that automatically find names in their various forms in data sources (Gerner et al. 2010, Thessen et al. 2012a, Le Guillarme and Thuiller 2022, Pafilis et al. 2013) and generate computable metadata that allows for resolution of taxonomic concepts over time. This requires more than a standard. This requires a “living” metadata file that can be automatically updated in a transparent, traceable way. Such a file will help to incorporate more biodiversity data into the growing body of computable biological knowledge.

Here we examine the feasibility of living metadata in biodiversity using GNFinder, a tool that can find scientific names in text with a high degree of precision and recall and return the corresponding current, valid name in JSON or CSV format. GNFinder was developed with the goal of processing everything ever published and is currently being used by the Biodiversity Heritage Library (BHL) (Mozzherin and Myltsev 2017), the Encyclopedia of Life (Thessen and Parr 2014), HathiTrust (Mozzherin et al. 2022), and TaxonWorks. For example, GNFinder processes the 60 million pages in BHL in seven hours on a laptop computer and the 6 billion pages in the HathiTrust in less than a day on an HPC cluster (Mozzherin et al. 2022). While GNFinder records the exact location of a taxonomic name in text, this study examined the utility of GNFinder for adding current taxonomic names as document-level metadata to improve data discoverability.

This paper seeks to determine the efficacy of GNFinder for adding taxonomic metadata to the published literature. As a result, annotations are made and results reported at the document level. Multiple instances of the same name string in a document were only counted once.

Description of the Data

Description of GNFinder

GNFinder is a web service that uses a combination of naive Bayes, rules, and lists to find scientific names in text (Mozzherin 2022a, Mozzherin et al. 2018). The rules create features that the Bayesian algorithm uses to calculate a score (Fig. 1). There are two types of rules:

Figure 1.

Example output from GNFinder showing the results of the heuristic and statistical rules used by the naive Bayes algorithm to calculate the final score. In this example GNFinder identified the name Canis familiaris with high odds of being a taxonomic name based on the following criteria. This name is in two separate “go” lists (A and B). Both the genus and the specific epithet have endings that are common in Latin (C and D). The length of the specific epithet and the genus are within expected values (E and G). The name is not an abbreviation (F). All of these features were used by a naive Bayes algorithm to calculate the final “odds” score, in this case, 11.56. The Bayesian prior was set at 0.1 (H).

1. heuristic, based on a “stop” list containing terms that are likely to appear with, but not be a part of a scientific name (such as “environmental sample”), a “caution” list containing words that are frequently used in European languages that also appear in scientific names, and a “go” list containing terms that are highly likely to only be used in a scientific name as a genus or a specific epithet, and
2. statistical, for example, “are the word endings common in Latin”.

The score (result of naive Bayes) is represented as “odds” instead of a probability. GNFinder output can be configured to show the results from each of the rules and the final Bayesian score (Fig. 1).

Once GNFinder has recognized a name in the text, the name string is parsed into its semantic elements such as genus name, specific epithet, year of publication, authorship, etc. using GNParser (Mozzherin 2022b, Mozzherin et al. 2017). Parsing is essential for matching different variants of taxonomic names (such as Canis familiaris Linneaus and Canis familiaris L.). This is a process of name string “normalization” that can establish the canonical form of the name. Normalization is essential for finding the current, valid name according to a user-chosen taxonomic reference using GNVerifier (Mozzherin 2022c).

GNVerifier compares the name string found by GNFinder to names in a list of over 200 reference taxonomies (Mozzherin 2022d). The default setting looks in all available lists and prioritizes results from Catalogue of Life (Hobern et al. 2021). Users can choose to view all available matches or only the best match. GNVerifier compares seven features of the found name and the matched name to calculate a score used to rank the matches (Fig. 2). In this way, if GNFinder finds a name that is no longer in contemporary use, it may be able to return the current name. This process of matching old names to current names is referred to as name resolution.

Figure 2.

Example GNVerifier score matching “Canis familiaris” found name string to Canis lupus familiaris Linnaeus, 1758 in the Catalogue of Life. The final score (G) is calculated based on the following seven attributes and used to sort results: A) Are the names uninomials, binomials, or trinomials? B) Do the names share an infraspecific rank, such as variety or form? C) Do the names match exactly? D) How carefully curated is the source of the matched name? E) Does the author and year information match? F) Is the found name a synonym of the matched name?

GNFinder can be accessed directly through the webpage (Mozzherin 2022a) or by using the API (Mozzherin 2022e). Complete documentation of GNFinder and description of the JSON output can be found on GitHub (Mozzherin et al. 2018).

Data Preparation

Testing Performance

Assessment of Metadata Creation

Annotator Agreement

GNFinder Performance

Error Analysis

Comparison to Other Tools

Metadata Coverage Assessment

Data are rendered non-discoverable because of the ways taxonomic names change over time and because of the idiosyncratic ways in which names are expressed. The Global Names project recognizes that names may be expressed in various forms, and the infrastructure has been designed so that we can extend GNFinder to parse additional variant forms (Patterson et al. 2016). The results can be mapped to the canonical form of a name (i.e., the Latin binomial, genus name capitalized with a single space separating it from the species epithet, no annotations and no authority information), and then track the canonical form to a currently accepted name through an understanding of synonymies. This process is illustrated in this study and with the iPlant TNRS service that uses the GNParser (Boyle et al. 2013).

GNFinder can find scientific names in text and resolve name strings to a current name in a user-chosen list. This is also known as Named Entity Recognition (NER) and is a very active area of research in the Natural Language Processing and Machine Learning fields (Goyal et al. 2018). In addition to GNFinder, there are other algorithms that perform NER for taxa (see results above). LINNAEUS does an excellent job of finding species names in text and resolving those names to concepts in the NCBI taxonomy with very high precision (Gerner et al. 2010). TaxoNERD uses a deep neural network (DNN) to find mentions of taxa in text with very high recall (Le Guillarme and Thuiller 2022). Our intention with GNFinder is to balance precision and recall, as is suggested by these results. Comparing the results of these different algorithms was difficult and can be misleading. The ideal method for comparing algorithm outputs is to compare their results against a publicly-available gold standard corpus. Two potential corpora exist, COPIOUS (Nguyen et al. 2019) and S800 (Pafilis et al. 2013), but none of these algorithms have processed both. Additionally, the algorithms have subtle differences in the scope of their results.

The name-resolution function performed by GNFinder also serves as quality control for resources like BHL, which have used Optical Character Recognition (OCR) as part of the digitization process. OCR can introduce errors in names at rates that depend heavily on the language and typography used (historical texts are particularly vulnerable) (Wei Q et al. 2010). When first checked over a decade ago, approximately 30% of taxonomic name strings in BHL contained an OCR error (Freeland 2009). Since then, OCR errors have been greatly reduced in BHL (Anonymous 2014, Mika 2017), but the need to find and correct misspellings in large corpora at scale has not disappeared. GNFinder can address these errors by resolving novel misspellings through its use of naive Bayes. In this way, a one-of-a-kind, erroneous name string can be recognized and resolved to a current name.

Not all of the 11,692 unique name strings identified by human annotators were properly formed scientific names and their regular abbreviated forms. A properly formed scientific name, for the purposes of this paper, includes a binomial (Panthera leo), trinomial (Felis silvestris lybica), or higher level taxon name with or without the authority and the regular abbreviation (P. leo). This is important because the semi-supervised portion of GNFinder relies on the rules of nomenclature to identify scientific names in text. Out of all of the documented ways a taxonomic name can be represented (Patterson et al. 2016), there were four types of taxon name modification that reduced GNFinder performance.

1. Irregular abbreviation. Irregular abbreviations were scientific names shortened by any means except: a) the first one or two letters of the generic name with the first capitalized, b) followed by a full stop and a space, c) followed by the specific epithet. Often these included names with strain designations or location information. While regular abbreviations were identified by GNFinder, they were not resolved by GNVerifier.

2. Unusual punctuation or spacing. Unconventional spacing and punctuation can be used to represent hybrids, species complexes, or unofficial specific epithets such as Aus bus × cus, Aus bus/cus, and Aus “bus”.

3. Improper capitalization. Publications will sometimes contain a genus name that is not capitalized or a specific epithet that is capitalized, such as E. Caballo. GNFinder needs the capitalization to recognize the genus name and specific epithet.

4. Adding two letter qualifier abbreviations. Manuscripts often have qualifiers added to the names, such as cf. aff. or sp. When these abbreviations occur within the name string, GNFinder will not recognize the binomial. When they occur after the name string, such as Bos sp., GNFinder will include the sp. in the returned name string.

The benefits of including all mentioned taxa as metadata are unclear because a paper may be about one specific taxon, but mention several; so, including all mentioned taxa could lead to less precise document retrieval. Author-supplied keywords and algorithms that can detect keywords from text (Huang et al. 2020) can increase the precision of document search, especially for publications that are about a single taxon; however, keywords are limited to 5-10 and many studies produced data for more than 5-10 taxa. Additionally, keyword extraction algorithms, like NER-RAKE, are not designed for taxonomic name extraction (Huang et al. 2020). GNFinder includes in its results the “mainTaxon” and “mainTaxonRank” for each document, which can partially address this issue by reporting the lowest rank taxon that includes at least 50% of all the mentioned species.

The utility of including mentions of taxa above the rank of genus is also unclear. The argument against this is that parent taxa can be automatically added from an authoritative hierarchy when a taxon is detected; thus, keeping the search criteria broad enough to include them decreases precision of the algorithm and the document search unnecessarily. The arguments for this are the cases where only higher level taxa are mentioned and in the cases where more than one genus has the same name. A path forward is to add both types of higher level taxa (i.e., found and inferred) to the metadata file and label them appropriately.

These results suggest that the majority of relevant taxa are mentioned in the publication and thus searching the publication file will generate most of the needed taxonomic metadata for the accompanying data. This argues that the first priority for future GNFinder development should be improving the extraction of names from published manuscripts, especially proper handling of names in figures. It is known that not all data are published (Heidorn 2008, Shin et al. 2020), but the proportion of digitized data that do not have an accompanying publication is unclear. The data in Table 3 argues that identifying vernacular names and reading files that are not in .txt format should be the next priorities. Name detection in text written in languages other than English was not discussed here, but is already part of future development plans for GNFinder.

Taxonomic names are useful metadata for finding, accessing, integrating, and reusing data, but only if they can be effectively resolved when there are changes in taxonomy, or when a name represents more than one species concept. GNFinder demonstrates good overall performance on finding name strings that occur in text representing taxa across the tree of life. In this study, approximately 14% of names used in publications 10–20 years old were out-of-date and were mapped to a current, valid name by GNVerifier. Furthermore, the speed of GNFinder makes it possible to apply names as living metadata to the entire body of published literature. Without name-finding algorithms, much biological content cannot be accessed by searches based on the taxon name. The use of GNFinder to tag files with appropriate taxonomic metadata improves discovery on an unprecedented scale. The major advance of GNFinder is the almost unlimited scalability and reliability, while still preserving reasonably high quality of name detection.

This work was supported by NSF grants ABI 1062387 Collaborative Research: ABI: Innovation: The Global Names Architecture, an infrastructure for unifying taxonomic databases and services for managers of biological information and ABI 1356347 ABI Development: Global Names Discovery, Indexing and Reconciliation Services. We thank Lakshmi Akella and Patrick Leary for access to software.

NSF ABI 1062387 and 1356347

Collaborative Research: ABI: Innovation: The Global Names Architecture, an infrastructure for unifying taxonomic databases and services for managers of biological information and ABI Development: Global Names Discovery, Indexing and Reconciliation Services.

Contributions made by David Shorthouse represent work initiated prior to his employment with Agriculture and Agri-Food Canada