Abstract

In recent years, the appearance of pre-trained language models has boosted the performance of several Natural Language Processing (NLP) models, achieving state of the art in many NLP tasks. Previous work in Named Entity Recognition (NER) has shown that using sentence-level context is not always enough to obtain high-quality contextualized representations while using document-level context contributes to significant improvements in the task. In this paper, we compared the performance of several domain-specific and general-domain language models to identify species mentions on the LivingNER shared task. Specifically, we fine-tuned these models using document-level context with the FLERT approach, which consists of creating the representation based on the context of the actual sentence and its neighboring sentences. Then, to obtain the codes of each entity mention, we used the output of the FLERT model and a Levenshtein distance-based approach. Finally, we trained NER models for real clinical use cases using a similar two-step system and combined these results to perform document-level classification and coding. Our submission results show that our models' performance is far superior to the average of other systems proposed, thus being an important contribution to species recognition and normalization. To reproduce our experiments, the source code of the system is freely available at https://github.com/plncmm/flert-matcher.