Cookies Policy
The website need some cookies and similar means to function. If you permit us, we will use those means to collect data on your visits for aggregated statistics to improve our service. Find out More
Accept Reject
  • Menu
Publications

Publications by Daniel Bouçanova Loureiro

2019

Language Modelling Makes Sense: Propagating Representations through Word Net for Full-Coverage Word Sense Disambiguation

Authors
Loureiro, D; Jorge, AM;

Publication
57TH ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS (ACL 2019)

Abstract
Contextual embeddings represent a new generation of semantic representations learned from Neural Language Modelling (NLM) that addresses the issue of meaning conflation hampering traditional word embeddings. In this work, we show that contextual embeddings can be used to achieve unprecedented gains in Word Sense Disambiguation (WSD) tasks. Our approach focuses on creating sense-level embeddings with full-coverage of WordNet, and without recourse to explicit knowledge of sense distributions or task-specific modelling. As a result, a simple Nearest Neighbors (k-NN) method using our representations is able to consistently surpass the performance of previous systems using powerful neural sequencing models. We also analyse the robustness of our approach when ignoring part-of-speech and lemma features, requiring disambiguation against the full sense inventory, and revealing shortcomings to be improved. Finally, we explore applications of our sense embeddings for concept-level analyses of contextual embeddings and their respective NLMs.

2019

LIAAD at SemDeep-5 Challenge: Word-in-Context (WiC)

Authors
Loureiro, D; Jorge, A;

Publication
Proceedings of the 5th Workshop on Semantic Deep Learning, SemDeep@IJCAI 2019, Macau, China, August 12, 2019

Abstract

2020

MedLinker: Medical Entity Linking with Neural Representations and Dictionary Matching

Authors
Loureiro, D; Jorge, AM;

Publication
Advances in Information Retrieval - 42nd European Conference on IR Research, ECIR 2020, Lisbon, Portugal, April 14-17, 2020, Proceedings, Part II

Abstract
Progress in the field of Natural Language Processing (NLP) has been closely followed by applications in the medical domain. Recent advancements in Neural Language Models (NLMs) have transformed the field and are currently motivating numerous works exploring their application in different domains. In this paper, we explore how NLMs can be used for Medical Entity Linking with the recently introduced MedMentions dataset, which presents two major challenges: (1) a large target ontology of over 2M concepts, and (2) low overlap between concepts in train, validation and test sets. We introduce a solution, MedLinker, that addresses these issues by leveraging specialized NLMs with Approximate Dictionary Matching, and show that it performs competitively on semantic type linking, while improving the state-of-the-art on the more fine-grained task of concept linking (+4 F1 on MedMentions main task). © Springer Nature Switzerland AG 2020.

2018

Affordance Extraction and Inference based on Semantic Role Labeling

Authors
Loureiro, D; Jorge, A;

Publication
Proceedings of the First Workshop on Fact Extraction and VERification (FEVER)

Abstract

2021

Analysis and Evaluation of Language Models for Word Sense Disambiguation

Authors
Loureiro, D; Rezaee, K; Pilehvar, MT; Camacho Collados, J;

Publication
COMPUTATIONAL LINGUISTICS

Abstract
Transformer-based language models have taken many fields in NLP by storm. BERT and its derivatives dominate most of the existing evaluation benchmarks, including those for Word Sense Disambiguation (WSD), thanks to their ability in capturing context-sensitive semantic nuances. However, there is still little knowledge about their capabilities and potential limitations in encoding and recovering word senses. In this article, we provide an in-depth quantitative and qualitative analysis of the celebrated BERT model with respect to lexical ambiguity. One of the main conclusions of our analysis is that BERT can accurately capture high-level sense distinctions, even when a limited number of examples is available for each word sense. Our analysis also reveals that in some cases language models come close to solving coarse grained noun disambiguation under ideal conditions in terms of availability of training data and computing resources. However, this scenario rarely occurs in real-world settings and, hence, many practical challenges remain even in the coarse-grained setting. We also perform an in-depth comparison of the two main language model-based WSD strategies, namely, fine-tuning and feature extraction, finding that the latter approach is more robust with respect to sense bias and it can better exploit limited available training data. In fact, the simple feature extraction strategy of averaging contextualized embeddings proves robust even using only three training sentences per word sense, with minimal improvements obtained by increasing the size of this training data.

2020

Don’t Neglect the Obvious: On the Role of Unambiguous Words in Word Sense Disambiguation

Authors
Loureiro, D; Camacho-Collados, J;

Publication
Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

Abstract

  • 1
  • 2