Cookies Policy
The website need some cookies and similar means to function. If you permit us, we will use those means to collect data on your visits for aggregated statistics to improve our service. Find out More
Accept Reject
  • Menu
Interest
Topics
Details

Details

  • Name

    Daniel Bouçanova Loureiro
  • Cluster

    Computer Science
  • Role

    External Research Collaborator
  • Since

    01st November 2017
003
Publications

2022

LMMS reloaded: Transformer-based sense embeddings for disambiguation and beyond

Authors
Loureiro, D; Mário Jorge, A; Camacho Collados, J;

Publication
Artificial Intelligence

Abstract

2021

Analysis and evaluation of language models for word sense disambiguation

Authors
Loureiro, D; Rezaee, K; Pilehvar, MT; Camacho Collados, J;

Publication
Computational Linguistics

Abstract
Transformer-based language models have taken many fields in NLP by storm. BERT and its derivatives dominate most of the existing evaluation benchmarks, including those for Word Sense Disambiguation (WSD), thanks to their ability in capturing context-sensitive semantic nuances. However, there is still little knowledge about their capabilities and potential limitations in encoding and recovering word senses. In this article, we provide an in-depth quantitative and qualitative analysis of the celebrated BERT model with respect to lexical ambiguity. One of the main conclusions of our analysis is that BERT can accurately capture high-level sense distinctions, even when a limited number of examples is available for each word sense. Our analysis also reveals that in some cases language models come close to solving coarse-grained noun disambiguation under ideal conditions in terms of availability of training data and computing resources. However, this scenario rarely occurs in real-world settings and, hence, many practical challenges remain even in the coarse-grained setting. We also perform an in-depth comparison of the two main language model-based WSD strategies, namely, fine-tuning and feature extraction, finding that the latter approach is more robust with respect to sense bias and it can better exploit limited available training data. In fact, the simple feature extraction strategy of averaging contextualized embeddings proves robust even using only three training sentences per word sense, with minimal improvements obtained by increasing the size of this training data.

2020

MedLinker: Medical Entity Linking with Neural Representations and Dictionary Matching

Authors
Loureiro, D; Jorge, AM;

Publication
Advances in Information Retrieval - 42nd European Conference on IR Research, ECIR 2020, Lisbon, Portugal, April 14-17, 2020, Proceedings, Part II

Abstract

2020

Don’t Neglect the Obvious: On the Role of Unambiguous Words in Word Sense Disambiguation

Authors
Loureiro, D; Camacho-Collados, J;

Publication
Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

Abstract

2019

Language Modelling Makes Sense: Propagating Representations through Word Net for Full-Coverage Word Sense Disambiguation

Authors
Loureiro, D; Jorge, AM;

Publication
57TH ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS (ACL 2019)

Abstract
Contextual embeddings represent a new generation of semantic representations learned from Neural Language Modelling (NLM) that addresses the issue of meaning conflation hampering traditional word embeddings. In this work, we show that contextual embeddings can be used to achieve unprecedented gains in Word Sense Disambiguation (WSD) tasks. Our approach focuses on creating sense-level embeddings with full-coverage of WordNet, and without recourse to explicit knowledge of sense distributions or task-specific modelling. As a result, a simple Nearest Neighbors (k-NN) method using our representations is able to consistently surpass the performance of previous systems using powerful neural sequencing models. We also analyse the robustness of our approach when ignoring part-of-speech and lemma features, requiring disambiguation against the full sense inventory, and revealing shortcomings to be improved. Finally, we explore applications of our sense embeddings for concept-level analyses of contextual embeddings and their respective NLMs.