Abstract
This paper summarises the setup and results of the shared task on onomastic wordplay translation at the CLEF 2025 JOKER Lab, an earlier version of which was run at JOKER 2022 as a pilot task. The objective of the task is to translate wordplay concerned with proper names from English to French. Such wordplay, widespread in classic and modern creative writing, is particularly challenging to translate due to its idiosyncratic nature and cultural references. Four teams participated in this year’s task, submitting 20 runs. We describe our construction of the data set using for training and testing, the methods employed by the participating teams, and the results obtained for the runs and a naïve baseline in terms of various manually and automatically applied measures of translation quality. Despite notable advances, we find that translation of onomastic wordplay remains highly challenging, with fewer than 10% of manually evaluated translations judged as acceptable alternatives. Recurrent errors included untranslated source wordplay, overfitting to the training data, omission of surnames, and nonsensical generations.

Abstract
This paper presents the details of Task 1 of the JOKER-2025 Track, an information retrieval task where the goal is to find relevant humorous in a collection of text documents. The intended use case is retrieving jokes on a specific topic, something that may benefit humanities research, second-language learning, and the writing or translation of comedic texts. We provide two document collections: one in English and another in European Portuguese. The English collection consists of 77,658 documents, of which 5,198 are annotated as humorous, and 219 queries with relevance judgments. The Portuguese collection contains 45,126 texts, including 1,199 humorous documents along with 98 queries. Together, these collections support cross-linguistic studies in humour detection and contribute to the development of more inclusive and language-aware retrieval systems. Nine teams submitted 62 runs in total for this task.

Abstract
This paper describes Task 2 of the CLEF 2025 JOKER track on the translation of puns from English into French. We outline the overall structure and setup of the shared task, discuss the approaches employed by the participants, and present and analyse the results they achieved. We also describe experiments with a promising new approach for the automatic evaluation of pun translation. Despite the significant improvements observed this year by participating systems, most of which used state-of-the-art large language models, we find wordplay translation to remain a complex and demanding task. Among the manually evaluated translations, 37.5% successfully preserved the meaning and involved wordplay, with success rates per English pun varying widely.

Abstract

Abstract

Abstract
Acquiring high-quality annotated data remains one of the most significant challenges in Natural Language Processing (NLP), especially for supervised learning approaches. In scenarios where pre-existing labeled data is unavailable, common solutions like crowdsourcing and zero-shot approaches often fall short, suffering from limitations such as the need for large datasets and a lack of guarantees regarding annotation quality. Traditionally, data for human annotation has been selected randomly, a practice that is not only costly and inefficient but also prone to bias, particularly in imbalanced datasets where minority classes are underrepresented. To address these challenges, this work introduces an automatic and informed data selection architecture designed to minimize the volume of required annotations while maximizing the diversity and representativeness of the selected data. Among the evaluated methods, Reverse Semantic Search (RSS) demonstrated superior performance, consistently outperforming random sampling in imbalanced scenarios and enhancing the effectiveness of trained classifiers. Furthermore, we compared RSS with other clustering-based approaches, providing insights into their respective strengths and weaknesses.