Cookies Policy
The website need some cookies and similar means to function. If you permit us, we will use those means to collect data on your visits for aggregated statistics to improve our service. Find out More
Accept Reject
  • Menu
About

About

I was born in Portugal in 1964. I graduated in mathematics from the Faculty of Sciences of the University of Porto and earned a Ph.D. in Computer Science from the same institution.
My current position is auxiliary professor at the Computer Science department of the Faculty of Sciences of the University of Porto. I am also affiliated with the Center for Research in Advanced Computing Systems (CRACS), an R&D unit of INESCTEC Research Laboratory, where I am an effective member.
My main research interests are technology enhanced learning, web adaptability, and semantic web.

Interest
Topics
Details

Details

  • Name

    José Paulo Leal
  • Role

    Senior Researcher
  • Since

    01st January 2009
003
Publications

2025

Can a large language model replace humans at rating lexical semantic relations strength?

Authors
André Fernandes dos Santos; José Paulo Leal;

Publication
Computational Linguistics

Abstract
Abstract This paper investigates the ability of large language models (LLMs) to evaluate semantic relations between word pairs by examining their alignment with human-generated semantic ratings. Semantic relations represent the degree of connection (e.g., relatedness or similarity) between linguistic elements and are traditionally validated against human-annotated datasets. Due to the challenges of building such datasets and recent progress in LLMs’ capacity to model human-like understanding, we explore whether LLMs can serve as reliable substitutes for traditional human ratings. We conducted experiments using multiple LLMs from OpenAI, Google, Mistral, and Anthropic, evaluating their performance across diverse English and Portuguese semantic relations datasets. We included in the analysis PAP900, a recently published dataset of semantic relations in Portuguese, to examine the influence of prior exposure to the dataset on LLM training. The results show that the LLM predictions correlate strongly with human ratings. The findings reveal the potential of LLMs to supplement or replace traditional semantic measure algorithms and crowd-sourced human annotations in semantic tasks.

2025

Osiris: A Multi-Language Transpiler for Educational Purposes

Authors
Marrao, B; Leal, JP; Queirós, R;

Publication
6TH INTERNATIONAL COMPUTER PROGRAMMING EDUCATION CONFERENCE, ICPEC 2025

Abstract
While server-side assessment of programming exercises, with its ease of installing diverse compilers and execution environments, is common, it presents three key limitations: the necessity of a constant Internet connection, increased bandwidth consumption, and centralized execution load. The alternative is to rely on JavaScript, the single programming language supported by all standard web browsers. This paper introduces Osiris, a pure JavaScript multi-language transpiler designed to enable the execution of diverse programming languages within web browsers. Targeted primarily at Virtual Learning Environments (VLE) for language programming education, Osiris employs a parser generator to translate small student programs into JavaScript based on language-specific grammars with semantic rules. It also includes a comprehensive, though not exhaustive, JavaScript library that emulates the standard libraries of its supported languages. Validation of Osiris indicates the pedagogical effectiveness of browser-based transpilation for introductory programming education.

2025

Designing a Multi-Narrative Gamified Learning Experience

Authors
Bauer, Y; Leal, JP; Queirós, R; Swacha, J; Paiva, J;

Publication
6TH INTERNATIONAL COMPUTER PROGRAMMING EDUCATION CONFERENCE, ICPEC 2025

Abstract
The combination of storytelling and gamification in educational settings has emerged as a method to enhance student engagement and learning outcomes. Through an overarching narrative, course content can be connected while providing context for gamified exercises, creating a motivating and competitive learning experience. However, a narrative that resonates with one student may not interest others. The presented solution to this problem is to offer multiple narratives for students to choose from. This enables the students to engage with the material in ways that align with their interests and motivations. Yet, managing multiple narratives presents several challenges. Each narrative must cover all syllabus topics equally, and every exercise must be available across all narratives while maintaining consistent difficulty levels and learning objectives. This paper presents a systematic approach for creating gamified courses with multiple narratives. The methodology includes the development of a base course template and its narrative variations, along with transformation processes to generate exercises in the FGPE Ecosystem, namely AuthorKit and FGPE PLE. The final output is a single Moodle MBZ file that can be imported into Moodle, a widely adopted learning management system.

2025

PAP900: A dataset of semantic relationships between affective words in Portuguese

Authors
dos Santos, AF; Leal, JP; Alves, RA; Jacques, T;

Publication
DATA IN BRIEF

Abstract
The PAP900 dataset centers on the semantic relationship between affective words in Portuguese. It contains 900 word pairs, each annotated by at least 30 human raters for both semantic similarity and semantic relatedness. In addition to the semantic ratings, the dataset includes the word categorization used to build the word pairs and detailed sociodemographic information about annotators, enabling the analysis of the influence of personal factors on the perception of semantic relationships. Furthermore, this article describes in detail the dataset construction process, from word selection to agreement metrics. Data was collected from Portuguese university psychology students, who completed two rounds of questionnaires. In the first round annotators were asked to rate word pairs on either semantic similarity or relatedness. The second round switched the relation type for most annotators, with a small percentage being asked to repeat the same relation. The instructions given emphasized the differences between semantic relatedness and semantic similarity, and provided examples of expected ratings of both. There are few semantic relations datasets in Portuguese, and none focusing on affective words. PAP900 is distributed in distinct formats to be easy to use for both researchers just looking for the final averaged values and for researchers looking to take advantage of the individual ratings, the word categorization and the annotator data. This dataset is a valuable resource for researchers in computational linguistics, natural language processing, psychology, and cognitive science. (c) 2025TheAuthors.

2025

Incremental Repair Feedback on Automated Assessment of Programming Assignments

Authors
Paiva, JC; Leal, JP; Figueira, A;

Publication
ELECTRONICS

Abstract
Automated assessment tools for programming assignments have become increasingly popular in computing education. These tools offer a cost-effective and highly available way to provide timely and consistent feedback to students. However, when evaluating a logically incorrect source code, there are some reasonable concerns about the formative gap in the feedback generated by such tools compared to that of human teaching assistants. A teaching assistant either pinpoints logical errors, describes how the program fails to perform the proposed task, or suggests possible ways to fix mistakes without revealing the correct code. On the other hand, automated assessment tools typically return a measure of the program's correctness, possibly backed by failing test cases and, only in a few cases, fixes to the program. In this paper, we introduce a tool, AsanasAssist, to generate formative feedback messages to students to repair functionality mistakes in the submitted source code based on the most similar algorithmic strategy solution. These suggestions are delivered with incremental levels of detail according to the student's needs, from identifying the block containing the error to displaying the correct source code. Furthermore, we evaluate how well the automatically generated messages provided by AsanasAssist match those provided by a human teaching assistant. The results demonstrate that the tool achieves feedback comparable to that of a human grader while being able to provide it just in time.