Cookies
O website necessita de alguns cookies e outros recursos semelhantes para funcionar. Caso o permita, o INESC TEC irá utilizar cookies para recolher dados sobre as suas visitas, contribuindo, assim, para estatísticas agregadas que permitem melhorar o nosso serviço. Ver mais
Aceitar Rejeitar
  • Menu
Sobre

Sobre

Nasci em Portugal em 1964. Sou licenciado em matemática pela Faculdade de Ciências da Universidade do Porto (FCUP) e tenho um doutoramento em ciência de computadores pela mesma instituição. Correntemente, sou professor auxiliar no departamento de ciências de computadores da FCUP. Também estou afiliado no Center for Research in Advanced Computing Systems (CRACS) uma unidade de investigação do INESC TEC, onde sou membro efectivo. Os meus interesses de investigação são o ensino mediado por computador, adaptabilidade web e web semântica.

Tópicos
de interesse
Detalhes

Detalhes

  • Nome

    José Paulo Leal
  • Cargo

    Investigador Sénior
  • Desde

    01 janeiro 2009
003
Publicações

2025

Can a large language model replace humans at rating lexical semantic relations strength?

Autores
André Fernandes dos Santos; José Paulo Leal;

Publicação
Computational Linguistics

Abstract
Abstract This paper investigates the ability of large language models (LLMs) to evaluate semantic relations between word pairs by examining their alignment with human-generated semantic ratings. Semantic relations represent the degree of connection (e.g., relatedness or similarity) between linguistic elements and are traditionally validated against human-annotated datasets. Due to the challenges of building such datasets and recent progress in LLMs’ capacity to model human-like understanding, we explore whether LLMs can serve as reliable substitutes for traditional human ratings. We conducted experiments using multiple LLMs from OpenAI, Google, Mistral, and Anthropic, evaluating their performance across diverse English and Portuguese semantic relations datasets. We included in the analysis PAP900, a recently published dataset of semantic relations in Portuguese, to examine the influence of prior exposure to the dataset on LLM training. The results show that the LLM predictions correlate strongly with human ratings. The findings reveal the potential of LLMs to supplement or replace traditional semantic measure algorithms and crowd-sourced human annotations in semantic tasks.

2025

Osiris: A Multi-Language Transpiler for Educational Purposes

Autores
Marrao, B; Leal, JP; Queirós, R;

Publicação
6TH INTERNATIONAL COMPUTER PROGRAMMING EDUCATION CONFERENCE, ICPEC 2025

Abstract
While server-side assessment of programming exercises, with its ease of installing diverse compilers and execution environments, is common, it presents three key limitations: the necessity of a constant Internet connection, increased bandwidth consumption, and centralized execution load. The alternative is to rely on JavaScript, the single programming language supported by all standard web browsers. This paper introduces Osiris, a pure JavaScript multi-language transpiler designed to enable the execution of diverse programming languages within web browsers. Targeted primarily at Virtual Learning Environments (VLE) for language programming education, Osiris employs a parser generator to translate small student programs into JavaScript based on language-specific grammars with semantic rules. It also includes a comprehensive, though not exhaustive, JavaScript library that emulates the standard libraries of its supported languages. Validation of Osiris indicates the pedagogical effectiveness of browser-based transpilation for introductory programming education.

2025

Designing a Multi-Narrative Gamified Learning Experience

Autores
Bauer, Y; Leal, JP; Queirós, R; Swacha, J; Paiva, J;

Publicação
6TH INTERNATIONAL COMPUTER PROGRAMMING EDUCATION CONFERENCE, ICPEC 2025

Abstract
The combination of storytelling and gamification in educational settings has emerged as a method to enhance student engagement and learning outcomes. Through an overarching narrative, course content can be connected while providing context for gamified exercises, creating a motivating and competitive learning experience. However, a narrative that resonates with one student may not interest others. The presented solution to this problem is to offer multiple narratives for students to choose from. This enables the students to engage with the material in ways that align with their interests and motivations. Yet, managing multiple narratives presents several challenges. Each narrative must cover all syllabus topics equally, and every exercise must be available across all narratives while maintaining consistent difficulty levels and learning objectives. This paper presents a systematic approach for creating gamified courses with multiple narratives. The methodology includes the development of a base course template and its narrative variations, along with transformation processes to generate exercises in the FGPE Ecosystem, namely AuthorKit and FGPE PLE. The final output is a single Moodle MBZ file that can be imported into Moodle, a widely adopted learning management system.

2025

PAP900: A dataset of semantic relationships between affective words in Portuguese

Autores
dos Santos, AF; Leal, JP; Alves, RA; Jacques, T;

Publicação
DATA IN BRIEF

Abstract
The PAP900 dataset centers on the semantic relationship between affective words in Portuguese. It contains 900 word pairs, each annotated by at least 30 human raters for both semantic similarity and semantic relatedness. In addition to the semantic ratings, the dataset includes the word categorization used to build the word pairs and detailed sociodemographic information about annotators, enabling the analysis of the influence of personal factors on the perception of semantic relationships. Furthermore, this article describes in detail the dataset construction process, from word selection to agreement metrics. Data was collected from Portuguese university psychology students, who completed two rounds of questionnaires. In the first round annotators were asked to rate word pairs on either semantic similarity or relatedness. The second round switched the relation type for most annotators, with a small percentage being asked to repeat the same relation. The instructions given emphasized the differences between semantic relatedness and semantic similarity, and provided examples of expected ratings of both. There are few semantic relations datasets in Portuguese, and none focusing on affective words. PAP900 is distributed in distinct formats to be easy to use for both researchers just looking for the final averaged values and for researchers looking to take advantage of the individual ratings, the word categorization and the annotator data. This dataset is a valuable resource for researchers in computational linguistics, natural language processing, psychology, and cognitive science. (c) 2025TheAuthors.

2025

Incremental Repair Feedback on Automated Assessment of Programming Assignments

Autores
Paiva, JC; Leal, JP; Figueira, A;

Publicação
ELECTRONICS

Abstract
Automated assessment tools for programming assignments have become increasingly popular in computing education. These tools offer a cost-effective and highly available way to provide timely and consistent feedback to students. However, when evaluating a logically incorrect source code, there are some reasonable concerns about the formative gap in the feedback generated by such tools compared to that of human teaching assistants. A teaching assistant either pinpoints logical errors, describes how the program fails to perform the proposed task, or suggests possible ways to fix mistakes without revealing the correct code. On the other hand, automated assessment tools typically return a measure of the program's correctness, possibly backed by failing test cases and, only in a few cases, fixes to the program. In this paper, we introduce a tool, AsanasAssist, to generate formative feedback messages to students to repair functionality mistakes in the submitted source code based on the most similar algorithmic strategy solution. These suggestions are delivered with incremental levels of detail according to the student's needs, from identifying the block containing the error to displaying the correct source code. Furthermore, we evaluate how well the automatically generated messages provided by AsanasAssist match those provided by a human teaching assistant. The results demonstrate that the tool achieves feedback comparable to that of a human grader while being able to provide it just in time.