Cookies Policy
The website need some cookies and similar means to function. If you permit us, we will use those means to collect data on your visits for aggregated statistics to improve our service. Find out More
Accept Reject
  • Menu
Publications

Publications by José Paulo Leal

2015

Odin: a service for gamification of learning activities

Authors
Leal, José Paulo; Paiva, José; Queirós, Ricardo;

Publication
Symposium on Languages, Applications and Technologies (SLATE), 4th

Abstract
Existing gami?cation services have features that preclude their use by e-learning tools. Odin is a gami?cation service that mimics the API of state-of-the-art services without these limitations. This paper describes Odin, its role in an e-learning system architecture requiring gami?cation, and details its implementation. The validation of Odin involved the creation of a small e-learning game, integrated in a Learning Management System (LMS) using the Learning Tools Interoperability (LTI) speci?cation.

2024

Authoring Programming Exercises for Automated Assessment Assisted by Generative AI

Authors
Bauer, Y; Leal, JP; Queirós, R;

Publication
5th International Computer Programming Education Conference, ICPEC 2024, June 27-28, 2024, Lisbon, Portugal

Abstract
Generative AI presents both challenges and opportunities for educators. This paper explores its potential for automating the creation of programming exercises designed for automated assessment. Traditionally, creating these exercises is a time-intensive and error-prone task that involves developing exercise statements, solutions, and test cases. This ongoing research analyzes the capabilities of the OpenAI GPT API to automatically create these components. An experiment using the OpenAI GPT API to automatically create 120 programming exercises produced interesting results, such as the difficulties encountered in generating valid JSON formats and creating matching test cases for solution code. Learning from this experiment, an enhanced feature was developed to assist teachers in creating programming exercises and was integrated into Agni, a virtual learning environment (VLE). Despite the challenges in generating entirely correct programming exercises, this approach shows potential for reducing the time required to create exercises, thus significantly aiding teachers. The evaluation of this approach, comparing the efficiency and usefulness of using the OpenAI GPT API or authoring the exercises oneself, is in progress. © Yannik Bauer, José Paulo Leal, and Ricardo Queirós;

2024

Clustering source code from automated assessment of programming assignments

Authors
Paiva, JC; Leal, JP; Figueira, A;

Publication
INTERNATIONAL JOURNAL OF DATA SCIENCE AND ANALYTICS

Abstract
Clustering of source code is a technique that can help improve feedback in automated program assessment. Grouping code submissions that contain similar mistakes can, for instance, facilitate the identification of students' difficulties to provide targeted feedback. Moreover, solutions with similar functionality but possibly different coding styles or progress levels can allow personalized feedback to students stuck at some point based on a more developed source code or even detect potential cases of plagiarism. However, existing clustering approaches for source code are mostly inadequate for automated feedback generation or assessment systems in programming education. They either give too much emphasis to syntactical program features, rely on expensive computations over pairs of programs, or require previously collected data. This paper introduces an online approach and implemented tool-AsanasCluster-to cluster source code submissions to programming assignments. The proposed approach relies on program attributes extracted from semantic graph representations of source code, including control and data flow features. The obtained feature vector values are fed into an incremental k-means model. Such a model aims to determine the closest cluster of solutions, as they enter the system, timely, considering clustering is an intermediate step for feedback generation in automated assessment. We have conducted a twofold evaluation of the tool to assess (1) its runtime performance and (2) its precision in separating different algorithmic strategies. To this end, we have applied our clustering approach on a public dataset of real submissions from undergraduate students to programming assignments, measuring the runtimes for the distinct tasks involved: building a model, identifying the closest cluster to a new observation, and recalculating partitions. As for the precision, we partition two groups of programs collected from GitHub. One group contains implementations of two searching algorithms, while the other has implementations of several sorting algorithms. AsanasCluster matches and, in some cases, improves the state-of-the-art clustering tools in terms of runtime performance and precision in identifying different algorithmic strategies. It does so without requiring the execution of the code. Moreover, it is able to start the clustering process from a dataset with only two submissions and continuously partition the observations as they enter the system.

2013

Ensemble: an e-learning framework

Authors
Queirós, Ricardo; Leal, José Paulo;

Publication
Journal of Universal Computer Science

Abstract
E-Learning frameworks are conceptual tools to organize networks of elearning services. Most frameworks cover areas that go beyond the scope of e-learning, from course to financial management, and neglects the typical activities in everyday life of teachers and students at schools such as the creation, delivery, resolution and evaluation of assignments. This paper presents the Ensemble framework - an e-learning framework exclusively focused on the teaching-learning process through the coordination of pedagogical services. The framework presents an abstract data, integration and evaluation model based on content and communications specifications. These specifications must base the implementation of networks in specialized domains with complex evaluations. In this paper we specialize the framework for two domains with complex evaluation: computer programming and computer-aided design (CAD). For each domain we highlight two Ensemble hotspots: data and evaluations procedures. In the former we formally describe the exercise and present possible extensions. In the latter, we describe the automatic evaluation procedures.

2024

Comparing Semantic Graph Representations of Source Code: The Case of Automatic Feedback on Programming Assignments

Authors
Paiva, JC; Leal, JP; Figueira, A;

Publication
COMPUTER SCIENCE AND INFORMATION SYSTEMS

Abstract
Static source code analysis techniques are gaining relevance in automated assessment of programming assignments as they can provide less rigorous evaluation and more comprehensive and formative feedback. These techniques focus on source code aspects rather than requiring effective code execution. To this end, syntactic and semantic information encoded in textual data is typically represented internally as graphs, after parsing and other preprocessing stages. Static automated assessment techniques, therefore, draw inferences from intermediate representations to determine the correctness of a solution and derive feedback. Consequently, achieving the most effective semantic graph representation of source code for the specific task is critical, impacting both techniques' accuracy, outcome, and execution time. This paper aims to provide a thorough comparison of the most widespread semantic graph representations for the automated assessment of programming assignments, including usage examples, facets, and costs for each of these representations. A benchmark has been conducted to assess their cost using the Abstract Syntax Tree (AST) as a baseline. The results demonstrate that the Code Property Graph (CPG) is the most feature -rich representation, but also the largest and most space -consuming (about 33% more than AST).

2023

Automated Assessment of Simple Web Applications (Short Paper)

Authors
Costa, LM; Leal, JP; Queirós, R;

Publication
4th International Computer Programming Education Conference, ICPEC 2023, June 26-28, 2023, Vila do Conde, Portugal

Abstract
Web programming education is an important component of modern computer science curricula. Assessing students’ web programming skills can be time-consuming and challenging for educators. This paper introduces Webpal, an automated assessment tool for web programming exercises in entry-level courses. Webpal evaluates web applications coded in HTML, CSS, and Javascript, and provides feedback to students. This tool integrates with Virtual Learning Environments (VLEs) through an API, allowing the creation, storage, and access to exercises while assessing student attempts based on the created exercises. The evaluation process comprises various subcomponents: static assessment, interface matching, functional testing, and feedback management. This approach aims to provide feedback that helps students overcome their challenges in web programming assignments. This paper also presents a demo showcasing the tool’s features and functionality in a simulated VLE environment. © Luís Maia Costa, José Paulo Leal, and Ricardo Queirós; licensed under Creative Commons License CC-BY 4.0.

  • 16
  • 25