Abstract
This paper describes ongoing research to develop a system to automatically generate exercises on document type validation. It aims to support multiple text-based document formalisms, currently including JSON and XML. Validation of JSON documents uses JSON Schema and validation of XML uses both XML Schema and DTD. The exercise generator receives as input a document type and produces two sets of documents: valid and invalid instances. Document types written by students must validate the former and invalidate the latter. Exercises produced by this generator can be automatically accessed in a state-of-the-art assessment system. This paper details the proposed approach and describes the design of the system currently being implemented.

Abstract
The work presented in this article is part of ongoing research on the automated assessment of simple web applications. The proposed algorithm compares two interfaces by mapping their elements, using properties to identify those with the same role in both interfaces. The algorithm proceeds in three stages: firstly, it selects the relevant elements from both interfaces; secondly, it refines elements' attributes, excluding some and computing new ones; finally, it matches elements based on attribute similitude. The article includes an experiment to validate the algorithm as an assessment tool. As part of this experiment, a set of experts classified multiple web interfaces. Statistical analysis found a significant correlation between classifications made by the algorithm and those made by experts. The article also discusses the exploitation of the algorithm's output to access both the layout and functionality of a web interface and produce feedback messages in an automated assessment environment, which is planned as future research.

Abstract
Abstract This paper investigates the ability of large language models (LLMs) to evaluate semantic relations between word pairs by examining their alignment with human-generated semantic ratings. Semantic relations represent the degree of connection (e.g., relatedness or similarity) between linguistic elements and are traditionally validated against human-annotated datasets. Due to the challenges of building such datasets and recent progress in LLMs’ capacity to model human-like understanding, we explore whether LLMs can serve as reliable substitutes for traditional human ratings. We conducted experiments using multiple LLMs from OpenAI, Google, Mistral, and Anthropic, evaluating their performance across diverse English and Portuguese semantic relations datasets. We included in the analysis PAP900, a recently published dataset of semantic relations in Portuguese, to examine the influence of prior exposure to the dataset on LLM training. The results show that the LLM predictions correlate strongly with human ratings. The findings reveal the potential of LLMs to supplement or replace traditional semantic measure algorithms and crowd-sourced human annotations in semantic tasks.

Abstract

Abstract

Abstract