2025
Autores
Brito, P; Silva, APD;
Publicação
ADVANCES IN DATA ANALYSIS AND CLASSIFICATION
Abstract
We present parametric probabilistic models for numerical distributional variables. The proposed models are based on the representation of each distribution by a location measure and inter-quantile ranges, for given quantiles, thereby characterizing the underlying empirical distributions in a flexible way. Multivariate Normal distributions are assumed for the whole set of indicators, considering alternative structures of the variance-covariance matrix. For all cases, maximum likelihood estimators of the corresponding parameters are derived. This modelling allows for hypothesis testing and multivariate parametric analysis. The proposed framework is applied to Analysis of Variance and parametric Discriminant Analysis of distributional data. A simulation study examines the performance of the proposed models in classification problems under different data conditions. Applications to Internet traffic data and Portuguese official data illustrate the relevance of the proposed approach.
2025
Autores
Vitorino, J; Maia, E; Praça, I; Soares, C;
Publicação
CoRR
Abstract
2025
Autores
Brito, A; Santos, M; Folgado, D; Soares, C;
Publicação
DISCOVERY SCIENCE, DS 2025
Abstract
Ensuring robustness in time series classification remains a critical challenge for safety-sensitive domains like clinical decision systems. While current evaluation practices focus on accuracy measures, they fail to address model stability under semantically meaningful input deformations. We propose tsMIST (Time Series Model Sensitivity Test), a novel morphing-based framework that systematically evaluates classifier resilience through controlled interpolation between adversarial class prototypes. By calculating the switchThreshold - defined as the minimal morphing distance required to flip predictions - our method reveals critical stability patterns across synthetic benchmarks with tunable class separation and 17 medical time series datasets. Key findings show convolutional architectures (ROCKET) maintain optimal thresholds near 50% morphing (48.2 +/- 3.1%), while feature-based models (Catch22) exhibit premature decision flips at 22.7% deformation (+/- 15.4%). In clinical scenarios, tsMIST detected critical ECG misclassifications triggered by <= 12% signal variation - vulnerabilities undetected by accuracy measures. Our results establish that robustness measures must complement accuracy for responsible AI in high-stakes applications. This work advances ML evaluation practices by enabling systematic sensitivity analysis, with implications for model auditing and deployment in safety-critical domains.
2025
Autores
Teixeira, C; Gomes, I; Soares, C; van Rijn, JN;
Publicação
DISCOVERY SCIENCE, DS 2025
Abstract
The growing deployment of artificial intelligence in critical domains exposes a pressing challenge: how reliably models make predictions for ambiguous data without exhibiting overconfidence. We introduce hubris benchmarking, a methodology to evaluate overconfidence in machine learning models. The benchmark is based on a novel architecture, ambiguous generative adversarial networks (AmbiGANs), which are trained to synthesize realistic yet ambiguous datasets. We also propose the hubris metric to quantitatively measure the extent of model overconfidence when faced with these ambiguous images. We illustrate the usage of the methodology by estimating the hubris of state-of-the-art pre-trained models (ConvNext and ViT) on binarized versions of public datasets, including MNIST, Fashion-MNIST, and Pneumonia Chest X-ray. We found that, while ConvNext is on average 3% more accurate than ViT, it often makes excessively confident predictions, on average by 10% points higher than ViT. These results illustrate the usefulness of hubris benchmarking in high-stakes decision processes.
2025
Autores
Soares, C; Azevedo, PJ; Cerqueira, V; Torgor, L;
Publicação
DISCOVERY SCIENCE, DS 2025
Abstract
A subgroup discovery-based method has recently been proposed to understand the behavior of models in the (original) feature space. The subgroups identified represent areas of feature space where the model obtains better or worse predictive performance when compared to the average test performance. For instance, in the marketing domain, the approach extracts subgroups such as: in groups of customers with higher income and who are younger, the random forest achieves higher accuracy than on average. Here, we propose a complementary method, Meta Subspace Analysis (MSA), MSA uses metalearning to analyze these subgroups in the metafeature space. We use association rules to relate metafeatures of the feature space represented by the subgroups to the improvement or degradation of the performance of models. For instance, in the same domain, the approach extracts rules such as: when the class entropy decreases and the mutual information increases in the subgroup data, the random forest achieves lower accuracy. While the subgroups in the original feature space are useful for the end user and the data scientist developing the corresponding model, the meta-level rules provide a domain-independent perspective on the behavior of the model that is suitable for the same data scientist but also for ML researchers, to understand the behavior of algorithms. We illustrate the approach with the results of two well-known algorithms, naive Bayes and random forest, on the Adult dataset. The results confirm some expected behavior of algorithms. However, and most interestingly, some unexpected behaviors are also obtained, requiring additional investigation. In general, the empirical study demonstrates the usefulness of the approach to obtain additional knowledge about the behavior of models.
2025
Autores
Santos, M; de Carvalho, A; Soares, C;
Publicação
INTERNATIONAL JOURNAL OF DATA SCIENCE AND ANALYTICS
Abstract
Time series forecasting is an important tool for planning and decision-making. Considering this, several forecasting algorithms can be used, with results depending on the characteristics of the time series. The recommendation of the most suitable algorithm is a frequent concern. Metalearning has been successfully used to recommend the best algorithm for a time series analysis task. Additionally, it has been shown that decomposition methods can lead to better results. Based on previously published studies, in the experiments carried out, time series components were used. This work proposes and empirically evaluates METAFORE, a new time series forecasting approach that uses seasonal trend decomposition with Loess and metalearning to recommend suitable algorithms for time series forecasting combinations. Experimental results show that METAFORE can obtain a better predictive performance than single models with statistical significance. In the experiments, METAFORE also outperformed models widely used in the state-of-the-art, such as the long short-term memory neural network architectures, in more than 70%\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$70\%$$\end{document} of the time series tested. Finally, the results show that the joint use of metalearning and time series decomposition provides a competitive approach to time series forecasting.
The access to the final selection minute is only available to applicants.
Please check the confirmation e-mail of your application to obtain the access code.