Cookies Policy
The website need some cookies and similar means to function. If you permit us, we will use those means to collect data on your visits for aggregated statistics to improve our service. Find out More
Accept Reject
  • Menu
Interest
Topics
Details

Details

  • Name

    João Nuno Fernandes
  • Role

    Assistant Researcher
  • Since

    10th January 2022
Publications

2024

Explaining Bounding Boxes in Deep Object Detectors Using Post Hoc Methods for Autonomous Driving Systems

Authors
Nogueira, C; Fernandes, L; Fernandes, JND; Cardoso, JS;

Publication
SENSORS

Abstract
Deep learning has rapidly increased in popularity, leading to the development of perception solutions for autonomous driving. The latter field leverages techniques developed for computer vision in other domains for accomplishing perception tasks such as object detection. However, the black-box nature of deep neural models and the complexity of the autonomous driving context motivates the study of explainability in these models that perform perception tasks. Moreover, this work explores explainable AI techniques for the object detection task in the context of autonomous driving. An extensive and detailed comparison is carried out between gradient-based and perturbation-based methods (e.g., D-RISE). Moreover, several experimental setups are used with different backbone architectures and different datasets to observe the influence of these aspects in the explanations. All the techniques explored consist of saliency methods, making their interpretation and evaluation primarily visual. Nevertheless, numerical assessment methods are also used. Overall, D-RISE and guided backpropagation obtain more localized explanations. However, D-RISE highlights more meaningful regions, providing more human-understandable explanations. To the best of our knowledge, this is the first approach to obtaining explanations focusing on the regression of the bounding box coordinates.

2024

Intrinsic Explainability for End-to-End Object Detection

Authors
Fernandes, L; Fernandes, JND; Calado, M; Pinto, JR; Cerqueira, R; Cardoso, JS;

Publication
IEEE ACCESS

Abstract
Deep Learning models are automating many daily routine tasks, indicating that in the future, even high-risk tasks will be automated, such as healthcare and automated driving areas. However, due to the complexity of such deep learning models, it is challenging to understand their reasoning. Furthermore, the black box nature of the designed deep learning models may undermine public confidence in critical areas. Current efforts on intrinsically interpretable models focus only on classification tasks, leaving a gap in models for object detection. Therefore, this paper proposes a deep learning model that is intrinsically explainable for the object detection task. The chosen design for such a model is a combination of the well-known Faster-RCNN model with the ProtoPNet model. For the Explainable AI experiments, the chosen performance metric was the similarity score from the ProtoPNet model. Our experiments show that this combination leads to a deep learning model that is able to explain its classifications, with similarity scores, using a visual bag of words, which are called prototypes, that are learned during the training process. Furthermore, the adoption of such an explainable method does not seem to hinder the performance of the proposed model, which achieved a mAP of 69% in the KITTI dataset and a mAP of 66% in the GRAZPEDWRI-DX dataset. Moreover, our explanations have shown a high reliability on the similarity score.

2023

MobileWeatherNet for LiDAR-Only Weather Estimation

Authors
da Silva, MP; Carneiro, D; Fernandes, J; Texeira, LF;

Publication
2023 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS, IJCNN

Abstract
An autonomous vehicle relying on LiDAR data should be able to assess its limitations in real time without depending on external information or additional sensors. The point cloud generated by the sensor is subjected to significant degradation under adverse weather conditions (rain, fog, and snow), which limits the vehicle's visibility and performance. With this in mind, we show that point cloud data contains sufficient information to estimate the weather accurately and present MobileWeatherNet, a LiDAR-only convolutional neural network that uses the bird's-eye view 2D projection to extract point clouds' weather condition and improves state-of-the-art performance by 15% in terms of the balanced accuracy while reducing inference time by 63%. Moreover, this paper demonstrates that among common architectures, the use of the bird's eye view significantly enhances their performance without an increase in complexity. To the extent of our knowledge, this is the first approach that uses deep learning for weather estimation using point cloud data in the form of a bird's-eye-view projection.

2023

Towards an airtightness compliance tool based on machine learning models for naturally ventilated dwellings

Authors
Cardoso, VEM; Simoes, ML; Ramos, NMM; Almeida, RMSF; Almeida, M; Sanhudo, L; Fernandes, JND;

Publication
ENERGY AND BUILDINGS

Abstract
Physical models and probabilistic applications often guide the study and characterization of natural phenomena in engineering. Such is the case of the study of air change rates (ACHs) in buildings for their complex mechanisms and high variability. It is not uncommon for the referred applications to be costly and impractical in both time and computation, resulting in the use of simplified methodologies and setups. The incorporation of airtightness limits to quantify adequate ACHs in national transpositions of the Energy Performance Building Directive (EPBD) exemplifies the issue. This research presents a roadmap for developing an alternative instrument, a compliance tool built with a Machine Learning (ML) framework, that overcomes some simplification issues regarding policy implementation while fulfilling practitioners' needs and general societal use. It relies on dwellings' terrain, geometric and airtightness characteristics, and meteorological data. Results from previous work on a region with a mild heating season in southern Europe apply in training and testing the proposed tool. The tool outputs numerical information on the air change rates performance of the building envelope, and a label, accordingly. On the test set, the best regressor showed mean absolute errors (MAE) below 1.02% for all the response variables, while the best classifier presented an average accuracy of 97.32%. These results are promising for the generalization of this methodology, with potential for application at regional, national, and European Union levels. The developed tool could be a complementary asset to energy certification programmes of either public or private initiatives. (c) 2023 The Author(s). Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).

2022

A labelling strategy to define airtightness performance ranges of naturally ventilated dwellings: An application in southern Europe

Authors
Cardoso, VEM; Simoes, ML; Ramos, NMM; Almeida, RMSF; Almeida, M; Fernandes, JND;

Publication
ENERGY AND BUILDINGS

Abstract
Energy efficiency and indoor air quality are frequently-two conflicting objectives when establishing the air change rate (ACH) of a dwelling. In Europe, the northern countries have a clear focus on energy conservation, leading to an obvious awareness of the importance of airtightness, which translates into a high level of regulation and implementation. Meanwhile, the southern counterparts experience a more com-plex challenge by having predominantly passive ventilation strategies and milder climates, which often results in a more permissive approach. This work proposes an innovative labelling methodology to classify the performance of naturally ventilated dwellings. A representative sample of a southern European national built stock is used in a stochastic process to create a pool of 43,200 unique dwellings. The simulation period refers to a month of the typical heating season in the southern European mild conditions. The results test the labelling methodology. With feature selection, ACH limits, and a labelling strategy, dwellings classify according to their ability to provide adequate ACHs. The terrain was the best splitter of the dataset from the applied categorical variables. Regarding continuous variables, the airtightness was the one explaining most of the variability of the outputted ACHs, followed by the floor area. From the best performing dwellings labelled as compliant (Com), the average airtightness level was 5.3 h(-1), with 4.9 h(-1) and 5.8 h(-1) in rural and urban locations.