2024
Authors
Eduard-Alexandru Bonci; Orit Kaidar-Person; Marília Antunes; Oriana Ciani; Helena Cruz; Rosa Di Micco; Oreste Davide Gentilini; Nicole Rotmensz; Pedro Gouveia; Jörg Heil; Pawel Kabata; Nuno Freitas; Tiago Gonçalves; Miguel Romariz; Helena Montenegro; Hélder P. Oliveira; Jaime S. Cardoso; Henrique Martins; Daniela Lopes; Marta Martinho; Ludovica Borsoi; Elisabetta Listorti; Carlos Mavioso; Martin Mika; André Pfob; Timo Schinköthe; Giovani Silva; Maria-Joao Cardoso;
Publication
Cancer Research
Abstract
2024
Authors
Zolfagharnasab, MH; Freitas, N; Gonçalves, T; Bonci, E; Mavioso, C; Cardoso, MJ; Oliveira, HP; Cardoso, JS;
Publication
Artificial Intelligence and Imaging for Diagnostic and Treatment Challenges in Breast Care - First Deep Breast Workshop, Deep-Breath 2024, Held in Conjunction with MICCAI 2024, Marrakesh, Morocco, October 10, 2024, Proceedings
Abstract
Breast cancer treatments often affect patients’ body image, making aesthetic outcome predictions vital. This study introduces a Deep Learning (DL) multimodal retrieval pipeline using a dataset of 2,193 instances combining clinical attributes and RGB images of patients’ upper torsos. We evaluate four retrieval techniques: Weighted Euclidean Distance (WED) with various configurations and shallow Artificial Neural Network (ANN) for tabular data, pre-trained and fine-tuned Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs), and a multimodal approach combining both data types. The dataset, categorised into Excellent/Good and Fair/Poor outcomes, is organised into over 20K triplets for training and testing. Results show fine-tuned multimodal ViTs notably enhance performance, achieving up to 73.85% accuracy and 80.62% Adjusted Discounted Cumulative Gain (ADCG). This framework not only aids in managing patient expectations by retrieving the most relevant post-surgical images but also promises broad applications in medical image analysis and retrieval. The main contributions of this paper are the development of a multimodal retrieval system for breast cancer patients based on post-surgery aesthetic outcome and the evaluation of different models on a new dataset annotated by clinicians for image retrieval. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2025.
2024
Authors
Freitas, N; Veloso, C; Mavioso, C; Cardoso, MJ; Oliveira, HP; Cardoso, JS;
Publication
Artificial Intelligence and Imaging for Diagnostic and Treatment Challenges in Breast Care - First Deep Breast Workshop, Deep-Breath 2024, Held in Conjunction with MICCAI 2024, Marrakesh, Morocco, October 10, 2024, Proceedings
Abstract
Breast cancer is the most common type of cancer in women worldwide. Because of high survival rates, there has been an increased interest in patient Quality of Life after treatment. Aesthetic results play an important role in this aspect, as these treatments can leave a mark on a patient’s self-image. Despite that, there are no standard ways of assessing aesthetic outcomes. Commonly used software such as BCCT.core or BAT require the manual annotation of keypoints, which makes them time-consuming for clinical use and can lead to result variability depending on the user. Recently, there have been attempts to leverage both traditional and Deep Learning algorithms to detect keypoints automatically. In this paper, we compare several methods for the detection of Breast Endpoints across two datasets. Furthermore, we present an extended evaluation of using these models as input for full contour prediction and aesthetic evaluation using the BCCT.core software. Overall, the YOLOv9 model, fine-tuned for this task, presents the best results considering both accuracy and usability, making this architecture the best choice for this application. The main contribution of this paper is the development of a pipeline for full breast contour prediction, which reduces clinician workload and user variability for automatic aesthetic assessment. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2025.
2024
Authors
Torto, IR; Cardoso, JS; Teixeira, LF;
Publication
Medical Imaging with Deep Learning, 3-5 July 2024, Paris, France.
Abstract
2024
Authors
Patrício, C; Neves, C; Teixeira, F;
Publication
ACM COMPUTING SURVEYS
Abstract
The remarkable success of deep learning has prompted interest in its application to medical imaging diagnosis. Even though state-of-the-art deep learning models have achieved human-level accuracy on the classification of different types of medical data, these models are hardly adopted in clinical workflows, mainly due to their lack of interpretability. The black-box nature of deep learning models has raised the need for devising strategies to explain the decision process of these models, leading to the creation of the topic of eXplainable Artificial Intelligence (XAI). In this context, we provide a thorough survey of XAI applied to medical imaging diagnosis, including visual, textual, example-based and concept-based explanation methods. Moreover, this work reviews the existing medical imaging datasets and the existing metrics for evaluating the quality of the explanations. In addition, we include a performance comparison among a set of report generation-based methods. Finally, the major challenges in applying XAI to medical imaging and the future research directions on the topic are discussed.
2024
Authors
Patricio, C; Teixeira, LF; Neves, JC;
Publication
IEEE INTERNATIONAL SYMPOSIUM ON BIOMEDICAL IMAGING, ISBI 2024
Abstract
Concept-based models naturally lend themselves to the development of inherently interpretable skin lesion diagnosis, as medical experts make decisions based on a set of visual patterns of the lesion. Nevertheless, the development of these models depends on the existence of concept-annotated datasets, whose availability is scarce due to the specialized knowledge and expertise required in the annotation process. In this work, we show that vision-language models can be used to alleviate the dependence on a large number of concept-annotated samples. In particular, we propose an embedding learning strategy to adapt CLIP to the downstream task of skin lesion classification using concept-based descriptions as textual embeddings. Our experiments reveal that vision-language models not only attain better accuracy when using concepts as textual embeddings, but also require a smaller number of concept-annotated samples to attain comparable performance to approaches specifically devised for automatic concept generation.
The access to the final selection minute is only available to applicants.
Please check the confirmation e-mail of your application to obtain the access code.