Abstract
Breast cancer treatments often affect patients' body image, making aesthetic outcome predictions vital. This study introduces a Deep Learning (DL) multimodal retrieval pipeline using a dataset of 2,193 instances combining clinical attributes and RGB images of patients' upper torsos. We evaluate four retrieval techniques: Weighted Euclidean Distance (WED) with various configurations and shallow Artificial Neural Network (ANN) for tabular data, pre-trained and fine-tuned Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs), and a multimodal approach combining both data types. The dataset, categorised into Excellent/Good and Fair/Poor outcomes, is organised into over 20K triplets for training and testing. Results show fine-tuned multimodal ViTs notably enhance performance, achieving up to 73.85% accuracy and 80.62% Adjusted Discounted Cumulative Gain (ADCG). This framework not only aids in managing patient expectations by retrieving the most relevant post-surgical images but also promises broad applications in medical image analysis and retrieval. The main contributions of this paper are the development of a multimodal retrieval system for breast cancer patients based on post-surgery aesthetic outcome and the evaluation of different models on a new dataset annotated by clinicians for image retrieval.

Abstract

Abstract
TPS621 Background: Breast cancer treatments often pose challenges in balancing efficacy with quality of life. The CINDERELLA Project pioneers an artificial intelligence (AI)-driven approach (CINDERELLA APP) for shared decision-making process, aiming to harmonise locoregional therapeutic interventions with breast cancer patients' expectations about aesthetic outcomes. The CINDERELLA clinical trial aims to establish a new standard in patient-centred care by bridging the gap between clinical treatment options and patient expectations through innovative technology. The trial focuses on evaluating the effectiveness of the CINDERELLA APP in improving patient satisfaction regarding locoregional treatment aesthetic outcomes, aligning patient expectations with real-world results, and assessing its impact on overall quality of life and psychological well-being. Methods: Trial design and statistical methods: This international multicentric interventional randomised controlled open-label clinical trial will recruit and randomise patients into two groups: one receiving standard treatment information and the other using the AI-powered CINDERELLA APP. The primary objective is to assess the levels of agreement among patients' expectations regarding the aesthetic outcome before and 12 months after locoregional treatment. The trial will also evaluate the aesthetic outcome level of agreement between the AI evaluation tool and self-evaluation. The impact of the intervention on aligning expectations with outcomes will be evaluated using the Wilcoxon signed-rank test. The improvement in classification of aesthetic results post-intervention will be measured by calculating the Weighted Cohen's kappa. Outcomes across different groups will be compared using statistical tests and bootstrap methods. CANKADO functions as the base system, allowing doctors to supervise APP content for patients and handle data gathering, while upholding principles of privacy, data security, and ethical AI practices. Intervention planned: Using the CINDERELLA APP, the patient will have access to supervised medical information approved by breast cancer experts, and the AI system will match patient's information to pictures showing the potential aesthetic outcome (spectrum of good-poor) according to different locoregional approach. Major eligibility criteria: Non-metastatic breast cancer patients eligible for either breast-conserving surgery or mastectomy with immediate reconstruction. Current enrollment: Recruitment is currently open at six study sites. The recruitment started on 8 August 2023, aiming to enroll at least 515 patients/arm. As of January 26, 2024, clinical study sites have successfully randomised 177 patients. Clinical trial information: NCT05196269 .

Abstract

Abstract

Abstract