Abstract

Abstract

Abstract
The coronavirus disease 2019 (COVID-19) pandemic has impacted healthcare systems across the world. Chest radiography (CXR) can be used as a complementary method for diagnosing/following COVID-19 patients. However, experience level and workload of technicians and radiologists may affect the decision process. Recent studies suggest that deep learning can be used to assess CXRs, providing an important second opinion for radiologists and technicians in the decision process, and super-human performance in detection of COVID-19 has been reported in multiple studies. In this study, the clinical applicability of deep learning systems for COVID-19 screening was assessed by testing the performance of deep learning systems for the detection of COVID-19. Specifically, four datasets were used: (1) a collection of multiple public datasets (284.793 CXRs); (2) BIMCV dataset (16.631 CXRs); (3) COVIDGR (852 CXRs) and 4) a private dataset (6.361 CXRs). All datasets were collected retrospectively and consist of only frontal CXR views. A ResNet-18 was trained on each of the datasets for the detection of COVID-19. It is shown that a high dataset bias was present, leading to high performance in intradataset train-test scenarios (area under the curve 0.55–0.84 on the collection of public datasets). Significantly lower performances were obtained in interdataset train-test scenarios however (area under the curve > 0.98). A subset of the data was then assessed by radiologists for comparison to the automatic systems. Finetuning with radiologist annotations significantly increased performance across datasets (area under the curve 0.61–0.88) and improved the attention on clinical findings in positive COVID-19 CXRs. Nevertheless, tests on CXRs from different hospital services indicate that the screening performance of CXR and automatic systems is limited (area under the curve < 0.6 on emergency service CXRs). However, COVID-19 manifestations can be accurately detected when present, motivating the use of these tools for evaluating disease progression on mild to severe COVID-19 patients. © 2022, The Author(s).

Abstract

Abstract
Abstract Funding Acknowledgements: Type of funding sources: Public grant(s) – National budget only. Main funding source(s): Health Research Council of New Zealand (HRC) National Heart Foundation of New Zealand (NHF) Segmentation of the left ventricular myocardium and cavity in 3D echocardiography (3DE) is a critical task for the quantification of systolic function in heart disease. Continuing advances in 3DE have considerably improved image quality, prompting increased clinical uptake in recent years, particularly for volumetric measurements. Nevertheless, analysis of 3DE remains a difficult problem due to inherently complex noise characteristics, anisotropic image resolution, and regions of acoustic dropout. One of the primary challenges associated with the development of automated methods for 3DE analysis is the requirement of a sufficiently large training dataset. Historically, ground truth annotations have been difficult to obtain due to the high degree of inter- and intra-observer variability associated with manual 3DE segmentation, thus, limiting the scope of AI-based solutions. To address the lack of expert consensus, we instead used labels derived from cardiac magnetic resonance (CMR) images of the same subjects. By spatiotemporally registering CMR labels to corresponding 3DE image data on a per subject basis (Figure 1), we collated 520 annotated 3DE images from a mixed cohort of 130 human subjects (2 independent single-beat acquisitions per subject at end-diastole and end-systole) consisting of healthy controls and patients with acquired cardiac disease. Comprising images acquired across a range of patient demographics, this curated dataset exhibits variation in image quality, 3DE acquisition parameters, as well as left ventricular shape and pose within the 3D image volume. To demonstrate the utility of such a dataset, nn-UNet, a self-configuring deep learning method for semantic segmentation was employed. An 80/20 split of the dataset was used for training and testing, respectively, and data augmentations were applied in the form of scaling, rotation, and reflection. The trained network was capable of reproducing measurements derived from CMR for end-diastolic volume, end-systolic volume, ejection fraction, and mass, while outperforming an expert human observer in terms of accuracy as well as scan-rescan reproducibility (Table I). As part of ongoing efforts to improve the accuracy and efficiency of 3DE analysis, we have leveraged the high resolution and signal-to-noise-ratio of CMR (relative to 3DE), to create a novel, publicly available benchmark dataset for developing and evaluating 3DE labelling methods. This approach not only significantly reduces the effects of observer-specific bias and variability in training data arising from conventional manual 3DE analysis methods, but also improves the agreement between cardiac indices derived from 3DE and CMR. Figure 1. Data annotation workflow Table I. Results