Author + information
- Ashley N. Beecy, MD,
- Qi Chang, MS,
- Khalil Anchouche, BA,
- Lohendran Baskaran, MD,
- Kimberly Elmore, MHA,
- Kranthi Kolli, PhD,
- Hao Wang, MS,
- Subhi Al’Aref, MD,
- Jessica M. Peña, MD, MPH,
- Ashley Knight-Greenfield, MD,
- Praneil Patel, MD,
- Peng Sun, PhD,
- Tong Zhang, PhD,
- Hooman Kamel, MD,
- Ajay Gupta, MD and
- James K. Min, MD∗ ()
- ↵∗Departments of Radiology and Medicine, Dalio Institute of Cardiovascular Imaging, New York-Presbyterian Hospital and Weill Cornell Medicine, 413 East 69th Street, Suite 108, New York, New York 10021
Stroke is the fourth leading cause of death and the leading cause of morbidity and long-term disability in adults (1). Early diagnosis of acute ischemic stroke (AIS) in patients is difficult and the need for this early diagnosis will increase as the population ages and acute therapies evolve. Deep learning (DL) is a novel machine learning approach that enables automated extraction and classification of imaging features. This study aims to use DL to enhance our ability to evaluate for AIS while offering both automation and confirmation of a diagnosis.
All patients with AIS admitted to the New York-Presbyterian Hospital/Weill Cornell Medical Center between 2011 and 2014 were prospectively registered in the Cornell Acute Stroke Academic Registry. A total of 114 patients with noncontrast-enhanced computed tomography (CT) scan evidence of acute brain infarction were randomly selected. Board-certified neuroradiologists, blinded to the derivation of the model, annotated the images by marking infarct area to obtain an expert consensus interpretation. Digital Imaging and Communication in Medicine data was split randomly into a training set and test set (80:20). A 3-dimensional multiscale, fully convolutional deep learning neural network was developed and trained on the training set of CT images (2). Neural networks are mathematical models built to recognize patterns in images and predict outcomes based on a predefined ground truth (3). The performance of this model was independently tested using the test set and compared with the expert consensus interpretation. Diagnostic accuracy, sensitivity, specificity, and area under the receiver-operating characteristic curve (AUC) for the DL algorithm were calculated at a voxel and image level on the test set. Computer-generated heat maps were created to denote the possibility of infarct (Figure 1).
The mean age of the study sample was 76 ± 13 years, and 62 (55%) patients were female. Imaging datasets were split for the 114 patients into a training set (n = 92) and a testing set (n = 22). In the training set of 5,888 images, infarction was present in 602 (10.2%) images. In the testing set of 920 images, infarction was present in 130 (14.1%) images. A total of 1.5 billion voxels were used to train the model.
The AUC for the DL algorithm for voxel accuracy was 0.973 (95% confidence interval: 0.972 to 0.974). Diagnostic accuracy, sensitivity, and specificity were 92%, 93%, and 92%, respectively. Positive predictive value (PPV) and negative predictive value (NPV) were 86% and 92%, respectively. The AUC for the DL model for automated diagnosis of infarction at an image level was 0.91 (95% confidence interval: 0.90 to 0.94). The corresponding diagnostic accuracy, sensitivity, and specificity were 88%, 65%, and 91%, respectively. PPV and NPV were 49% and 95%.
These results demonstrate that DL-based neural networks can be trained to identify acute brain infarction on CT scan. Our present results reveal a diagnostic accuracy of 93% compared with expert interpretation by blinded neuroradiologists. Of importance, at a cutpoint AUC of 0.91, we observed a NPV of 95% with a lower PPV of 49%. These findings suggest that the DL algorithms may allow for better ruling out of AIS, and that future methods are required for verification of acute brain infarction by machine learning.
To our knowledge, this study represents the largest to date to evaluate deep learning methods for autodiagnosis of acute brain infarct on CT scan, with levels of overall diagnostic accuracy comparable to board-certified specialists. Our results support the use of machine-learning methods for automated diagnosis of stroke to improve the diagnostic efficiency of radiologists and health systems. Future studies may also allow classification of any CT-visualized feature, as well as other types of pathologies.
The study is not without limitations. This study was performed at a single tertiary care academic center, and it is possible that the study results would prove different if CT scans were judged by less well-trained radiologists, a possibility that can be mitigated by the DL solutions. Also, the DL model in this study was trained by supervised learning methods on a generally small patient dataset and lacked a control group for comparison. It is important for future prospective analysis to be performed to validate our study findings.
In conclusion, machine-learned models using novel DL techniques enable highly accurate automated diagnosis of acute brain infarction. These algorithms have the potential to assist radiologists while improving patient outcomes.
Please note: This work was supported by a generous gift from the Dalio Institute of Cardiovascular Imaging and the Michael Wolk Foundation. Dr. Min serves on the scientific advisory board of Arineta; has ownership in MDDX; and has a research agreement with GE Healthcare. All other authors have reported that they have no relationships relevant to the contents of this paper to disclose. Sherif Nagueh, MD, served as the Guest Editor for this paper. Drs. Beecy and Chang contributed equally to this work and are joint first authors.