Author + information
- Received October 26, 2009
- Accepted November 12, 2009
- Published online May 1, 2010.
- Troy M. LaBounty, MD⁎,⁎ (, )
- Robert J. Kim, MD⁎,
- Fay Y. Lin, MD, MA⁎,
- Matthew J. Budoff, MD‡,
- Jonathan W. Weinsaft, MD† and
- James K. Min, MD†
- ↵⁎Reprint requests and correspondence:
Dr. Troy M. LaBounty, Weill Cornell Medical College—New York Presbyterian Hospital, Starr Pavilion 4th Floor, 520 East 70th Street, New York, New York 10021
Objectives This study assessed the diagnostic performance of coronary computed tomography angiography (CTA) for the detection and exclusion of significant coronary artery stenosis as remotely interpreted on a mobile handheld device with dedicated medical imaging software.
Background Recent advances in technology now permit remote interpretation of medical imaging studies on mobile handheld devices, although the diagnostic performance of this approach is unknown.
Methods We evaluated 102 patients with stable chest pain and both 64-detector row coronary CTA and quantitative invasive coronary angiography. The diagnostic performance of remote coronary CTA interpretation was assessed using a mobile handheld device and employing dedicated software. The coronary CTA studies were examined in an intent-to-diagnose manner for the presence or absence of coronary artery stenosis ≥50% on a per-artery and per-patient level; results were compared with quantitative invasive coronary angiography. Two blinded imagers independently interpreted coronary CTA studies, with a third imager achieving consensus for discordance. Coronary CTAs were re-interpreted in random order to determine interobserver agreement. Finally, coronary CTAs were evaluated on a dedicated 3-dimensional imaging workstation; results were compared to mobile handheld device findings for intertechnology agreement.
Results The prevalence of significant coronary artery stenosis was 25% (26 of 102) at the per-patient level and 10% (40 of 405) at the per-artery level. Per-patient and per-artery sensitivity, specificity, and positive and negative predictive values were: 100% (26 of 26), 78% (59 of 76), 60% (26 of 43), and 100% (59 of 59), respectively; and 95% (38 of 40), 85% (310 of 365), 41% (38 of 93), and 99% (310 of 312), respectively. At the per-artery level, interobserver, intraobserver, and intertechnology agreement was 0.74, 0.89, and 0.75, respectively (p < 0.01 for all).
Conclusions The interpretation of coronary CTA using a mobile handheld device with dedicated software for medical image evaluation possesses high diagnostic accuracy for detection and exclusion of significant coronary stenosis.
Coronary computed tomography angiography (CTA) possesses high diagnostic accuracy for detection and exclusion of significant coronary artery stenosis (1–3). Historically, coronary CTA interpretation has been limited to dedicated stand-alone 3-dimensional (3D) imaging workstations, which are expensive and generally unavailable outside performing institutions. Recently, improvements in mobile handheld device technology have resulted in enhanced computer processing power, superior screen resolution, enriched user interfaces, and improved networking (4). In parallel with these advances, dedicated software that permits visualization and post-processing of medical images has been developed. The combination of these hardware and software improvements creates the potential for novel platforms that permit remote viewing of medical imaging studies by mobile handheld devices when dedicated 3D imaging workstations are not available. Although such platforms may enhance point-of-care for medical imaging studies, the diagnostic performance of medical image interpretation by mobile handheld devices has not been evaluated to date.
The aim of the present study was to compare the diagnostic accuracy of remote coronary CTA interpretation using a mobile handheld device to quantitative coronary angiography (QCA) for the detection and exclusion of significant coronary artery stenosis in chest pain patients without known coronary artery disease (CAD). We assessed the diagnostic performance at the per-patient and per-vessel level in an intent-to-diagnose fashion, including all patients and all vessels for final efficacy analysis.
We evaluated patients with stable chest pain who underwent 64-detector row coronary CTA and invasive coronary angiography (ICA) with QCA measurement, as part of the multicenter ACCURACY (Assessment by Coronary Computed Tomographic Angiography of Individuals Undergoing Invasive Coronary Angiography) Trial (1). The inclusion criteria of the ACCURACY trial included: age ≥18 years; typical or atypical chest pain symptoms; and referral for a nonemergent ICA. Exclusion criteria included: established iodinated contrast allergy; baseline renal insufficiency (creatinine ≥1.7 mg/dl); resting tachycardia (heart rate >100 beats/min); irregular heart rhythm; contraindication to beta-blocker, calcium-channel blocker, or nitroglycerin; pregnancy; or established history of CAD. The study was performed at 16 centers in the U.S., with approval of the institutional review board from each site. Details of the study design have been previously reported (1). For the present study, we included 102 patients from the ACCURACY trial who were randomly selected from the overall study cohort of 230 patients. The study was limited to this number based on the storage capacity of the handheld device.
Coronary CTA image acquisition
Subjects underwent coronary CTA by a standard protocol (1). Patients received oral and/or intravenous beta-blockers as needed to achieve a heart rate <65 beats/min and were given 0.4-mg sublingual nitroglycerin before the study. All scans were performed with 64-detector CT scanners (Lightspeed VCT, GE Healthcare, Milwaukee, Wisconsin) using a triple-phase contrast protocol: 60-ml iodixanol (GE Healthcare, Princeton, New Jersey), followed by a 40-ml 50:50 mixture of iodixanol and saline, followed by a 50-ml saline flush. The scan parameters included 64 × 0.625 mm collimation, tube voltage 120 mV, 350 to 780 mA, and dose-modulated retrospective electrocardiogram-gating. Radiation dose reduction algorithms using automodulation of tube current and electrocardiography-modulated imaging was performed. After coronary CTA scan completion, images were reconstructed at 75% of the R-R interval for the present study.
The coronary CTA images were reviewed using MIM Pro software version 1.0.4 (MIM Vista, Cleveland, Ohio) on an iPhone 3G device (Apple Inc., Cupertino, California), with a diagonal screen size of 8.9 cm and a screen resolution of 480 × 320 pixels. Full-resolution images were transferred over an encrypted 802.11 Wi-Fi network to the handheld device. Each study was transferred and uncompressed on the device in less than 5 min. As the MIM Pro software (MIM Vista) used for the present study did not have post-processing reconstruction capability, coronary CTA image interpretations were restricted to axial images at high resolution (0.625-mm in-plane slice thickness).
Two coronary CTA imagers independently interpreted studies in blinded fashion; a third blinded reader achieved consensus in cases of discordance. All 3 readers were level III–certified in coronary CTA performance and interpretation, and each reader had experience interpreting >1,000 coronary CTA examinations. Coronary CTA imager #1 had experience in using the particular handheld device for coronary CTA viewing and had previously interpreted more than 50 studies on the device as part of a separate study. Coronary CTA imager #2 had no experience using the handheld device prior to the study. To assess intraobserver variability, 1 imager blindly reinterpreted all coronary CTA examinations using the mobile handheld device in a random order, in blinded fashion, and at a minimum of 4 weeks after initial interpretation. To assess intertechnology variability, 1 reader blindly interpreted all coronary CTA examinations using a dedicated 3D imaging workstation (AW 4.4 Advantage Workstation with CardIQ software; GE Healthcare, Milwaukee, Wisconsin) using only axial images. One blinded, experienced reader interpreted coronary CTA studies using the same dedicated 3D imaging workstation and all necessary post-processing reconstruction techniques, including axial images, maximum intensity projections, multiplanar reformats, cross-section analyses, and volume-rendered imaging. To minimize recall bias, all studies were re-read in different sequences and at least 4 weeks apart. All studies on handheld devices were interpreted while sitting in a variety of lighted rooms, including rooms with both fluorescent and incandescent lighting.
The coronary CTA studies were assessed for maximal stenosis on a per-patient and per-artery level, and comparisons were performed between coronary CTA and QCA for the detection of significant coronary artery stenosis. The handheld device and software permitted touch-screen optimization of windowing and leveling (by moving 2 fingers in the x- and y-axes on the screen) (Fig. 1), scrolling in the z-axis (by using 1 finger on a scrollbar on the edge of the screen), image zooming (by pinching 2 fingers together or apart on the screen) (Fig. 2), and panning in the x-y axes (by dragging 1 finger on the screen) (Fig. 3). Each artery was examined by centering the artery on the screen and scrolling throughout the arterial course on axial images, with zoom and windowing at the discretion of the reader.
Studies were evaluated for the presence of significant coronary artery stenosis, defined as ≥50% luminal diameter stenosis, using the most severe stenosis per vessel (Figs. 4 and 5). Stenosis estimation was based upon comparisons of the luminal size at the site of stenosis of greatest severity to the nearest proximal site with the most normal appearance. Per-artery evaluation was performed for the left main, left anterior descending, left circumflex (including ramus intermedius), and right coronary arteries. For purposes of evaluation, the diagonal and septal branches were considered as part of the left anterior descending artery. The obtuse marginal, left posterolateral, and left posterior descending branches were considered as part of the left circumflex artery. The right ventricular marginal branches, right posterior descending artery, and right posterolateral branches were considered part of the right coronary artery. Each artery was examined in its entirety, and if any portion of an artery was considered nonevaluable, the entire artery was considered nonevaluable for the purposes of analysis. All portions of arteries were included for analysis, irrespective of luminal diameter size.
The ICA studies were performed in a standardized manner, as previously described, with all studies blindly interpreted by an independent reader using QCA (1).
Analyses were performed to determine the diagnostic performance of coronary CTA as interpreted by a mobile handheld device as compared with QCA for the detection and exclusion of ≥50% maximal diameter stenosis. Similar comparisons were performed using dedicated workstations limited to axial image review as well as dedicated workstations using post-processing reconstruction techniques. Nonevaluable arteries were examined in an intent-to-diagnose fashion, including all patients and all vessels for the final efficacy analysis. Arteries in which portions of the vessels were considered nonevaluable by expert coronary CTA imagers were considered as having ≥50% diameter stenosis. A secondary analysis was performed that excluded arteries and patients in which portions of the arteries were deemed nonevaluable. Patients in whom coronary CTA demonstrated at least 1 nonevaluable artery but in whom a ≥50% stenosis was noted in separate evaluable artery were included in the latter analysis.
Comparisons between groups were performed with McNemar and Fisher exact tests for paired and nonpaired categorical variables, respectively. Kappa tests were used to compare intraobserver, interobserver, and intertechnology agreement. To determine the 95% confidence intervals for proportions, the efficient-score method was used. SPSS version 17.0 (SPSS Inc., Chicago, Illinois) was used for all analyses. A 2-tailed p value <0.05 was deemed significant.
The study cohort included 102 stable chest pain patients who underwent coronary CTA before nonemergent ICA. Patient demographics are listed in Table 1. A total of 3 arteries from 3 patients could not be evaluated by QCA; these were excluded from further analysis, and the remaining 405 arteries were included in the study. Significant coronary artery stenosis at the 50% threshold was observed by QCA in 26 of 102 (26%) patients and 40 of 405 (10%) arteries, with significant coronary artery stenosis in the left main, left anterior descending, left circumflex, and right coronary arteries in 1%, 11%, 12%, and 16% of patients, respectively.
On coronary CTA, 17 of 405 (4%) arteries from 12 patients were considered nonevaluable by expert coronary CTA imagers. Significant coronary artery stenosis was observed in separate evaluable arteries in 7 of these patients, resulting in 5 patients where the presence or absence of significant CAD could not be determined at the per-patient level. The frequency of nonevaluable arteries was 0%, 2%, 4%, and 11% for the left main, left anterior descending, left circumflex, and right coronary arteries, respectively.
Using dedicated workstations, there were 10 arteries deemed nonevaluable for analysis on axial image review (p = 0.07 vs. handheld device), and 10 arteries deemed nonevaluable after use of post-processing reconstruction techniques (p = 0.19 vs. handheld device).
The diagnostic performance of coronary CTA as interpreted by a remote mobile handheld device in comparison to QCA is provided in Tables 2 and 3⇓. For the primary intent-to-diagnose analyses, all nonevaluable arteries on coronary CTA were assumed to exhibit significant coronary artery stenosis. Secondary analyses were performed with nonevaluable arteries excluded; if at least 1 artery was nonevaluable but significant CAD was identified in other arteries, the patient was included in the per-patient analysis. Per-patient and per-artery analyses revealed high sensitivity and negative predictive values, with moderate and fair specificity and positive predictive values, respectively.
Our coronary CTA imager #1 had prior experience using the handheld device to interpret coronary CTA studies, whereas coronary CTA imager #2 had no prior experience. The sensitivity of each coronary CTA imager was identical (100% [26 of 26] for each, p = 1.0), whereas coronary CTA imager #1 had higher specificity (78% [59 of 76] vs. 64% [49 of 76], p = 0.01). The negative predictive value was identical between readers (100% [59 of 59] vs. 100% [49 of 49], p = 1.0), with no significant difference in the positive predictive value (60% [26 of 43] vs. 49% [26 of 53], p = 0.31).
Impact of variables on diagnostic performance
Additional analyses were performed to assess the impact of specific variables on study interpretability and the diagnostic performance (Table 4). No differences in diagnostic performance of coronary CTA as interpreted by the mobile handheld device were noted for patients with higher coronary artery calcium scores or body mass index; however, both a heart rate ≤65 beats/min (p < 0.01) and intrascan heart rate variability <10 beats/min (p = 0.03) were associated with a significant increase in study interpretability, as well as improved or a trend toward improved specificity.
Comparison between interpretation techniques
There was good interobserver agreement between the coronary CTA imagers, and very good intraobserver agreement (Table 5). Intertechnology agreement comparing interpretation on the mobile handheld device to the dedicated 3D imaging workstation demonstrated good agreement.
A comparison of the diagnostic performance between coronary CTA interpreted on the handheld device and interpretations using a dedicated workstation demonstrates that both axial-only and complete post-processing image reconstruction result in higher specificity and positive predictive values, although the differences were not statistically significant (Table 6).
This study represents the first evaluation of the diagnostic performance of remote medical image interpretation via a mobile handheld device with dedicated medical imaging software. We employed coronary CTA examinations from the prospective multicenter ACCURACY trial, which included individuals with chest pain syndrome without known CAD. Our results demonstrate high diagnostic sensitivity and negative predictive value for a mobile handheld device to detect and exclude significant coronary artery stenosis, albeit with lower specificity and positive predictive values. These results demonstrate diagnostic performance characteristics that are on par with other noninvasive imaging modalities (such as stress echocardiography and myocardial perfusion scintigraphy).
In our study, we noted differences in the diagnostic accuracy of coronary CTA interpretation when comparing 2 coronary CTA imagers with different levels of familiarity with the mobile handheld technology. In particular, coronary CTA imager #1 had previously used the device for another study evaluating coronary CTA images in a different study population that consisted of 50 cases and had extensive experience using the particular device for additional applications. In contrast, coronary CTA imager #2 had never previously used the mobile handheld device for either medical image viewing or for other applications. These differences were associated with a reduced specificity with the less-experienced coronary CTA imager #2. The interobserver agreement was lower than the intraobserver agreement, which is consistent with these findings. Despite these differences, both readers' interpretations demonstrated very high sensitivity and negative predictive value, suggesting that the “learning curve” of this mobile handheld device is generally rapid and easy. Nevertheless, future studies evaluating the length of the “learning curve” of coronary CTA interpretation on mobile handheld devices should be performed.
The current version of the medical image viewing software was limited to review and processing of axial images alone; 3D post-processing was unavailable. In contrast, prior studies evaluating the diagnostic accuracy of coronary CTA for detection and exclusion of coronary artery stenosis have used dedicated imaging workstations capable of 3D post-processing techniques and have included interpretation using not only axial images, but also oblique images, multiplanar reformations, maximum intensity projections, and volume-rendered surface-shaded displays. Indeed, current scientific guidelines encourage the use of multiple 3D post-processing techniques when interpreting coronary CTA studies (5). A recent comparison of the diagnostic accuracy of these different techniques reported a sensitivity and specificity of 91% and 88% for axial images; in comparison, 3D oblique maximum intensity projections and multiplanar reformations had similar sensitivity, but higher specificity (6). As such, future iterations of medical image viewing software with 3D post-processing capability may result in slightly different results than what we observed in the current study. When compared to the diagnostic accuracy of coronary CTA interpreted on dedicated workstations (both analysis limited to axial images and analysis using post-processing techniques), there was a nonsignificant trend to reduced specificity using the handheld device. Nevertheless, the findings of the present analysis suggest that current-generation remote handheld device medical image review restricted to the axial plane does permit interpretation with high sensitivity and negative predictive value, and can be thus useful for detection and exclusion of significant coronary artery stenosis.
We used a mobile handheld phone device for this study, which possesses a screen resolution of 480 × 320 pixels. This small screen may theoretically influence diagnostic performance. Further, the number of shades of gray that can be discerned by the mobile handheld device has not been reported by the manufacturer, and any screen with finite grayscale discrimination that is less than the approximately 4,000 shades of gray of the CT Hounsfield unit scale may theoretically influence diagnostic performance. In our study, we noted that the intertechnology agreement was high between the mobile handheld device and the dedicated 3D imaging workstation. Given differences in screen resolution, bit depth and grayscales on different mobile handheld devices, it remains unknown whether the device used in this study differs from other handheld devices with respect to diagnostic accuracy. Future studies should be performed to evaluate these differences.
Stratification of patients by coronary artery calcium score and body mass index did not result in differences in coronary CTA study interpretability or diagnostic performance. In contrast, heart rates >65 beats/min and heart rate variability ≥10 beats per coronary CTA scan were associated with a significant reduction in study interpretability; whereas the diagnostic sensitivity was unaffected, the specificity was poorer for patients with faster heart rates and greater heart rate variability. In our study population, the number of patients with poorly controlled or highly variable heart rates was low, emphasizing the importance of patient selection and pre-medication with beta-blockers in coronary CTA examinations. Further, we employed prospectively acquired data from the multicenter ACCURACY trial, which restricted enrollment to those individuals without known CAD in whom a decision to perform invasive coronary angiography had already been made. The ACCURACY study cohort, as is reflected in the present study, identified an intermediate prevalence of significant coronary artery stenosis. As such, the diagnostic performance of the mobile handheld device technology in other study cohorts—such as those with known CAD or with alternate prevalence of significant coronary artery stenosis—remains unknown.
Finally, we employed beta software for interpretation of coronary CTA studies on a mobile handheld device. Although an application for this software has been submitted to the U.S. Food and Drug Administration, it is not currently approved for coronary CTA interpretation and our present results should be interpreted as proof-of-concept pending their approval.
To enhance point-of-care for individuals undergoing medical imaging studies, remote access to medical images has expanded beyond dedicated 3D imaging workstations. Numerous potential solutions have been developed that include off-site 3D imaging workstations with dedicated broadband access, thin-client servers, and more recently, mobile handheld devices. Contemporary advances in the field of mobile handheld technology now permit viewing and processing of medical images anywhere an Internet connection exists, which may permit sharing of studies for consultation with additional readers and which may enable remote medical image evaluation if on-site readers are unavailable. These developments carry the potential of improving the efficiency of diagnostic imaging evaluation, given the potentially expanded access to medical images. Further, the effects of medical image viewing via remote mobile device technology may influence patient factors such as medical compliance and lifestyle modification, as these images may be easily demonstrated to patients. Future studies should be performed to examine the clinical utility of mobile handheld devices in relation to workflow and patient care.
Importantly, the handheld device was limited to review of axial images, and the ability to perform 3D reconstructions may improve the diagnostic performance of coronary artery assessment on handheld devices. Additionally, noncoronary findings were not evaluated, and the diagnostic performance for detection of pulmonary emboli, aortic dissection, or other findings were not assessed. Finally, whereas the current study was limited to coronary CTA exams, remote image assessment may be feasible and should be explored with other imaging modalities including cardiac magnetic resonance imaging, echocardiography, nuclear perfusion imaging, and invasive coronary angiography.
The current handheld device represents an early step in the use of remote imaging technology in medical diagnosis, and as such is preliminary data. The confidentiality of patient data and images will need to be maintained both during remote transfer and while on the handheld device in compliance with government regulations. Further advances (such as the ability to perform 3D reconstructions) and additional research evaluating these devices may permit their future use in clinical care.
The present study demonstrates that remote interpretation of coronary CTA via a mobile handheld device with dedicated medical image evaluation software possesses high diagnostic performance for detection and exclusion of significant coronary artery stenosis at a per-patient and per-vessel level.
Dr. Min has a research grant and is on the Speakers' Bureau for GE Healthcare, has a research grant from Vital Images, and is a consultant for the Cordis Corporation. Dr. Budoff has research support and is on the Speakers' Bureau for GE Healthcare. The MIM Pro software (MIM Vista, Cleveland, Ohio) on the iPhone 3G device (Apple, Inc., Cupertino, California) is not currently approved for clinical use by the U.S. Food and Drug Administration.
- Abbreviations and Acronyms
- coronary artery disease
- computed tomographic angiography
- invasive coronary angiography
- quantitative coronary angiography
- Received October 26, 2009.
- Accepted November 12, 2009.
- American College of Cardiology Foundation
- Budoff M.J.,
- Dowe D.,
- Jollis J.G.,
- et al.
- Meijboom W.B.,
- Meijs M.F.,
- Schuijf J.D.,
- et al.