Author + information
- Received December 10, 2013
- Accepted December 12, 2013
- Published online March 1, 2014.
- M. Khalid Mojadidi, MD∗∗ (, )
- Scott C. Roberts, BS∗,
- Jared S. Winoker, BS∗,
- Jorge Romero, MD∗,
- David Goodman-Meza, MD∗,
- Rubine Gevorgyan, MD† and
- Jonathan M. Tobis, MD†
- ∗Division of Cardiology, Montefiore Medical Center, Albert Einstein College of Medicine, Bronx, New York
- †Program in Interventional Cardiology, Division of Cardiology, David Geffen School of Medicine, University of California at Los Angeles, Los Angeles, California
- ↵∗Reprint requests and correspondence:
Dr. M. Khalid Mojadidi, Department of Medicine, Albert Einstein College of Medicine, 1400 Pelham Parkway South, Building 1, Rm 3N1, Bronx, New York 10461.
Objectives The aim of this meta-analysis was to determine the accuracy of transcranial Doppler (TCD) compared with transesophageal echocardiography (TEE) as the reference.
Background Right-to-left shunting (RLS), usually through a patent foramen ovale (PFO), has been associated with migraine, cryptogenic stroke, and hypoxemia. With emerging observational studies and clinical trials on the subject of PFO, there is a need for accurate diagnosis of PFO in patients with these conditions, and those being considered for transcatheter closure. Although a TEE bubble study is the current standard reference for diagnosing PFO, the TCD bubble study may be a preferable alternative test for RLS because of its high sensitivity and specificity, noninvasive nature, and low cost.
Methods A systematic review of Medline, the Cochrane Library, and Embase was done to look for all the prospective studies assessing intracardiac RLS using TCD compared with TEE as the reference; both tests were performed with a contrast agent and a maneuver to provoke RLS in all studies.
Results A total of 27 studies (29 comparisons) with 1,968 patients (mean age 47.8 ± 5.7 years; 51% male) fulfilled the inclusion criteria. The weighted mean sensitivity and specificity for TCD were 97% and 93%, respectively. Likewise, the positive and negative likelihood ratios were 13.51 and 0.04, respectively. When 10 microbubbles was used as the embolic cutoff for a positive TCD study, TCD produced a higher specificity compared with when 1 microbubble was used as the cutoff (p = 0.04); there was, however, no significant change in sensitivity (p = 0.29).
Conclusions TCD is a reliable, noninvasive test with excellent diagnostic accuracies, making it a proficient test for detecting RLS. TCD can be used as a part of the stroke workup and for patients being considered for PFO closure. If knowledge of the precise anatomy is required, then TEE can be obtained before scheduling a patient for transcatheter PFO closure.
Patent foramen ovale (PFO) is a remnant of the fetal circulation that is present in 20% to 25% of the population (1–3). Transient right-to-left shunting (RLS), usually through a PFO, has been implicated in the pathophysiology of stroke, migraine, and hypoxemia (3–6). A meta-analysis of observational studies and a recent meta-analysis of the CLOSURE 1 (Closure or Medical Therapy for Cryptogenic Stroke with Patent Foramen Ovale), RESPECT (Closure of Patent Foramen Ovale Versus Medical Therapy After Cryptogenic Stroke), and PC (Percutaneous Closure of Patent Foramen Ovale in Cryptogenic Embolism) trials suggest that PFO occluding devices reduce the recurrence of stroke and transient ischemic attack at higher rates than conventional medical treatment alone (pooled hazard ratio: 0.59, 95% confidence interval [CI]: 0.36 to 0.97; p = 0.04) (7,8). These data, along with the evaluation of patients with severe migraines or other PFO-associated conditions, make it essential to accurately diagnose RLS in patients being considered for PFO closure.
Whereas contrast transesophageal echocardiography (TEE) is considered the gold standard for diagnosing PFO (9,10), contrast transcranial Doppler (TCD) is increasingly being used for safe, noninvasive, and cost-effective screening of intracardiac RLS (11–37). The aim of this study was to expand on prior reviews of TCD to provide the first meta-analysis that methodically assesses the diagnostic accuracy of TCD in evaluating for an intracardiac RLS.
Relevant citations were searched for on Medline, the Cochrane Library, and Embase. The search was completed in August 2013, yielding literature since 1913. The terms used in the search were “PFO” OR “patent foramen ovale” OR “right to left shunt” OR “atrial septal defect” AND “TCD” OR “transcranial Doppler” OR “TEE” OR “echo” OR “transesophageal echo” OR “transesophageal echocardiogram” OR “transesophageal echocardiography.”
The references of all primary studies as well as those from known reviews were analyzed to find cited studies that were not found by initial searches. No restrictions were used regarding publication language. Abstracts lacking peer-reviewed manuscripts were omitted because they would not have enough data required for the meta-analysis.
Selection of studies
Studies that were identified were analyzed by 3 independent reviewers (M.K.M., S.C.R., and J.S.W.). Each study was screened for pre-set inclusion criteria:
1. Original prospective studies (reviews, abstracts, isolated cases, commentaries, editorials, and letters were excluded)
2. Subject age ≥18 years
3. Studies were selected if they included at least 20 patients with suspected intracardiac RLS who were screened by TCD and confirmed by TEE as a reference. If a study conducted both TCD and TEE, but did not consider TEE as the gold standard, we calculated the appropriate parameters assuming TEE as the reference comparison.
4. TCD and TEE accuracies calculated utilizing a provocation maneuver.
5. Able to interpret diagnostic accuracies by adequate demonstration of true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN).
6. If a study compared different TCD protocols (such as comparing accuracy of different contrast injection sites or different types of contrast) and also provided the variables to calculate the different accuracies (i.e., the TP, FP, FN, and TN), then each methodology was considered a separate comparison in the final analysis. A sensitivity analysis was then conducted to demonstrate the effect of varying methodologies on accuracy of TCD.
The data were extracted onto a spreadsheet with information regarding study design, cohort size, age, sex, TCD/TEE indication, contrast type, method of provocation (Valsalva maneuver or cough), microbubble cutoff used for a positive TCD/TEE study, and test accuracy results (TP, FP, FN, and TN).
The quality of each study was assessed by evaluating items considered relevant to the review topic, on the basis of the Quality Assessment of Diagnostic Accuracy Studies (version 2) instrument (QUADAS-2) (38). Three reviewers (M.K.M., S.C.R., and J.R.) independently assessed the quality items, and discrepancies were resolved by consensus.
Sensitivities and specificities were calculated for every study using a more recently developed bivariate random effects model (39). The bivariate approach assumed that logit transforms of sensitivity and specificity from individual studies are from a bivariate normal distribution. The bivariate approach is considered superior to the standard summary receiver-operating characteristic (ROC) approach (40) because: 1) it assesses heterogeneity across studies, providing a summary estimate of sensitivity and specificity; 2) it models sensitivity and specificity jointly so that a 95% confidence ellipse around the summary estimate can be calculated; 3) it allows calculation of positive and negative likelihood ratios; 4) it allows one to directly compare sensitivity and specificity between methods; and 5) several choices are available to obtain a summary ROC curve (39). In this study, the summary ROC curve was obtained by transforming the regression line of logit sensitivity on logit specificity into ROC space. Publication bias was assessed for each analysis using Deeks's method (41). Post-test probabilities were calculated using Bayes' normogram, which requires converting pre-test probabilities to odds and backtracking post-test odds to probabilities.
We assessed between-study heterogeneity visually, by plotting sensitivity and specificity in the ROC curves (42). The analyses were conducted using STATA 12 (Metandi Syntax, StataCorp LP, College Station, Texas), and the figures were generated using STATA graph editor.
We further evaluated whether the performance of each technique depends on features of the technique and patient characteristics. A logistic regression for each technique was used to model the sensitivity of these factors.
We identified 174,961 reports, of which 53 studies were considered for detailed evaluation and 27 studies met the inclusion criteria (11–37). Two studies demonstrated 2 different accuracies for TCD by comparing different protocols; we therefore included 29 comparisons in our final analysis consisting of 1,968 patients (mean age 47.8 ± 5.7 years; 51% male).
Twenty-six studies were excluded from the final analysis because they did not meet the inclusion criteria; the excluded studies, along with the reasons for exclusion, are provided in the Online Appendix. Figure 1 describes the study selection method used for this analysis.
There was no publication bias using Deeks's method, with a p value of 0.51 (Fig. 2).
Using the items for evaluating diagnostic studies with QUADAS-2, the risk of bias and applicability concerns for all the studies were assessed (Fig. 3). Most studies had high methodological quality with very minimal concerns regarding applicability of the test in clinical practice (Fig. 3). Uninterpretable results and withdrawals that represented the “flow and timing” section were unclear in 86% (25 of 29) of the comparisons. Data on these 2 parameters are often not reported in diagnostic accuracy studies, with the uninterpretable results and withdrawals simply removed from the analysis. This may lead to a biased assessment of test characteristics. Whether or not bias will arise depends on the possible correlation between uninterpretable test results and the true disease status. Uninterpretable results frequently occur randomly and are unrelated to the true disease status of the individual. Therefore, in theory, these should not have any effect on test performance. Likewise, 3 of 29 comparisons (10%) were not conducted in a blinded fashion; this may have led to review bias in these particular studies. Some did not clearly specify whether blinding occurred in the index or reference tests. A review bias may potentially lead to inflated measures of diagnostic accuracy.
A total of 29 comparisons met all inclusion criteria and were used for further meta-analytic calculations. Table 1 describes the characteristics of the included studies, and Table 2 describes the diagnostic accuracies of the studies. The major clinical indication for performing a TCD in most of the studies was stroke followed by migraine headache. Of the 29 comparisons that performed TCD and TEE with contrast, 12 (41%) used agitated saline as the contrast agent, 10 (35%) used Echovist (Schering, Berlin, Germany), 4 (14%) used a gelatin-based solution, and 3 (10%) used 2 different contrast agents. The Valsalva maneuver was used as the provocation method in 86% (25 of 29) of the comparisons, Valsalva with cough was used in 10% (3 of 29), and the provocation method used was unknown in 3% (1 of 29) of the comparisons. The majority of the comparisons used ≥1 microbubble as the embolic cutoff for a positive TCD (62%; 18 of 29) or TEE (52%; 15 of 29).
When all eligible studies were pooled into the diagnostic accuracy meta-analysis, the sensitivity of TCD for the diagnosis of intracardiac RLS was 97.0% (95% CI: 94.0% to 98.0%; I2 = 71.02%) (Fig. 4A), the specificity was 93.0% (95% CI: 86.0% to 97.0%; I2 = 89.8%) (Fig. 4B), the positive likelihood ratio (LR+) was 13.51 (95% CI: 6.54% to 27.92%; I2 = 89.1%) (Fig. 5A), and the negative likelihood ratio (LR−) was 0.04 (95% CI: 0.02% to 0.07%; I2 = 69.3%) (Fig. 5B). The studies were heterogeneous in their estimates of sensitivity, specificity, LR+, and LR− (p < 0.01). The hierarchical summary ROC curves are illustrated in Figure 6.
We performed a sensitivity analysis for different contrast agents used, different provocation maneuvers, different microembolic cutoffs for a positive index (TCD) and reference test (TEE), different timings of provocation maneuver, and unilateral versus bilateral middle cerebral artery (MCA) insonation (Table 3). There was no significant difference in sensitivity or specificity when different contrast agents (agitated saline, Echovist, and gelatin-based solutions) were utilized (p > 0.05). However, there was a trend towards Echovist producing a higher sensitivity (95% sensitivity) compared with when gelatin-based solutions (94% sensitivity) were used (p = 0.06). Studies that used Valsalva with cough did not produce a higher sensitivity or specificity compared with studies that only used Valsalva as their provocation maneuver (p > 0.7). When 10 microbubbles was used as the embolic cutoff for a positive TCD study, TCD produced a higher specificity compared with when 1 microbubble was used as the cutoff (p = 0.04); there was, however, no significant change in sensitivity (p = 0.29). There was no significant difference in sensitivity or specificity between studies that used 1 microbubble compared with studies that used 3 microbubbles as the cutoff for a positive TEE (p > 0.1). There were no significant differences in sensitivity or specificity in studies that performed the provocation maneuver for ≤5 s compared with >5 s (p > 0.50). Lastly, there was a trend towards insonation of the unilateral MCA producing a higher specificity (95% specificity) compared with when bilateral MCA insonation (89% specificity) was used (p = 0.09). There was no significant difference in sensitivity between unilateral and bilateral MCA insonation (p = 0.15).
Figure 7 demonstrates the pre- and post-test probabilities of detecting an intracardiac RLS with TCD in the general population, and in our study cohort consisting mainly of patients with stroke or migraine. Because PFO is present in 20% to 25% of the adult population and in approximately 50% to 55% of patients with migraine or cryptogenic stroke (1–3,14), these respective prevalences were assumed to demonstrate the likelihood of detecting a RLS by TCD in the 2 populations.
With a LR+ of 14 and LR− of 0.04, a TCD performed in the general population consisting of a 20% RLS prevalence will have 77% probability of a positive result being a TP and a 1% probability of a negative result being a FN (Fig. 7A). These probabilities significantly change in a population of patients with stroke or migraine who undergo TCD. With a LR+ of 14 and LR− of 0.04, a TCD performed in patients with migraine or stroke consisting of a 50% to 55%% RLS prevalence will have 93% to 94% probability of a positive result being a TP and a 4% probability of a negative result being a FN (Figs. 7B and 7C).
Our study demonstrates that TCD detects intracardiac RLS with a sensitivity of 97% and a specificity of 93% when TEE is used as the reference. TCD has an excellent LR+ of 14 and LR− of 0.04, making it a proficient test to rule out or rule in RLS in the stroke or migraine population (Fig. 7). Increasing the microembolic threshold for a positive TCD from 1 to 10 microbubbles increases the specificity of TCD without compromising sensitivity. This is the first meta-analysis that assesses the accuracy of TCD for detecting intracardiac RLS compared with TEE as the reference. It is also the first meta-analysis that compares different protocols of TCD for detecting intracardiac RLS.
In the evaluation of patients who may have a PFO, several methods are available to determine whether a RLS is present: TCD, transthoracic echocardiography, TEE, or intracardiac echo. Although TEE is considered the gold standard for diagnosing PFO (9,10), studies that compared TEE with autopsy or intraoperative detection of PFO demonstrated that the diagnosis is sometimes missed by TEE (43,44). Studies that compared the accuracy of TEE in the detection of PFO with that of catheterization and/or surgery have demonstrated a sensitivity of 91% to 100% and accuracy of 88% to 97% (45,46). Thus, our results may have underestimated the sensitivity of TCD; Spencer et al. (46) demonstrated a sensitivity of 98% and accuracy of 94% when TCD was compared with PFO detection during catheterization as the reference. In addition, some stroke patients may have dysfunctional swallowing or poor cooperation, making TEE difficult to perform, affecting the test accuracy because of an inadequate Valsalva maneuver. FN TEE tests may also be explained by ineffective Valsalva secondary to sedation or by the presence of the TEE probe in the patient's esophagus (37,47). Thus, the decreased specificity of TCD may reflect some true shunts that are not recognized by TEE.
Currently, TEE is often utilized when routine diagnostics cannot identify a stroke etiology, especially in young patients (48). However, TEE may be uncomfortable and time-consuming for some patients. Although unusual, severe complications such as esophageal bleeding or perforation may occur. Contraindications of TEE such as esophageal varices, Barrett's esophagus, Zenker's diverticulum, esophageal or pharyngeal carcinoma, strictures, Mallory-Weiss tears, or patients with a serious bleeding risk make it important to have a reliable alternative in contemporary clinical practice (49).
TCD is an alternative method for indirectly diagnosing PFO by assessing the presence of a RLS. It employs the functional assessment of the shunt using insonation of at least 1 MCA during a venous injection bubble study and Valsalva maneuver. Although intracardiac shunting through a PFO is usually directed from the left atrium to the right atrium, the release of the Valsalva maneuver allows right atrial pressure to briefly exceed left atrial pressure, resulting in transient reversal of flow. After contrast injection, bubbles enter the systemic circulation during this transient RLS, resulting in microembolic signals in the cerebral arteries that are detected by TCD. TCD utilizes a pulsed Doppler transducer that detects the velocity and intensity of cerebral arterial blood flow by spectral analysis. Although less common, color duplex TCD may also be used in addition to spectral TCD to confirm the positivity of an exam; 4 of the included studies also utilized color TCD (26,29,31,34). TCD provides good patient tolerance and excellent accuracies, making it a useful alternative for detecting RLS in patients with stroke or migraine, or in patients being considered for PFO closure.
It is thought that the accuracy of TCD can vary, depending on the effectiveness of the provocative maneuver, skill of the operator, location of the intravenous needle, positioning of the ultrasound probes, type of contrast, and the TCD model (Power M-mode vs. single-gated TCDs). The Consensus Conference of Venice outlined some key standards for performing a TCD including the use of an 18-gauge needle in the cubital vein, preferable utilization of agitated saline as the contrast, and application of the Valsalva maneuver as the provocation maneuver for more than 10 s (50). Although these guidelines are based on data derived from observational studies before the year 2000, some differences in methodology exist even now, depending on the institution. In addition, the Consensus Conference of Venice did not establish any microbubble number cutoff to define a positive TCD study. According to our sensitivity analysis, using a different contrast agent, provocation maneuver, insonation of unilateral versus bilateral MCAs, and a different microbubble cutoff for a positive shunt with TEE do not significantly affect the accuracy of TCD. However, increasing the TCD threshold for a positive shunt from 1 microbubble to 10 microbubbles significantly increased the specificity of TCD from 89% to 100% (p = 0.04) without affecting sensitivity (from 98% to 97%; p = 0.29). Our data support use of a higher microembolic threshold for a positive shunt by TCD; this finding is supported by previously published literature that demonstrates that using a higher cutoff for a positive TCD, such as with the Spencer Logarithmic Scale, which uses 30 microbubbles as a cutoff instead of 1 microbubble, can decrease the number of FP TCDs that occur because of clinically insignificant shunts (46). TCD has a higher sensitivity than specificity as it is difficult for a TCD to differentiate between pulmonary and intracardiac shunts, which may sometimes be misinterpreted on a TEE as well. TCD does not show the operator the anatomic position of the RLS, nor can it distinguish between a PFO, an atrial septal defect, or a pulmonary arteriovenous fistula. Thus, patients who have a clinical indication for PFO closure and have a positive TCD may be sent to the catheterization laboratory, where a TEE, or alternatively, intracardiac echocardiography can be performed before transcutaneous closure. For a suspected pulmonary arteriovenous fistula, a pulmonary artery injection of echo contrast or a pulmonary angiogram may confirm the diagnosis.
Of 29 included comparisons, 3 (10%) were not conducted in a blinded fashion; this may have led to review bias in these particular studies.
Another limitation of our review was the lack of studies that utilized Power M-mode TCD. The sensitivity of Power M-mode TCD has been demonstrated to be higher than older single-gated TCDs for the diagnosis of intracardiac RLS when catheterization was used as the reference (46).
This study was also limited by the heterogeneity of the included studies. In this meta-analysis, we attempted to perform a sensitivity analysis on different protocols, where possible, to assess the effect of changing the TCD protocol on accuracy of the test. However, Hamann et al. (20) described 2 different accuracies of TCD by comparing contrast injection through the antecubital versus the femoral veins. We were unable to compare antecubital versus femoral injection in this meta-analysis because of the lack of other studies utilizing the femoral injection site. It has been reported that femoral injection of agitated saline produces a higher sensitivity for the detection of RLS using TCD (51).
TCD is a reliable, noninvasive alternative to TEE for the diagnosis of RLS, with an excellent sensitivity and specificity of 97% and 93%, respectively. Increasing the microembolic threshold for a positive TCD from 1 microbubble to 10 microbubbles significantly improves the specificity of TCD without compromising sensitivity. With a LR+ of 14 and LR− of 0.04, TCD is the test of choice for detecting RLS through a PFO in patients with cryptogenic stroke or migraine, having a post-test probability of 93% to 94% when testing positive, and a post-test probability of 4% when testing negative.
For an expanded Methods section, please see the online version of this paper.
Dr. Tobis is a consultant for St. Jude Medical, Inc. and W.L. Gore, Inc. All other authors have reported that they have no relationships relevant to the contents of this paper to disclose.
- Abbreviations and Acronyms
- confidence interval
- false negative
- false positive
- likelihood ratio
- middle cerebral artery
- patent foramen ovale
- Quality Assessment of Diagnostic Accuracy Studies
- right-to-left shunt
- receiver-operating characteristic
- transcranial Doppler
- transesophageal echocardiography
- true negative
- true positive
- Received December 10, 2013.
- Accepted December 12, 2013.
- American College of Cardiology Foundation
- Agarwal S.,
- Bajaj N.S.,
- Kumbhani D.J.,
- Tuzcu E.M.,
- Kapadia S.R.
- Rengifo-Moreno P.,
- Palacios I.F.,
- Junpaparp P.,
- Witzke C.F.,
- Morris D.L.,
- Romero-Corral A.
- Pearson A.C.,
- Labovitz A.J.,
- Tatineni S.,
- Gomez C.R.
- Jauss M.,
- Kaps M.,
- Keberle M.,
- Haberbosch W.,
- Dorndorf W.
- Droste D.W.,
- Reisener M.,
- Kemény V.,
- et al.
- Droste D.W.,
- Kriete J.U.,
- Stypmann J.,
- et al.
- Heckmann J.G.,
- Niedermeier W.,
- Brandt-Pohlmann M.,
- Hilz M.J.,
- Hecht M.,
- Neundörfer B.
- Droste D.W.,
- Jekentaite R.,
- Stypmann J.,
- et al.
- Droste D.W.,
- Lakemeier S.,
- Wichter T.,
- et al.
- González-Alujas T.,
- Evangelista A.,
- Santamarina E.,
- et al.
- Littenberg B.,
- Moses L.E.
- Harloff A.,
- Handke M.,
- Reinhard M.,
- Geibel A.,
- Hetzel A.
- Gevorgyan R.,
- Perlowski A.,
- Shenoda M.,
- Mojadidi M.K.,
- Agrawal H.,
- Tobis J.M.