Cardiovascular imaging is a source of innovation and controversy for the health care community. Cardiologists and radiologists are now capable of obtaining high quality images that describe myocardial function and perfusion, define risk of major clinical events, and show coronary anatomy without need for invasive instrumentation (1). At the same time, there is concern that the rapid dissemination of cardiovascular imaging is a prime example of a costly technology that is enthusiastically embraced without appropriate supporting scientific evidence (2- 3).

During the past 5 years, medical imaging has grown substantially, with Medicare Part B costs alone increasing from $6.89 billion in 2000 to $14.11 billion in 2005 (105%) of which an estimated one-third is cardiovascular (3- 4). In addition, there is inconsistent use, with some areas of the country having utilization rates 10 times those of others (5). There is no clear explanation for the rapid growth; it cannot be ascribed entirely to aging of the population, changing disease rates, or improved outcomes (3- 4). The “value” of imaging in terms of improved health outcomes or reduced cardiovascular events remains subjective, with limited evidence, often generated with flawed research methodology (6- 7). There are also concerns that imaging can cause harm (8- 9), that there are few rigorous regulatory controls, and that utilization is at least in part driven by self-referral (10) and, in some cases, even direct-to-consumer advertising (11).

A commonly cited model for efficacy in imaging describes 6 hierarchical tiers of evidence: 1) technical efficacy; 2) diagnostic accuracy; 3) diagnostic thinking; 4) therapeutic efficacy; 5) patient outcome; and 6) societal efficacy (12- 14). A recently convened American College of Cardiology–Duke University think tank on imaging quality in cardiovascular medicine (15), noted that imaging research has primarily focused on diagnostic and prognostic accuracy, with little work directed at determining the direct impact of imaging on patient outcomes. As a result, among 745 recommendations for cardiovascular imaging in American College of Cardiology and American Heart Association guidelines, only 1% are based on Level of Evidence: A (16). In contrast, in cancer medicine, randomized trials have been completed or are under way for assessing the ability of imaging technologies to prevent major clinical events due to breast (17) or lung cancer (18).

Though it may seem logical that diagnosing disease with “better” imaging tests will yield better outcomes, there are reasons why this may not be so. For example, some disease detected by sensitive technologies in fact reflects subclinical disease that if left alone would never become clinically manifest (19). This was discovered during large-scale studies of mass screening for neuroblastoma in children (20). Another unintended consequence of advanced imaging may be the detection of “nontarget” findings, such as noncalcified lung nodules, that may not have clinical relevance but require additional testing and/or procedures. Therefore, a number of scientists have argued that a preferred way to definitively determine whether or not any new diagnostic test improves outcomes is through properly designed randomized trials using clinical events as outcomes (21). However, there are a number of major methodological difficulties in designing and implementing randomized trials in which imaging tests themselves are the target of randomization (6). Effects, by definition, have to be indirect as tests do not directly affect clinical status. Instead we must presume that they lead clinicians and patients to modify behavior, which hopefully will lead to fewer clinical events.

Several issues represent important considerations when planning trials to determine if imaging can affect outcomes.

The initial consideration is whether one is testing a strategy of performing an imaging test versus not performing any imaging, or whether a comparison is desired between distinct imaging modalities. As an example of the latter design, 103 patients with chronic coronary artery disease (CAD) and left ventricular (LV) dysfunction being considered for revascularization (22) were randomized to either single-photon emission computed tomography (SPECT), myocardial perfusion imaging (MPI) or positron emission tomography (PET) for determination of viability. The imaging information was provided to clinicians for decision making blinded with regard to the imaging modality (with polar maps showing areas of ischemia, infarction, and the like) and patients were followed for 2- to 3-year outcomes. There was no difference in event-free survival between the 2 groups, suggesting that the use of either imaging modality to inform revascularization decisions results in similar outcomes. An ongoing study that represents the “imaging versus no imaging” approach is the WOMEN (What is the Optimal Method for Ischemia Evaluation in WomeN?) study, in which women with suspected CAD are randomized to an initial evaluation strategy of SPECT MPI versus an initial exercise electrocardiography (ECG) testing strategy, with the end point of 2-year negative predictive value for outcome events (23). These studies demonstrate that it is feasible to subject imaging modalities to the same rigorous comparisons that are standard for therapeutics.

An area of substantial uncertainty in the evaluation of imaging outcomes is the appropriate end points for use in trials. Ideally, end points would involve important natural history outcomes such as death, cardiac death or composites of cardiac death, and nonfatal cardiovascular events including myocardial infarction (MI). However, the many decisions made “downstream” from the imaging results have a highly significant effect on outcomes, such that the imaging results themselves are only 1 of many influences on outcomes, and thus challenging to isolate. This has led to considerations of other end points occurring over a shorter time horizon, including such metrics as cost-to-diagnosis, cost-to-predict event, cost-to-prevent nonfatal events, and behavior change with risk factor modification.

Efficacy refers to the performance characteristics of a test under ideal conditions performed and interpreted by experts. Effectiveness refers to test performance under “real-life” conditions (24). An efficacious test does not necessarily translate into an effective test, and ideally imaging modalities would be subject to both types of analysis. Stowers et al. (25) reported SPECT imaging efficacy in a small study of 46 emergency department (ED) patients randomized to resting SPECT perfusion imaging or conventional clinical strategy. Length of stay and costs were lower in the imaging strategy arm. Effectiveness of rest perfusion imaging was studied in the ERASE Chest Pain (Emergency Room Assessment of Sestamibi for the Evaluation of Chest Pain) trial, in which over 2,500 patients were randomized to an initial ED evaluation strategy of resting SPECT perfusion imaging, in addition to standard testing, or to a nonimaging standard evaluation strategy (26). The results demonstrated a reduction in unnecessary hospital admissions associated with the imaging strategy, suggesting significant effectiveness of imaging in this situation.

The National Heart, Lung, and Blood Institute (NHLBI) recently released its strategic plan for “Shaping the Future of Research” (27). The importance of optimizing diagnostic tests for improving outcomes is explicitly recognized in the plan, which states that “research is needed to evaluate the extent to which risk stratification and application of personalized approaches can improve effectiveness” (Challenge 3.1.a); that “studies are needed to reduce the inappropriate used of diagnostic tests and treatments” (Challenge 3.1.c); and that there is a need to “evaluate the risks, benefits, and costs of diagnostic tests and treatments in representative populations and settings” (Challenge 3.2.a).

Therefore, on July 21 and July 22, 2008, the NHLBI convened experts in noninvasive cardiovascular imaging, outcomes research, statistics, and clinical trials to develop a vision for imaging research that transcends current reliance on diagnostic and prognostic end points to a new paradigm that focuses on preventive and therapeutic value, where value implies an improved clinical outcome and/or reduced costs. The panel was specifically charged to develop a set of recommendations for future analyses and possible research funding by NHLBI, including sample trial designs for 4 pre-defined clinical scenarios commonly encountered in clinical practice. The 4 scenarios were: 1) screening the asymptomatic patient for CAD; 2) assessment of stable angina; 3) identification of acute coronary syndromes in the emergency room; and 4) assessment of heart failure patients with chronic CAD with reduced LV ejection fraction. The panel was asked to identify need, assess feasibility, and determine 1 to 2 examples of possible trial concepts for each scenario. Given the time limitations, it was recognized that these trial overviews would subsequently require substantial statistical and logistical analysis to become formal, detailed, and actionable trial designs.

Forty years ago, the World Health Organization (28) first published principles around which screening programs can be justified (Table 1), and many of these principles also apply to vascular diseases such as CAD. Screening for abdominal aortic aneurysm is now an accepted practice for some patient groups based on multiple randomized controlled trials (29- 31). However, there are also a number of unknowns that have blunted enthusiasm for screening for CAD (32- 33). Controversy has arisen regarding whether imaging-based risk classification improves selection of patients for treatments and whether outcomes after screening are improved compared with traditional risk factor measurements and risk-factor based treatments (34- 36). It is also unclear from existing data which patients to screen and how frequently to perform screening tests.

Cohort studies using coronary calcium measurement have shown the ability of cardiac computed tomography (CT) to identify high-risk asymptomatic patients (37). For example, the MESA (Multi-Ethnic Study of Atherosclerosis) found that coronary calcium (CAC) scores were strongly and incrementally (compared with Framingham risk score [FRS]) associated with clinical vascular outcomes in 45- to 84-year-old subjects (38). Compared with CAC scores equal to 0, a CAC score of >300 was associated with a >6-fold higher odds of a major coronary event and a >9-fold higher odds of any CAD event. Consensus panels (37,39) have concluded that CAC scores are capable of stratifying patients into low-, intermediate-, and high-risk groupings. Similar predictive information has been published regarding carotid intima-media thickness measurements (40), although the MESA trial suggested that CAC is a stronger predictor of cardiovascular events (41). Thus, the use of CAC might be the preferred imaging strategy, independent of other considerations such as cost, availability, or impact of incidental scan findings, as a single imaging test for screening purposes.

Despite the demonstrated predictive value of CAC and intima-media thickness, enthusiasm among consensus panels for routine screening is limited (32- 34), in part because of the absence of clinical trials data (34,42- 43). The possibility that screening can cause harm in the form of radiation exposure (for CAC) and false reassurance for people with high risk factor scores but low levels of anatomic disease are often mentioned as reasons for caution in the adoption of screening using imaging tests (34,42- 43). However, others have suggested that such data are absent for many other forms of screening and that trials of this sort are expensive and unlikely to be undertaken (35- 36). In the absence of better outcomes data for use of an imaging strategy for screening and risk assessment, the controversy between screening advocates and screening detractors cannot be easily resolved.

Workshop participants considered several sample study overviews designed to reach more definitive conclusions on the roles of imaging tests for cardiovascular screening. The first design was an effectiveness study of asymptomatic men and women with intermediate FRS (Figure 1). The hypothesis was that CAC testing will improve risk stratification resulting in improved risk factor modification and leading to both reduced events and lower costs. Patients would be randomized to receive an invitation for coronary calcium testing versus no invitation for CAC testing. All patients would receive an individualized risk assessment and associated risk interpretation including FRS, which would be provided to all patients and to their doctors for subsequent treatment without specific guidance.

Inclusion criteria would be asymptomatic individuals with intermediate FRS (>6% but <20%) and without known CAD, cardiovascular disease, peripheral artery disease, or renal disease. The primary outcome would be a combined end point consisting of major clinical events (MI, stroke, congenital heart disease [CHD] death). Major secondary end points would include total health care costs estimated from hospitalization and doctor and ED visits, medications, additional tests, quality of life measurements, behavior changes after testing, cardiovascular drug use, risk factor changes, clinically indicated coronary revascularization, and other CHD events. The workshop discussants proposed a 10% to 20% reduction in major cardiovascular disease end points as study design goals. Similar trials in different populations, such as in asymptomatic type 2 diabetes mellitus, commonly considered a CHD risk equivalent, were also considered.

To address concerns that reliance on usual physician care may increase the likelihood of a negative result, a fully managed trial testing the efficacy of a guideline-based treatment approach versus a CAC plus risk factor-based approach to individualized therapy of cardiovascular risk was proposed. The trial would have similar inclusion and exclusion criteria and end points as already discussed.

Imaging modalities for use in patients with suspected or known CAD have generally been evaluated on the basis of accuracy for the detection of angiographic CAD. All of the contemporary imaging modalities—SPECT or PET MPI, stress echocardiography, cardiac computed tomographic angiography (CCTA), and cardiac magnetic resonance (CMR)—perform to a clinically acceptable standard. The next tier of evaluation focusing on prognostic or risk-stratification studies has generally demonstrated that greater abnormalities on SPECT MPI and stress echo imaging are associated with a higher risk of an incident cardiovascular event during follow-up (44- 45), documenting the “incremental value” of the imaging data over previously available and less expensive to obtain clinical or stress ECG information (46- 48). Only a very few imaging randomized controlled studies have been performed to date of the kind that might be considered to constitute “higher level evidence” from the prism of therapeutic trials (26); however, these demonstrate that it is feasible to subject imaging modalities to the same rigorous analysis that is standard for therapeutics.

In patients without known CAD who present with symptoms suggestive of CAD and requiring initial diagnostic/prognostic work-up, workshop participants proposed a trial design randomizing patients to an initial CCTA strategy as compared with an initial functional-based testing strategy (Figure 2). The primary hypotheses were that CCTA would result in noninferior 12-month major adverse cardiovascular event rates and would be cost-efficient. Secondary end points could include rates of invasive angiography, effective biological radiation dosages received by patients, cost-effectiveness in the low likelihood group (hypothesizing that CCTA is more cost-effective), and cost-effectiveness in the high likelihood group (hypothesizing that functional imaging is more cost-effective).

To address the impact of imaging in clinically stable patients with known CAD and previous myocardial revascularization, workshop participants proposed randomizing clinically stable patients >2 years after revascularization to either routine late “screening” for recurrent ischemia with stress imaging versus symptom-driven testing (Figure 3). The hypothesis would be that periodic imaging after revascularization is noninferior for major adverse cardiac events and cost-effective. For patients randomized to the initial imaging strategy group (including any functional imaging test such as SPECT MPI, PET MPI, stress echo, stress CMR), the results would be provided to their physicians to act on as they see fit. Secondary end points could evaluate the “yield” of routine late post-revascularization stress functional imaging, the clinical predictors of a positive test (to potentially enhance the yield of imaging), and the influence of time from revascularization.

Emergency department visits in the U.S. for suspected acute coronary syndrome patients exceed 10 million individuals (8%), 6.24 million of whom undergo a fairly extensive evaluation. Of the latter, 50% are ultimately determined to have a noncardiac diagnosis. Unfortunately, current technology is often inadequate to differentiate the roughly 85% of patients with noncardiac problems from the small minority with an acute cardiovascular disease presentation (49). Risk factors (50), risk scores (e.g., Thrombolysis In Myocardial Infarction [TIMI]) (51), the physical exam, chest radiography, and even the arrival ECG are nondiagnostic in 98% of patients (49), and even interpretation of the patient's symptoms is constrained by language barriers, recall quality, and the fact that as many as one-third of confirmed MI patients do not have chest pain (52).

Given far greater risk associated with an inappropriate discharge as compared with additional diagnostic testing or hospitalization, test sensitivity is critical. Highly specific testing, though valuable when positive, may be inadequate for safe discharge. Unfortunately, currently available biomarker tests have high specificity but sensitivity as low as 10% (53), although a “chest pain center” strategy of serial markers and selective stress testing decreases mortality and increases discharges by 37% and 36%, respectively, compared with usual care (54). Thus, use of this model has sky rocketed (55- 59), despite tremendous cost, average length of hospitalization of 17 h, and great inconvenience to the patient. Adding the use of imaging technology to usual care may improve the system, but prospective data are sparse. This limitation has resulted in vague guideline statements that suggest “the potential benefit of noninvasive coronary angiography is likely to be greatest in symptomatic patients who are at intermediate risk for coronary artery disease after initial risk stratification” (60). Future research is required before general use of ED imaging can be adopted.

In patients presenting with symptoms suggestive of acute coronary syndromes, workshop participants proposed randomizing patients to an initial cardiac marker and CCTA strategy as compared with usual care (Figure 4). The primary hypothesis would be that use of a biomarker plus CTA discharge strategy is safe and effective compared with the present standard of care. Eligible patients would be those presenting to the ED with ischemic symptoms and negative cardiac marker determinations. Major exclusion criteria would include cardiac catheterization indications (diagnostic ECG changes or local positive troponin) and CCTA exclusions (reduced renal function, known CAD, or significant arrhythmia). The primary end point would be a combination (major adverse cardiac events) of death, coronary revascularization intervention, or of heart failure. Secondary end points could include additional clinical outcomes and resource implications of a marker/CTA strategy such as ED process time, time to accurate diagnosis, rates of noncardiac diagnoses, and percutaneous coronary interventions (PCI) performed as a consequence of strategy used, dye load and complications, radiation exposure, patient satisfaction, and revisits.

Imaging plays several important roles in the current management of patients with LV systolic dysfunction including: to assess its severity, to identify those with underlying CAD, to determine the extent and severity of myocardial ischemia, and to identify the magnitude of dysfunctional but viable myocardium. Only LV ejection fraction has been studied in prospective randomized clinical trials. Among the many candidate clinical trials in patients with ischemic LV dysfunction discussed at this workshop, the assessment of myocardial viability and the role of imaging in ischemic mitral regurgitation (MR) were selected for consideration, as these 2 topics have both clinical need and potential public health impact.

Numerous studies have demonstrated the potential of PET, SPECT, dobutamine echo, and contrast-enhanced CMR to identify viable myocardium in patients with CAD and LV dysfunction, and to predict recovery of LV function following percutaneous (PCI) or surgical (coronary artery bypass graft [CABG]) revascularization (61- 63), as well as improved survival and symptomatic status compared with the results of medical therapy (62- 64). However, these studies were all retrospective in nature, often with treatment biases based on the results of the imaging tests, and the medically treated patients often did not receive aggressive evidence-based medical management.

Even the ongoing NHLBI-funded STICH (Surgical Treatment for Ischemic Heart Failure) trial (65) will leave unresolved a number of important questions as patients were randomized to revascularization versus aggressive medical management in patients with ischemic LV dysfunction independent of imaging results. Further studies are needed to address whether an imaging strategy is useful in actually guiding management decisions in patients with ischemic LV dysfunction.

A possible trial in this area could test the effectiveness of routine viability imaging versus nonimaging strategy in the management of patients with symptomatic heart failure and multivessel CAD by improving outcomes, health status, and/or quality of life and reducing cost. Following angiography, patients with 2- or 3-vessel CAD and ejection fraction 35% or less would be randomized to standard of care therapy (optimal medical therapy or PCI or CABG) without imaging or to a strategy of imaging followed by standard of care (Figure 5). Ideally, a second randomization would be performed within the imaging arm, in which patients would be randomized to one of 4 imaging strategies—PET, SPECT, dobutamine stress echo, or CMR—to determine the relative effect of each of these tests. Patient care would be determined at the discretion of the treating physician, but physicians are encouraged to follow the results of the viability data in patients randomized to imaging. The clinical end points could include cardiovascular mortality and cardiac readmissions for MI, unstable angina, heart failure, and late revascularization (excluding planned PCI or CABG based on initial testing).

Patients with ischemic cardiomyopathy who have MR have a worse outcome in terms of mortality, development of heart failure, and hospitalization than patients without MR (66- 67). In this situation, MR develops secondary to LV dysfunction with dilation and displacement of the papillary muscles, mitral annular dilation and tethering of the mitral valve leaflets (68). It is unclear whether the resulting “functional” MR is merely a marker of a greater degree of LV dysfunction or whether it contributes actively to progression of LV dysfunction. It is also unclear whether surgery to repair or replace the mitral valve leads to a better outcome (69- 70), or whether mild to moderate MR should be repaired at the time of CABG.

This clinical trial proposal involves using the infrastructure developed by the NHLBI Cardiothoracic Surgery Clinical Research Network to assess whether mild to moderate MR should be repaired at the time of CABG in patients with LV dysfunction and could include 3-dimensional echocardiography and CMR as part of the prospective evaluation of patients enrolled in such studies. This study would examine the importance of imaging in identifying which patients with ischemic MR benefit from mitral valve repair at the time of CABG through follow-up echocardiography and CMR at 6 months and 2 years after surgery (Figure 6). The study would also examine whether and how imaging influences operative decision making and outcomes of all-cause mortality and hospital readmissions, including whether imaging is helpful in determining which patients will benefit.

The sample trials considered by the workshop share some common themes and limitations that, taken together, provide a practical lesson in how to think about outcomes research in imaging. Most trial designs focused on real-world populations and were large practical trials intended to assess effectiveness and not efficacy. All but 1 specified the use of “usual care” in which decisions regarding further testing or therapeutic intervention were left to the patients' care team following randomization. In other words, the trials did not specify how physicians were to act upon imaging results. Some were based on randomization to the use of imaging or not and others randomized patients between imaging strategies.

In general, the sample trial designs advocated for use of “hard end points” such as death or myocardial infarction over at least a 1-year period for primary end points, rather than relying on softer outcomes such as clinical worsening or use of medications. Most also included a broad variety of secondary end points such as radiation exposure, assessment of quality of life, behavior change with risk factor modification, and economic analyses, including such metrics as cost-to-diagnosis, cost-to-predict event, and cost-to-prevent MI. Motivation for these additional metrics included the wish to incorporate end points that may be less challenging to develop over a shorter time horizon, as well as interest in the variables themselves. They also reflect the broad range of concerns around imaging use.

Many limitations common to the sample trials are noted. The trial designs were not subject to rigorous evaluation of feasibility, in part due to acknowledged time constraints during the workshop and lack of analytic expertise needed to construct detailed clinical protocols, but also by intent so as not to limit creativity. These challenges also extend beyond the time constraints of the workshop format as it is difficult to properly estimate a sample size in the absence of reliable community-based data on prevalence of disease, test performance, end point occurrence, effect size, and cross over rates, among other concerns. It is possible that initial pilot studies or simulations may be helpful in more detailed planning.

Another concern was the duration of time required to perform such studies, especially related to the rapid pace of technologic change, and whether the results would still be relevant at the time of trial completion. Finally, all of the sample trials would be “large” and “expensive” trials and that the cost of even 1 such trial would be quite high, perhaps even prohibitive, an especially important consideration for NHLBI, as the convener of the conference and for any future Request For Applications (RFA) that might arise as a result of the workshop deliberations. Several alternative solutions in addition to conventional federal funding were discussed including pooling resources from the private sector (industry, payers) with National Institutes of Health funds, using only clinically indicated (and therefore “covered”) testing or creating other incentives for enrollment that might mitigate this concern. Other strategies proposed to minimize costs included combining the emergency room and stable angina trials, with identical end points to allow pooling of data, and administrative approaches to achieve economies of scale such as using a single coordinating center and using common sites and/or data collection forms for several trials.

Because such practical considerations will be critical going forward, and many of these would require additional thought by multiple stakeholders, an Imaging Outcomes Consortium was proposed to facilitate further, in-depth exploration of these strategies. Such a consortium could also be used to further review trial proposals developed at the workshop, engage key stakeholders, conduct large or smaller trials substudies or registries, and provide ongoing oversight and support to the emerging outcomes research standard for imaging.

Given that Medicare spends over $14 billion per year on Part B imaging alone, about one-third of which is cardiovascular imaging (4), it is imperative that a robust effort be made to understand the scientific basis for the use of imaging and its contribution to the nation's health. Fortunately the research paradigm regarding imaging is changing, with growing recognition that there is both urgent need and great opportunity in this area (6,15). Future imaging trials must address actual patient outcomes, instead of sensitivity/specificity and prognostic value. The workshop deliberations, as summarized in this proceedings document, amply demonstrate both a commitment on the part of multiple stakeholders to this goal and a shared belief that this is feasible and timely. There is much work remaining to be done, from creating more detailed and practical trial designs to determining sources of funding. It is hoped that, in the near future, clinicians ordering cardiovascular imaging tests will have a clear idea of their value in improving the health of their patients.

For a list of workshop participants, please see the online version of this article.

Outcomes Research in Cardiovascular Imaging: Report of a Workshop sponsored by the National Heart Lung and Blood Institute