Author + information
- Received March 21, 2008
- Revision received July 11, 2008
- Accepted July 14, 2008
- Published online September 1, 2008.
- R. Parker Ward, MD, FACC⁎ (, )
- Ibrahim N. Mansour, MD,
- Nicole Lemieux, MD,
- Nitin Gera, MD,
- Rupa Mehta, MD and
- Roberto M. Lang, MD, FACC
Reprint requests and correspondence to:
Dr. R. Parker Ward, Associate Professor of Medicine, University of Chicago Medical Center, 5841 S. Maryland Avenue, RM 611B, MC6080, Chicago, Illinois 60637
We sought to prospectively evaluate the clinical application of the American College of Cardiology Foundation/American Society of Echocardiography Appropriateness Criteria (AC) for transthoracic echocardiography in a single-center university hospital. Indications for transthoracic echocardiograms (TTE) were prospectively determined for consecutive studies by 2 reviewers and categorized, according to the AC for TTE, as appropriate (A) or inappropriate (I). The overall level of agreement in characterizing appropriateness between reviewers was high (kappa = 0.83). Among the 1,553 studies for which a primary indication was determined, 89% were covered in the AC for TTE. Of these studies, 89% were A, and 11% were I. New important TTE abnormalities were more common on A compared with I studies (40% vs. 17%, p < 0.001), and noncardiac specialists more frequently ordered I studies (13% vs. 9%, p = 0.04). In conclusion, the AC for TTE encompasses the majority of clinical indications for TTE and appears to reasonably stratify TTE ordering. However, revisions will be needed to fully capture and stratify appropriate clinical practice.
During the last decade, there has been a dramatic increase in the use of cardiovascular diagnostic imaging. Diagnostic imaging services reimbursed under Medicare's physician fee schedule have grown more rapidly than any other type of physician service from 1999 to 2003 (1,2). This increased use has resulted in increased scrutiny of the appropriate use of cardiac imaging services (1,2). In an effort to guide physicians and reimbursement agencies in determining a rational approach to the use of diagnostic imaging in the delivery of high-quality care, the American College of Cardiology Foundation (ACCF) in conjunction with imaging subspecialty societies have published Appropriateness Criteria (AC) for selected patient indications for a variety of imaging modalities (3–5).
Recently, the ACCF/American Society of Echocardiography (ASE) AC for Transthoracic and Transesophageal Echocardiography have been published (3). As with other AC documents, the authors of these criteria used a standardized methodology in which they combined available evidence with expert opinion to identify common indications for echocardiographic procedures and to determine their level of appropriateness (3–5). These criteria attempt to “identify common scenarios encompassing the majority of clinical practice” and caution that the AC should not be “considered substitutes for sound clinical judgment or practice experience” (3). Inherent in this methodology is that the application of these criteria will need to be tested, both to fully describe their application on current clinical practice and to shed light on their potential impact on the delivery of high quality of medical care. Thus, the purpose of this study was to test the feasibility of prospective clinical application of the published AC for transthoracic echocardiography and to describe their application to current clinical practice at a single-center university hospital.
All patients referred for a complete transthoracic echocardiogram (TTE) to the echocardiography laboratories at the University of Chicago Medical Center between July and September 2007 were eligible for inclusion. The protocol was approved by the Institutional Review Board, and all patients provided informed consent. For each study, patient demographic information, referring physician specialty, outpatient versus inpatient status, and the primary indication for the study were prospectively determined and the results of the TTE recorded.
For each study, written requisitions and hospital/practice records were prospectively reviewed, and previous echocardiographic testing or other previous relevant imaging testing was recorded. A primary indication for each study was determined independently by 2 investigators who were blinded to the results of the echocardiogram. Investigators were asked to select an indication number or category for each study from the following 53 options: any of the indications for TTE (indication numbers 1 to 51) listed in the AC for echocardiography, not addressed (NA; i.e., primary indication determinable but not addressed in AC for echocardiography), or undetermined (UD; i.e., insufficient data to determine a primary indication). A primary consensus indication for each study was then determined for all further analysis.
For studies in which the 2 investigators were in agreement, the indication/category (numbers 1 to 51, NA, or UD) selected served as the final primary consensus indication. For those studies in which there was not agreement between the 2 investigators on the indication number or category (NA or UD), a third investigator independently reviewed the data and chose 1 of the 2 initial selections as a final primary consensus indication.
Complete 2-dimensional and color Doppler echocardiograms, including pulsed-wave Doppler examination of both mitral and pulmonary vein inflows and tissue Doppler imaging for the septal mitral valve annulus, were performed in all patients with the use of a full-platform echocardiographic instrument (Philips iE33, Philips Medical Systems, Andover, Massachusetts). All echocardiograms were performed and interpreted in accordance with the preferred recommendations of the ASE (6,7), as is standard in our laboratory. All studies were interpreted by one of 7 expert echocardiographers, and the findings that were used for analysis in this study represent those reported on the final clinical echocardiogram report.
As is standard in our laboratory, left ventricular (LV) function was assessed with the use of visual or quantitative methods, and ASE-recommended definitions for LV dysfunction (left ventricular ejection fraction [LVEF] ≤54%), moderate LV dysfunction (LVEF 30% to 44%), and severe LV dysfunction (LVEF <30%) were used (6). The severity of valvular heart disease was determined with the use of a combination of expert visual opinion and the preferred quantitative methodology and definitions of the ASE (7). Right ventricular systolic function was determined by expert visualization. Regional wall motion abnormalities were determined by expert opinion and defined as a hypokinesis, akinesis, or dyskinesis reported in any of the 17 myocardial segments.
The presence of pulmonary hypertension was defined as an estimated right ventricular systolic pressure of ≥35 mm Hg, as determined from the maximum tricuspid regurgitation velocity and estimated right atrial pressure. Moderate or greater pulmonary hypertension was defined as an estimated right ventricular systolic pressure of ≥50 mm Hg. Diastolic dysfunction was determined based on expert review of mitral, pulmonary, and tissue Doppler data and is reported as mild (impaired relaxation, grade I), moderate (impaired relaxation with moderately elevated filling pressures or “pseudonormal,” grade II), or severe (impaired relaxation with marked elevation of filling pressures, grade III or IV).
Classification of TTE findings
A composite end point of “any TTE abnormality” was defined as LV dysfunction (LVEF ≤54%); aortic stenosis (aortic valve area <1.5 cm2); a regional wall motion abnormality; right ventricular dysfunction; any pulmonary hypertension; mild or greater mitral, aortic, or tricuspid regurgitation; diastolic dysfunction; or other significant abnormality (mitral stenosis [mitral valve area <1.5 cm2], moderate or greater pulmonary valve regurgitation, moderate or greater pericardial effusion, or any other significant abnormality, i.e., thrombus, vegetation, tumor).
A “major TTE abnormality” was defined as moderate or greater LV dysfunction (LVEF <45%); moderate or greater mitral, aortic, or tricuspid regurgitation; aortic stenosis (aortic valve area <1.5 cm2); a regional wall motion abnormality; right ventricular dysfunction; moderate or greater pulmonary hypertension; moderate or severe diastolic dysfunction; or other significant abnormality (mitral stenosis [mitral valve area <1.5 cm2]); moderate or greater pulmonary valve regurgitation; moderate or greater pericardial effusion; any other significant abnormality (i.e., thrombus, vegetation, tumor).
Studies for which a previous TTE study had been performed were compared with the previous study. A “new TTE abnormality” and a “new major TTE abnormality” were defined respectively as “any TTE abnormality” or “any major TTE abnormality” that was not previously known or in which there had been a change of at least one severity grade from a previous TTE (i.e., previously mild LV dysfunction, now moderate LV dysfunction).
Comparisons between study and patient characteristics, echocardiographic findings, and levels of appropriateness were performed with chi-square tests or Fisher exact tests for categorical data as appropriate, and Student t test for continuous data with the use of a 2-tailed p value <0.05 for statistical significance. Interobserver variability in the determination of indication/category and level of appropriateness is expressed as percent agreement between 2 independent reviewers and with the use of kappa statistics. For indication/category determination, interobserver comparisons represent the frequency in which the reviewers agreed on the indication number or category (i.e., indication numbers 1 to 51, NA, or UD). For level of appropriateness determination, interobserver comparisons represent the frequency in which reviewers selected indications with a matching level of appropriateness (A vs. I), although not necessarily an identical indication number. Studies for which both reviewers chose UD or NA (thus, no appropriateness level was available for either reviewer selection) were excluded from appropriateness level comparisons.
Overall, 1,580 echocardiographic studies performed on 1,431 patients enrolled in the study, with 52% (n = 814) being outpatient studies and 48% (n = 766) being inpatient studies. The mean patient age for all studies was 58.8 ± 16.9 years, and 53% (n = 837) of the studies were performed on women. Compared with patients who received only one study (n = 1,314) in the 3-month enrollment period, those who received more than one study (n = 117) were not significantly different in age (57.4 ± 16.9 years vs. 59.0 ± 17 years, p = 0.32) or gender (50% vs. 53% women, p = 0.53). When previously enrolled patients (n = 149) underwent duplicate studies, they were significantly more likely to be inpatients compared with those patients undergoing first-time studies (74% vs. 46%, p < 0.001). Cardiac specialists (48%; defined as cardiologists [44%] or cardiac surgeons [4%]) were the most common referring specialty. The other referring specialties included internal medicine physicians (36%), noncardiac surgical specialties (8%), neurology (4%), anesthesiology (2%), and other (2%).
Of the 1,580 studies included, a primary consensus indication could not be determined (UD) in 1.7% (n = 27). Of the remaining 1,553 studies, 89.2% (n = 1,385) were ordered for indications outlined in the AC for echocardiography (indication numbers 1 to 51), whereas 10.8% (n = 168) were not. The frequency of studies ordered for the most common 10 indications as outlined in the AC document is summarized in Table 1. Overall, the most common indication for obtaining a TTE study was indication number 1 (“symptoms potentially due to suspected cardiac etiology…”). Of the 1,385 studies for which the AC document could be applied, 88.7% (n = 1,228) were ordered for appropriate (A) indications, whereas 11.3% (n = 157) were ordered for inappropriate (I) indications. The patient and study characteristics according to level of appropriateness are listed in Table 2. Compared with A studies, I studies were significantly more likely to be ordered on younger patients (55.9 ± 18.7 years vs. 59.9 ± 16.7 years, p = 0.005), outpatients (78% of I studies vs. 45% of A studies, p < 0.001), and those with a previous TTE (57% of I studies vs. 31% of A studies, p < 0.001). It was found that I studies were less likely to be ordered by cardiac specialists compared with A studies (42% of I studies vs. 51% of A studies, p = 0.04), and they were more likely to be ordered by internal medicine physicians compared with A studies (47% of I studies vs. 37% of A studies, p = 0.02). Overall, noncardiac specialists ordered a greater frequency of I studies than cardiac specialists (13% vs. 9%, p = 0.04).
Transthoracic echocardiogram findings according to level of appropriateness are listed in Table 3. Overall, a TTE abnormality was found in 68% of studies, including 31% with LV dysfunction. Of the studies with an abnormal TTE finding, 29% were unchanged from previous TTE, leaving a total of 48% of all studies with a new TTE abnormality. The frequency of any abnormal TTE finding was similar when comparing A and I studies (70% vs. 65%, p = 0.23). However, new echocardiographic abnormalities were significantly more common on A compared with I studies (52% vs. 29%, p < 0.001). A major TTE abnormality was found on 54% of all studies, including 21% with moderate or greater LV dysfunction. Major TTE abnormalities were present with similar frequency among A and I studies (56% vs. 52%, p = 0.30), although new major TTE abnormalities were significantly more common among A compared with I studies (40% vs. 17%, p < 0.001).
An analysis of the individual I indications is summarized in Table 4. The most common I indication was indication number 42 (“Routine [yearly] re-evaluation of patients with heart failure [systolic or diastolic] in which there was no change in clinical status.”). The I indication for which the most new major TTE abnormalities were identified was indication 21 (“Routine [yearly] re-evaluation of an asymptomatic patient with mild native AS or mild-moderate native MS and no change in clinical status”). Indication 42 (68% by cardiac specialists vs. 32% by noncardiac specialists) and indication 21 (61% by cardiac specialists vs. 39% by noncardiac specialists) were also the only I indications more likely to be ordered by cardiac compared with noncardiac specialists.
On review of individual studies ordered for indication number 42, 29% were ordered as a follow-up of a previous TTE, with a recorded LVEF on the previous study of <35%, to assess a patient's candidacy for implantable defibrillator placement after a trial of maximal medical therapy or a coronary revascularization. This clinical setting was not specifically addressed in the AC document, yet reviewers consistently placed these studies under indication 42 based on clinical information consistent with this indication as written. Among studies ordered for Indication 21, 72% were ordered to follow up the documentation of previously mild aortic stenosis.
The indications for which TTE studies were ordered that are not addressed in the AC document are listed in Table 5. The most common indication was pre-operative evaluation, accounting for 50% of all NA studies, with 27% before solid organ transplant and 20% before general noncardiac surgery. Indications related to heart failure or native or prosthetic valvular disease for which the severity or frequency of follow-up is not addressed in the AC document accounted for 30%. Overall, 43% of studies with indications not addressed were related to solid organ transplant programs at our institution.
Our analysis of the interobserver variability for indication determination revealed that the 2 initial independent reviewers had 84% agreement (kappa = 0.82) on the determination of indication/category (indication numbers 1 to 51, NA, or UD) and 97% agreement (kappa = 0.83) when selecting indications with the same level of appropriateness (A or I).
In this study, we found that the AC for echocardiography encompasses the majority of indications for TTE that are ordered in routine clinical practice at a single-center university hospital and that a large majority of the studies ordered for indications addressed in the AC document are found to be A. The AC for echocardiography also appear to reasonably stratify TTE test ordering because A studies were found to have significantly more newly recognized echocardiogram abnormalities than I studies. There remain a small number of studies (11%) in our clinical practice that are not addressed by the AC for echocardiography, which suggests that additional study and revisions of this document will be necessary to fully encompass and stratify the appropriate clinical practice of echocardiography.
This study represents the first published prospective study of the clinical application of any of the AC documents and the first study of any kind to evaluate the clinical application of AC for echocardiography. We used prospective methodology in an attempt to identify the “true” indication of the study. Using this approach, we found that it was possible to identify a primary indication for the vast majority of clinically ordered studies. The small fraction (2%) for which a primary indication could not be determined were primarily related to the inability to access the most recent medical records. We also found that although there was good agreement between independent reviewers in assigning a primary indication (84%), for a number of studies, more than one indication was supported by the clinical data. This finding illustrates the sometimes-subjective nature of indication assignment and carries potential implications if these criteria gain widespread use for reimbursement determinations. Mitigating this concern was our finding that reviewers demonstrated excellent agreement (97%) in selecting any A versus any I indication. Thus, although studies may have more than one reasonable primary indication, the consensus on whether the study is ultimately deemed A or I by the AC for echocardiography is high.
Studies ordered for indications not addressed by the AC document represented a small but significant fraction (11%) of those ordered at our institution. Although some were triggered by programs specific to a large university hospital (e.g., solid-organ transplant program), which would be expected to represent only a fraction of the broad clinical practice of echocardiography, others illustrate gaps in the AC for echocardiography that will need to be addressed for the document to more completely encompass the common clinical practice of TTE. For example, indications related to preoperative cardiac evaluation were the most common group of indications not addressed.
Since the development of the AC for echocardiography, revised American College of Cardiology/American Heart Association guidelines for pre-operative evaluation before noncardiac surgery have been published and include more definitive recommendations on the pre-operative assessment of LV function, which is frequently assessed with TTE (8). Thus, revisions of the AC for echocardiography aimed at incorporating the pre-operative use of TTE may now be more feasible.
Additionally, a variety of indications related to the severity or frequency of follow-up of patients with common conditions such as heart failure or valvular heart disease are not addressed in the AC for echocardiography. These omissions likely relate to the lack of clear evidence or consensus regarding the appropriate use of TTE in these clinical settings. It is notable that overall we find that studies ordered for indications not addressed in the AC document were significantly less likely to have a new TTE abnormality or a new major TTE abnormality than studies deemed to be A by the AC document. Further study with larger numbers of studies for these indications will be needed to establish the level of appropriateness of TTE in each of these clinical settings.
Studies determined to be I by the AC document were more commonly ordered on younger patients, outpatients, and those with a previous TTE study. It is not surprising that outpatient studies were more likely than inpatient studies to be ordered for I indications. The reason for inpatient studies is frequently related to new signs or symptoms, or other changes in clinical status, which all are key clinical features used throughout the AC document for differentiating A from I indications. The fact that the recipients of I studies more commonly had a previous TTE is also an expected result, given that a majority of I indications as written specifically involve follow up of a previous TTE study (54%) (3). This fact also explains the high rate of composite TTE abnormalities among I studies (65%), because many were performed to follow up a known echocardiogram abnormality. Because fewer of the A indications involve re-evaluation of a previous TTE study (26%) (3) and, thus, fewer were performed to follow up known echocardiogram abnormalities, it is also not surprising that we did not find a significant difference in overall TTE abnormalities between A and I studies.
We do find that the AC document stratifies indications according to the likelihood of finding a newly recognized TTE abnormality, because new TTE abnormalities were significantly more common in A compared with I studies. Because there may be discussion about the clinical significance of some of the minor echocardiogram abnormalities, we chose to further study “major” echocardiogram abnormalities. Although the definition of a “major TTE abnormality” can be debated, we chose those findings we thought would be sure to prompt treatment, work-up, or serial follow-up. Using our definition, we found that new major TTE abnormalities were significantly more prevalent in A compared with I studies, suggesting that the AC for echocardiography successfully stratifies these studies. It is important to note, however, that echocardiogram findings should not be the only determinant of whether an echocardiogram was necessary or appropriate, because a normal study may provide important clinical information that may change patient management whereas an incidental finding unrelated to study indication may not.
The analysis of studies according to ordering physician specialty reveals that cardiac specialists are significantly less likely than noncardiac specialists to order studies for I indications. Although all the reasons for this cannot be determined by our study, it should be noted that the AC documents are designed to parallel evidence-based guidelines. Thus, a greater familiarity with cardiovascular guidelines for the use of cardiac imaging among cardiac specialists might be expected and may contribute to this discrepancy. This finding also represents an opportunity for targeted education of ordering physicians on the use of TTE, which may improve appropriate use and lower costs without impacting high-quality care.
Among the I indications, it was notable that indication 42 (“routine [yearly] re-evaluation of patients with heart failure [systolic or diastolic] in whom there is no change in clinical status”), and indication 21 (“routine [yearly] re-evaluation of an asymptomatic patient with mild native aortic stenosis or mild to moderate native mitral stenosis and no change in clinical status”) were the only indications more frequently ordered by cardiac specialists (3). Among indication 42 studies, 29% were ordered as a follow-up to a previous LVEF <35% to determine candidacy for implantable cardioverter-defibrillator after a trial of maximal medical therapy or coronary revascularization, an indication current guidelines would support (9). This result highlights an example of a specific clinical situation not addressed, yet one for which reviewers thought an indication in the AC document broadly applies. It also points to the challenges of clinical application of the AC document and the need for an appeals process, particularly if this document becomes widely used for reimbursement decisions.
There are limitations of this study that deserve mention. First, because we evaluated the clinical practice of echocardiography at a single center university hospital, the results may be different in other practice types. For example, solid-organ transplant programs at our institution accounted for 43% of studies ordered for indications not addressed by the AC documents. Second, our study does not address the clinical impact of echocardiographic findings and, thus, whether they change patient management, which is the ultimate determinant of the true appropriateness of a diagnostic imaging study. Third, because reviewers were not blinded to patient characteristics such as gender and referring physician specialty, we cannot exclude the possibility that this introduced bias into the process of indication determination. Finally, it should be noted that our study only addressed the clinical practice of TTE, whereas transesophageal echocardiography also is covered in the AC for echocardiography. Therefore, further study will be needed to address the clinical application of the AC for echocardiography on the practice of transesophageal echocardiography.
The ACCF/ASE AC for Echocardiography was published in an effort to establish a rational approach to the use of echocardiography in the delivery of high-quality care. Because these criteria may ultimately be used for reimbursement determinations, it is critical that they reflect and stratify the appropriate clinical practice of echocardiography. Our study examined the clinical application of these criteria, and we found that they encompass the majority clinical practice of echocardiography in our institution. The application of these criteria was found to be feasible and reproducible, and they appear to successfully stratify test ordering according to likelihood of finding important TTE abnormalities. However, we also identify a number of revisions of the AC for echocardiography that will need to be addressed to allow routine widespread clinical application of these criteria.
This study was sponsored by a grant from the American Society of Echocardiography
- Received March 21, 2008.
- Revision received July 11, 2008.
- Accepted July 14, 2008.
- American College of Cardiology Foundation
- Medicare Payment Advisory Commission (MedPAC)
- Medicare Payment Advisory Commission (MedPAC)
- Douglas P.S.,
- Khandheria B.,
- Stainback R.F.,
- et al.
- Brindis R.G.,
- Douglas P.S.,
- Hendel R.C.,
- et al.
- Hendel R.C.,
- Patel M.R.,
- Kramer C.M.,
- et al.
- Lang R.M.,
- Bierig M.,
- Devereux R.B.,
- et al.
- Zoghbi W.A.,
- Enriquez-Sarano M.,
- Foster E.,
- et al.,
- American Society of Echocardiography Guidelines and Standards Committee
- Fleisher L.A.,
- Beckman J.A.,
- Brown K.A.,
- et al.
- Hunt S.A.,
- Abraham W.T.,
- Chin M.H.,
- et al.