Author + information
- Received October 21, 2016
- Revision received January 24, 2017
- Accepted February 17, 2017
- Published online May 17, 2017.
- Oana Mirea, MD, PhDa,
- Efstathios D. Pagourelias, MD, PhDa,
- Jurgen Duchenne, MSca,
- Jan Bogaert, MD, PhDb,
- James D. Thomas, MDc,
- Luigi P. Badano, MD, PhDd,
- Jens-Uwe Voigt, MD, PhDa,∗ (, )
- EACVI-ASE-Industry Standardization Task Force
- aDepartment of Cardiovascular Diseases, University Hospital Leuven, Leuven, Belgium
- bDepartment of Radiology, University Hospital Leuven, Leuven, Belgium
- cBluhm Cardiovascular Institute, Northwestern University, Chicago, Illinois
- dCardiac, Thoracic and Vascular Sciences, University Padua, Padua, Italy
- ↵∗Address for correspondence:
Prof. Dr. Jens-Uwe Voigt, Department of Cardiovascular Diseases, University Hospital Gasthuisberg, Herestraat 49, 3000 Leuven, Belgium.
Objectives The purpose of this study was to compare the accuracy of vendor-specific and independent strain analysis tools to detect regional myocardial function abnormality in a clinical setting.
Background Speckle tracking echocardiography has been considered a promising tool for the quantitative assessment of regional myocardial function. However, the potential differences among speckle tracking software with regard to their accuracy in identifying regional abnormality has not been studied extensively.
Methods Sixty-three subjects (5 healthy volunteers and 58 patients) were examined with 7 different ultrasound machines during 5 days. All patients had experienced a previous myocardial infarction, which was characterized by cardiac magnetic resonance with late gadolinium enhancement. Segmental peak systolic (PS), end-systolic (ES) and post-systolic strain (PSS) measurements were obtained with 6 vendor-specific software tools and 2 independent strain analysis tools. Strain parameters were compared between fully scarred and scar-free segments. Receiver operating characteristic curves testing the ability of strain parameters and derived indexes to discriminate between these segments were compared among vendors.
Results The average strain values calculated for normal segments ranged from −15.1% to −20.7% for PS, −14.9% to −20.6% for ES, and −16.1% to −21.4% for PSS. Significantly lower values of strain (p < 0.05) were found in segments with transmural scar by all vendors, with values ranging from −7.4% to −11.1% for PS, −7.7% to −10.8% for ES, and −10.5% to −14.3% for PSS. Accuracy in identifying transmural scar ranged from acceptable to excellent (area under the curve 0.74 to 0.83 for PS and ES and 0.70 to 0.78 for PSS). Significant differences were found among vendors (p < 0.05). All vendors had a significantly lower accuracy to detect scars in the basal segments compared with scars in the apex (p < 0.05).
Conclusions The accuracy of identifying regional abnormality differs significantly among vendors.
The accurate noninvasive evaluation of regional myocardial function is essential in clinical practice. In heart failure patients with conduction delays, myocardial function can differ significantly between walls, and its accurate assessment could be of potential use for selecting suitable candidates for resynchronization therapy, which would have implications for therapy success and survival (1). After myocardial infarction (MI), the early detection of ischemic injury and the evaluation of myocardial viability directly determines patient management and can prevent serious complications such as sudden cardiac death, left ventricular (LV) remodeling, or arrhythmias (2–4).
In clinical cardiology, cardiac ultrasound is the method of choice for the noninvasive evaluation of regional LV function. In clinical practice, the assessment of regional function abnormalities still relies mostly on the visual interpretation of wall motion abnormalities, which makes it subjective and dependent on skills and experience.
Two-dimensional speckle tracking echocardiography allows the objective quantification of LV global and regional function in a convincingly simple and feasible way. The method has been validated against sonomicrometry and cardiac magnetic resonance (CMR) (5), and global longitudinal strain has proved to be a clinically reliable and reproducible parameter of LV systolic function (6). Likewise, segmental strain measurements have been shown to detect the presence and extent of ischemia (7–13) and distinguish between viable and nonviable myocardial segments (14), which makes them a potential alternative to the conventional “eye-balling” analysis. It is not clear, however, whether current tracking software is sufficiently accurate for this and to what extent the performance of software from different vendors differs.
The present study was designed to evaluate and compare in a clinical setting the accuracy of different software packages to detect regional functional abnormality.
Sixty-three patients, age >18 years, with regular heart rhythm and a previous MI were selected for the study. In all patients, a late gadolinium enhancement (LGE) CMR study had been performed after an MI (without other ischemic events or cardiac interventions before the image acquisitions for this study). Five patients had to be replaced by healthy young volunteers who were on stand-by because of no-show or paroxysmal atrial fibrillation. The study was approved by the ethics commission of the University Hospitals Leuven, and all subjects gave written informed consent before inclusion.
Industry partner recruitment
Seven industry partners (Hitachi, Tokyo, Japan; Esaote, Florence, Italy; GE Vingmed Ultrasound, Horten, Norway; Philips, Andover, Massachusetts; Samsung, Seoul, South Korea; Siemens, Mountain View, California; and Toshiba, Otawara, Japan) provided an ultrasound machine, post-processing software, and an application specialist responsible for optimizing the machine settings according to requirements for speckle tracking analysis. Additionally, 2 manufacturers of generic software solutions for speckle tracking analysis (Epsilon, Ann Arbor, Michigan and TOMTEC, Unterschleissheim, Germany) participated in the comparison. One company withdrew from the study for technical reasons. A list of participants and the versions of software used for analysis is provided in Table 1.
The echocardiographic image acquisitions were completed in 5 days during 9 sessions of 2 to 3 h each. Each patient was scanned by an expert examiner. Seven patients were examined per session each by 1 expert echocardiographer on all 7 ultrasound machines. Application specialists from all companies ensured optimal machine settings during the scans. Images were obtained with the subject in the left lateral decubitus position. Three consecutive cardiac cycles from the apical views (4-, 2-, and 3-chamber) were obtained during breath hold. Additionally, pulsed wave Doppler recordings of the mitral inflow and aortic outflow were acquired for timing measurements. All image data were stored as raw data in a proprietary company format and in the standard DICOM (Digital Imaging and Communications in Medicine) format to allow post-processing with the independent software packages.
All patients had undergone a CMR study on a 1.5-T Philips Intera-CV (Philips, Best, the Netherlands) using dedicated cardiac software, a phased-array surface receiver coil, and electrocardiography triggering. The CMR acquisitions were performed no sooner than 4 days after the MI. Cine images in horizontal, vertical, and short-axis views were acquired using a breath-hold cine steady-state free-precession sequence. Post-contrast breath-hold T1-weighted 3-dimensional inversion-recovery imaging was used for detection and quantification of LGE 10 min after intravenous bolus of 0.2 mmol/kg gadolinium-tetraazacyclododecane-tetraacetic acid (Dotarem, Guerbet, Villepinte, France), with the same views that were used for cine images. The inversion time was individually adapted to suppress the signal of normal myocardium (220 to 350 ms). Extent and transmurality of scar were documented per segment using the standard 18-segment model recommended for functional echocardiographic measurements (15).
LV ejection fraction was measured on GE images using the modified Simpson rule (15). Myocardial scar was visually assessed in the 2-dimensional images of each ultrasound machine using an LV 18-segment model and the following criteria: 1) thinned wall; 2) hypokinesia or akinesia; and 3) increased reflectivity of the myocardium. For the 2 independent software tools, scar readings from 1 vendor (GE) were used.
Speckle tracking echocardiography
Images were analyzed by a single observer (O.M.). All strain measurements were performed on vendor-specific speckle tracking software. For the 2 independent software providers, DICOM images acquired with the GE system were used. For all strain analysis, the LV was divided into 18 segments, as recommended for regional function analysis (15).
End diastole was manually set as the R peak of the electrocardiogram. If the software did not allow that, the automatic settings of the software were used instead. Aortic valve closure (AVC) was calculated from the pulsed wave Doppler recordings of the LV outflow. The endocardial borders were traced manually whenever possible. Otherwise, the region of interest was created by automated recognition according to the requirements of the respective software.
Tracking accuracy of the different software solutions was assessed visually. Segments were excluded from further analysis when the tracking did not accurately follow the underlying myocardial motion after at least 2 attempts to readjust the region of interest. Because we had 2 sets of apical views from each vendor, we used the datasets with the best image quality for the scar detection analysis. Feasibility of strain measurements, however, was calculated considering both acquisitions (2,268 segments per vendor).
Zero strain was defined at end diastole as described above. For each segmental strain curve, peak systolic strain (PS), end-systolic strain (ES), and the post-systolic strain (PSS) peak were measured. The peaks were defined as follows: PS, maximum (positive or negative) strain value before AVC; ES, strain value at AVC; and PSS, maximum negative deformation after AVC, if more negative than ES (Figure 1). Additionally, the post- systolic index (PSI) was calculated as (PSS − ES)/PSS × 100.
The CMR images were analyzed by consensus of 2 observers (O.M., E.D.P.). The readings were supervised by a CMR specialist (J.B.). We used an 18-segment model of the LV. Segmentations of CMR and echocardiographic datasets were carefully aligned by use of anatomic landmarks such as papillary muscles or the intersection of the right ventricular free wall with the interventricular septum. The presence of LGE was visually assessed from 3 short-axis projections (base, midventricular, apex) representing the middle of the echocardiographic segment.
Segments were rated as “nontransmural scar” when 1% to 75% of the segment showed hyperenhancement in the transmural direction; otherwise, they were labeled as “no scar” or “transmural scar.” Segments were rated as “partial scar” when >20% but <80% of the segment showed hyperenhancement in the circumferential or longitudinal direction. Although no CMR was available, all segments from healthy volunteers were assigned a no-scar status for the statistical analysis.
For comparison with echocardiography data, we exclusively used segments with no scar and transmural scar. Therefore, in the following, the term scarred segment will be used in the sense of transmural scar, unless explicitly stated otherwise.
The healthy volunteers did not undergo CMR. The 90 segments analyzed from the volunteers were included in the final analysis as no scar.
We used SPSS version 23.0 (SPSS Inc., Chicago, Illinois) for statistical analysis. The normal distribution of the data was tested with the Kolmogorov-Smirnov test. Continuous variables are expressed as mean ± SD. Comparison of strain values between scarred and normal segments was performed with a 2-tailed Student t test. A p value <0.5 was considered significant. Receiver operating characteristic (ROC) curves for PS, ES, PSS, and PSI were calculated to evaluate the discrimination capacity between normal segments and scars for each vendor. Area under the curve (AUC) of ROC was interpreted as follows (16): 1) AUC = 0.5, no discrimination; 2) 0.6 ≤ AUC < 0.7, poor discrimination; 3) 0.7 ≤ AUC < 0.8, acceptable discrimination; 4) 0.8 ≤ AUC < 0.9, excellent discrimination; and 5) AUC > 0.9, outstanding discrimination. The comparison between ROCs was performed with a De Long test (17) (MedCalc Software, Mariakerke, Belgium).
Conventional echocardiographic parameters
Ejection fraction ranged from 28% to 75% (mean 54 ± 10%).
The CMR examinations were performed on average 12 ± 8 months before the study and no sooner than 4 days after the MI. Of 1,044 segments analyzed, 241 had a transmural scar (23%). No LGE was present in 658 (63%) of the segments. The remaining 145 segments (14%) had nontransmural or partial scar. An average of 5.0 segments with transmural scar per patient (28% of the total myocardium) were present on LGE readings. Transmural scar was most prevalent in the apex (one-third of the apical segments) (Figure 2).
Feasibility of strain analysis
The feasibility of segmental strain measurements ranged from 77.1% to 92.9% and differed significantly between vendors (ANOVA p < 0.05) (Online Figure 1).
The average strain values calculated for normal segments ranged from −15.1% to −20.7% for PS (Figure 3), −14.9% to −20.6% for ES, and −16.1% to −21.4% for PSS (Online Figures 2 and 3). Significant differences were found among vendors (ANOVA, p < 0.05). The scarred segments had lower strain values for all vendors (p < 0.05), with average values ranging from −7.4% to −11.1% for PS, −7.7% to −10.8% for ES, and −10.5% to −14.3% for PSS (Figure 3 and Online Figures 2 and 3, respectively). Also here, significant differences were found among vendors (ANOVA, p < 0.05). The highest difference of PS strain between normal and scarred segments was 11.8% (GE) and the lowest 7.7% (Esaote, Toshiba). The respective data for ES and PSS are displayed in Online Figures 2 and 3.
Post-systolic shortening was found in both normal and scarred segments for all vendors. However, the PSI differed significantly between normal and scarred segments (p < 0.01) (Figure 4).
Distinction of segments with and without SCAR
The ROC analysis for the ability of PS, ES, and PSS to distinguish segments with and without scar is shown in Figure 5. AUCs differed significantly among vendors (p < 0.05). The discrimination capacity was acceptable to excellent, with AUCs ranging from 0.74 to 0.83 for PS and ES and 0.70 to 0.78 for PSS. For all vendors, the accuracy of PS and ES was similar, whereas PSS demonstrated lower discrimination capacity (p < 0.05). Transmural scars limited to 1 segment (surrounded by normal myocardium) demonstrated higher strain values (−21.6 ± 1.8) than segments with transmural scar that had 2 or more transmural scar neighbors (−0.8 ± 4.6) (Figure 6).
The ability of PSI to distinguish between segments with and without scar was also different among vendors (p < 0.05) and ranged from poor to excellent (AUC between 0.68 and 0.85) (Figure 7). All vendor software had significantly more difficulty detecting scars in the basal segments (p < 0.05) (Figure 8). If scar was defined according to echocardiographic criteria only instead of CMR LGE, the detection of scar by strain parameters was found to be more accurate in all vendors (p < 0.05) (Figure 9).
The purpose of the present study was to compare the ability and accuracy to detect abnormal regional myocardial function among 6 vendor-specific and 2 independent strain software tools. We chose myocardial infarct scar as a model of regional dysfunction because it can be defined by an external gold standard method (CMR LGE). To avoid uncertainties in infarct border zones and to minimize the impact of segmentation misalignment between modalities, only segments with full transmural scar and segments free of scar were considered for the analysis.
The main findings of our study are as follows: 1) All vendors measured lower strain values in segments with scar than in segments free of scar. 2) The strain difference between scar and no-scar segments differed significantly among vendors. 3) The ability to distinguish scarred and nonscarred segments differs significantly between vendors. 4) Scars located in the basal segments were most difficult to detect. 5) PSI, PS, and ES were the best parameters to detect scar, whereas PSS performed less well.
Scar and regional deformation
The presence of transmural scar implies loss of contractile function and a reduction of systolic deformation. Accordingly, in our study, strain values were significantly lower in scarred segments with all vendor software, which is in agreement with previous studies (18,19). Although only segments with transmural scar were considered, a considerable degree of systolic deformation was still measured in the scarred segments (Figure 3). This can be explained in part by mechanical interaction between the scar tissue and the surrounding myocardium. Tethering forces induce passive deformation, which is biggest in small scars and lowest in the middle of a large scar region. Figure 6 demonstrates that a 1-segment transmural scar can hardly be detected by deformation measurements, because (passive) deformation of the tissue is similar to normal myocardium. It is further supported by the finding that scar definition by visual analysis of echocardiographic images leads to better scar detection results (higher AUC values) by strain measurements than if CMR LGE is used as the gold standard for scar definition (Figure 9 vs. Figure 5). This is understandable, because the visual echocardiographic analysis considers predominantly functional abnormalities (which are also measured by strain parameters), whereas CMR LGE reflects histological changes in the myocardium.
Ability to detect regional dysfunction
Nevertheless, the strain difference between scarred and nonscarred segments differed significantly between vendors (Figure 3). We interpret a high difference as a higher fidelity of the software to follow local deformation, a property that is presumably related to the amount of spatial smoothing in the tracking algorithm. Figure 6 shows that a scar that extends over more than 2 segments can lead to clearly abnormal values if the software has a good ability to detect local functional abnormality.
Scar detection by strain parameters
The ability to discriminate between segments with and without scar was at least acceptable in all software packages. Nevertheless, significant differences were observed among vendors, with AUC values for PS and ES between 0.74 and 0.83 (Figure 5). PSI, a parameter that combines both PS and PSS with the intention of describing properties of the curve shape, had the best results, but AUC values showed an even wider range between 0.68 and 0.85 (Figure 7). We presume that these strain parameters are affected by a combination of technical characteristics such as spatial smoothing, temporal smoothing, and local tracking fidelity, as well as image quality.
It was surprising that all vendors had difficulty detecting basal scars (Figure 8). This result could be related in part to the fact that most scar segments were in the apex, where scars stretch over several segments, which makes the likelihood for tethering effects by surrounding normal myocardium lowest. Furthermore, the quality of speckle tracking in the basal region was lowest, which might also contribute to worse distinction between scar and nonscar segments. However, we also noted that the difference between the software performance in the basal and apical regions differed between vendors, with some having reasonable results in all LV levels and others only performing well in the apex.
In the present study, we compared only endocardial strain measurements, because those were the only strain component that could be provided by all vendors. To what extent an analysis of midwall strain would generate different results remains to be determined.
In our study, we used infarct scar as a model of regional myocardial dysfunction. Other regional pathology, such as dyssynchrony in dilated cardiomyopathy, might present a different spectrum of challenges for a software package (thinner walls, more focus on temporal resolution, less focus on spatial resolution) and might lead to slightly differing results. We believe, however, that the possibility of using an external gold standard method (CMR LGE) for the definition of scar location and extent made our study more objective, which outweighs its potential disadvantages.
In contrast to global strain measurements, which are highly reproducible and also comparable among vendors (6), the ability and accuracy of detecting regional function abnormalities of the LV differ significantly among vendors. This implies that the technical characteristics of the tested software solutions have a considerable impact on the measurement results. Further standardization and pre-release testing, potentially including hardware or software phantoms, is needed to make regional strain analysis a clinically reliable tool that can produce interchangeable results regardless of the software used.
COMPETENCY IN MEDICAL KNOWLEDGE: In this study, we have tested the accuracy of 6 vendor-specific and 2 independent software packages for strain analysis to identify scarred myocardial segments. Our results indicate there is a significant difference in performance between vendors. Moreover, the localization of the scar and the number of adjacent affected neighbors were shown to influence the ability of the software packages to depict scars.
TRANSLATIONAL OUTLOOK: The fidelity of regional myocardial function assessment by means of segmental longitudinal strain measurements not only depends on type, localization, and extent of the abnormality but also has relevant vendor-specific differences. As of today, a perfect match of good software, good image quality, and careful post-processing and data interpretation with pathophysiologic understanding is still mandatory for meaningful clinical conclusions, which leaves little tolerance for the small imperfections in image acquisition and post-processing, as well as operator skills, that unavoidably exist in daily practice. Although global longitudinal strain has reached a level of development at which it can be safely recommended for clinical use, further improvements in the robustness of regional strain measurements would be desirable to make this great tool more feasible in daily practice.
For supplemental figures, please see the online version of this paper.
Dr. Mirea is permanently affiliated to the Department of Cardiology, University Hospital of Craiova, Romania. This study was supported by a dedicated grant from the American Society of Echocardiography. Dr. Mirea has received a research grant from the European Association of Cardiovascular Imaging. Dr. Pagourelias holds a research grant from the European Association of Cardiovascular Imaging. Dr. Thomas has received honoraria and consulting fees from Edwards, Abbott, and GE. Dr. Voigt holds a personal research mandate from the Flemish Research Foundation; and has received a research grant from the University Hospital Gasthuisberg. All other authors have reported that they have no relationships relevant to the contents of this paper to disclose.
- Abbreviations and Acronyms
- area under the curve
- aortic valve closure
- cardiac magnetic resonance
- late gadolinium enhancement
- left ventricle
- myocardial infarction
- peak systolic
- post-systolic index
- post-systolic strain
- receiver operating characteristic
- Received October 21, 2016.
- Revision received January 24, 2017.
- Accepted February 17, 2017.
- 2017 American College of Cardiology Foundation
- Stankovic I.,
- Prinz C.,
- Ciarka A.,
- et al.
- Scott P.A.,
- Morgan J.M.,
- Carroll N.,
- et al.
- Chan J.,
- Khafagi F.,
- Young A.,
- Cowan B.,
- Thompson C.,
- Marwick T.
- Kwon D.H.,
- Halley C.M.,
- Carrigan T.P.,
- et al.
- Amundsen B.H.,
- Helle-Valle T.,
- Edvardsen T.,
- et al.
- Tennant R.,
- Wiggers C.
- Becker M.,
- Hoffmann R.,
- Kühl H.P.,
- et al.
- Sjøli B.,
- Ørn S.,
- Grenne B.,
- Ihlen H.,
- Edvardsen T.,
- Brunvand H.
- Aarsaether E.,
- Rösner A.,
- Straumbotn E.,
- Busund R.
- Lang R.M.,
- Badano L.P.,
- Mor-Avi V.,
- et al.
- Hosmer D.W.,
- Lemeshow S.
- Rösner A.,
- Avenarius D.,
- Malm S.,
- Iqbal A.,
- Bijnens B.,
- Schirmer H.
- Cimino S.,
- Canali E.,
- Petronilli V.,
- et al.