Author + information
- Received October 24, 2016
- Revision received December 12, 2016
- Accepted December 15, 2016
- Published online March 6, 2017.
- aMorristown Medical Center, Morristown, New Jersey
- bCentre for Heart Valve Innovation UBC, Vancouver, British Columbia, Canada
- cMedStar Health Research Institute and Georgetown University, Washington, DC
- ↵∗Address for correspondence:
Dr. Linda D. Gillam, Department of Cardiovascular Medicine, Gagnon Cardiovascular Institute, Morristown Medical Center, 100 Madison Avenue, Morristown, New Jersey 07960.
Cardiovascular imaging is an integral component of many clinical trials beyond those for which the primary goal is to evaluate or validate imaging technologies. The scope of such trials is broad, ranging from those in which a medical, surgical, or interventional cardiovascular device or drug is being evaluated to those in which there is concern about cardiovascular adverse events complicating treatment for noncardiac conditions. This paper discusses study design as it pertains to the incorporation of imaging elements, the important role played by imaging core laboratories, the rationale for and approaches to involvement of imagers in clinical trials, and guidance by the U.S. Food and Drug Administration on imaging endpoints in clinical trials.
- cardiac computed tomography
- cardiac magnetic resonance imaging
- clinical trials
- core laboratory
- nuclear cardiology
Cardiovascular imaging is an integral component of many clinical trials beyond those for which the primary goal is to evaluate or validate imaging technologies. The scope of such trials is broad, ranging from those in which a medical, surgical, or interventional cardiovascular device or drug is being evaluated to those in which there is concern about cardiovascular adverse events complicating treatment for noncardiac conditions. The present paper discusses study design as it pertains to the incorporation of imaging elements, the important role played by imaging core laboratories, the rationale for and approaches to involvement of imagers in clinical trials, and guidance by the U.S. Food and Drug Administration (FDA) on imaging endpoints in clinical trials. Table 1 describes the multiple ways in which imaging interfaces with other elements of a clinical trial.
Design of Imaging Assessments
The design of the imaging elements of multicentered cardiovascular clinical trials must take into consideration the technical and diagnostic capabilities of different imaging modalities, their accuracy and reproducibility, relative risks and costs, availability, and, the likelihood that the expertise exists at each of the study sites to provide adequate images for analysis. Where site-reported rather than core laboratory–adjudicated imaging results are to be used, the skill of the interpreting physicians at study sites must also be a consideration. For these reasons, the study design team should include or consult with those who are not only imaging experts but who also have an awareness of what might reasonably be expected in terms of local site technical and interpretive expertise. These considerations can influence overall trial design and the decision to include imaging data as principle study endpoints. As an example, the degree of left ventricular reverse remodeling as determined by transthoracic echocardiographically derived indexed left ventricular end-systolic volume was chosen as the primary endpoint in the Cardiothoracic Surgery Network studies of mitral regurgitation (1,2).
The selection of imaging elements in a trial must, of course, be based on the degree to which imaging can provide a robust and reproducible assessment of parameters that are relevant to the questions being posed by the study. At times, there may be multiple options among available modalities (e.g., when the objective is to measure left ventricular ejection fraction). At other times, the options may be more limited. For example, there is no practical alternative to transthoracic echocardiography for multiple assessments of transvalvular gradients after aortic valve replacement. When multiple options exist, a thoughtful comparison of modalities as to accuracy and reproducibility, risks, costs, and site capabilities can be used to finalize modality selection. Considerations of accuracy and reproducibility should include issues of spatial and temporal resolution, interobserver and intraobserver variability, and performance of the modality under consideration relative to a gold or reference standard.
Depending on the question being posed of the imaging data, test-retest and beat-to-beat variability should also be considered. For some parameters (e.g., echocardiographically derived strain), inter-vendor variability may also be a consideration that could drive a decision to ensure that all images are acquired with equipment of a single manufacturer. All other considerations being equal, the modality or application within a modality (3-dimensional [3D] vs. 2-dimensional, contrast vs. noncontrast imaging) that provides the most accurate and reproducible findings should be used. Risks that must be evaluated include those associated with vascular access and contrast agents, esophageal intubation (transesophageal echocardiography), and radiation exposure. Both risk and cost considerations are particularly important if study-mandated imaging cannot be considered standard of care.
Modern computed tomography (CT), afforded by its isotropic voxels, is capable of reconstructed images in any plane without compromising its spatial resolution. In addition, the technical aspects are highly standardized with well-defined and established acquisition protocols. Imaging is relatively independent of local expertise even when local interpretation may not be feasible. This relative operator independence and standardized acquisitions make it an excellent tool to allow a core laboratory to provide noninvasive imaging procedural guidance and for its images to serve as an anatomic endpoint in clinical trials. Although these features make CT scanning an increasingly popular imaging tool to support cardiovascular clinic trials, significant limitations remain with this technique, particularly the lack of hemodynamic data, the inability to integrate CT scanning at the time of a procedure in the absence of meaningful fusion imaging, and the need to administer iodinated contrast medium. Nonetheless, CT is essential in trials of transcatheter aortic and mitral valve replacement, for example, in which precise and reproducible anatomic measurements are required.
Magnetic resonance imaging (MRI), conversely, offers opportunities in which CT is lacking. Although not as standardized and far more operator dependent, MRI offers hemodynamic evaluation and does not rely on iodinated contrast material or ionizing radiation, thus making it a robust imaging tool for patients with abnormal renal function. MRI has become the reference standard for ventricular volumes and mass as well as myocardial fibrosis.
In contrast, echocardiography has important limitations due to its operator and subject dependence, particularly when 3D acquisitions are required for volumetric assessments. However, it has high spatial and temporal resolution and remains the procedure of choice for assessing valvular function and hemodynamic variables, including pulmonary artery systolic pressure. As a technique that is radiation free, relatively inexpensive, and capable of real-time imaging, it is widely used in trials of native and prosthetic valves as well as those for which an impact on valvular structure and function might be anticipated (e.g., anorectic drugs), and those in which subtle alterations of global ventricular function might be encountered (e.g., strain imaging for assessing the impact of chemotherapeutic agents). Although there is active work in expanding the clinical and research capabilities of radioisotope molecular imaging, in trials for which imaging serves as a tool rather than the major focus of the investigation, nuclear techniques are largely limited to the assessment of myocardial perfusion at rest or with stress.
In some situations, the framework for the use of imaging has been established by independent expert consensus (3,4) or through the need to provide information similar to that in relevant historical trials. As discussed in the following section, the FDA may also have input into the design of imaging elements of trials.
An overarching consideration in imaging study design should be the degree to which a change detected by imaging is meaningful, recognizing that statistical significance can be reached even when the measured changes are within the sampling error of the technique. Where valid quantitative alternatives exist, subjective semi-quantitative measures should not be used. However, there may be instances in which subjective measures are clinically relevant and the best choice (e.g., regional wall motion assessed by echocardiography, cardiac MRI, cardiac CT, nuclear cardiology or invasive left ventriculography) as opposed to the more complex quantitative approaches that reference endocardial movement toward a ventricular centroid.
Once the modality and parameters to be measured are decided, imaging time points must be determined. In general, study visits are rarely scheduled exclusively for imaging, with the norm being that imaging will be incorporated into visits scheduled for clinical follow-up. Decision-making here must incorporate the predicted time at which change might occur as well as considerations of patient safety and convenience, especially where imaging is associated with risk. Consideration should also be given to the possibility of dropouts, particularly when a critical imaging parameter is measured at a time that is remote from subject enrollment. In such cases, redundancy might be built into the imaging schedule with intermediate points, which increase the possibility that imaging data will be available for analysis. Furthermore, the data acquired at intermediate time points may provide important insights into the time course of change.
Role of Imaging Core Laboratories
Over the past decade, as the role of cardiovascular imaging in clinical trials has grown, there has been increasing recognition of the importance of core laboratory adjudication of imaging findings. While there remains a role for early single-site studies in which local interpretation of imaging findings may suffice, the potential for bias as well as the unpredictability of local expertise in image acquisition and interpretation argue strongly for the superiority of using core laboratories to provide standardized unprejudiced imaging results. With the recognition of the discordance between site interpretations and core laboratory assessments in large randomized coronary artery disease trials (5,6), as well as in other settings, the role of imaging core laboratories has appropriately expanded to improve the consistency and reliability of the imaging results. Although important in the early days of interventional procedural development, core laboratory endpoint adjudication becomes essential as progressively lower risk subjects are being enrolled in clinical trials (7–9). In such instances, as clinical outcomes including peri-procedural mortality improve, post-implant imaging has become an increasingly important discriminator of procedural success.
The need for standardization
Standardized definitions of various procedural and imaging endpoints are the first step toward providing translatable and meaningful data from post-procedural imaging. The Valve Academic Research Consortium (3,10) has significantly advanced the field by providing definitions, particularly for those imaging elements that were either rarely or never previously encountered. It should be emphasized, however, that by their very definition, these are living documents requiring adaptation based on what is learned from thoughtful and open-minded evaluation of the ever-growing multicenter and randomized trial data. Importantly, these definitions and the core laboratory data derived by using them not only inform the trials they support but equally importantly provide large datasets to allow procedural/device development and evolution.
Similarly, lessons learned from operationalizing these definitions in image evaluation and comparing imaging results with other study endpoints should, in turn, be considered in the next iteration of the definitions. Paravalvular regurgitation, for example, was a relative footnote in most surgical trials owing to its relative rareness in surgical aortic valve replacement, as has been confirmed in recent trials comparing surgical and transcatheter aortic valve replacement (TAVR) (11). Now, through nearly a decade of work in the TAVR space, paravalvular regurgitation has been re-defined in the context of TAVR, with a focus not only on its identification and quantification but also on developing a connection to adverse clinical outcomes. Although clear definitions to inform the work of imaging core laboratories are essential for the adjudication of procedural endpoints, ultimately linking these imaging findings to downstream clinical outcomes is the true value.
For robust imaging support of a trial, there are certain unifying elements of imaging core laboratories that are needed. To start, core laboratories must be highly organized and have clear and standardized operating procedures (SOPs), many of which are mandated to meet the requirements of sponsor and FDA audits. Examples include those that define all steps in the handling of images, including those taken to ensure confidentiality, image evaluation and reporting of results, staff assignments and training, as well as meticulous quality assessment. In addition, there may be requirements for SOPs related to approaches to corrective and preventive action to minimize errors, planned deviations from core laboratory protocols that may be needed to address unexpected trial outcomes or unanticipated deadlines, and study closeout that must address the approach to post– study completion data storage.
Document management systems that track all updates to SOPs are also important. Alternatively, a single document (the image review charter) may encompass all these elements. Thus, the function of the core laboratory includes many factors beyond simply providing skilled and experienced interpretation of the images, as might be available at the study site. Core laboratories must also be highly responsive to and speak the language of the investigators running the trials they support. It should go without saying that all core laboratories should adhere to good clinical practice standards.
The issue of reproducibility in image interpretation is a critical one. By definition, one cannot be accurate if one is not reproducible. Thus, a key function of core laboratories is to routinely and rigorously assess interobserver and intraobserver variability. Input from a biostatistician will ensure that an appropriate statistical approach is taken and that analyses are conducted correctly. By providing reproducible measurements through clearly articulated processes, core laboratories can help not only define procedural outcomes but also help educate the field as to best practices for image acquisition and analysis. This approach will ultimately allow learning born out of clinical trials to make its way to the clinical setting. Measurements that can only be made in a core laboratory ultimately die with the trial and will have modest if any value to the sponsor, the field, or the patients we serve.
Imager Involvement and Collaboration
For the successful conduct of any trial in which imaging serves an important function, it is essential that imagers be involved on multiple levels, including study design and protocol development, engagement, education and ongoing support of site imagers by the study core laboratory, and oversight of actual image acquisition and transmission at each study site by site imagers. Such involvement is particularly true for studies in which imaging provides primary or secondary endpoints, is essential to establish subject eligibility for a trial (inclusion and exclusion criteria), and/or is needed to guide study-related procedures such as device implantation.
Study design, protocol development, and site and core laboratory selection
The decision to include imaging in a trial is typically made before a core laboratory is selected, perhaps in response to an FDA mandate or simply because imaging is essential to the conduct of the investigation. It is important that those involved in study design at this level understand the capabilities of the different imaging modalities and, within modalities, the different applications. For example, obtaining certain types of information with cardiac magnetic resonance may require sequences that are not universally available, and echocardiographic guidance of device implantation may require advanced 3D transesophageal echocardiographic ability that, again, may not exist at all sites. This information can influence overall study design or site selection. For example, a study may be modified to eliminate imaging data for which there is a low expectation of adequate acquisition, or site screening may be modified to include an assessment of the capabilities of the site imagers, including test cases. Input at this level may be provided by an expert imaging consultant, an employee of the trial sponsor, or a member of the trial steering committee or imaging core laboratory if one has already been selected.
As critical as it is that study sites have the necessary imaging capability, it is also important that core laboratories have experience in the acquisition and analysis of these images. Without experience in image acquisition, it will be difficult for imaging core laboratory staff to refine imaging-related components of the protocol, develop imaging protocols, deliver site training, perform quality assessment of site images, or provide ongoing support for image quality improvement. Similarly, certain images may require analytic tools and expertise that are not universally available. It is expected that these considerations will be important in developing the request for applications for core laboratory services, vetting applicants, and ultimately selecting a core laboratory.
Although there is no formal process for accrediting core laboratories as exists for clinical laboratories, the American Society of Echocardiography has published standards for echocardiographic core laboratories (12), and corporate study sponsors often evaluate core laboratories during the selection process and regularly thereafter with a formal intense auditing process that is similar, in some ways, to the approach they would take in selecting a manufacturing subcontractor. These audits emphasize the existence of and close adherence to SOPs that cover a broad range of topics including staff training, image analysis, results reporting, and quality assessment and improvement. Although smaller start-up sponsors may not have these resources, it is expected that the core laboratory will apply the same high standards to all studies that it undertakes. The American Society of Echocardiography provides an online listing of academic imaging core laboratories that are associated with not-for-profit institutions and which have performed core laboratory functions in at least 1 multicenter trial, either funded by the National Institutes of Health or with data submitted to the FDA (13).
Core laboratory involvement
Once the core laboratory is selected, depending on the complexity of the imaging elements of the trial, it may be helpful for the core laboratory to provide input into the study design and report elements. At a minimum, it is expected that the core laboratory will develop a study-specific imaging manual that provides detailed direction to sites regarding image acquisition, storage, and transmission. This document typically will form the basis for additional training in the form of webinars or live presentations at investigator meetings. These are designed to confirm that site imagers understand what is being requested of them, and, ideally, the rationale for the requests. Such interactions with site imagers are critical to the overall degree to which sites will provide adequate images for core laboratory analysis. Test images may be helpful to ensure that individual sites are capable of following the protocol and archiving images in the appropriate format. In addition, imaging core laboratories play an important role in educating sites and disseminating knowledge around standardization of image acquisition and optimization of image quality.
As the trial is conducted, the imaging core laboratory should collaborate and be in communication with both the study leadership and study sites. Communication with study leadership can be in the form of regular status updates or conference calls; these will keep both parties aware of study progress and concerns. Communication with the study sites will typically include formal written feedback on the adequacy of submitted images ideally on both an individual study and aggregate basis. Where recurrent deficiencies are detected, remedial training may be necessary. Such core laboratory–site interactions have been shown to improve the quality of image acquisition (Figure 1) (14).
Another area in which collaboration between the core laboratory and study leadership is important is in the development of manuscripts in which core laboratory imaging data are used. At a minimum, the core laboratory should ensure that description of imaging methods and reporting of imaging results are accurate. Where imaging data provide important results, it is reasonable to include representatives of the core laboratory staff as manuscript authors or members of the publication committee. There should be an opportunity for the core laboratory to generate suggestions for manuscripts and encourage the participation of active site imaging coinvestigators as coauthors.
Site imager involvement and collaboration
It is human nature to be more vested in doing a good job if one’s role and contribution are recognized. For this reason, it is important that site imagers who will have ownership over the study imaging be identified. Ideally, such individuals will serve as site coinvestigators, acknowledgment that may be helpful in, for example, promotion or academic appointment in an academic medical center. Not engaging with the local site imaging teams runs the risk of marginalizing them, and their support to help optimize image quality may be lost, thereby negatively affecting the trial.
For studies in which advanced imaging is required, identifying individuals with this expertise at each site is essential. However, having a site imaging point person is also important when the study imaging elements are more basic (e.g., left ventricular wall thickness or mitral inflow patterns). Even when the requested images are those that are performed on a regular basis by clinical laboratories, one should not assume that they will be done well or according to professional guidelines and standards. For most imaging modalities, there can be variability in the skill set or commitment of the technicians (sonographers) or physicians acquiring the images, and one of the roles of the site imaging coinvestigators is to build a strong site team.
Another consideration is the equipment assigned to the study. Ideally, the best imaging systems will be used, and in areas in which there are recognized inter-vendor differences (e.g., echocardiographic strain), it may additionally be important to ensure that the same system is used for each study. Failure to assign ownership of the site images to an on-site imager will likely result in poor or, at best, inconsistent image quality that, in turn, will result in missing or misleading data points for the trial. This outcome is particularly true for echocardiography, an extremely operator-dependent modality. For example, when a sonographer effectively demonstrates a regurgitant jet that another sonographer has failed to capture, one may incorrectly conclude that new valve regurgitation has occurred. An engaged site imager will not only help ensure the technical adequacy of the images but their storage in an approved format. The latter is particularly important if the study protocols deviate from those that are typically used clinically or where the analyses require access to raw imaging data or volumetric data sets.
Although cardiac CT scans are generally somewhat less operator dependent, adherence to best practice is very important to optimize image quality and ensure the best results. Lack of engagement results in shortcuts being taken often because of a lack of understanding of what is important for a trial such as poor adherence to heart rate control protocols for coronary evaluation and not acquiring multiphase data for CT examinations before TAVR. Depending on the importance of the imaging findings to the trial, it may be reasonable to support face-to-face imaging investigator meetings either in conjunction with larger principal investigator meetings or as stand-alone gatherings.
For studies that involve cardiac CT scanning, MRI, or, in some cases, nuclear cardiology, collaboration between cardiology and radiology may be required. In such instances, there should be consideration of including representatives from both specialties in the site imaging leadership. All those involved in providing or supporting imaging for subjects in a trial must speak the same language and be aware of the study design and aims.
Regulatory Perspective: FDA Guidance on Imaging Endpoints
The discussion of core imaging laboratory practices, standards, reproducibility, and staff responsibilities addresses logistic and technical considerations necessary to have high-quality imaging in clinical trials. Many of these areas are addressed in-depth by professional society guidelines (12) with clear recommendations. Similarly, the FDA has released draft standards for the Clinical Trial Imaging Endpoints Process (15). Although this guidance document was released as a draft open for comment in March 2015 without a final version to follow, it has become, as is, a reference standard for industry and sponsors of clinical trials.
In brief, the draft guidance document outlines the issues a sponsor should consider to ensure optimal imaging data quality from a clinical trial intended to support the approval of a new drug or biologic product. Some sponsors have extrapolated this guideline to also apply to trials intended to support the approval of a new device. Although it covers an array of categories, much of the document is focused on image acquisition, image display, data archiving, and interpretation process standards. The draft does not mandate that all of the details for these categories be documented in an imaging charter; however, it suggests that the sponsor should take into consideration the extent and complexity of the imaging to determine if a charter (or detailed SOPs) is warranted. Certainly, if the imaging process standards extend beyond those typically performed at the clinical site (i.e., there are study-specific imaging requirements), most would recommend that there is not just a study-specific imaging protocol but an entire charter or set of SOPs to address all the components of image processing.
The draft FDA guidance document also asks sponsors to consider the following key attributes in image acquisition and analysis and, from this assessment, to determine the need for and role of a core laboratory:
1. Centralized image interpretation: this factor will be determined by the role of the imaging data (primary endpoint vs. peripherally important), the variability and susceptibility to bias in the image interpretation, and by the imaging modality–specific qualities. When bias is possible, when variability of the imaging modality is high, and when the requirements for advanced training/experience are high, centralized image interpretation becomes essential. From a practical point of view, very few clinical trials today do not use centralized image interpretation because of these factors.
2. Blinded review: this option becomes important to reduce bias and enhance the credibility of the image assessments. Often with blinding, there is greater consistency of image assessments. Therefore, blinding is always used unless not feasible (i.e., the intervention vs. control group can be determined from the image itself).
3. Frequency of image acquisition: this rate will be determined by the disease/device/drug being studied and the likelihood of events or changes in imaged parameters during the clinical trial. An imaging evaluation should always be performed at baseline and at sufficient frequency to provide a reasonably precise measure of the time to the expected clinical endpoint (or change detected on imaging).
4. Timing of image interpretation: interpretation of images can be conducted by using several different approaches ranging from near real-time (i.e., as the imaging studies are done, they are sent to the core laboratory and interpreted immediately) to bulk reading at the end of the study. There are clear advantages and disadvantages to both approaches. For example, real-time imaging provides up-to-date data that can be used to make decisions throughout the study, such as an interim analysis or data to be used by a data safety monitoring board. However, real-time data analysis is subject to bias (i.e., interpreter knows which studies are baseline and which are likely follow-up) and temporal drift, in which the approach of the reader to making measurements or thresholds for labeling abnormalities as mild, moderate, or severe may change over time. Bulk reading at the end of the study, with blinding to date and sequence, removes the possibility of temporal drift but does not provide any data until the study is complete. In addition, it precludes the possibility of making adjustments (i.e., re-training) if image quality is poor from specific clinical sites. From the perspective of core laboratories, it may be difficult to plan laboratory staffing to handle the large bolus of studies inherent in the bulk reading approach. For these reasons, most sponsors (and core laboratories) favor some variant of real-time interpretations.
5. Standardization of imaging procedures: this feature is the primary focus and goal of the draft guidelines and the area most clinical trials concentrate on when using imaging endpoints. This approach is especially true when the imaging requirements go beyond standard clinical imaging practices. Standards should be considered for imaging modality, modality equipment performance features, qualifications of imaging staff, calibration requirements, specific image acquisition features and expectations, quality control of image acquisition, image display requirements, image interpretation standards and definitions, quality control of image interpretation, image and data-handling procedures, and image and measurement archiving.
When imaging endpoints are used in clinical trials, it is imperative that the data acquired from images be accurate, reproducible, and unbiased. Only then can they support the overall conclusions drawn from the trial and, in the case of trials related to devices and drugs, support applications for device/drug approval. To ensure that imaging data meet these requirements, imaging input into trial design, communication between site imagers and study representatives, and optimized image acquisition and interpretation are all essential. The imaging collaboration continues after all trial images have been analyzed through study closeout. For most trials, use of a core laboratory and with it clearly articulated standard operative procedures, including those for quality assessment and improvement, will be necessary.
Dr. Gillam provides core laboratory services through Morristown Medical Center to Edwards Lifesciences, Medtronic, and Middlepeak. Dr. Leipsic is a consultant for Edwards Lifesciences, Circle CVI, Valtech, and HeartFlow; and provides core laboratory services through UBC to Ancora, Edwards Lifesciences, Medtronic, Neovasc, and Tendyne. Dr. Weissman provides core laboratory services through Medstar to Abbott Vascular, Boston Scientific, Edwards Lifesciences, St. Jude Medical, Medtronic, LivaNova, JenaValve, and Direct Flow Medical. Pamela Douglas, MD, served as the Guest Editor for this paper.
- Abbreviations and Acronyms
- computed tomography
- Food and Drug Administration
- magnetic resonance imaging
- standard operating procedure
- transcatheter aortic valve replacement
- Received October 24, 2016.
- Revision received December 12, 2016.
- Accepted December 15, 2016.
- American College of Cardiology Foundation
- Acker M.A.,
- Parides M.K.,
- Perrault L.P.,
- et al.
- Stone G.W.,
- Vahanian A.S.,
- Adams D.H.,
- et al.
- Kataoka A.,
- Scherrer-Crosbie M.,
- Senior R.,
- et al.
- Shaw L.J.,
- Weintraub W.S.,
- Maron D.J.,
- et al.
- Webb J.,
- Gerosa G.,
- Lefèvre T.,
- et al.
- Leon M.B.,
- Piazza N.,
- Nikolsky E.,
- et al.
- Douglas P.S.,
- DeCara J.M.,
- Devereux R.B.,
- et al.
- ↵American Society of Echocardiography. Academic Imaging Core Labs: Reference Guide for Academic Imaging Core Labs. Available at: http://asecho.org/what-we-do/research-resources/academic-imaging-core-labs/. Accessed February 21, 2017.
- ↵Food and Drug Administration. Clinical Trial Imaging Endpoint Process Standards: Guidance for Industry. Available at: http://www.fda.gov/downloads/drugs/guidancecomplianceregulatoryinformation/guidances/ucm268555.pdf. Accessed February 21, 2017.