Author + information
- Received March 3, 2013
- Revision received August 2, 2013
- Accepted August 9, 2013
- Published online December 1, 2013.
- Wei-Hua Yin, MD∗,
- Bin Lu, MD∗∗ (, )
- Nan Li, MD∗,
- Lei Han, MD∗,
- Zhi-Hui Hou, MD∗,
- Run-Ze Wu, MD†,
- Yong-Jian Wu, MD‡,
- Hong-Xia Niu, MD‡,
- Shi-Liang Jiang, MD∗,
- Aleksander W. Krazinski, BS§,
- Ullrich Ebersberger, MD§,
- Felix G. Meinel, MD§ and
- U. Joseph Schoepf, MD§
- ∗Department of Radiology, State Key Laboratory of Cardiovascular Disease, Fuwai Hospital, National Center for Cardiovascular Diseases, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, People's Republic of China
- †Siemens Healthcare, CT Division, Beijing, People's Republic of China
- ‡Department of Cardiology, State Key Laboratory of Cardiovascular Disease, Fuwai Hospital, National Center for Cardiovascular Diseases, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, People's Republic of China
- §Department of Radiology and Radiological Science and Division of Cardiology, Department of Medicine, Medical University of South Carolina, Charleston, South Carolina
- ↵∗Reprint requests and correspondence:
Dr. Bin Lu, Fuwai Hospital, #167 Bei-Li-Shi Street, Xi-Cheng District, Beijing, 100037, People's Republic of China.
Objectives This study sought to determine whether a 50%-reduced radiation dose protocol using iterative reconstruction (IR) preserves image quality and diagnostic accuracy at coronary computed tomography angiography (CTA) as compared with a routine dose protocol using traditional filtered back projection (FBP).
Background IR techniques show promise to decrease radiation requirements at coronary CTA. No study has performed a direct head-to-head, intraindividual comparison of IR algorithms with FBP vis-à-vis diagnostic accuracy and radiation dose at coronary CTA.
Methods Sixty consecutive subjects (45 men, 53.3 ± 9.4 years of age) prospectively underwent coronary catheter angiography (CCA) and 2 coronary CTA scans. One coronary CTA acquisition used routine radiation dose settings and was reconstructed with FBP. For another scan, the tube current–time product was reduced by 50%, and data were reconstructed with IR. Studies were blindly and randomly interpreted. Image quality, radiation dose, and diagnostic accuracy were compared using CCA as the reference standard.
Results Sensitivity and specificity for diagnosing ≥50% coronary artery stenosis on a per-segment level were 88.5% and 92.1% with FBP and 84.2% and 93.4% with IR, respectively. On a per-patient level, sensitivity and specificity were 100% and 93.1% with FBP and 96.8% and 89.7% with IR, respectively (all p > 0.05). With FBP versus IR, the area under the receiver-operating characteristic curve was 0.903 (95% confidence interval [CI]: 0.881 to 0.922) and 0.888 (95% CI: 0.864 to 0.909) on a per-segment level, and 0.966 (95% CI: 0.883 to 0.996) and 0.932 (95% CI: 0.836 to 0.981) on a per-patient level, respectively (p = 0.290 and 0.330). Compared with FBP, the iterative series showed no significant (p > 0.05) differences in image quality analyses. Median dose–length product was 52% lower for the IR protocol compared with the FBP protocol (109.00 [interquartile range: 82.00 to 172.50] mGy·cm vs. 52.00 [interquartile range: 39.00 to 84.00] mGy·cm, p < 0.001).
Conclusions Compared with a routine radiation dose FBP protocol, 50% reduced dose acquisition using IR preserves image quality and diagnostic accuracy at coronary CTA.
Concerns over radiation doses from coronary computed tomography angiography (CTA) have hastened efforts at dose optimization. Conventional filtered back projection (FBP), the traditional method of image reconstruction at computed tomography (CT), incurs a trade-off between spatial resolution and image noise (1), which limits the options for further reduction of the radiation dose. Because of improved computer power, several different iterative CT reconstruction techniques have become available over recent years that show promise of decoupling the relationship between radiation requirements and spatial resolution to some extent, and as a common hallmark, of substantially reducing image noise (1–5). Accordingly, these algorithms are currently under intense evaluation with the aim of further reducing radiation requirements without incurring a loss in diagnostic image quality.
The iterative reconstruction (IR) algorithm used in this study primarily operates in the raw data domain. It uses initial voxel attenuation coefficients to predict projection data and compares these predictions with actual data. Attenuation is iteratively modified until the error between actual and estimated data is acceptable. Several iterations and updated correction loops aim at improving image quality and suppressing image noise, with the goal of enabling the use of lower radiation dose protocols, while maintaining diagnostic quality. Reduced radiation, achieved through manipulation of tube voltage and current, decreases the density and energy of penetrating photons. Decreased photon density and energy result in higher image noise, subsequently corrected with IR, reducing radiation requirements. Several benefits of IR algorithms for use with coronary CTA and other CT applications have been described (1–7). However, an intraindividual comparison of the performance of IR algorithms with FBP protocols vis-à-vis diagnostic accuracy and radiation dose requirements at coronary CTA has not been performed to date.
Accordingly, we undertook this prospective investigation to test, in the same patient population, our hypothesis that IR algorithms, applied to reduced radiation dose acquisitions, can maintain image quality and diagnostic accuracy at coronary CTA compared with coronary catheter angiography (CCA) with substantial radiation dose reduction over traditional FBP.
This single-center study was approved by our hospital's institutional ethics committee. From May to October 2012, a total of 64 consecutive symptomatic patients (45 men, age 53.3 ± 9.4 years) who had been clinically referred for elective CCA because of suspected coronary artery disease (CAD) were approached for participation, after permission of their attending physician of record had been obtained. Inclusion criteria were subject age between 30 and 70 years and sinus rhythm. Subject exclusion criteria were known CAD, history of prior percutaneous intervention or bypass surgery, prior reaction to iodinated contrast materials, impaired renal function (serum creatinine >120 μmol/l), inability to hold their breath; and acute coronary syndrome. Of the 64 patients approached for participation, 4 refused, so that the final study cohort consisted of 60 patients who provided written informed consent.
CT scanning protocols
One or 2 days before their scheduled CCA procedure, all patients underwent coronary CTA on a second-generation dual-source CT scanner (SOMATOM Definition Flash, Siemens Healthcare, Forchheim, Germany). Acquisition parameters were: 2 × 64 × 0.6-mm detector collimation and 280-ms gantry rotation time. All studies were acquired in a craniocaudal direction at end-inspiration. Attenuation-based tube current modulation (CARE Dose4D, Siemens Healthcare) was applied per default. For contrast medium enhancement, automated bolus tracking was used in a region of interest within the ascending aorta, with a signal attenuation trigger threshold of 100 Hounsfield units (HU) and a 6-s scan delay. We used a triple-phase contrast medium injection protocol, which consisted of 50 to 60 ml of undiluted contrast agent (iopromide [Ultravist] 370 mgI/ml, Bayer Healthcare, Berlin, Germany) followed by a 30-ml 30%:70% mixture of contrast medium and saline, and a 40-ml saline chaser bolus, all injected with flow rates of 4 to 5 ml/s (8). All patients underwent 2 coronary CTA acquisitions within 10 min, the order of which was determined by statistical randomization software. With the exception of the tube current–time product, all acquisition parameters, including the contrast medium injection protocol, were kept constant between the 2 scans. The scan protocol was chosen with the goal of minimizing radiation dose in accordance with the “as low as reasonably achievable” (ALARA) principle. In patients with heart rates of ≤60 beats/min, the prospectively electrocardiogram (ECG)-triggered, high-pitch spiral scan mode (9) was used with a trigger phase at 60% of the R-R interval. In patients with a heart rate between 60 beats/min and 70 beats/min, we used the prospectively ECG-triggered, sequential acquisition mode at a diastolic 70% to 80% of the R-R interval. With heart rates ≥70 beats/min, we used the prospectively ECG-triggered, sequential acquisition mode at a systolic 35% to 45% of the R-R interval. This approach reflects our clinical practice, where we avoid administration of beta-blockers below heart rates of 90 beats/min, but rather compensate for faster heart rates by adaptation of the acquisition method. No nitroglycerin was administered as part of the study protocol. X-ray tube potential settings were adjusted individually for each patient, depending on body mass index (BMI). The routine x-ray tube settings for reconstruction with FBP consisted of 120-kV tube potential with a 320-mAs tube current–time product for patients with a BMI ≥25 kg/m2, 100 kV with 370 mAs for BMI ≥20 kg/m2 and <25 kg/m2, and 80 kV with 400 mAs for BMI <20 kg/m2 (Fig. 1). For the scan acquisitions to be reconstructed with IR, the x-ray tube current–time products were reduced by 50%, that is, to 160 mAs with 120 kV, 186 mAs with 100 kV, and 200 mAs with 80 kV.
All coronary CTA datasets were reconstructed with 0.75-mm section thickness, 0.5-mm reconstruction increment, and a 200-mm field of view. FBP series were reconstructed with a medium-smooth tissue convolution algorithm (B26f), whereas IR used the corresponding I26f kernel. In addition, series were reconstructed with the higher spatial frequency B46f and I46f algorithms, because of the previously described beneficial effects of these kernels on the evaluation of heavily calcified vessels (10). Of the 5 noise suppression presets (strength 1 to 5) available with our IR algorithm, we consistently chose the medium-strength level of 3 in accordance with a recent report (11).
CT volume dose index and dose–length product (DLP) for both coronary CTA acquisitions were recorded from the automatically generated patient dose report. Radiation dose from CCA was not estimated in the conduct of this study.
Subjective image quality analysis
All image datasets were transferred to a picture archiving and communication systems (PACS) diagnostic workstation (Advantage Windows, GE Healthcare, Waukesha, Wisconsin). Two observers (with 22 and 9 years of experience in cardiovascular imaging, respectively), who were blinded to the reconstruction technique and unaware of CCA results, independently evaluated all coronary CTA studies in random order; however, FBP and iteratively reconstructed image series of the same patient were presented approximately 6 weeks apart in order to minimize reader recall bias. Both observers used transverse sections and standardized window settings (window level 300 HU, width 800 HU) to assess image quality. The overall image quality was graded on a 4-point scoring system (9): 1, excellent, no artifact; 2, good, minor artifacts; 3, moderate, artifacts, but diagnosis still possible; and 4, poor, nondiagnostic. Image quality was assessed on a per-vessel level, including the left main, left anterior descending, left circumflex, and right coronary arteries. For evaluation of coronary artery luminal integrity, the observers used transverse sections as well as secondary visualization methods provided by the interpretation platform, such as maximum intensity projections, curved multiplanar reformats, and 3-dimensional volume-rendered technique. The presence of luminal stenosis ≥50% was assessed on a per-segment, per-vessel, and per-patient level, using the 16-segment modified American Heart Association classification (12). A final consensus read was performed to resolve discrepancies in interpretation between the 2 observers regarding image quality scores and coronary artery stenosis.
CCA was performed according to standard Judkins technique, using 5-F or 6-F diagnostic catheters. At least 5 projections of the left and 2 projections of the right coronary artery were acquired. Two experienced cardiologists (with 23 and 16 years of experience in CCA interpretation, respectively), who were blinded to coronary CTA results, visually evaluated the presence of coronary luminal stenosis and followed the same procedure as described for coronary CTA in the preceding text in order to resolve discrepancies in interpretation.
At both coronary CTA and CCA interpretation, the observers recorded the reasons for nondiagnostic segments, such as “blooming” artifacts due to heavy calcifications, motion artifacts, or a low contrast-to-noise ratio (13). Segments that were behind complete ostial occlusion of a vessel, congenitally absent vessels, or vessels with an ostial diameter of <1.5 mm were labeled as not available per protocol.
The 2 coronary CTA image datasets acquired in each patient were displayed side by side as transverse sections, using the standardized window settings as described in the preceding text. A single observer (with 2 years of experience in cardiac CT) prescribed circular region of interests of 100 mm2 centrally within the aortic root at the level of the right coronary artery ostium in order to measure CT attenuation (in HU) and image noise, expressed as the SD of CT attenuation. Care was taken that all measurements were performed in the same location of the respective image series. Signal-to-noise ratio was defined as the mean CT attenuation divided by image noise measured in the aorta. In addition, regions of interest of 2 mm2 to 4 mm2 were placed in a proximal portion of the left main coronary artery lumen, as well as in the pericoronary adipose tissue. Contrast-to-noise ratio was defined as the difference between the mean CT attenuation within the coronary artery lumen and within the pericoronary fat, divided by image noise. We avoided placing regions of interests in areas with plaque or heterogeneous regions of pericoronary fat. All region-of-interest measurements were performed in triplicate and their average used for statistical evaluation.
Data were analyzed using SPSS version 16.0 (SPSS, Chicago, Illinois). Continuous variables were expressed as mean ± SD, and categorical variables as frequencies or percentages. Ordinal variables were summarized as medians and 25th and 75th percentiles. Normally distributed variables were compared using the paired Student t test, non-normally distributed variables using the Wilcoxon matched-pairs signed rank test. The McNemar test was used for categorical variables. p < 0.05 was considered statistically significant. Inter-reader agreement between the 2 observers for the assessment of coronary artery stenosis at their initial interpretation before reconciliation was determined using Cohen's Kappa coefficient. Levels of agreement on k values were defined as follows: k = 0.81 to 1.00, excellent; k = 0.61 to 0.80, good; k = 0.41 to 0.60, moderate; k = 0.20 to 0.40, fair; k < 0.20, poor. Receiver-operating characteristic (ROC) analyses were performed to compare diagnostic performance. The areas under the ROC curves along with the corresponding 95% confidence intervals (CIs) were calculated in MedCalc for Windows (version 12.5, MedCalc Software, Ostend, Belgium) and compared using the method described by DeLong et al. (14).
In the 60 subjects who agreed to participate, all coronary CTA studies were successfully completed without complications, resulting in a total of 120 datasets for analysis. Demographics and risk factors of the study population are provided in Table 1. The mean total contrast media volume that subjects received during the 2 scan acquisitions was 124 ± 5 ml. There was no significant difference in mean heart rate during the 2 scan acquisitions (60 ± 10 beats/min vs. 61 ± 8 beats/min, p > 0.05).
Of the 960 coronary artery segments, 782 (81%) could be evaluated on both coronary CTA acquisitions and CCA. Fifteen segments (1.6%) were reconciled for a mismatch in terminology between coronary CTA and CCA segmental designations. One hundred thirty-six segments (14.2%) were labeled not available per protocol on 1 or more of the 3 imaging studies obtained in each patient and were excluded from the statistical analysis: 107 segments (11.1%) were <1.5 mm in diameter at the origin, 29 segments (3.0%) were inadequately visualized on CCA because of proximal occlusion. Forty-two segments (4.4%) were deemed nondiagnostic, consistently by both observers: 14 segments (1.5%) could not be confidently evaluated on coronary CTA due to a blooming artifact from heavy calcification, and 28 segments (2.9%) had motion artifacts. These 42 segments were classified as positive for ≥50% stenosis and were included for statistical analysis.
Luminal stenosis ≥50% by CCA was found in 31 of 60 patients (52%), 73 of 240 coronary vessels (30%), and 139 of 824 (17%) coronary segments. Of 60 patients, 4 had left main disease, 14 had 3-vessel disease, 10 had 2-vessel disease, and 7 had single-vessel disease. Accuracy, sensitivity, specificity, positive predictive value, and negative predictive value of coronary CTA were calculated on per-segment, per-vessel, and per-patient levels (Table 2). There were no statistically significant (p > 0.05) differences between standard radiation dose FBP acquisitions and 50% reduced dose, IR reconstructed series (Figs. 2 to 4⇓⇓) in their performance for detecting ≥50% luminal stenosis by CCA on any level.
The area under the ROC curve analysis was 0.903 (95% CI: 0.881 to 0.922) and 0.888 (95% CI: 0.864 to 0.909) on a per-segment level, 0.918 (95% CI: 0.876 to 0.949) and 0.907 (95% CI: 0.863 to 0.941) on a per-vessel level, and 0.966 (95% CI: 0.883 to 0.996) and 0.932 (95% CI: 0.836 to 0.981) on a per-patient level for FBP versus IR series, respectively. Thus, according to ROC analysis, FBP and IR protocols showed similar diagnostic performance for detecting ≥50% luminal stenosis on every analysis level (p = 0.290, 0.551, and 0.330) (Fig. 5). Kappa values of 0.792, 0.755, and 0.768 between the 2 observers indicated good initial inter-reader agreement regarding assessment for coronary artery stenosis on the per-segment level.
There was no significant difference in mean CT attenuation (419.56 ± 102.63 HU vs. 418.23 ± 112.16 HU), image noise (24.54 ± 6.42 vs. 24.58 ± 6.78), signal-to-noise ratio (17.48 ± 3.69 vs. 17.53 ± 4.12), and contrast-to-noise ratio (34.25 ± 11.20 vs. 32.67 ± 12.24) (all p > 0.05) between the 2 acquisition series (Table 3).
Of the 240 coronary arteries analyzed, 154 (64%) received an image quality score of 1 and 73 (30%) had a score of 2 in FBP series. In the iteratively reconstructed coronary CTA studies, 150 (63%) vessels received a score of 1 and 76 (32%) were scored as 2. There was no statistically significant difference in the distribution of image quality scores between the 2 acquisition series (p = 0.552) (Table 4).
Prospectively ECG-triggered, high-pitch spiral acquisitions were performed in 38 (63%) patients; prospectively ECG-triggered, sequential acquisitions during diastole were performed in 13 patients and during systole in 9 patients. The median DLP was 109.00 mGy·cm (82.00 to 172.50) in FBP-reconstructed, standard radiation dose acquisitions and 52.00 mGy·cm (39.00 to 84.00) in 50%-reduced tube current–time product, iteratively reconstructed coronary CTA studies, corresponding to an overall reduction of 52% in radiation dose within our patient cohort. Radiation dose descriptors are summarized in Table 5 and Figure 6. Broken down by acquisition technique, the mean DLP from prospectively ECG-triggered, high-pitch spiral acquisitions reconstructed with FBP was 88.39 ± 32.19 mGy·cm versus 42.87 ± 15.22 mGy·cm in iteratively reconstructed high-pitch series. For sequential, prospectively ECG-triggered acquisitions in systole and diastole, mean DLP with FBP was 226.68 ± 95.74 mGy·cm versus 110.14 ± 46.06 mGy·cm with IR.
Our direct, intraindividual head-to-head comparison in the same patient population shows that compared with a routine radiation dose FBP protocol, a 50% reduced radiation dose image acquisition using IR preserves image quality and, more importantly, diagnostic accuracy at coronary CTA. These findings illustrate the potential of substantial radiation dose reduction across the population for coronary CTA studies, enabled by a reduced tube current–time product when iteratively reconstructed, without jeopardizing the diagnostic yield of the examination.
Since its inception, radiation doses associated with noninvasive coronary CTA had risen with each new generation of multidetector-row CT systems and had reached its zenith with the launch of the first-generation 64-slice scanners in the year 2004. With these platforms, radiation doses from coronary CTA were reported to be as high as approximately 30 mSv, with an estimated median of approximately 12 mSv (15). Since then, vigorous efforts have been undertaken to reduce the radiation dose from this test. The adaptation of prospectively ECG-triggered acquisition techniques (16), refinements in automated anatomical tube current modulation (17), BMI-adjusted x-ray tube potentials (18), volume CT acquisition (19), and the high-pitch spiral technique (20) are just a few of the technical innovations that have incrementally lowered the radiation dose from cardiac CT in recent years.
Although the aforementioned approaches reduce the dose on the data acquisition side, IR algorithms are a post-processing option available for further reducing radiation dose requirements at CT. These techniques were introduced over 3 decades ago; however, for the purpose of conventional CT, they have only recently become clinically available owing to algorithm improvements and increased computer processing power. Previous work has demonstrated image noise reduction and improved diagnostic accuracy in the evaluation of heavily calcified coronary arteries via reduction of blooming artifacts with the use of IR (21). In addition, several previous investigations explored the potential for radiation dose reduction by comparing image quality and diagnostic accuracy between FBP-reconstructed full radiation dose series and iteratively reconstructed series based on image raw data containing only 50% of the original photon count (1,6,22,23). However, these prior studies all used specialized software to simulate lower radiation dose acquisitions from full-dose image raw data in lieu of scanning their patients twice. A direct head-to-head, intraindividual comparison within the same patient population for evaluating the radiation-dose savings potential of IR techniques, or of any of the previously described technical options for radiation protection for that matter, had not been performed to date.
The previous lack of such investigations is likely founded in ethical concerns related to dual subject exposure to radiation and iodinated contrast material. However, the advent of rapid, low-radiation CT examination strategies as used in this current study should increasingly enable the conduct of methodologically strong and ethically sound comparative investigations for evaluating novel techniques aimed at enhanced patient protection, even if they involve scanning the same subject twice. More specifically, the mean total contrast media volume of approximately 125 ml and the median DLP of approximately 161.00 mGy·cm that subjects received from the 2 scan acquisitions combined are very well within the limits of routine clinical applications, as well as of common research settings. Dual investigation of the same subjects is likely methodologically stronger than studies involving, for example, randomized population designs, because the former enables a more specific and realistic evaluation of a limited set of variables as in this current study with the diagnostic performance of IR techniques in lower radiation dose environments while all other variables are kept constant. A wide spectrum of radiation protection strategies were applied in our patient cohort, resulting in an already rather low median baseline radiation dose of 109.00 mGy·cm. Our study design enabled us to demonstrate that the addition of IR techniques allows further reduction of the median dose from coronary CTA acquisitions to 52.00 mGy·while maintaining strong diagnostic performance vis-à-vis CCA as the reference standard.
Several limitations to our investigation merit consideration. First, because of the scope and nature of our single-center investigation, our patient cohort was limited in size. Larger multicenter studies would be desirable to further validate our results. In this context, it is also of consideration that we investigated an Asian population, who generally tend to have leaner body types than average patients in Western societies. However, the usefulness of IR techniques in obese patients has already been suggested by prior investigations (1,24). Second, although care was taken to present studies in a randomized fashion and to minimize reader recall, it is not possible to effectively blind observers to reconstruction technique, because the image characteristics of FBP and IR (described as “oil painting” by one of our readers) remain rather readily distinguishable. Additionally, CCA studies were visually evaluated by 2 experienced cardiologists. A quantitative approach would be preferable in future studies. Some of our research methods (e.g., reconciliation reads between the 2 readers, along with performance of measurements in triplicate) do not correspond to those that would be applied in practice, which may influence the results. Last, we arbitrarily prescribed a 50% reduction in the tube current–time product and the resultant radiation dose, which we considered a conservative, realistic clinical goal. We did not explore whether radiation doses could be lowered even further while still preserving diagnostic accuracy. Meanwhile, coronary CTA with an estimated effective radiation dose equivalent of <0.1 mSv has been suggested to provide sufficient image quality in selected patients through the combination of prospectively ECG-triggered, high-pitch spiral acquisition and the raw data–based IR algorithm used in our investigation (25). However, these promising preliminary observations are pending evaluation in more general patient cohorts and determination of their diagnostic accuracy in comparison with outside reference standards.
Our study results demonstrate that the combination of currently available radiation protection strategies enables acquisition of highly accurate coronary CTA studies at exceedingly low radiation doses. Our direct head-to-head comparison within the same patient population shows that the addition of IR techniques allows further reduction of these already low radiation doses by 52%, without penalties in image quality or, more importantly, in diagnostic accuracy compared with CCA as an outside reference standard.
This study was supported in part by a research grant provided by Bayer Healthcare, Berlin, Germany. Dr. R-Z Wu is an employee of Siemens Healthcare. Dr. Schoepf is a consultant for and/or receives research support from Bayer, Bracco, GE Healthcare, Medrad, and Siemens Healthcare. All other authors have reported that they have no relationships relevant to the contents of this paper to disclose.
- Abbreviations and Acronyms
- body mass index
- coronary artery disease
- coronary catheter angiography
- computed tomography angiography
- confidence interval
- computed tomography
- dose–length product
- filtered back projection
- iterative reconstruction
- receiver-operating characteristic
- Received March 3, 2013.
- Revision received August 2, 2013.
- Accepted August 9, 2013.
- American College of Cardiology Foundation
- Moscariello A.,
- Takx R.A.,
- Schoepf U.J.,
- et al.
- Gosling O.,
- Loader R.,
- Venables P.,
- et al.
- Achenbach S.,
- Marwan M.,
- Ropers D.,
- et al.
- Maruyama T.,
- Takada M.,
- Hasuike T.,
- Yoshikawa A.,
- Namimatsu E.,
- Yoshizumi T.
- Dewey M.,
- Zimmermann E.,
- Deissenrieder F.,
- et al.