Artificial intelligence in coronary artery calcium score: rationale, different approaches, and outcomes

Recently, several clinical guidelines endorsed the use of CACS to up or down-stratify asymptomatic, middle-aged patients at intermediate risk of ASCVD [15] and to guide therapeutic decisions (Table 2). Nonetheless, the benefits of CACS have not been fully exploited yet. Recent evidence showed that CACS-weighted pre-test probability models better stratified the risk of obstructive CAD compared to those based on conventional risk factors in patients with typical and atypical chest pain [30]. Additionally, by reclassifying 54% of patients to a lower CAD pre-test probability based on their CACS, it reduced downstream diagnostic testing [30].

Of note, while the “power of 0” is still debated [31], the 10-year risk of all-cause mortality of CACS 0 is < 1% [19]. Conversely, minimal increases (CACS 1-10) are associated with a two-fold increase in the overall mortality rates [32]. Moreover, individuals with a CACS ≥ 1000 have five times the risk of ASCVD and roughly three times the risk for all-cause mortality of those with a CACS 0 after adjusting for conventional risk factors [33]. Even more alarming is the absence, of a disease-specific and all-cause mortality plateau effect in this patient population [33].

Although CACS was developed on ECG-gated images, a meta-analysis of 661 adults showed a high correlation between scores derived from ECG-gated and non-gated, non-contrast images [34]. This may improve ASCVD risk assessment in specific cohorts of patients, namely candidates for lung cancer (LC) screening and oncologic patients sharing risk factors with patients with ASCVD. Former or active smokers enrolled in the control arm of an LC screening randomized controlled trial had equivalent 10-year mortality rates of LC and ASCVD (24% and 21%, respectively) [35]. Indeed, older age and a heavier smoking history increased the likelihood of undergoing LC screening in a population-based study involving more than 14,500 adults, of whom 67% were overweight or obese and 22% had diabetes [36]. Further, 62% of patients enrolled in a cross-sectional LC screening study were CACS positive, 7% had values > 1000, and 57% of those qualifying for primary prevention statin therapy were not actively treated [37]. Tailor et al. corroborated this evidence, confirming that a minority of LC screening patients carried a diagnosis of ASCVD (31%), while most (74%) were eligible for statin therapy but not under active treatment [38].

However, performing CACS on non-gated images may underestimate the Agatston score. Xie et al. showed that 9% of CACS-positive and 19% of CACS ≥ 400 patients on ECG-gated images were downgraded to CACS 0 or < 400 on non-gated images, respectively [34]. A possible solution to the problem of accurate CACS quantification in non-gated images could be qualitative analysis. CAC-DRS also supports the qualitative evaluation of the presence and extent of coronary tree calcification on non-gated images [24, 39]. Qualitative CAC-DRS are divided into the following categories: very low (CAC-DRS 0), mildly increased (CAC-DRS 1), moderately increased (CAC-DRS 2), and moderate to severely increased (CAC-DRS 3), mirroring their quantitative counterpart, and providing similar ASCVD risk stratification and treatment recommendations [24].

Neither the Agatston score nor CAC-DRS considers plaque location. However, involvement of the left main trunk and left anterior descending artery (LAD) is associated with a worse outcome [40‐42]. Additionally, the higher the number of affected vessels, the higher the risk of CAD [43]. Hence, the SCCT suggested reporting CAC location, irrespective of the evaluation method applied [24]. Scan-rescan studies showed the complexity of measuring a modifiable parameter. Those with a short interscan period (23 ± 27 days) demonstrated low measurement variations [44]. Conversely, a sub-study of the MESA with an interscan period between 1 and 5 years showed that baseline CACS values, body mass index, and scanner factors must be considered when quantifying CACS variations in longitudinal studies [45]. Of the 2,832 patients with CACS > 0 at baseline, 85% showed an increase in CACS at follow-up scans [45]. However, 52% of the variation was attributable to the previous factors, and factor-adjusted analysis demonstrated that only 32% of the initial patients had increased CACS values [45]. An additional point not yet taken into consideration is the difference in CAC profiles between sexes. Women have fewer calcified plaques and less calcified vessels, ultimately generating a lower CAC volume [46]. Additionally, the proportion of women with detectable CAC typically rises at the age of 46, nearly a decade after that observed in men [46]. Nonetheless, CACS-positive women have an overall 1,3-fold higher relative risk of ASCVD-related death compared to men. This relationship increases with CACS values, being ~ 1,8-fold higher in women with CACS > 100 [46]. Interestingly, a sub-study of the MESA involving 2,456 postmenopausal women showed that those with early menopause had a lower prevalence of CACS 0 (55%) than those without early menopause (60%) [47]. Further, in the coronary artery risk development in young adults study, the prevalence of women with CACS > 0 was 18%, 21%, and 13% for premature menopause, menopause ≥ 40 years, and premenopausal, respectively [48]. Of note, the results of these studies did not reach statistical significance, leaving the association between CACS and menopausal status unanswered. Finally, it is worth noting that, irrespective of sex, ~ 10% of CACS-negative individuals have non-calcified plaques, 1% have obstructive non-calcified plaques, and ~ 0.5% develop cardiac events at long-term follow-up (≥ 42 months) [49].

Installation and availability of dual-energy and photon-counting CT (PCCT) brought exciting perspectives into CACS [39, 50]. These techniques allow the reconstruction of virtual non-contrast (VNC) images from contrast-enhanced exams [18]. Therefore, CACS could be quantified from contrast-enhanced scans. However, VNC images proportionally underestimated CACS values derived from non-contrast images primarily because of an underestimation of plaque volume and density, requiring ad-hoc conversion factors to yield accurate results [51, 52]. Additionally, ultra-low dose non-enhanced CACS images acquired using a dual-energy scanner underestimated CACS-based risk categories in 17% of patients compared to their standard-dose counterparts [39]. PCCT, a pioneering technique directly converting the energy of X-ray photons into an electric pulse, can increase the contrast-to-noise ratio (CNR) and detect smaller and less calcified objects [50, 53]. PCCT scanners implemented in clinical routine have detectors of 2 mm thickness and a ~ 200 μm pixel dimension at the isocenter [54], lower than the ~ 1000 μm of energy integrating detectors [55]. While the foreseeable benefits of PCCT adoption are huge, studies comparing CACS between normal (energy integrating) scanners and PCCT showed a systematic reduction in CACS values in PCCT, leading to the reclassification of 5% of patients [56].

Datasets, model fitting, and model performances are crucial notions to understand the complexities and nuances of ML, DL, and CNNs. The training dataset is used to build the model/algorithm, whereas the validation dataset serves to tune and improve the learning performance of the algorithm and the testing dataset to evaluate the model’s performance (Fig. 2, i.e., accuracy, precision, recall, etc.), respectively [61, 62]. It is not surprising that the size of the training dataset should be tailored to the underlying task, with more complex tasks requiring larger datasets. Larger training datasets also obviate false pattern recognition due to imbalances in variables (i.e., sex, age, smoking status, etc.) used to build the algorithm [58]. Training, validation, and testing datasets need to be independent, with no overlap. Further, to increase the algorithm’s generalizability, the testing dataset should, ideally, be external [61]. These steps help reduce overfitting, which refers to an algorithm working exceptionally well on the data (images) it was trained on (training dataset) but then fails to generalize adequately to new, unseen data (testing dataset) [61].

Several performance metrics can be used to interpret the quality and accuracy of the results of AI algorithms. Accuracy evaluates the number of correct predictions within the whole dataset. Although useful, accuracy is generic and unable to disentangle the performance in different classes (i.e., sensitivity and specificity) [72]. The confusion matrix and the area under the receiver-operating curve improve the understanding of the performance of the classifier [72], while the F1-score evaluates the harmonic average between recall and precision rates [66]. Jaccard index and DICE similarity index are commonly used to grade image segmentation, measuring the degree of proximity between two segmentations on a pixel-wise analysis [72, 73]. Their values always lie between 0 and 1, with the two extremes representing the lack of or the exact correspondence between the segmentation generated by the algorithm and the ground truth [73]. Finally, signal-to-noise ratio (SNR), CNR, modulation transfer function, and noise power spectrum are commonly used to quantify the image quality. The mathematical formulas used to calculate these metrics are reported in Table 3.

CAC detection and segmentation strategies rely either on the intrinsic high density of calcified plaques or on the anatomic location of coronary arteries (Fig. 3) [74]. While these approaches seem robust, the presence of high-density cardiac (i.e., aortic and valvular calcification) and non-cardiac (i.e., lymph nodes, metal structures, noise, etc.) mediastinal structures, as well as the low coronary arteries-to-myocardium contrast difference on non-contrast images, are inherent difficulties encountered in automatic CAC segmentation algorithms creation [75, 76]. To date, the following approaches have been used in ECG-gated images:

In non-gated CT images, the cardiac motion artifacts preclude the possibility of applying segmentation-based approaches [74]. Lessemann et al. overcame this shortcoming by proposing a two-stage CNN approach [74]. The first CNN’s large receptive field applied image thresholding, categorizing all voxels exceeding 130 HU and subdividing them according to the presumed coronary artery they belonged to, while the second CNN’s smaller receptive field refined the results of the previous subdividing authentic calcification from other high-density structures (false positive voxels) [74].

Fully automated CACS algorithms offered high performances on both ECG-gated and non-gated scans. In ECG-gated images, the mean difference between manually and AI-calculated Agatston scores was − 2.86 to 3.24 [83] in older algorithms and 0.0 to 1.3 in more recent ones [84]. Additionally, experts-to-AI intraclass correlation coefficient (ICC) values ranging from 0.84 to 0.99 were reported by several authors [84‐88], while the expert-to-expert ones were 0.84, 0.85, and 0.77 [84]. Interestingly, the availability of additional datasets rather than adopting multiple models (in the same dataset) improved the AI-based accuracy [89]. While these results already proved the potential of AI, the adoption of U-net++, an ameliorated version of U-net [90], reduced the CACS error from 5.5 to 0.48, indicating a bright future for this application [91]. On non-gated images, ICC values of 0.99 and 0.90 were reported using chest CT scans [92] and low-dose chest CT scans acquired for LC screening [93], respectively. Similarly, AI and expert CACS evaluation had a good correlation on Bland-Altmann plots using non-gated chest scans [94]. However, the agreement of risk categorization between AI and expert evaluation varied according to the test dataset used, being between k = 0.58 and k = 0.80 on routine chest CT using external datasets [83] and between k = 0.85 [93] and k = 0.91 [74] on low-dose chest CT using internal ones. Similarly, the agreement between risk categories based on ECG-gated and non-gated images ranked from k = 0.52 to 0.82, with lower values obtained with external datasets [83, 95]. This proves the important connection between the algorithm’s performance and the dataset-specific characteristics (i.e., scanner, field-of-view, reconstruction filter, slice thickness, etc.) in non-gated images, resulting in a rapid performance degradation varying the input data [96]. In the study by Lessmann et al., the overall sensitivity of the algorithm, trained on soft reconstruction kernels only, decreased from 91 to 54% when applied to images reconstructed with a sharp kernel [74]. To overcome these issues, van Velzen et al. retrained their algorithm with a small additional set of representative data-specific examinations [96]. Supplementing the network with data-specific scans generated narrower confidence intervals (CIs) on Bland-Altman plots, indicating an improved correlation between the Agatston values calculated by the software and the reference standard [96]. Further, the combined reliability of risk category assignment improved from k = 0.90 to 0.91 [96]. However, the amount of supplementary data-specific scans needed to ensure optimal performance is unknown. Besides these drawbacks, a recent study on 5,678 adults without known ASCVD transitioned AI usage from research into a clinical scenario by utilizing a DL-based algorithm to analyze CACS on non-gated images, showing that adults with DL-CACS ≥ 100 had an increased risk of death (adjusted hazard ratio: 1.51; 95% CI: 1.28 to 1.79) compared to those with DL-CACS 0 [97]. A fascinating, additional perspective was provided by a cloud-based DL CACS evaluator showing high ICC value (0.88; 95% CI: 0.83 to 0.92) between ECG-gated and non-gated images derived from 18 F-fluorodeoxyglucose positron emission tomography (PET) [95]. Although these results were not replicated in a similar setting [98], cloud-based tools have the potential to broaden the users of AI-based CACS evaluation beyond university and tertiary hospitals, helping to reach its full potential.

Total CACS is currently used by international guidelines to guide therapeutic decisions; however, it is worth noting that CAC is unevenly distributed, mostly located in the LAD or the right coronary artery [41]. However, as previously discussed, heavily calcified left main trunk or LAD correlates with a higher mortality risk [41, 42]. Therefore, detailing algorithm performances on a vessel-based level is of uttermost importance but often overlooked or done by using dissimilar metrics, as described in Table 4.

Algorithm architecture, use of graphic processing unit, and the number of cores of the computer processing unit strongly impacted the computational time taken to quantify CACS, generating heterogeneous results (mean computational time: 3 min, range: 2 s to 10 min, Fig. 4) [74, 81, 83‐86, 96, 98‐100]. Irrespective of the computational time, these results show that automatizing CACS calculation may reduce its costs and streamline the workflow of imaging departments.

The growing awareness of the potential risks associated with ionizing radiation usage forced CT manufacturers and the medical community to adopt low-dose protocols [101]. Low-dose images inherently exhibit higher image noise and artifacts than standard-dose images. Hence, in the last five years, vendors have introduced deep learning reconstruction (DLR) to address these problems [102, 103]. To date, three DLR methods have been developed: TrueFidelity, AiCE, and Precise Image. These algorithms differ based on the input data used in the training phase, the framework they were built on, and the ground truths adopted to compare their performances [102]. Inputs included raw data (sinograms), filtered back projections (FBP), or iterative reconstruction IR images (specifically model-based iterative reconstruction), while frameworks distinguished between direct and indirect. Direct frameworks generated DLR-optimized images in a single step by applying DL algorithms to sinograms (TrueFidelity and Precise Image). Indirect frameworks either generate DL-optimized sinograms before reconstructing images or optimize reconstructed images using DL algorithms (AiCE) [102]. A more comprehensive and detailed description of the strategies used to create DLR can be found in Koetzier et al. [102].

The use of DLR led to a reduction in image noise and a concomitant increase in SNR and CNR compared to FBP [104‐106]. However, a phantom study using standard acquisition parameters showed that, compared with other image reconstruction techniques, DLR failed to detect calcifications ≤ 1.2 mm [107]. While most studies showed Agatston scores being comparable throughout different reconstruction techniques, its values were reduced with increasing DLR strengths compared to FBP [104‐106]. This translated into a downward reclassification of risk scores in 2 to 8% of patients [104‐107]. The sole study comparing 3 mm FBP-reconstructed ECG-gated images to 1 mm, low-dose, non-gated DLR-reconstructed images showed that the latter underestimated the CACS (94 ± 249 vs. 105 ± 249) and had 90% accuracy in classifying different risk classes (Fig. 5) [108]. These results prove that DLRs are a promising tool. However, their intrinsic tendency to down-quantify the Agatston score may profoundly impact treatment strategies. Hence, their implementation in CACS evaluation needs additional studies or correction factors.

Although CACS images rely on a limited field-of-view, a holistic AI algorithm aiming to automatize CACS reporting should not overlook possible extracardiac findings. A systematic review including more than 11,000 patients undergoing cardiac CT showed that 41% of them had extracardiac findings (ECFs) with an average prevalence of clinically significant ECFs of 16% [109]. Although these results may not be entirely transferable to patients undergoing CACS, they highlight the need for additional automated AI-based evaluation of ECFs. Suspicious lung nodules, hiatal hernia, emphysema, enlarged lymph nodes, and pleural effusion accounted for 63% of clinically significant ECFs [109] and are readily diagnosable on non-enhanced CT images. The sole study exploring the automated detection of CAC and solid lung nodules on low-dose CT images had ambiguous results [110]. The AI-to-expert agreement was excellent in discriminating between patients with CACS 0 and those with CACS > 0 (k = 0.85), whereas the performance of nodule detection was suboptimal (k = 0.42) [110]. These results prove that further research is needed to improve the performance of AI algorithms in handling multiple inputs.