Deep learning segmentation results in precise delineation of the putamen in multiple system atrophy

This retrospective bicentric study included patients who underwent MRI for the differential diagnosis of neurodegenerative Parkinson syndromes or the evaluation of advanced treatment options between 01/2018 and 12/2021. Patients in whom MSA or PD was either clinically established or probable were recruited. Clinical diagnoses were validated by two experts in movement disorders (M.R., board-certified neurologist; W.H.J., board-certified neurologist; N.S., > 7 years of neurology training) based on current diagnostic criteria; all available medical records were used for the best clinical lifetime diagnosis [17, 18].

We included age and sex-matched healthy controls (HC) as a comparison group. Subjects in the HC group had no known neurological medical condition, no neurological deficits on clinical examination, and no family history of neurodegenerative Parkinson syndromes. Furthermore, this group of subjects did not show neuropsychological impairment according to the Montreal Cognitive Assessment [19].

The study was approved by the Institutional Review Board (Ethics Committee – University of Freiburg; EK 22/1270) and carried out in accordance with the Declaration of Helsinki and its later amendments. Due to the retrospective nature of this study, the need for written informed consent was waived.

MRI was performed with a 3-Tesla scanner (MAGNETOM Prisma, Siemens Healthcare) with a 64-channel head and neck coil. Patients with Parkinson syndromes were in the ON-state during acquisition. T1‐weighted (T1w) images were acquired with a three‐dimensional (3D) magnetization‐prepared 180° radio frequency pulses and rapid gradient‐echo (MPRAGE) sequence (repetition time: 2500 ms, echo time: 2.82 ms, flip angle: 7°, TI = 1100 ms, GRAPPA factor = 2, 1.0 mm³ isotropic voxels, 192 contiguous sagittal slices). In addition, 3D T2w FLAIR and 2D SWI data were obtained as given in Supplementary Table 1.

All data processing was implemented within our in-house post-processing platform (NORA; www.​nora-imaging.​com). T1w imaging datasets were automatically segmented into white and gray matter and spatially normalized using CAT12 (http://​www.​neuro.​uni-jena.​de/​cat/​). From this, the AAL3 atlas was employed to create the ROI of the putamen [20].

We obtained an automatic segmentation of the putamen from the FreeSurfer toolbox [21] Version 6.0 and employed the recently introduced Fastsurfer approach [10] after parcellation according to the Desikan-Killiany atlas (DKT) [22].

Left and right putamen were manually segmented on 3D reformatted T1w MPRAGE images under simultaneous consideration of the 3D FLAIR and SWI by an experienced neuroradiologist (A.R., 5 years of experience in neuroimaging) blinded to clinical diagnosis as depicted in Fig. 1 to serve as ground truth for further processing and analyses. Primary segmentation of the putaminal outline was manually done on axial slices and correction of the outline was performed on coronal and sagittal reformats. This manual segmentation served as ground truth for further DNP training and testing of the different methods. The data leakage was omitted due to the patient-level separation of training and test data. A single case per class was randomly drawn from the training sample for monitoring the training process as validation cases. DNP was used as a segmentation framework that is based on hierarchical and nested stacking of patch-based 3D networks of fixed matrix size but decreasing physical input size. The size of the hierarchical pyramid was selected such that it allowed for a reasonable 3D field-of-view of each dimension of 150 mm in the coarsest layer and a very high spatial resolution in the smallest layer with an isotropic resolution of 1 mm. The matrix size of 32³ voxels was selected in a way that would map representative portions of the anatomy. These settings were limited by the available hardware capacity for sample size and pyramid size. The architecture of the basis U-Net applied in this project is close to the default U-Net configuration with feature dimensions (8, 16, 16, 32, 64) and maximum pooling in the encoding layers and transposed convolutions in the decoding layers. The network is trained with the Adam optimizer with a learning rate of 0.001. As a loss function, a binary cross-entropy variant of the top-K loss was used. The Patchwork CNN was trained for 5 million patches. The training took around 20 h with a batch size of 150 images in the graphic unit ‘s memory. Training was performed on a GPU-accelerated server system using an RTX A6000 graphic unit (NVIDIA). During training, patches were randomly sampled so that approximately 80% of the finest patches contained at least one label. No systematic tuning was done with the settings adapted to prior established values [16]. To assess the performance of the hierarchical structure of the DNP, we trained a more classical single-level 3D U-Net with a matrix size of 128³, consequently reducing the spatial resolution to 2×2×1.5 mm [23]. The batch size had to be limited to 10 samples due to the high hardware demands of such a large 3D U-Net.

To create the results in the separate test dataset, a random patching scheme was used, combined with a tree-like branching sampling method. In detail, six nested patches were randomly drawn from the coarsest level, and the three patches with the highest probability of containing a label were selected for further analysis at the next nested level. This process was repeated overall levels, allowing for efficient coverage of relevant image parts. The prediction of a full volume was performed on a 16-core CPU machine (without a GPU), which, corresponding to the used image volume, took around 2 min.

Statistical analyses were performed using R (version 4.1.0, https://​www.​R-project.​org/​). Data are presented as the mean and standard deviation for continuous variables and as absolute frequencies and percentages for categorical variables. The Shapiro–Wilk test was used to assess the normal distribution of data. Group differences were assessed by the chi-squared test and Mann–Whitney-U test. Intergroup differences were assessed through analysis of variance (ANOVA), followed by Tukey’s honest significance test. The performance of the AAL3 atlas, the Freesurfer- and Fastsurfer-approach, and DNP in comparison to the human labeling was assessed via the dice coefficient and the 95% Hausdorff distance. We plotted the ROC curves of the putaminal volume for the differential diagnosis of MSA vs. PD and HC within the test cohort and the AUC and cut-off values were calculated. DeLong’s test was employed to compare the AUCs. We additionally assessed the respective algorithm´s performance in Bland–Altman plots. The significance threshold was set to p < 0.05.

We report on 32 HC, 61 patients with MSA, and 158 patients with PD. There were no group differences in terms of age (p = 0.49) between the HC, MSA, or PD groups, whereas more male patients were included in the PD group than in the HC or MSA group (p < 0.001). The disease duration in the PD group was significantly longer than in the MSA group (p < 0.001). Further patient characteristics are provided in Table 1.

Prior to DNP training, we randomly split our cohort into a training set (N = 131: 17HC, 30MSA, 84PD) and a test set (N = 120: 15HC, 31MSA, 74PD) that did not differ in terms of age and sex (p > 0.05).

Human ground truth labeling resulted in significantly lower putaminal volume in MSA (2.62 ± 1.03 mL) compared to HC (4.61 ± 0.54 mL; p < 0.001) and PD (4.34 ± 0.53 mL; p < 0.001), while no difference between HC and PD was found (p = 0.36). We successfully obtained putaminal segmentations from all employed algorithms, i.e. AAL3, Freesurfer, Fastsurfer, and DNP as depicted in Fig. 2, and observed no side differences within the respective algorithm outputs (all p > 0.05, please see Supplementary Fig. 1).

All algorithms revealed significantly lower putaminal volume in MSA in comparison to HC and PD, too, as given in Fig. 3. Regarding the mean putaminal volume within all groups, compared to human ground truth labeling (3.93 ± 1.04 mL), the AAL3 output resulted in a structurally larger volume (6.17 ± 1.04 mL; p < 0.001), whereas Freesurfer and Fastsurfer resulted in structurally smaller volumes (2.81 ± 0.85 mL and 2.98 ± 0.94, respectively; both p < 0.001). Good agreement was found between the ground truth and the DNP (3.88 ± 0.96 mL; p = 0.99).

This is corroborated by a significantly lower dice coefficient of 0.72 ± 0.08 for AAL3 (p < 0.001), 0.82 ± 0.06 for Freesurfer (p < 0.001), and 0.84 ± 0.05 for Fastsurfer (p < 0.001), compared to 0.96 ± 0.02 for the DNP as well as better 95% Hausdorff distance of the DNP (0.89 ± 0.30) compared to AAL3 (3.76 ± 1.28; p < 0.001), Freesurfer (2.79 ± 1.16; p < 0.001), and Fastsurfer (2.76 ± 1.16; p < 0.001). The more classical single-level 3D-Unet also performed inferior to the DNP, as it reached dice-coefficients of 0.90 ± 0.04 (p < 0.001).

A more detailed analysis after the separation of the diagnoses revealed that AAL3, Freesurfer, Fastsurfer, and the DNP had lower dice scores in MSA compared to HC or PD (all p < 0.001). More details are presented in Table 2. The dice coefficient results are corroborated by the Bland–Altman plots (please see Supplementary Fig. 1).

To access the diagnostic value of the respective segmentations, we calculated the AUC-ROC for the comparison of MSA vs. PD and HC. The AAL3-derived segmentation revealed an AUC of 0.88 (95% CI 0.81–0.94; cut point 5.51 mL), 0.86 for Freesurfer (95% CI 0.79–0.92; cut point 2.33 mL), 0.85 for Fastsurfer (95% CI 0.78–0.91; cut point 2.61 mL), and 0.93 for the DNP (95% CI 0.87–0.97; cut point 3.77 mL), whereas the human ground truth labeling’s AUC was 0.93 (95% CI 0.87–0.97; cut point 3.57 mL). DeLong’s test revealed that the AUC of the DNP was superior to AAL3 (p = 0.02), Freesurfer (p = 0.048), and Fastsurfer (p = 0.04).