Next Article in Journal
Controlled Synthesis of Up-Conversion NaYF4:Yb,Tm Nanoparticles for Drug Release under Near IR-Light Therapy
Previous Article in Journal
Improving Generation of Cardiac Organoids from Human Pluripotent Stem Cells Using the Aurora Kinase Inhibitor ZM447439
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Biomedicines
Volume 9
Issue 12
10.3390/biomedicines9121951
biomedicines-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Systematic Review

The Effect of a Multimodal Occupational Therapy Program with Cognition-Oriented Approach on Cognitive Function and Activities of Daily Living in Patients with Alzheimer’s Disease: A Systematic Review and Meta-Analysis of Randomized Controlled Trials

by
Min-Joo Ham
1,†,
Sujin Kim
2,3,†,
Ye-Ji Jo
1,
Chisoo Park
1,
Yunkwon Nam
2,
Doo-Han Yoo
1,3,* and
Minho Moon
2,3,*
1
Department of Occupational Therapy, Konyang University, 158, Gwanjeodong-ro, Seo-gu, Daejeon 35365, Korea
2
Department of Biochemistry, College of Medicine, Konyang University, 158, Gwanjeodong-ro, Seo-gu, Daejeon 35365, Korea
3
Research Institute for Dementia Science, Konyang University, Daejeon 35365, Korea
*
Authors to whom correspondence should be addressed.
†
These authors contributed equally to this work.
Biomedicines 2021, 9(12), 1951; https://doi.org/10.3390/biomedicines9121951
Submission received: 26 November 2021 / Revised: 10 December 2021 / Accepted: 17 December 2021 / Published: 20 December 2021
(This article belongs to the Section Neurobiology and Clinical Neuroscience)
Browse Figures
Versions Notes

Abstract

:
Non-pharmacological intervention, which includes a broad range of approaches, may be an alternative treatment for Alzheimer’s disease (AD). Multimodal non-pharmacological intervention alleviates cognitive dysfunction and the impairment of activities of daily living (ADL) in AD patients. However, it is still unclear which combination of non-pharmacological interventions is preferred. We selected a non-pharmacological intervention combined with occupational therapy (OT). We investigated the effect of a multimodal OT program with cognition-oriented approach on cognitive dysfunction and impairments of ADL in patients with AD. Four electronic databases were searched from January 2000 to August 2020. The studies were assessed for heterogeneity, quality assessment, effect size and publication bias. A total of seven randomized controlled trials examining multimodal OT programs with cognition-oriented approach in AD patients were included in the meta-analysis. Compared with the control group, the multimodal OT program with cognition-oriented approach group was statistically beneficial for cognitive dysfunction (95% CI: 0.25–0.91). However, compared with the control group, the multimodal OT program with cognition-oriented approach group tended to be beneficial for basic ADL, and instrumental ADL. These results suggest that the multimodal OT program with cognition-oriented approach might be the optimal multimodal non-pharmacological intervention for improving cognitive dysfunction in AD patients.

1. Introduction

Dementia is defined as a state in which daily living is severely disrupted by a combination of symptoms, such as memory impairment, aphasia, apraxia, agnosia, or impaired executive function caused by an acquired brain disease [1]. Alzheimer’s disease (AD) accounts for 55–70% of all dementia cases and is clinically characterized by gradually worsening severe memory loss that causes impairment of daily living [2]. Treatment approaches for patients with AD are divided into pharmacological and non-pharmacological approaches. Interestingly, according to the DSM-5 [3], ADL is mentioned as an important diagnostic criterion along with cognitive factors in the definition of dementia. Research has been expanding on non-pharmacological treatment as a strategy to improve the performance of activities of daily living (ADL) and maintain cognitive function in patients with AD [4,5]. Indeed, the number of studies on non-pharmacological interventions has rapidly increased in the last 15 years [6].
Non-pharmacological interventions include occupational therapy (OT), reminiscence therapy, reality orientation therapy, validation therapy, sensory stimulation, multisensory stimulation, art therapy, music therapy, aromatherapy, light therapy, physical activity, exercise, cognitive stimulation, cognitive training, and cognitive rehabilitation. In addition, there are four approaches to non-pharmacological interventions: (1) cognition-oriented, (2) emotion-oriented, (3) behavior-oriented, and (4) stimulation-oriented approaches. The cognition-oriented approach aims to improve cognitive dysfunction, the most prevalent symptom of AD. In particular, cognition-oriented approaches are widely used in combination with other non-pharmacological interventions, such as OT, to optimize cognitive function for ADL improvement [7,8,9,10]. Moreover, risk factors for AD include aging, cardiovascular disease, and diabetes, as well as socio-demographical variables, psychological variables, hearing loss, and anxiety [11]. Therefore, for AD treatment, not only treating the disease, but also non-drug interventions that can affect many aspects with a variety of approaches can be helpful.
Several studies have suggested the potential of OT in the treatment of AD [12,13,14,15]. OT seeks not only simply to treat the disease, but also to understand and respect patients with AD as human beings by applying a ‘social-psychological model’ [15,16]. These aspects are clearly different from pharmacological approaches or other non-pharmacological interventions. In particular, numerous studies show the beneficial effect of OT intervention on dementia patients [17,18,19]. Moreover, the meta-analysis about OT interventions based on sensory stimulation on dementia patients reveals the improvement of behavioral disorders of dementia patients [18]. Moreover, OT intervention to maintain body function was showed to improve the ADL ability of dementia patients [20]. One of the studies reported that OT-including cognitive interventions improved the abilities of AD patients to perform ADL [21]. Interestingly, multimodal OT interventions are used to alleviate occupational imbalances, which is the loss of balance while engaging in daily activities caused by dementia [4]. Moreover, in previous studies, multimodal OT programs with cognition-oriented approaches have been reported to have positive effects on cognitive function and ADL in patients with dementia [8,22]. Therefore, OT is strongly encouraged in the treatment of patients with mild-to-moderate dementia [23,24].
Previous studies have demonstrated the suitability and effectiveness of the multimodal OT program with cognition-oriented approach in patients with dementia [8,22]. However, to the best of our knowledge, no study has been conducted meta-analysis regarding the effectiveness of multimodal OT program with cognition-oriented approach. In this study, we performed a systematic literature review and meta-analysis to investigate whether the multimodal OT program with a cognition-oriented approach has any effects on cognitive dysfunction and ADL impairment, which are considered major problems for patients with AD. Our study could provide clinicians in the field of dementia with evidence of a multimodal OT program with a cognition-oriented approach.

2. Materials and Methods

2.1. Search Strategy to Identify Relevant Studies and Criteria for Inclusion

We conducted a search for studies using a multimodal OT program with cognition-oriented approach in patients with AD. Studies with publication dates between January 2000 and August 2020 were retrieved from the MEDLINE, CINAHL, PubMed, and Academic Search Complete databases. The main search terms were as follows: Alzheimer, Dementia, Intervention, Cognitive stimulation, Cognitive training, Cognitive rehabilitation, Multimodal, Multidisciplinary, Multistimulation, Occupational therapy, Cognitive function, Activities of daily living, and Randomized controlled trial.
The inclusion criteria were as follows: (1) studies on patients with AD; (2) studies on multimodal OT program with cognition-oriented approach; (3) studies measuring changes in overall cognitive function or ADL; (4) randomized controlled trials (RCTs); (5) studies published in English; and (6) studies that indicated the sizes of the intervention and control groups, the means and standard deviations, or the standardized mean difference (SMD) scores. The exclusion criteria were as follows: (1) graduation theses, (2) case studies, (3) literature reviews, (4) pilot studies that did not lead to the main studies, and (5) studies result on caregivers were excluded. The criteria for multimodal OT program with cognition-oriented approach studies as follows: (1) studies that applied cognitive stimulation, cognitive training, and cognitive rehabilitation interventions simultaneously, clearly including OT, and (2) studies involving occupational therapists.

2.2. Data Extraction

Based on the Preferred Reporting Items for Systematic reviews and Meta-Analysis (PRISMA) [25], we have diagrammed the step-by-step process of study selection in a flowchart (Figure 1) (This review was not registered in PRISMA). A total of 2149 studies were retrieved from the initial literature search, and 155 duplicate studies were excluded. Of the remaining 1994 studies, 1779 studies were excluded based on the inclusion/exclusion criteria by reviewing the titles and abstracts, and 131 studies were excluded because they did not include patients with AD. The full texts of the remaining 84 studies were inspected, and 47 studies were excluded because they did not assess cognition or ADL. Finally, studies that were non-RCTs that did not thoroughly describe the characteristics of OT, or that did not show means or SMD were excluded, and the remaining seven studies were selected for the final analysis.
During data selection and review, two researchers performed the literature search in independent locations and completed the prepared results. If there was any disagreement between the researchers during this process, a consensus result was obtained by a research meeting in which the researchers reviewed the main text together.

2.3. Evidence Table Construction

In this study, we classified the characteristics of the seven selected RCTs based on the PICOTS (patient, intervention, comparison, outcome, time, and setting/study design) criterion. The results of patients with AD were completed in the following order: sex and age, sample sizes, intervention methods, number of treatment sessions, location of treatment sessions, and instruments used to assess outcomes (Table 1).
ADAS-Cog, Alzheimer’s Disease Assessment Scale-Cognitive subscale; ADLs, Activities of daily living; BADL, Basic activities of daily living; CG, Control group; EG, Experimental group; COPM, Canadian occupational performance measure; CR, Cognitive rehabilitation; DSM-4, Diagnostic and Statistical Manual of Mental Disorders; ICD-10: International Classification of Diseases; IADL, Instrumental activities of daily living; ILS, Independent living scale; K-MMSE, Korean version of mini mental status examination; MBI, Modified Barthel index; MCP, Multidomain cognitive program; MIP, Multicomponent intervention program; MMSE, Mini mental state examination; MRI, Multimodal rehabilitative intervention; MRP, Multidisciplinary rehabilitation program; NINCDS-ADRDA: National Institute of Neurological and Communicative Disorders and Stroke; N/A, Not assessment; N/I, Not intervention; OT, Occupational therapy; RDRS-2, Rapid Disability Rating Scale-2

2.4. Assessing the Quality of Research

Cochranés risk of bias (ROB) is an instrument used to evaluate potential bias that can occur in the design of RCTs. The ROB assesses seven domains: random sequence generation, allocation concealment, blinding of participants and researchers, blinding of outcome assessors, incomplete outcome data, selective outcome reporting, and other potential sources of bias that threaten the validity [32]. The results of quality assessment can be classified according to the Scottish Intercollegiate Guidelines Network levels of evidence [33]. In this study, the quality assessment was performed independently by two researchers, and when the results differed, the final quality grade was determined through discussion.

2.5. Statistical Analysis

For meta-analysis, we performed data coding and analysis using R version 4.1.2 meta and metafor package [34] and RevMan 5.3, provided by the Cochrane collaboration. We performed a meta-analysis using the change value of the mean score following interventions, the change value in the SMD following the interventions, and the group size for the test and control groups in each study. If the study provided the change value of the mean or SMD following the interventions, the raw data were used without any changes. If the change value was not provided, the change was calculated from the final value [35].

2.5.1. Statistical Heterogeneity

For statistical heterogeneity, Cochran’s Q-test (chi-square test) and Higgins I2-statistic were used to verify the homogeneity of the groups with regard to effect size in the results of each study. When the Q-test’s p-value is ≥0.1, a fixed-effects model can be used, but when it is <0.1, a random-effects model can be used [35]. For the Higgins I2-statistic, I2 < 50% indicates that a fixed effects model can be used, while I2 > 50% indicates that a random-effects model can be used [36].

2.5.2. Calculation of Effect Size

The effect size was calculated through the comparison of dependent variables between the test group and the control group of the seven RCTs studies selected, and the change in mean and SMD and the number of participants were used according to Cohen’s interpretation criteria [37]. In the interpretation of the overall effect size, ≤0.2 is considered a small effect, around 0.5 is considered a moderate effect, and ≥0.8, is considered a large effect [37]. The confidence interval (CI) was set at 95% of SMD.

2.5.3. Publication Bias

We constructed a contour-enhanced funnel plot using the metafor package for meta-analyses in R [34] to statistically test for the risk of publication bias. A contour-enhanced funnel plot suppresses non-significant results and is known to be more effective at detecting publication bias [38]. In accordance with Sterne and Egger [39], the vertical axis was labeled as the standard error, and the horizontal axis as the observed outcome. Additionally, the statistical significance of publication bias was calculated using fail-safe N statistics.

3. Results

3.1. Included Studies Characteristics

3.1.1. Diagnostic Criteria and Research Design

The seven studies selected involved participants with AD, and all studies had an RCTs design. All subjects (100%) were patients with AD. The six studies used diagnostic criteria of NINCDS-ADRDA 4 (57%), DSM-4 1 (14%), and ICD-10 1 (14%). One study did not support the detailed information on the diagnostic criteria.

3.1.2. Demographic of Study Participants

A total of 453 participants were included in the meta-analysis. The countries where the study was conducted were Korea (28%), Italy (14%), Germany (14%), Brazil (14%), and Spain (14%). The mean age was ≥70 years and <80 years in six studies, and ≥60 years and <70 years in one study. Six studies presented the mean age separately for the test and control groups, and one study showed only the overall mean age.

3.1.3. Intervention Type

Five studies applied an OT-based combination intervention program, but also included other treatment modalities, and two studies implemented a combination intervention program using only OT. The primary outcomes were cognitive function and ADL, with six studies including cognitive function, four studies including BADL, and three studies including IADL. Other therapeutic characteristics regarding the times or settings are described in detail in Table 1.

3.2. ROB Quality Assessment

In the Cochrane ROB analysis, there were three studies that showed a moderate ROB and four studies that showed a high ROB. Among the subdomains, random sequence generation was used in all studies, but two studies did not clearly describe their methods of random sequence generation. Second, three studies did not use allocation concealment, and in one study, the concealment method was unclear. Third, in terms of blinding of participants and researchers, there was one study that performed blinding, five studies did not discuss this matter, and one study did not perform blinding. Fourth, although most studies performed blinding of outcome assessors, two studies did not clearly report theirs. Fifth, in terms of incomplete outcome data, four studies were judged to have a low ROB due to missing or insufficient measurements, and three studies were judged to have a high ROB due to differences in the number of assessed participants in the test and control groups. Sixth, in terms of selective outcome, only one study was judged to have a high ROB due to insufficient information about the planned intervention process. Seventh, all the studies were judged to have a low risk of other sources of bias, threatening the validity of the study (Figure 2).

3.3. Meta-Analysis for the Effect of Multimodal OT Program with Cognition-Oriented Approach

3.3.1. Meta-Analysis of the Effect of a Multimodal OT Program with the Cognition-Oriented Approach on Cognitive Decline in Patients with AD

Across six studies, 211 participants estimated the effect on cognitive decline in patients with AD. The heterogeneity for the overall effect size was I2 = 53% (p = 0.06), and a random-effects model was used. The overall effect of multimodal OT program with cognition-oriented approach on cognitive dysfunction was moderate and statistically significant, 0.58 (95% CI: 0.25–0.91). In addition, the effect size was positive (+) in five of the six studies. In summary, compared to controls, the multimodal OT program with the cognition-oriented approach showed significant improvement in cognitive dysfunction in patients with AD (Figure 3).

3.3.2. Meta-Analysis of the Effect of a Multimodal OT Program with Cognition-Oriented Approach on BADL in Patients with AD

Four studies that reported 133 participants analyzed the effect on BADL in patients with AD. The heterogeneity for the overall effect size indicated I2 = 81% (p < 0.01), and a random-effects model was used. The effect size of multimodal OT program with cognition-oriented approach on BADL was moderate but not statistically significant, 0.76 (95% CI: −0.24–1.76). The effect size was positive in three of the four studies. However, one study did not show a positive effect. In total, compared to controls, the multimodal OT program with cognition-oriented approach group revealed a trend toward improving the impairment of BADL in patients with AD (Figure 4).

3.3.3. Meta-Analysis of the Effect of a Multimodal OT Program with Cognition-Oriented Approach on IADL in Patients with AD

Three studies that investigated 51 participants were available for IADL outcomes. The heterogeneity of the overall effect size was I2 = 81% (p < 0.01), and a random-effects model was used. The effect size of multimodal OT program with cognition-oriented approach on IADL was moderate but not statistically significant, 0.46 (95% CI: −0.37–1.29). The effect size was positive in two of the three studies. However, one study showed a negative effect. Collectively, compared to controls, the multimodal OT program with cognition-oriented approach group showed trends for maintenance, improvement, or degradation of IADL in patients with AD (Figure 5).

3.4. Publication Bias

We conducted a publication bias analysis to test the validity of all the study results (Figure 6). Since there were fewer than 10 studies included in our sample, we used a contour-enhanced funnel plot to visually inspect the left-right symmetry of the distribution of effect estimates. The aim of the method is to visually investigate the presence of small-study effects and to determine whether there is a relationship between precision (vertical axis) and effect size (horizontal axis) among the individual studies included in the meta-analysis [40]. In the assessment of cognitive function, six studies showed a symmetrical distribution. In addition, for BADL, three studies showed a relatively symmetrical distribution, but one study was located in the missing data region. For IADL, the three studies showed asymmetrical distribution. As a result of fail-safe N statistical analysis, publication bias was not observed as 94 in the case of cognitive function. Unfortunately, it was confirmed that publication bias exists with the safety coefficients (fail-safe N) of BADL and IADL of 30 and 25.

4. Discussion

In this study, we examined the effect of a multimodal OT program with cognition-oriented approach on cognitive dysfunction and impairment of ADL in patients with AD using meta-analysis. Seven RCT studies analyzed the effect size and heterogeneity of cognitive decline and impairment of BADL and IADL. In addition, to improve the reliability of the results, we assessed the quality of the selected studies and tested for publication bias. Cochrane’s ROB results suggested that selected studies have a relatively high level of quality, and the intervention effect can be very close to the actual intervention effect [41] (Figure 2). Furthermore, our study results showed that the multimodal OT program with cognition-oriented approach has beneficial effects on cognitive dysfunction and impairment of BADL and IADL in patients with AD (Figure 3, Figure 4 and Figure 5). These results may provide occupational therapists and dementia-related professionals with clinical evidence to support the use of non-pharmacological interventions.
Some previous studies have reported that combining two or more non-pharmacological interventions is more likely to slow cognitive decline more effectively than a non-pharmacological intervention alone [42]. Among the various combinations of multimodal non-pharmacological interventions, we investigated the effect of a multimodal OT program with cognition-oriented approach that seeks a patient-centered approach. The results of the meta-analysis demonstrated that the multimodal OT program with cognition-oriented approach had a significant effect on the improvement of cognitive dysfunction in patients with AD (Figure 3). Surprisingly, our results are consistent with a previous study that showed that multimodal intervention combined with OT, such as art therapy, language therapy, physical training, and computer-based cognitive training, for mild AD patients at day-hospital facilities which help to improve cognitive dysfunction [43]. Furthermore, although there was no statistical difference, the results of the meta-analysis indicated that the multimodal OT program with cognition-oriented approach tended to improve BADL disability in patients with AD (Figure 4). Interestingly, we found that the studies that provided the interventions one to three times for 16 weeks showed a greater improvement in BADL than studies that provided the interventions once per week for 8 weeks. In rehabilitation therapy for patients with neurodegenerative diseases, repetition training can be considered the most important factor for learning and recovery of skills [44]. Thus, we found that increasing the duration and frequency of the multimodal OT program with cognition-oriented approach helped to positively influence the BADL of patients with AD. Unfortunately, we did not find robust evidence that the multimodal OT program with cognition-oriented approach improved IADL disability (Figure 5), nor did we identify enough studies reporting on the outcomes of IADL to synthesize the results in a meta-analysis. We speculated that the various assessment tools used to measure IADL influenced the statistics. Moreover, these results have concerns that a small sample size, short treatment period, various treatment environments, and low sensitivity of evaluation tools may appear as methodological problems due to the nature of non-pharmacological intervention studies. To supplement these limitations in future meta-analyses, it will be necessary to focus on studies using the same assessment instruments, or to include additional related studies to increase the number of studies analyzed.
The reduction of various experiences and engagement, such as social networks, mental activity, and exercise, due to aging and disease, is associated with cognitive decline and the onset or progression of AD [45,46]. The increase in experience, engagement, and various stimuli through non-pharmacological intervention provides an opportunity for the brain to efficiently utilize and recruit existing neural networks and alternative neural networks [47,48,49]. One of the possible mechanisms by which non-pharmacological interventions, including OT, may affect AD could be related to cognitive reserve. In particular, cognitive reserve is the one of the important concepts of brain changes in neurodegenerative diseases, such as AD [50]. It is well known that in the AD brain reduces the number of neural precursor cells and decreases the generation of new neurons, causing a deficiency in hippocampal plasticity and cognitive reserve [51,52]. Interestingly, an increase in experience through non-pharmacological stimulus enhanced the rate of hippocampal progenitor cell proliferation in AD transgenic mice compared with that in the control group [53]. Furthermore, an increase in experience restored spatial memory in APP transgenic mice compared to the control group [54]. Thus, we suggest that experience-induced cognitive reserve through the complexity or novelty of non-pharmacological stimuli could be affected by AD-related cognitive dysfunction.
To the best of our knowledge, this is the first systematic review and meta-analysis of RCTs investigating multimodal OT program with cognition-oriented approach in patients with AD. Our findings will be able to provide practical information about non-pharmacological interventions for dementia patients. This study is useful because we investigated clinical evidence for the effects of OT, which is a representative non-pharmacological treatment modality for patients with dementia, even among other combination cognitive interventions. OT aims to help the patient participate in valuable tasks and perform significant ADL; to this end, it improves cognitive function and ADL, and can ultimately help to improve the patient’s quality of life [55]. For this reason, when non-pharmacological interventions are provided in a clinical setting to improve quality of life in patients with dementia, a multimodal OT program with cognition-oriented approach can be suggested as an effective therapeutic strategy for patients with AD. In particular, given that there was a statistically significant effect size for cognitive function and there was no publication bias on a scale that would result in an obvious error, we believe that the findings of this meta-analysis can provide highly objective and reliable clinical information. Nevertheless, our study had several limitations. First, I2, a value representing the heterogeneity of the overall effect size, was high (≥50%). Given that statistical heterogeneity was observed, we looked for the possible source of the variance and determined whether it was appropriate to combine studies [56]. Thus, we selected the random-effects model in this study. Moreover, to reduce heterogeneity, we also considered various factors, such as dementia type, intervention methods, and dependent variables, in the study selection stage. However, due to the overall low number of studies collected, it was not possible to control the same types of assessment instruments. In future studies, it will be necessary to expand the scope of data collection to include non-English and gray literature. Moreover, if a meta-analysis that was limited in scope to a specific assessment instrument was conducted, we believe that it would be possible to reduce the problems of heterogeneity that we observed. Second, although it is possible to combine effect sizes as long as there are two or more studies, for more accurate results, it is recommended to combine at least five studies [36]. In our analysis, while there were enough studies for cognitive function, for the outcomes of BADL and IADL, we had to conduct a meta-analysis using only three to four studies, which is not sufficient. In addition, publication errors were observed in the areas of BADL and IADL. However, in the results of this study, BADL and IADL did not show statistically significant effects. Thus, we speculate that the results of BADL and IADL will vary significantly not depending on the presence or absence of publication bias. Thus, the results need to be interpreted with caution, considering not only the possibility of publication bias but also a small number of studies. Therefore, it is necessary to accurately identify the publication bias even with a small amount of literature by utilizing various methods to check the publication bias [57]. If further studies are conducted on this topic, we propose to increase the number of experimental studies examining the effects of a multimodal OT program with cognition-oriented approach on ADL disability in patients with dementia. Third, since there were few studies available to compare the changes following the multimodal OT program with cognition-oriented approach, we were not able to show the extent of changes in the dependent variables with differences in the intervention schedule, time of assessment, or the intervention environment. Fourth, although the evaluation of the quality of research using ROB is high, among the seven ROB subdomains, allocation concealment and blinding of participants and researchers showed a high or uncertain risk of bias in more than half of the studies. For this reason, although the data in this meta-analysis show a high level of evidence because of the use of RCT designs, given the ROB, careful interpretation is required for actual clinical applications. Finally, we cannot exclude the possibility that studies on this research topic might have been missed during the literature search and selection process, and thus the results of this meta-analysis should be interpreted with caution.

5. Conclusions

In this study, to investigate the effects of a multimodal OT program with cognition-oriented approach for patients with AD, which is the most common form of dementia, we performed a systematic review analyzing the characteristics of the participants and quality of research. We performed a meta-analysis to examine the effects of a multimodal OT program with cognition-oriented approach on cognitive dysfunction and ADL disability in patients with AD. The results of the meta-analysis showed a statistically significant improvement in cognitive dysfunction. In addition, although the measured values of BADL and IADL did not show statistically significant results, the results of each study showed a tendency to maintain and improve cognitive function. Based on the above results, we propose that OT can be used very effectively as part of a combination intervention to improve cognitive dysfunction in patients with AD. In future research, it will be necessary to classify OT methods in detail depending on the stage of dementia progression, and to strictly control the factors, such as the assessment instruments and the intervention duration.

Author Contributions

Conceptualization; D.-H.Y., M.M. and M.-J.H.; Data curation; M.-J.H., S.K. and Y.-J.J.; Formal analysis; Y.-J.J. and C.P.; Funding acquisition; D.-H.Y. and M.M.; Investigation; S.K. and Y.N.; Methodology; M.-J.H. and S.K.; Project administration; D.-H.Y. and M.M.; Resources; M.-J.H., Y.-J.J. and C.P.; Software; S.K. and Y.N.; Supervision; D.-H.Y. and M.M.; Validation; M.-J.H., Y.-J.J. and C.P.; Visualization; S.K. and Y.N.; Roles/Writing—original draft; M.-J.H. and S.K.; Writing—review & editing; D.-H.Y. and M.M. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported by the Basic Science Research Program of the National Research Foundation of Korea (NRF), which is funded by the Ministry of Science, ICT & Future Planning (NRF-2018R1D1A3B07041059 to M.M.; NRF-2019R1G1A1004010 to D.-H.Y.) and by the Cooperative Research Program for Agriculture Science and Technology Development (Project No. PJ01319901 and PJ01428603), Rural Development Administration, Republic of Korea.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders: DSM-IV; American Psychiatric Association: Arlington, VA, USA, 1994. [Google Scholar]
  2. Alzheimer’s Association. 2019 Alzheimer’s disease facts and figures. Alzheimer Dement. 2019, 15, 321–387. [Google Scholar] [CrossRef]
  3. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders: DSM-5; American Psychiatric Association: Arlington, VA, USA, 2013. [Google Scholar]
  4. Grasel, E.; Wiltfang, J.; Kornhuber, J. Non-drug therapies for dementia: An overview of the current situation with regard to proof of effectiveness. Dement. Geriatr. Cogn. Disord. 2003, 15, 115–125. [Google Scholar] [CrossRef]
  5. Onor, M.L.; Trevisiol, M.; Negro, C.; Signorini, A.; Saina, M.; Aguglia, E. Impact of a multimodal rehabilitative intervention on demented patients and their caregivers. Am. J. Alzheimers Dis. Other Demen. 2007, 22, 261–272. [Google Scholar] [CrossRef] [PubMed]
  6. Sopina, E.; Sorensen, J. Decision modelling of non-pharmacological interventions for individuals with dementia: A systematic review of methodologies. Health Econ. Rev. 2018, 8, 8. [Google Scholar] [CrossRef] [Green Version]
  7. Clare, L.; Woods, R.T. Cognitive training and cognitive rehabilitation for people with early-stage Alzheimer’s disease: A review. Neuropsychol. Rehabil. 2004, 14, 385–401. [Google Scholar] [CrossRef]
  8. Takeda, M.; Tanaka, T.; Okochi, M.; Kazui, H. Non-pharmacological intervention for dementia patients. Psychiatry Clin. Neurosci. 2012, 66, 1–7. [Google Scholar] [CrossRef]
  9. Huntley, J.D.; Gould, R.L.; Liu, K.; Smith, M.; Howard, R.J. Do cognitive interventions improve general cognition in dementia? A meta-analysis and meta-regression. BMJ Open 2015, 5, e005247. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  10. Grazia D’Onofrio, D.S.; Seripa, D.; Ricciardi, F.; Giuliani, F.; Panza, F.; Greco, A. Non-Pharmacological Approaches in the Treatment of Dementia; Update on Dementia; InTech Open: London, UK, 2016. [Google Scholar] [CrossRef] [Green Version]
  11. Santabarbara, J.; Bueno-Notivol, J.; Lipnicki, D.M.; de la Camara, C.; Lopez-Anton, R.; Lobo, A.; Gracia-Garcia, P. A Novel Score for Predicting Alzheimer’s Disease Risk from Late Life Psychopathological and Health Risk Factors. Int. J. Environ. Res. Public Health 2021, 18, 1802. [Google Scholar] [CrossRef]
  12. Olazaran, J.; Reisberg, B.; Clare, L.; Cruz, I.; Pena-Casanova, J.; Del Ser, T.; Woods, B.; Beck, C.; Auer, S.; Lai, C.; et al. Nonpharmacological therapies in Alzheimer’s disease: A systematic review of efficacy. Dement. Geriatr. Cogn. Disord. 2010, 30, 161–178. [Google Scholar] [CrossRef]
  13. Brodaty, H.; Arasaratnam, C. Meta-analysis of nonpharmacological interventions for neuropsychiatric symptoms of dementia. Am. J. Psychiatry 2012, 169, 946–953. [Google Scholar] [CrossRef]
  14. Laver, K.; Cumming, R.G.; Dyer, S.M.; Agar, M.R.; Anstey, K.J.; Beattie, E.; Brodaty, H.; Broe, T.; Clemson, L.; Crotty, M.; et al. Clinical practice guidelines for dementia in Australia. Med. J. Aust. 2016, 204, 191–193. [Google Scholar] [CrossRef] [Green Version]
  15. Ojagbemi, A.; Owolabi, M. Do occupational therapy interventions improve quality of life in persons with dementia? A meta-analysis with implications for future directions. Psychogeriatrics 2017, 17, 133–141. [Google Scholar] [CrossRef]
  16. Kitwood, T.; Bredin, K. Towards a theory of dementia care: Personhood and well-being. Ageing Soc. 1992, 12, 269–287. [Google Scholar] [CrossRef]
  17. Bennett, S.; Laver, K.; Voigt-Radloff, S.; Letts, L.; Clemson, L.; Graff, M.; Wiseman, J.; Gitlin, L. Occupational therapy for people with dementia and their family carers provided at home: A systematic review and meta-analysis. BMJ Open 2019, 9, e026308. [Google Scholar] [CrossRef] [Green Version]
  18. Kim, S.Y.; Yoo, E.Y.; Jung, M.Y.; Park, S.H.; Park, J.H. A systematic review of the effects of occupational therapy for persons with dementia: A meta-analysis of randomized controlled trials. NeuroRehabilitation 2012, 31, 107–115. [Google Scholar] [CrossRef]
  19. Graff, M.J.; Vernooij-Dassen, M.J.; Thijssen, M.; Dekker, J.; Hoefnagels, W.H.; Olderikkert, M.G. Effects of community occupational therapy on quality of life, mood, and health status in dementia patients and their caregivers: A randomized controlled trial. J. Gerontol. A Biol. Sci. Med. Sci. 2007, 62, 1002–1009. [Google Scholar] [CrossRef] [Green Version]
  20. Gitlin, L.N.; Winter, L.; Corcoran, M.; Dennis, M.P.; Schinfeld, S.; Hauck, W.W. Effects of the home environmental skill-building program on the caregiver-care recipient dyad: 6-month outcomes from the Philadelphia REACH Initiative. Gerontologist 2003, 43, 532–546. [Google Scholar] [CrossRef] [Green Version]
  21. Graff, M.J.; Vernooij-Dassen, M.J.; Thijssen, M.; Dekker, J.; Hoefnagels, W.H.; Rikkert, M.G. Community based occupational therapy for patients with dementia and their care givers: Randomised controlled trial. BMJ 2006, 333, 1196. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  22. Regan, B.; Wells, Y.; Farrow, M.; O’Halloran, P.; Workman, B. MAXCOG-Maximizing Cognition: A Randomized Controlled Trial of the Efficacy of Goal-Oriented Cognitive Rehabilitation for People with Mild Cognitive Impairment and Early Alzheimer Disease. Am. J. Geriatr. Psychiatry 2017, 25, 258–269. [Google Scholar] [CrossRef]
  23. Schaber, P. Occupational Therapy Practice Guidelines for Adults with Alzheimer’s Disease and Related Disorders; AOTA Press: Bethesda, MA, USA, 2010. [Google Scholar]
  24. Yuill, N.; Hollis, V. A systematic review of cognitive stimulation therapy for older adults with mild to moderate dementia: An occupational therapy perspective. Occup Ther. Int. 2011, 18, 163–186. [Google Scholar] [CrossRef] [PubMed]
  25. Page, M.J.; McKenzie, J.E.; Bossuyt, P.M.; Boutron, I.; Hoffmann, T.C.; Mulrow, C.D.; Shamseer, L.; Tetzlaff, J.M.; Akl, E.A.; Brennan, S.E.; et al. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ 2021, 372, n71. [Google Scholar] [CrossRef]
  26. Clare, L.; Linden, D.E.; Woods, R.T.; Whitaker, R.; Evans, S.J.; Parkinson, C.H.; van Paasschen, J.; Nelis, S.M.; Hoare, Z.; Yuen, K.S.; et al. Goal-oriented cognitive rehabilitation for people with early-stage Alzheimer disease: A single-blind randomized controlled trial of clinical efficacy. Am. J. Geriatr. Psychiatry 2010, 18, 928–939. [Google Scholar] [CrossRef] [PubMed]
  27. Kurz, A.; Thone-Otto, A.; Cramer, B.; Egert, S.; Frolich, L.; Gertz, H.J.; Kehl, V.; Wagenpfeil, S.; Werheid, K. CORDIAL: Cognitive rehabilitation and cognitive-behavioral treatment for early dementia in Alzheimer disease: A multicenter, randomized, controlled trial. Alzheimer Dis. Assoc. Disord. 2012, 26, 246–253. [Google Scholar] [CrossRef] [PubMed]
  28. Kim, S. Cognitive rehabilitation for elderly people with early-stage Alzheimer’s disease. J. Phys. Ther. Sci. 2015, 27, 543–546. [Google Scholar] [CrossRef] [Green Version]
  29. Kim, H.J.; Yang, Y.; Oh, J.G.; Oh, S.; Choi, H.; Kim, K.H.; Kim, S.H. Effectiveness of a community-based multidomain cognitive intervention program in patients with Alzheimer’s disease. Geriatr. Gerontol. Int. 2016, 16, 191–199. [Google Scholar] [CrossRef]
  30. Santos, G.D.; Nunes, P.V.; Stella, F.; Brum, P.S.; Yassuda, M.S.; Ueno, L.M.; Gattaz, W.F.; Forlenza, O.V. Multidisciplinary rehabilitation program: Effects of a multimodal intervention for patients with Alzheimer’s disease and cognitive impairment without dementia. Arch. Clin. Psychiatry 2015, 42, 153–156. [Google Scholar] [CrossRef] [Green Version]
  31. Fernandez-Calvo, B.; Contador, I.; Ramos, F.; Olazaran, J.; Mograbi, D.C.; Morris, R.G. Effect of unawareness on rehabilitation outcome in a randomised controlled trial of multicomponent intervention for patients with mild Alzheimer’s disease. Neuropsychol. Rehabil. 2015, 25, 448–477. [Google Scholar] [CrossRef]
  32. Higgins, J.P.; Altman, D.G.; Gotzsche, P.C.; Juni, P.; Moher, D.; Oxman, A.D.; Savovic, J.; Schulz, K.F.; Weeks, L.; Sterne, J.A.; et al. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ 2011, 343, d5928. [Google Scholar] [CrossRef] [Green Version]
  33. Scottish Intercollegiate Guidelines Network (SIGN). SIGN 50: A Guideline Developer’s Handbook. 2011. Available online: https://www.sign.ac.uk/our-guidelines/sign-50-a-guideline-developers-handbook/ (accessed on 10 December 2021).
  34. Viechtbauer, W. Conducting Meta-Analyses in R with the metafor Package. J. Stat. Softw. 2010, 36, 1–48. [Google Scholar] [CrossRef] [Green Version]
  35. Rosenthal, R.; Rubin, D. A simple general purpose display of magnitude of experimental effect. J. Educ. Psychol. 1982, 74, 166–169. [Google Scholar] [CrossRef]
  36. Rothstein, M.B.; Hedges, L.V.; Higgins, J.P.T.; Rothstein, H.R. Introduction to Meta-Analysis; John Wiley and Sons: Hoboken, NJ, USA, 2009; pp. 103–106. [Google Scholar] [CrossRef]
  37. Cohen, J. Statistical power analysis for the behavioral sciences. In Preface to the Original Edition; Cohen, J., Ed.; Academic Press: Cambridge, MA, USA, 1977; pp. xiii–xv. [Google Scholar] [CrossRef]
  38. Peters, J.L.; Sutton, A.J.; Jones, D.R.; Abrams, K.R.; Rushton, L. Contour-enhanced meta-analysis funnel plots help distinguish publication bias from other causes of asymmetry. J. Clin. Epidemiol 2008, 61, 991–996. [Google Scholar] [CrossRef] [PubMed]
  39. Sterne, J.A.; Egger, M. Funnel plots for detecting bias in meta-analysis: Guidelines on choice of axis. J. Clin. Epidemiol. 2001, 54, 1046–1055. [Google Scholar] [CrossRef]
  40. Tom, M.; Palmer, J.L.P.; Alex, J.S. Contour-enhanced funnel plots for meta-analysis. Stata J. 2008, 8, 242–254. [Google Scholar] [CrossRef] [Green Version]
  41. Balshem, H.; Helfand, M.; Schunemann, H.J.; Oxman, A.D.; Kunz, R.; Brozek, J.; Vist, G.E.; Falck-Ytter, Y.; Meerpohl, J.; Norris, S.; et al. GRADE guidelines: 3. Rating the quality of evidence. J. Clin. Epidemiol. 2011, 64, 401–406. [Google Scholar] [CrossRef] [PubMed]
  42. Zhu, X.; Yin, S.; Lang, M.; He, R.; Li, J. The more the better? A meta-analysis on effects of combined cognitive and physical intervention on cognition in healthy older adults. Ageing Res. Rev. 2016, 31, 67–79. [Google Scholar] [CrossRef]
  43. Viola, L.F.; Nunes, P.V.; Yassuda, M.S.; Aprahamian, I.; Santos, F.S.; Santos, G.D.; Brum, P.S.; Borges, S.M.; Oliveira, A.M.; Chaves, G.F.; et al. Effects of a multidisciplinary cognitive rehabilitation program for patients with mild Alzheimer’s disease. Clinics 2011, 66, 1395–1400. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  44. Lee, J.; Kim, B. Role of Intensity and Repetition in Rehabilitation Therapy. Brain Neurorehabilit. 2012, 5, 6–11. [Google Scholar] [CrossRef]
  45. Mortimer, J.A. Brain reserve and the clinical expression of Alzheimer’s disease. Geriatrics 1997, 52 (Suppl. S2), S50–S53. [Google Scholar]
  46. Zhao, C.; Noble, J.M.; Marder, K.; Hartman, J.S.; Gu, Y.; Scarmeas, N. Dietary Patterns, Physical Activity, Sleep, and Risk for Dementia and Cognitive Decline. Curr. Nutr. Rep. 2018, 7, 335–345. [Google Scholar] [CrossRef]
  47. Kempermann, G.; Kuhn, H.G.; Gage, F.H. More hippocampal neurons in adult mice living in an enriched environment. Nature 1997, 386, 493–495. [Google Scholar] [CrossRef]
  48. Kempermann, G.; Kuhn, H.G.; Gage, F.H. Experience-induced neurogenesis in the senescent dentate gyrus. J. Neurosci. 1998, 18, 3206–3212. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  49. Grońska-Pęski, M.; Gonçalves, J.T.; Hébert, J.M. Enriched Environment Promotes Adult Hippocampal Neurogenesis through FGFRs. J. Neurosci. 2021, 41, 2899–2910. [Google Scholar] [CrossRef] [PubMed]
  50. Stern, Y. Cognitive reserve in ageing and Alzheimer’s disease. Lancet Neurol. 2012, 11, 1006–1012. [Google Scholar] [CrossRef] [Green Version]
  51. Coleman, P.D.; Yao, P.J. Synaptic slaughter in Alzheimer’s disease. NeuroBiol. Aging 2003, 24, 1023–1027. [Google Scholar] [CrossRef]
  52. DeKosky, S.T.; Scheff, S.W.; Styren, S.D. Structural correlates of cognition in dementia: Quantification and assessment of synapse change. Neurodegeneration 1996, 5, 417–421. [Google Scholar] [CrossRef]
  53. Wen, P.H.; Hof, P.R.; Chen, X.; Gluck, K.; Austin, G.; Younkin, S.G.; Younkin, L.H.; DeGasperi, R.; Gama Sosa, M.A.; Robakis, N.K.; et al. The presenilin-1 familial Alzheimer disease mutant P117L impairs neurogenesis in the hippocampus of adult mice. Exp. Neurol. 2004, 188, 224–237. [Google Scholar] [CrossRef]
  54. Arendash, G.W.; Garcia, M.F.; Costa, D.A.; Cracchiolo, J.R.; Wefes, I.M.; Potter, H. Environmental enrichment improves cognition in aged Alzheimer’s transgenic mice despite stable beta-amyloid deposition. Neuroreport 2004, 15, 1751–1754. [Google Scholar] [CrossRef]
  55. Barbara, A.; Schell, G.G.; Marjorie, S.; Cohn, E.S. Willard and Spackman’s Occupational Therapy; Wolters Kluwer: Philadelphia, PA, USA, 2003. [Google Scholar]
  56. Thompson, S.G. Why sources of heterogeneity in meta-analysis should be investigated. BMJ 1994, 309, 1351–1355. [Google Scholar] [CrossRef]
  57. Rothstein, H.R.; Sutton, A.J.; Borenstein, M. Publication Bias in Meta-Analysis: Prevention, Assessment and Adjustments; John Wiley & Sons: Hoboken, NJ, USA, 2006. [Google Scholar]
Biomedicines 09 01951 g001 550
Figure 1. Flow diagram of the study selection process.
Figure 1. Flow diagram of the study selection process.
Biomedicines 09 01951 g001
Biomedicines 09 01951 g002 550
Figure 2. Risk of bias assessment across randomized controlled trials. Risk of bias graph (A) and summary (B).
Figure 2. Risk of bias assessment across randomized controlled trials. Risk of bias graph (A) and summary (B).
Biomedicines 09 01951 g002
Biomedicines 09 01951 g003 550
Figure 3. Forest plot for the effects of the multimodal occupational therapy program with cognition-oriented approach on cognitive decline in patients with AD.
Figure 3. Forest plot for the effects of the multimodal occupational therapy program with cognition-oriented approach on cognitive decline in patients with AD.
Biomedicines 09 01951 g003
Biomedicines 09 01951 g004 550
Figure 4. Forest plot for the effects of the multimodal occupational therapy program with cognition-oriented approach on BADL in patients with AD.
Figure 4. Forest plot for the effects of the multimodal occupational therapy program with cognition-oriented approach on BADL in patients with AD.
Biomedicines 09 01951 g004
Biomedicines 09 01951 g005 550
Figure 5. Forest plot for the effects of the multimodal occupational therapy program with cognition-oriented approach on IADL in patients with AD.
Figure 5. Forest plot for the effects of the multimodal occupational therapy program with cognition-oriented approach on IADL in patients with AD.
Biomedicines 09 01951 g005
Biomedicines 09 01951 g006 550
Figure 6. Funnel plot of the results of the studies included (A) funnel plot of cognitive function, (B) funnel plot of BADL, and (C) funnel plot of IADL.
Figure 6. Funnel plot of the results of the studies included (A) funnel plot of cognitive function, (B) funnel plot of BADL, and (C) funnel plot of IADL.
Biomedicines 09 01951 g006
Table
Table 1. Characteristics of included studies.
Table 1. Characteristics of included studies.
StudyDiagnosis
Criteria		ParticipantsIntervention/ComparisonTime
(Duration/Frequency)		SettingMeasurement of Outcomes
CountryGenderAge
(M ± SD)
EG
CG		EG (n)
CG (n)		Intervention
Format		Experimental GroupControl GroupCognitive FunctionADLs
1Onor et al., 2007 [5]DSM-4ItalyMale
Female		68.0 ± 6.5
72.0 ± 5.2		8
8		GroupMRI
(including OT)		N/I1 h
/1 session per weeks
/16 weeks		Clinical psychiatry
Visiting patient’s home		MMSEBADL
IADL
2Clare et al., 2010 [26]NINCDS-ADRDAUnited KingdomMale
Female		77.7
(Average)		22
22		IndividualOT based CRN/I1 h
/1 session per weeks
/8 weeks		Community based settingN/AILS
(IADL)
3Kurz et al., 2012 [27]ICD-10GermanyMale
Female		72.4 ± 8.5
75.0 ± 7.0		92
97		Individual4 thematic modules:
neurorehabilitation & psychotherapy
(Including OT)		N/I1 h
/1 session per weeks
/12 weeks		HospitalMMSEB-ADL
4Kim, 2015 [28]N/ISouth KoreaMale
Female		70.4 ± 7.9
71.4 ± 8.2		22
21		Group
Individual		OT based CRWatching videos
Conversation with the examiner		1 h
/1 session per weeks
/8 weeks		No informationMMSEMBI (BADL)
COPM
(IADL)
5Kim et al., 2016 [29]NINCDS-ADRDASouth KoreaMale
Female		78.4 ± 1.0
78.5 ± 1.7		32
21		GroupMCP
(including OT)		Routine pharmacotherapy5 h
/1 session per weeks
/24 weeks		Regional
dementia center		K-MMSEN/A
6Santos et al., 2015 [30]NINCDS-ADRDABrazilMale
Female		75.7 ± 5.6
74.8 ± 4.7		46
16		GroupMRP
(including OT)		N/I5 h
/2 sessions per weeks
/12 weeks		Regional
dementia center		MMSEN/A
7Fernandez-Calvo et al., 2015 [31]NINCDS-ADRDASpainMale
Female		74.3 ± 3.9
72.3 ± 3.7		25
30		IndividualMIP
(including OT)		N/I90 min
/3 sessions per week
/16 weeks		HomeADAS-CogRDRS-2
(BADL)
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Ham, M.-J.; Kim, S.; Jo, Y.-J.; Park, C.; Nam, Y.; Yoo, D.-H.; Moon, M. The Effect of a Multimodal Occupational Therapy Program with Cognition-Oriented Approach on Cognitive Function and Activities of Daily Living in Patients with Alzheimer’s Disease: A Systematic Review and Meta-Analysis of Randomized Controlled Trials. Biomedicines 2021, 9, 1951. https://doi.org/10.3390/biomedicines9121951

AMA Style

Ham M-J, Kim S, Jo Y-J, Park C, Nam Y, Yoo D-H, Moon M. The Effect of a Multimodal Occupational Therapy Program with Cognition-Oriented Approach on Cognitive Function and Activities of Daily Living in Patients with Alzheimer’s Disease: A Systematic Review and Meta-Analysis of Randomized Controlled Trials. Biomedicines. 2021; 9(12):1951. https://doi.org/10.3390/biomedicines9121951

Chicago/Turabian Style

Ham, Min-Joo, Sujin Kim, Ye-Ji Jo, Chisoo Park, Yunkwon Nam, Doo-Han Yoo, and Minho Moon. 2021. "The Effect of a Multimodal Occupational Therapy Program with Cognition-Oriented Approach on Cognitive Function and Activities of Daily Living in Patients with Alzheimer’s Disease: A Systematic Review and Meta-Analysis of Randomized Controlled Trials" Biomedicines 9, no. 12: 1951. https://doi.org/10.3390/biomedicines9121951

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop