Contributions to the literature
-
Our systematic scoping review gives the first comprehensive overview of randomized controlled trials in de-implementation.
-
De-implementation trials have focused on primary care and drug treatments; however, there is dire lack of research on diagnostics, surgical treatments, and in secondary/tertiary care.
-
Most trials were limited by complex intervention design, human intervention deliverer, small number of clusters in cluster trials, and lack of theoretical background and tailoring.
-
Major improvements in methodology are needed to find reliable evidence on effective de-implementation interventions. We provided recommendations on how to address these issues.
Introduction
Methods
Data sources and searches
Eligibility criteria
Outcomes and variables
Risk of bias and quality indicators
Study selection and data extraction
Intervention categorization and outcome hierarchy
Name | Rationale and definitions | Examples |
---|---|---|
Health outcomes | De-implementing a clinical practice should improve (or at least have no negative effect on) health outcomes. Health outcomes can therefore be considered measuring the safety of de-implementation | Mortality, morbidity, quality of life, symptoms |
Low-value care use | The primary aim of a de-implementation intervention is to reduce low-value care. Predefined low-value care use should therefore be (one of) the primary outcome(s) of de-implementation effectiveness. Typically, the definition of low-value care is based on diagnoses or clinical criteria that represent low-value care in combination with a specific clinical practice. Data is often gathered from individual patient records or administrative databases. Individual patient records usually contain more specific information on clinical decisions and may therefore yield more accurate information | Antibiotic use for viral upper respiratory infections Use of radiological imaging in patients with acute low back pain without “red-flag” symptoms |
Appropriate care use | Can be used as an outcome when a medical practice can be either appropriate or inappropriate. For instance, in patients with respiratory infection, use of antibiotics can be either appropriate or inappropriate. Change in appropriate care use measures unintended consequences of de-implementation and can therefore be considered as a measure of safety of de-implementation | Antibiotic use for confirmed pneumonia Use of radiological imaging in patients with low back pain and “red-flag” symptoms |
Total volume of care | Total volume includes both appropriate and inappropriate care and is an indirect measure of low-value care. It may sometimes be justifiable to use in very large samples if it is impossible to differentiate between appropriate and inappropriate care and if using individual patient records is not possible. Outcomes that are based on diagnoses often include both appropriate and inappropriate care and should therefore be considered as total volume care, not as low-value care, outcomes | Total use of antibiotics in upper respiratory tract infections Use of radiological imaging in low-back pain |
Intention to reduce the use of low-value care | Intention is the first step to change but does not reliably describe actual change in use of low-value care. As intention can be measured earlier than other outcomes, it may sometimes be justifiable to use as a preliminary assessment of the effectiveness of a de-implementation intervention. It is often used after educational interventions and when the data is gathered through surveys | Intention to reduce the use of inappropriate antibiotic use in upper respiratory tract infections Intention to reduce use of inappropriate radiological imaging in low-back pain |
Analysis
Results
Study characteristics
Characteristics | Aim and rationale | Outcomes | |||
---|---|---|---|---|---|
Settinga | n (%) | Aima | n (%) | Outcome categoriesa | n (%) |
Primary care — outpatient | 149 (66%) | Abandon | 0 (0%) | Health outcomes | 58 (26%) |
Primary care — inpatient | 3 (1%) | Reduce | 225 (99%) | Low-value care use | 63 (28%) |
Secondary/tertiary care — outpatient | 28 (12%) | Replace | 42 (19%) | Appropriate care use | 34 (15%) |
Secondary/tertiary care — inpatient | 40 (18%) | Unclear | 2 (1%) | Total volume of care | 194 (85%) |
Other | 22 (10%) | Rationalea | Intention to reduce the use of low-value care | ||
Randomization unit | Evidence suggests little or no benefit from treatment or diagnostic test | 115 (51%) | 17 (7%) | ||
Cluster | 145 (64%) | Measured costsa | |||
Individual | 82 (36%) | Evidence suggests another treatment is more effective or less harmful | 13 (6%) | Intervention costs | 20 (9%) |
Medical interventiona | Healthcare costs | 45 (20%) | |||
Prevention | 9 (4%) | Evidence suggests more harms than benefits for the patient or community | 145 (64%) | Reported effectiveness | |
Diagnostic imaging | 29 (13%) | (Some) desired effect | 186 (82%) | ||
Laboratory tests | 28 (12%) | Cost-effectiveness | 70 (31%) | No desired effect | 41 (18%) |
Drug treatment | 163 (72%) | Patient(s) do not want the intervention | 2 (1%) | Theoretical basis and tailoringa | |
Operative treatments | 7 (3%) | Theory-based interventions | 48 (21%) | ||
Rehabilitation | 2 (1%) | Not reported/unclear | 20 (9%) | Tailored interventions | 40 (18%) |
Other | 7 (3%) | Intervention complexityb | |||
Target groupa | Multicomponent | 152 (67%) | |||
Public | 5 (2%) | Simple | 84 (37%) | ||
Patients | 42 (19%) | ||||
Caregivers | 17 (7%) | ||||
Physicians | 193 (85%) | ||||
Nurses | 37 (16%) | ||||
Other | 23 (10%) |
Risk of bias
Study outcomes
Conflicts of interest and funding
Quality indicators
Intervention categorization
Discussion
Problem | Explanation and elaboration | Recommendation | Evidence (identified in our scoping review) |
---|---|---|---|
Complex interventions Studying very complex interventions increases challenges in feasibility, replication, and evaluation of individual factors that affect the success of de-implementation | To progress the understanding of what works in de-implementation and making interventions more feasible, simpler interventions should be conducted. Simpler intervention means that there are fewer factors potentially affecting the success of de-implementation. When conducting simpler interventions, it is also easier to separate effective from ineffective factors. When conducting more complex interventions, process evaluation can improve the feasibility and help separate the important factors | Prefer simpler intervention designs | 67% of studies had multiple intervention components, which usually leads to higher intervention complexity |
Human intervention deliverer Generalizability decreases when the “human factor” (personal characteristics of the deliverers) affects the results of de-implementation | A human deliverer of the intervention may introduce confounding characteristics that affect the success of de-implementation. To improve the applicability of the results, studies should aim for higher number of intervention deliverers. When reporting the results, article should specify the number and characteristics of the deliverers used | Aim for larger number of intervention deliverers and describe the number and characteristics of the deliverers | 50% of studies tested an intervention with educational sessions using a human intervention deliverer |
Small number of clusters A small number of clusters decreases the reliability of effect estimates | The intra-cluster correlation coefficient is used to adjust sample sizes for between-cluster heterogeneity in treatment effects. This adjustment is often insufficient in small cluster randomized trials, as they produce imprecise estimates of heterogeneity, which may lead to unreliable effect estimates and false-positive results [21, 22]. Probability of false-positive results increases with higher between-cluster heterogeneity and smaller number of clusters (especially under 30 clusters) [21, 22]. Analyses may be corrected by small sample size correction methods, resulting in decreased statistical power. If the number of clusters is low, higher statistical power in individually randomized trials may outweigh the benefits acquired from cluster RCT design, avoiding contamination [23] | If the eligible number of clusters is low, consider performing an individually randomized trial. If number of clusters is small, consider using small sample size correction methods to decrease the risk of a false-positive result. Take the subsequent decrease in statistical power into account when calculating target sample size | In 145 cluster randomized trials, the median number of clusters was 24 |
Dropouts Dropouts of participants may lead to unreliable effect estimates | Trials should report dropouts for all intervention participants, including participants that were targeted with the de-implementation intervention and participants used as the measurement unit. Trials should separate between intervention participants that completely dropped out and who were replaced by new participants. To minimize dropouts, randomization should occur as close to the intervention as possible | Report dropouts for all intervention participants. Randomize as near to the start of the intervention as possible | Missing data led to a high risk of bias in 60% of studies, of which 76% were due to unreported data |
Heterogeneous study contexts Diverse contextual factors may affect the outcome | Behavioral processes are usually tied to “local” context, including study environment and characteristics of the participants. These factors may impact participants’ behavior. Tailoring the intervention facilitates designing the intervention to target factors potentially important for the de-implementation. Examples include assessing barriers for change (and considering them in the intervention design) and including intervention targets in planning the intervention | Tailor the intervention to the study context | 82% of the studies did not tailor the intervention to the study context |
Heterogeneous mechanisms of action De-implementation interventions have diverse mechanisms of action | Theoretical knowledge helps to understand how and why de-implementation works. A theoretical background may not only increase chances of success but also improve the understanding of what works (and what does not work) in de-implementation. Examples include describing barriers and enablers for the de-implementation or describing who are involved and how they contribute to process of behavioral change | Use a theoretical background in the planning of the intervention | 79%of the studies did not report a theoretical basis for the intervention |
Randomization unit Randomization at a different level from the target level where the intervention primarily happens may result in loss of the randomization effect | Reducing the use of medical practices happens at the level of the medical provider. Therefore, if randomization happens at the level of the patient, the trial will not provide randomized data on provider-level outcomes. Even when the intervention target is the patient, the provider is usually involved in decision-making. Therefore, the intervention effect will occur on both provider and patient levels. Randomization is justified at the patient level when patient-level outcomes are measured or when the number of providers is large, representing several types of providers | Randomize at the same level as the intervention effect is measured | 12% of the studies had provider-level outcome(s) but were randomized at the patient level |
Outcomes Total volume of care outcomes may not represent changes in low-value care use | Total volume of care outcomes (including diagnosis-based outcomes) are vulnerable to bias, such as seasonal variability and diagnostic shifting [24]. Changes in these outcomes may not represent changes in actual low-value care use as the total volume of care includes both appropriate and inappropriate care. When measuring low-value care, comparing its use relative to the total volume of care or to appropriate care can help mitigate these biases | Use actual low-value care use outcomes whenever possible | 28% of the studies measured actual low-value care use |
Cluster heterogeneity Practice level variability in use of low-value care may be large | Baseline variability in low-value care use may be large [25]. As such, if the number of clusters is low, the baseline variability might lead to biased effect estimates | Compare low-value care use between the baseline and after the intervention | 24% of the studies did not report baselines estimates or differences between the baseline and after the intervention |