Bias should be taken into account when assessing clinical trials. It can occur in various forms in clinical studies and might influence the results in different directions. Bias can occur through the selection of study patients, the investigators, the type of data, and the analysis of the data. Different study types suffer from different potential biases. The aim of this paper is to describe common types of clinical trials and to illustrate their potential biases.
Hinweise
Redaktion
D. Günther, Köln
E. Herbst, Münster
×
Scan QR code & read article online
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Introduction
The quality and relevance of a research project depends on the choice of the appropriate study design. The study design, in turn, is determined by the respective research question and must be defined before conducting the study. In general, a distinction is made between original studies including basic research, observational studies and interventional studies, and review articles and meta-analyses. In 1981, Dr. David Sackett (McMaster University, Canada) and colleagues laid the groundwork for evidence-based medicine (EBM) [2]. From its inception, the goal of EBM has been to integrate the best available scientific evidence into clinical decision making. In this process, it is necessary to evaluate the available literature on a specific topic to critically assess the validity of the presented data. Therefore, when evaluating clinical trials, classification according to study type into different levels of evidence is an almost omnipresent concept. The classification is made according to the risk that bias is present in the data [3, 4]. Blinded randomized controlled trials (RCTs) have the highest scientific quality, as they are at least subject to relevant bias. In the following, the levels of evidence for different research questions (therapeutic, prognostic, diagnostic, economic) and disciplines have been adapted so that a classification can be made for each clinical study (see Fig. 1).
Fig. 1
Evidence levels of medical studies
×
Original research
Basic research (non-human research)
In this area, a fundamental distinction is made between theoretical and applied basic research. Theoretical basic research primarily comprises method development and material testing, such as the development/improvement of analytical (determination of markers, enzymes, genes) or imaging measurement procedures (computed tomography, magnetic resonance imaging). Applied basic research includes experiments on animals, cells, biochemical and genetic studies, and material development. This experimental research usually involves studying the effects of changing at least one independent variable on the dependent variable. By performing standardized experiments, high internal validity can be achieved, with usually limited extrapolation of study results to clinical reality.
Anzeige
Clinical research
In clinical (human) research, a basic distinction is made between observational and interventional studies. Observational studies are characterized by the fact that the scientist cannot control the assignment of the treatment to the subjects, which can lead to certain biases in the study results. These include cohort studies, case–control studies, cross-sectional and longitudinal studies, as well as case series and single-case presentations. Interventional clinical trials, on the other hand, include studies in which the researcher can assign participants directly to an intervention, usually by randomization. These include, in particular, RCTs (Table 1).
Table 1
Observational and interventional clinical studies
Observation
(prospective or retrospective)
Intervention
(prospective)
Cohort studies
Case-control studies
Cross-sectional studies
Longitudinal studies
Case series
Case reports
Interventional clinical studies
Clinical trial (according to AMG/MPG)
±controlled (e.g., placebo, healthy control, standard therapy …)
Observational studies aim to gain insights into possible associations between certain diseases, their treatment, and other factors (e.g., individual life circumstances) of the study participants. Here, study participants are merely observed, so no specific intervention takes place as part of the study. However, observational studies can follow patients after predefined surgeries. Mostly, observational studies involve large groups of participants (healthy, patients/experienced, or even former patients), who are usually evaluated by means of questionnaires. These observational studies can be either retrospective or prospective in nature (Fig. 2). Overall, the study types described below can also be summarized as non-randomized studies (NRS).
Fig. 2
Difference between retrospective and prospective observational studies
×
Cohort studies follow patients based on specific exposures (e.g., a particular surgery), so they distinguish between exposed and unexposed subjects. They can be both prospective (level 2) and retrospective (level 3). In a prospectively designed cohort study, the research question is formulated before the start of the study and patients are followed longitudinally according to the study design. In a retrospective cohort study, the research question is formulated and answered on the basis of existing data (e.g., register, in-hospital database).
Case–control studies examine patients based on a clinical event (e.g., re-rupture of the anterior cruciate ligament). Patients in whom the event occurred (cases) are compared to patients without the event (controls) to find possible differences. In the literature, the classifications as case–control study and retrospective cohort study are often incorrectly used as interchangeable. Here, retrospective studies that follow patients based on a therapy (exposure), i.e., cohort studies, are incorrectly referred to as case–control studies ([1]; Fig. 3).
Fig. 3
Difference between case–control (top, light grey) and cohort (bottom, dark grey) studies
×
Case series also examine a group of patients on the basis of an event. However, in comparison to the case–control study, there is no control group, so that only descriptive results can be derived from case series. Hypotheses arising from such data must subsequently be tested in further clinical trials of higher levels of evidence. Therefore, case series have the lowest level of evidence compared to the other study designs.
In a cross-sectional study, the survey, observation, or measurement is conducted once, so it reflects only a snapshot in time. Cross-sectional studies can be used both descriptively and analytically. In contrast, in a longitudinal study, the same survey, observation, or measurement is conducted at multiple timepoints, so it reflects processes of change over time (Fig. 4).
Fig. 4
Difference between cross-sectional and longitudinal studies
×
Selection bias in non-randomized trials.
The key feature of NRS is that the included patients originate from current common clinical practice and are not subject to any study-specific intervention. Regardless of whether data were collected prospectively or retrospectively, the physician decides on the patient’s treatment [6]. In addition to the available evidence, the physician’s decision is influenced by their own experience, the patient’s needs, and other factors. For example, young, athletically active patients with high physical demands may be more likely to be recommended a reconstruction of the anterior cruciate ligament than less active, or older patients. If surgically and conservatively treated patients are now compared to each other with regard to their clinical outcome, these confounding variables (“confounders”) between the patient groups can lead to bias in the study results. The conscious or subconscious selection of patients for study groups based on parameters that are not the variable of interest (e.g., intervention) is referred to as “selection bias” [7]. This can lead to overestimation of the effects of an intervention in an NRS. Thus, meta-analyses that include NRS and RCTs frequently show a more pronounced effect of an intervention in NRS compared to RCTs [8, 9]. In particular, small significant effects in observational studies should be interpreted with caution, as they may be influenced by a relevant selection bias.
The most effective way to minimize selection bias is randomization, as described below. However, it is also possible to compensate for confounding variables. A well-known possibility is the implementation of a “matched-pair” analysis. Here, the study groups are composed in such a way that they are similar in previously defined basic variables that are known to have an influence on the results (e.g., age, gender, body mass index, smoking habits) [10]. However, matching can only compensate for known confounding variables. Thus, selection bias can at best be reduced, but not fully eliminated. In addition to matching, selection bias can also be reduced by applying specific statistical methods, e.g., through propensity score matching. As explaining propensity score matching would be outside the scope of this article, we refer to exemplary literature [10, 11]. Regardless of whether specific efforts were made to adjust for confounding variables, each scientific article should have a description of the study groups [12]. This should include, in addition to epidemiological parameters (age, sex, body mass index, smoking status, etc.), possible study-specific confounding variables (e.g., meniscal or cartilage injuries in anterior cruciate ligament reconstruction studies). Thus, the reader can see for himself to what extent the study groups are similar.
Retrospective assessment of preoperative parameters and the recall bias.
A main drawback of retrospective studies is the fact that patients find it difficult to adequately reconstruct their medical history. When subjective parameters are queried, this can lead to biases known as “recall bias.” For example, Crutchfield and colleagues showed that when patients retrospectively answered patient-centered questionnaires after hip arthroscopy, this recall bias caused patients to rate their preoperative subjective condition significantly worse than it actually was. This then leads to overestimation of the effect of an intervention postoperatively [13]. Similar observations were shown in shoulder and knee surgery [14, 15]. Thus, a prospective survey of outcome parameters, as performed in prospective cohort studies, is preferable.
Outcome parameter collection.
Outcome parameters in clinical studies can be assessed in various ways. While objective parameters (e.g., quantification of knee joint stability using the Rolimeter® (Aircast Europa, Neubeuern, Germany)) have to be collected by an investigator, there are different ways to collect subjective patient-centered parameters. In particular, the regularly queried patient-reported outcome measures (PROMs) are suitable for collection using web-tools or by telephone, provided they have been validated for this purpose. However, the manner of collection can have significant influences on the outcome. Acosta et al. showed in their meta-analysis that PROMs collected by telephone showed significantly higher scores than online-based or face-to-face collection [16]. This is not necessarily a problem as long as the same methods are used consistently throughout the study. However, it regularly occurs that the initial survey was conducted during a face-to-face presentation (at inclusion), but the subsequent final evaluation, especially in the case of long-term follow-up, was conducted by telephone. This could bias postoperative values towards larger effects.
Intervention studies
Prospective RCTs are generally accepted as the gold standard of clinical intervention studies (level 1) [17]. However, even this type of study is subject to certain risks of bias that may call the study results into question, and not every RCT is of equally high quality [5].
Randomization as a solution to selection bias.
The main feature of RCTs is randomization. During randomization, patients who meet the inclusion criteria defined before the start of the study are randomly assigned to one of the study groups. This ensures an equal or random distribution of potential confounders between the study groups. Selection bias, as described above, is thus largely excluded or its risk is reduced [17]. Randomization can be performed with various methods. In addition to the classic coin flip, commonly accepted methods include allocation using computer-generated sequences [18, 19]. Non-accepted strategies are randomization by sex, date of birth, date of inclusion, or other patient- or clinic-specific factors, as they do not guarantee adequate randomization. It is also important that allocation remains unknown to the including personnel, hence preventing (conscious or unconscious) allocation to a preferred group (blinding of allocation) [20].
One problem that regularly arises in surgical intervention trials is patient (non-)compliance. For example, patients randomized to surgery may decide not to undergo surgery, thereby losing the original randomization. One way to account for this is to perform an intention-to-treat (ITT) analysis, in which patients are still analyzed in the group to which they were originally randomized, regardless of the actual therapy. For a detailed discussion of the specifics of ITT, as well as possible alternatives, we refer the reader to the relevant AGA Research Committee article by Laky et al. [21].
The problem of blinding in orthopedic RCTs.
Blinding in orthopedic studies can occur at different levels. Blinding of the surgeon is not possible for obvious reasons. In the case of intervention studies, implementation of studies blinded to the patient is only possible by performing a sham operation, which presents relevant ethical hurdles, so that this can only be justified in very rare cases [22]. Lastly, the person collecting the clinical endpoints (e.g., physical examination at follow-up) can be blinded. If this is not the case, it may lead to observer bias, as knowledge about the patient’s treatment may consciously or unconsciously influence the observer’s assessments [23]. Both the lack of blinding of the patient and the person following up can lead to significant bias in the reported results. Usually, this leads to overestimation of the effect of a treatment [24, 25]. The challenges that can arise from this problem, as well as possible solutions, have already been reviewed in a previous article from the research committee of Günther et al. [26].
Loss to follow-up and attrition bias.
Ideally, all patients included at the beginning of a study are available for follow-up at the end of data collection. However, as patients may move away, die, become unreachable, or no longer be interested in participating in the study, collection of all datasets is unrealistic, especially in longer-term studies. However, subjects lost to follow-up may differ significantly in terms of their characteristics from the remaining patients. Randomization in an RCT, as described above, ensures homogeneity of study participants between trial groups. If the loss-to-follow-up differs between the study groups, this may mean that the homogeneity of the groups obtained by randomization is no longer guaranteed, which may lead to a difference between the groups that can no longer be explained by the intervention. This bias in study results due to the asymmetric omission of patients for the final analysis of a study is referred to as attrition bias [27]. The relevance of attrition bias is highlighted in the work of Akl and colleagues [28]. In their systematic review, up to one third of the included studies lost their significant difference between groups when a significant difference was simulated for patients who were no longer available for follow-up compared to the remaining patients. Generally applicable thresholds for the extent at which the loss of study patients becomes relevant cannot be found in the literature. As a rule of thumb, it is stated that a loss below 5% is indicative of a low risk of attrition bias, whereas a loss above 20% is considered a high risk [27, 29]. However, the ratio of the loss to follow-up between study groups is more important than the absolute number of subjects lost for final analysis [29].
Secondary research (review articles and meta-analyses)
In secondary research, previously published work is summarized as simple (narrative) or systematic reviews with or without meta-analysis. In a systematic review, all previously published primary studies are systematically identified, selected, and critically evaluated for a clearly defined research question. The results are then summarized descriptively or quantitatively using statistical methods in the form of a meta-analysis. Meta-analysis is thus a statistical method for quantitatively combining the results of studies dealing with the same research question into an overall result within the framework of a systematic review. This is intended to increase the informative value compared to the respective individual studies. A simple (narrative) review, on the other hand, is based on a non-systematic selection process, so that a subjective partial selection of the studies can occur.
Anzeige
Conclusion
Evidence levels of clinical studies are based on the susceptibility of a study type to bias.
The primary limitation of non-randomized studies (NRS) is that study groups differ in terms of the distribution of known and unknown confounders, which can influence the study results (selection bias).
NRS tend to overestimate the effect of an intervention, which can be due to various reasons.
Randomized controlled trials (RCTs) represent the highest level of evidence, as they are the least susceptible to bias due to their methodology. However, randomized controlled trials can also be subject to bias, which can influence the study results.
The difficulty of blinding in orthopedic RCTs poses a problem.
Declarations
Conflict of interest
A. Deichsel, L.N. Münch, B. Laky, and the AGA Research Committee declare that they have no competing interests.
For this article no studies with human participants or animals were performed by any of the authors. All studies mentioned were in accordance with the ethical standards indicated in each case.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Mit e.Med Orthopädie & Unfallchirurgie erhalten Sie Zugang zu CME-Fortbildungen der Fachgebiete, den Premium-Inhalten der dazugehörigen Fachzeitschriften, inklusive einer gedruckten Zeitschrift Ihrer Wahl.
Der Einsatz von Operationsrobotern für den Einbau von Totalendoprothesen des Kniegelenks hat die Präzision der Eingriffe erhöht. Für die postoperative Zufriedenheit der Patienten scheint das aber unerheblich zu sein, wie eine Studie zeigt.
Mit dem Hebelzeichen-Test lässt sich offenbar recht zuverlässig feststellen, ob ein vorderes Kreuzband gerissen ist. In einer Metaanalyse war die Vorhersagekraft vor allem bei positivem Testergebnis hoch.
Extreme Arbeitsverdichtung und kaum Supervision: Dr. Andrea Martini, Sprecherin des Bündnisses Junge Ärztinnen und Ärzte (BJÄ) über den Frust des ärztlichen Nachwuchses und die Vorteile des Rucksack-Modells.
Bewegungs-, Dehnungs- und Entspannungsübungen im Wasser lindern die Beschwerden von Patientinnen mit Fibromyalgie besser als das Üben auf trockenem Land. Das geht aus einer spanisch-brasilianischen Vergleichsstudie hervor.
Update Orthopädie und Unfallchirurgie
Bestellen Sie unseren Fach-Newsletter und bleiben Sie gut informiert.
