Key terms relating to risk of bias and critical appraisal
| Key terms | Explanation |
| Risk of bias | Bias is a systematic deviation from the truth in the results of a research study. Bias may occur due to limitations in study design, conduct, analysis, or reporting.3 Bias is associated with underestimated or overestimated study findings. Multiple sources of bias exist, and different biases can vary in direction and magnitude. Assessing bias can never conclusively determine whether a study’s findings underestimate or overestimate a true result because study findings can be unbiased despite methodological limitations. Therefore, risk of bias, rather than bias, is assessed to judge the risk that bias is present. Risk of bias is synonymous with the term internal validity. |
| Study quality | Study quality is the extent to which a study is conducted to the highest methodological standards possible. Study quality evaluates multiple constructs of study methodology including reporting completeness, ethical approval, statistical power, precision, and internal and external validity.7 9 |
| Risk of bias and study quality | The terms ‘risk of bias’ and ‘study quality’ are often used interchangeably. However, both terms are distinct constructs. Discrepancies between study quality and risk of bias are highlighted when performing a risk of bias assessment. Blinding participants in randomised controlled trials (RCTs) can be challenging and often impossible in SEM research (eg, randomising professional football players to receive a Nordic hamstring exercise programme, or not). An RCT that cannot blind participants might be considered high-quality because it may be the only way for trial investigators to conduct such a RCT. However, risk of bias targets the extent to which study findings should be believed, irrespective of researchers’ (in)ability to prevent methodological shortcomings that may affect study findings. Because participants were not blinded, the trial outcome is at ‘high’ risk of bias—this fact is inescapable.3 |
| Reporting quality | Reporting quality refers to the extent to which an original research article provides complete and transparent information about the design, conduct, analysis, and results of a study. Complete reporting facilitates a comprehensive assessment of a study’s internal and external validity, and study design-specific reporting guidelines exist to guide systematic review reporting.23 High-quality or low-quality reporting in a study does not imply that the study’s outcomes are at ‘low’ or ‘high’ risk of bias, respectively.24 For example, there is a difference between reporting whether a methodological procedure, such as randomisation, was performed, and whether it was performed appropriately to sufficiently minimise risk of bias (eg, by using simple randomisation from a computer-generated random numbers table, with an equal allocation ratio). |
SEM, sport and exercise medicine.