Article Text

Download PDFPDF

Seven sins when interpreting statistics in sports injury science
  1. Rasmus Oestergaard Nielsen1,
  2. Cassandra M Chapman2,
  3. Winnifred R Louis2,
  4. Steven D Stovitz3,
  5. Mohammad Ali Mansournia4,5,
  6. Johann Windt6,
  7. Merete Møller7,
  8. Erik Thorlund Parner1,
  9. Adam Hulme8,
  10. Michael Lejbach Bertelsen1,
  11. Caroline F Finch9,
  12. Marti Casals10,11,
  13. Evert Verhagen12
  1. 1 Department of Public Health, Aarhus University, Aarhus, Denmark
  2. 2 School of Psychology, University of Queensland, Brisbane, Queensland, Australia
  3. 3 Department of Family Medicine and Community Health, University of Minnesota, Minneapolis, Minnesota, USA
  4. 4 Department of Epidemiology and Biostatistics, School of Public Health, Tehran University of Medical Sciences, Tehran, Iran
  5. 5 Sports Medicine Research Center, Neuroscience Institute, Tehran University of Medical Sciences, Tehran, Iran
  6. 6 Experimental Medicine Program, University of British Columbia, Vancouver, British Columbia, Canada
  7. 7 Department of Sports Science and Clinical Biomechanics, University of Southern Denmark, Odense, Denmark
  8. 8 Centre for Human Factors and Sociotechnical Systems, University of the Sunshine Coast, Queensland, Australia
  9. 9 Australian Centre for Research into Injury in Sport and its Prevention, Federation University Australia, Ballarat, Australia
  10. 10 Sport Performance Analysis Research Group, University of Vic, Barcelona, Spain
  11. 11 Research Centre Network for Epidemiology and Public Health (CIBERESP), Barcelona, Spain
  12. 12 Amsterdam Collaboration on Health and Safety in Sports, Public and Occupational Health, VU University Medical Center, Amsterdam, The Netherlands
  1. Correspondence to Professor Mohammad Ali Mansournia, Department of Epidemiology and Biostatistics, School of Public Health, Tehran University of Medical Sciences, Tehran, Iran; mansournia_ma{at}yahoo.com

Statistics from Altmetric.com

Introduction

The British Journal of Sports Medicine has introduced a series of editorials and infographics that explain the value of using appropriate methodology in sports injury research.1–6 Indeed, proper methodology is necessary for understanding why sports injuries develop, how best to prevent them and which therapeutic interventions will be most effective. Without correctly applying and interpreting statistics, subjective intuitions could lead to incorrect conclusions. In this editorial, we present seven common ‘statistical sins’ made in research and discuss how to present research findings in such a way as to help athletes, coaches and clinicians avoid drawing flawed conclusions when attempting to interpret causality in sports injury research. The sins have been adapted from an article originally published in The Conversation.7

Sin #1: trusting coincidence

Did you know that NFL teams with an animal team logo (eg, Denver Broncos and Carolina Panthers) have a dramatic 15% reduced risk of concussions compared with NFL teams without animal logos (eg, Tennessee Titans and Pittsburgh Steelers)?8 If one looks hard enough, apparently interesting associations and spurious correlations between phenomena can be found almost everywhere. However, simply because two things happen to change in parallel, or follow a similar pattern, does not mean they are causally related.

To avoid this sin, one must thoughtfully consider whether the association is likely to be causal or non-causal. Most sports injury studies simply examine non-causal associations, which are sometimes wrongly interpreted as being causal. Think of concussions and animal logos next time you read a sports injury article: are the reported relationships likely to be causally related? To better delineate causation from association, recent work in the sports injury research context has highlighted the need to prioritise training load in causal analyses.3 9

Sin #2: getting causation backwards

When two sports injury-related factors are associated, for example, foot pronation and sport injury, it might be tempting to see an ‘obvious’ causal path such that foot pronation is associated with sports injury development. However, sometimes the causal path goes in the opposite direction. Possibly, sports injury leads to certain movement patterns such as foot pronation.

You can avoid this sin by remembering to think about the direction of cause and effect when you see an association.10 In research, temporality is important: the effect has to occur after the cause.11 Ask yourself: which was there first, the chicken or the egg? Could the influence go in the other direction? Alternatively, could it go both ways, creating a feedback loop over time? This is especially important when using particular study designs, such as cross-sectional or traditional case–control.12

Sin #3: forgetting to consider multifactorial nature of sports injury development

People often fail to evaluate the multifactorial nature of sports injury development.10 Exposures are often viewed as confounders, and multiple covariates are included in the regression model using a stepwise selection algorithm. This places an over-reliance on p-values. This is not a good practice if the aim is to estimate causal effects.3 13 In sports injury science, more attention towards the use of causal diagrams (eg, directed acyclic graphs) is required for visualising complex relations between variables, since it is rarely the case that only one exposure influences sports injury aetiology, prevention or treatment.14

You can avoid this sin by remembering to think about the multifactorial nature of sports injury development whenever you see an association. Could other exposures influence the causal relationship in different ways beyond simple confounding (ie, mediators or effect-measure modifiers)?3 15

Sin #4: avoiding absolute measures of association

Researchers should choose their statistical analyses based on the type of injury data. For the most part, data on sports injuries—whether based on a time loss, medical attention or severity-based definition—rely on dichotomous (eg, injury yes/no) or categorical (eg, no injury, minor injury and substantial injury) distinctions. The informed reader should be aware of the pitfalls related to proportion-based and rate-based measures of association. Briefly summarised, drawing conclusions based solely on relative measures of association is not good practice, because this approach cannot account for the true, clinically meaningful differences between athletes who sustain injury and those who do not.2 Therefore, to avoid this sin, absolute measures of association, despite rarely being featured in sports injury research, should be used and given priority whenever possible.2 This should also be given priority in reliability studies where, for example, intraclass correlation and Pearson correlation, as measures of relative reliability, should, at times, be replaced by measures of absolute reliability, such as the Bland-Altman limits of agreement.16

Sin #5: relying on p-values, not considering minimal relevant differences

How do you know if a study’s results are important to your team, your patients or your community? It is a common belief that a given statistically significant association is equivalent to a clinically significant one. However, a statistically significant difference in injury risk does not necessarily mean that a meaningful target for sports injury prevention has been identified. Relying on p-values alone is not good practice.4 13 You can avoid the p-value sin by demanding to see the estimate and 95% confidence interval, since it provides more information than a single hypothesis test. Given this information, you must consider whether you consider this effect greater than the ‘minimal relevant difference’.2

Sin #6: deceptive graphs

A lot of mischief occurs in the scaling and labelling of the axes on graphs.17 The labels should show the full meaningful range of injury risk or rate values. Sometimes a graph designer chooses a narrower range to make a small difference or association look more impactful. On a scale from 0 to 100, two columns might appear to have similar y-values (figure 1). Contrastingly, if you graph the same data and only show a range of 32.5–33.5 on the y-scale, they might look drastically different. You can avoid this sin by taking care to note the appropriate labels and label spacing for each axis. Be especially sceptical of unlabelled graphs.

Figure 1

Hypothetical example demonstrating the impact of varying y-axes. Graphs tell a story, which can be made more or less impressive depending on scale. Imagining that the y-axes represent the proportion of athletes with injuries, compare the ‘Training program A’ versus ‘Training program B’ groups in the two graphs.

Sin #7: not considering subgroup differences while discussing training dose

Overweight individuals sustain more injuries than normal-weight athletes with a similar training load.3 18 Importantly, this does not imply that overweight athletes, as a group, should be asked to train less. Indeed, overweight athletes included in a specific study are, on average, at an increased risk. Nevertheless, certain subgroups of overweight athletes may be able to tolerate similar, and perhaps even higher, training doses compared with normal-weight individuals.3 Consequently, interpreting an average ‘overweight’ effect into injury preventive training advice for an individual is exceedingly difficult. To avoid committing this sin, consider the possibility of subgroup differences. Unfortunately, only few studies within sports science report stratified subgroup analyses using other covariates as effect measure modifiers, likely because of sample size restrictions. Nevertheless, this lack of data is an obstacle that the sports science community must overcome to improve the applicability of our research to clinical practice. In the future, researchers should consider gathering large-scale data, which allows for subgroup analyses.

Conclusion

Committing one of the ‘seven statistical sins’ is easy to do in sports injury research. We have, therefore, briefly explained how practitioners and readers of science can avoid falling victim to statistical misrepresentation when interpreting articles in the sports injury research context. We encourage researchers who produce and disseminate findings for the benefit of athletes and sport participants, coaches, parents, sport administrators and clinicians to carefully consider the content presented in this editorial when conducting investigations or reviewing their next paper. After all; methods matter.6

References

View Abstract

Footnotes

  • Contributors RON, CMC and WRL composed the first draft. All coauthors worked on revisions for major intellectual content.

  • Competing interests None declared.

  • Provenance and peer review Not commissioned; externally peer reviewed.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.