Article Text

Adding confidence to our injury burden estimates: is bootstrapping the solution?
  1. Sean Williams1,
  2. Joseph W Shaw2,3,
  3. Carolyn Emery4,
  4. Keith A Stokes1,5
  1. 1 Centre for Health, Injury and Illness Prevention in Sport, University of Bath, Bath, UK
  2. 2 Faculty of Sport, Technology and Health Sciences, St Mary's University Twickenham, London, UK
  3. 3 Ballet Healthcare, Royal Opera House, London, UK
  4. 4 Sport Injury Prevention Research Centre, University of Calgary, Calgary, Alberta, Canada
  5. 5 Rugby Football Union, London, UK
  1. Correspondence to Dr Sean Williams, Centre for Health, and Injury & Illness Prevention in Sport, University of Bath, Bath, BA2 7AY, UK; S.Williams{at}bath.ac.uk

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Introduction

Injury burden is a composite measure of injury incidence and mean severity that can be used to understand the overall impact of injuries and help identify priority areas for injury prevention.1 Injury burden has been used within rugby union epidemiological studies since the early 2000s,2 but it is now recognised and recommended within other sports, including the most recent International Olympic Committee consensus statement for the recording and reporting of epidemiological data on injury and illness.3 Injury burden is normally reported as athlete days absence per 1000 athlete-hours and is derived from the product of injury incidence (expressed as injuries sustained/1000 athlete-hours) and severity (expressed as the mean severity of injury in days).1 While the value of injury burden as an output measure from injury surveillance studies is evident, there appears to be some confusion in the literature regarding its calculation. For instance, some authors have used median severity to calculate injury burden rather than mean severity, as discussed in a recent critical review.4 In addition, there appears to be no clear guidance within the sports medicine literature regarding the most appropriate way to calculate confidence intervals (CIs) for this metric.

Estimating uncertainty

CIs convey the uncertainty about a point estimate (in this case, an injury burden rate) and are thus crucial for interpreting results and informing decision-making. Injury burden estimates have typically been modelled using the Poisson distribution.5 Using this approach, the precision around the burden estimate is driven entirely by ‘N’ in Equation 1.5

Equation 1.

Embedded Image

Embedded Image

However, several different values for ‘N’ in Equation 1 have been used within the literature. Many studies, including the International Olympic Committee consensus statement,3 use the total number of athlete days absence, while others have used the injury burden rate itself6 or the injury count7; each approach produces a different CI width (table 1) and could therefore hugely impact on the interpretation of the burden rate. For example, using the total number of injury days as the ‘N’ value produces a CI that is extremely narrow when the total number of injury days is large, greatly increasing the likelihood of making a type I error. Of these approaches, using the injury count as the ‘N’ value appears most appropriate as it produces a CI width that changes proportionately in line with the sample size (ie, the number of injury events). Yet, there is no consensus on this in the sports medicine literature. Moreover, the appropriateness of assuming a Poisson distribution when modelling injury burden data may be questioned given the extreme positive skew that is often present, and this approach does not account for the multiple sources of random error (ie, from both injury incidence and average injury severity).

Table 1

Injury burden CIs calculated using various approaches to derive CI width

Enter bootstrapping

Bootstrapping is a statistical resampling technique that involves sampling with replacement from an original dataset (at least 1000 iterations are recommended), with confidence limits derived from percentiles of the sampled values.8 Bootstrapping enables the estimation of the sampling distribution of a statistic without making assumptions about the underlying population distribution. As such, it appears a promising solution for producing robust CIs around an injury burden estimate. Bootstrapping is now readily available for descriptive statistics in statistical programmes such as SPSS and R (see online supplemental materials for worked examples). Note, a burden value is required for each injury record in the dataset, which may be obtained by dividing each injury severity value by the athlete’s exposure time (and then multiplying by 1000 to express as days absence per 1000 athlete-hours); where individual exposure time is not available, the total/team exposure time should be divided by the total injury count. As shown in table 1, the bootstrapping method changes proportionately in line with the sample size and produces the most conservative CI width. By avoiding the need to make assumptions about the distribution from which the data were obtained, we believe the bootstrapped CIs are the most trustworthy estimates of the precision around the burden estimate.

Supplemental material

The ‘Delta Method’ is another viable solution for obtaining a valid SE that accounts for both sources of uncertainty when calculating a burden value.8 Further considerations are needed when modelling injury burden as an outcome variable in regression models (eg, the use of a negative binomial distribution and/or bootstrapping approaches to account for clustering),5 but this is beyond the scope of the current editorial.

Moving forward with confidence

The uncertainty associated with output measures from injury surveillance studies should always be presented and used to guide decisions (eg, whether a meaningful change has occurred between seasons). Although a full methodological review of the proposed methods is warranted, bootstrapping appears to provide a robust and appropriate estimate of uncertainty for burden rate estimates, and this key methodological point should be considered by sports injury researchers.

Ethics statements

Patient consent for publication

Ethics approval

Not applicable.

References

Supplementary materials

  • Supplementary Data

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

Footnotes

  • Twitter @statman_sean, @JosephShaw_, @CarolynAEmery

  • Contributors SW drafted the manuscript. KAS, JWS and CE provided input. All authors edited and reviewed the final version.

  • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

  • Competing interests None declared.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.