Injuries are often recurrent, with subsequent injuries influenced by previous occurrences and hence correlation between events needs to be taken into account when analysing such data.

This paper compares five different survival models (Cox proportional hazards (CoxPH) model and the following generalisations to recurrent event data: Andersen-Gill (A-G), frailty, Wei-Lin-Weissfeld total time (WLW-TT) marginal, Prentice-Williams-Peterson gap time (PWP-GT) conditional models) for the analysis of recurrent injury data.

Empirical evaluation and comparison of different models were performed using model selection criteria and goodness-of-fit statistics. Simulation studies assessed the size and power of each model fit.

The modelling approach is demonstrated through direct application to Australian National Rugby League recurrent injury data collected over the 2008 playing season. Of the 35 players analysed, 14 (40%) players had more than 1 injury and 47 contact injuries were sustained over 29 matches. The CoxPH model provided the poorest fit to the recurrent sports injury data. The fit was improved with the A-G and frailty models, compared to WLW-TT and PWP-GT models.

Despite little difference in model fit between the A-G and frailty models, in the interest of fewer statistical assumptions it is recommended that, where relevant, future studies involving modelling of recurrent sports injury data use the frailty model in preference to the CoxPH model or its other generalisations. The paper provides a rationale for future statistical modelling approaches for recurrent sports injury.

Sports injuries are often recurrent in that some people experience more than one sports injury over time. There is wide recognition that subsequent injury (of either the same or a different type) can be strongly influenced by previous injury occurrences.

A key statistical challenge inherent in analysing recurrent injury data is that the probability of injury occurrence is likely to be influenced by earlier injuries, even when they are not of exactly the same type; this can be manifest as an injury either raising or lowering the rate of further injury. This is important because analyses that incorrectly treat different within-person injuries as statistically independent run the risk of generating misleading results. Ignoring potential within-person event dependency leads to reported greater precision than is warranted and possible biasing of results away from the null. A second statistical issue is that many naïve statistical approaches implicitly restrict the baseline probability of injury, and the influence of covariates on this, to be the same across all injuries when, in fact, they vary across people and different injury types. Across people, this variability implies that some will have inherently higher or lower rates of different subsequent injuries. Together, these statistical issues mean that in any recurrent injury dataset there will be different within-person correlations across people and that the within-person injury times will be dependent. Any correlation among injuries (whether produced by event dependence or variability) will violate assumptions that the timing of injuries is independent, and result in problems of estimation and incorrect inference if not properly taken into account.

Despite many studies documenting the incidence of sports injuries, and recognition of the recurrent nature of many injuries,

Several event history model variations based on the Cox proportional hazards (CoxPH) model

In the general recurrent event literature, extensions of the CoxPH model are popular because they enable all events for each individual to be analysed. Application of four prominent regression models (Andersen-Gill (A-G),

The aim of this paper is to (a) summarise the issues that need to be considered when modelling recurrent sports injury data where the time before/between injuries is of interest and (b) to assess and compare the suitability of the CoxPH model and its extensions for modelling such data. The methods and model comparison are demonstrated on Australian National Rugby League (NRL) injury data to provide defensible guidance on how to appropriately model recurrent sports injury data.

To demonstrate and compare the applicability of different extensions of the CoxPH model to a real-world data example, injury data were obtained on 35 players from a professional rugby league club competing in the 2008 Australian NRL competition. Injury and participation data were collected from 29 matches (including all trial, fixture and finals matches). Injuries were defined as conditions associated with pain or disability that occurred during match participation, irrespective of the need for first aid, medical attention or time loss.

The CoxPH model and four recurrent event generalisations were applied to the sports injury data. The time variable was taken to be the match number (range 1–29). Major statistical challenges with this sort of data are how to address the number of recurrent events and the number of players at risk appropriately. Four components were considered for the recurrent event model:

Statistical specifications and assumptions in relation to the risk interval, risk set, baseline hazard and within-person correlation in the extended Cox proportional hazards (CoxPH) models

Components | Andersen-Gill (A-G) | Frailty | Wei-Lin-Weissfeld total time (WLW-TT) marginal | Prentice-Williams-Petersen gap time (PWP-GT) conditional |
---|---|---|---|---|

Risk interval | Duration since starting observation | Duration since starting observation | Duration since starting observation | Duration since previous injury |

Risk set for injury k attime t | Independent injuries (any given injury occurrence is not affected by previous injuries) | A random effect (or frailty) term is used to account for the within-player correlation between injuries to enable modelling of the phenomenon by which some players are intrinsically more or less susceptible to experiencing a given injury than others are | All players who have not experienced injury k at time t | All players who have experienced injury k−1, and have not experienced injury k at time t |

Baseline hazard | Common/same baseline hazard across all injuries | Heterogeneity is directly incorporated via a random effect so that the baseline hazard is allowed to vary with each injury | Common baseline hazard for all injuries within a player | Stratifies the data by injury so that the baseline hazard is allowed to vary with each injury |

Within-person correlation | The within-person injuries are independent | Captures within-person correlation due to both injury dependence and heterogeneity | The within-person injuries are independent | The current injury is unaffected by earlier injuries that occurred to the player |

Comment | A-G model is recommended when there is no injury dependence and no covariate/injury effects | The frailty approach accounts for heterogeneity. The random effect (the frailty) has a multiplicative effect on the baseline hazard function and the mixture of individuals with different injury risks | At any time point (matches), WLW-TT describes all players who have not yet experienced k injuries are assumed to be at risk for the kth injury which is not realistic in the sports setting injury data | PWP-GT model takes into account the ordering of events |

The following three risk intervals were considered: (a) gap (or interoccurrence) time representing the time from the prior injury event and not relative to the actual timeline of observation; (b) calendar time which uses the same timescale for all events, referenced to a fixed point in time, but does not allow an overlap in risk periods across events for a given player; and (c) total time representing the time from the start of the player follow-up. In each case, the interval ends with the current injury. In both the gap time and calendar time representations, the player is at risk for the same length of time. Gap and calendar time models are conditional since a player is at risk of a new injury, conditioned on having sustained a previous injury. For total time, the clock does not reset for each event and the beginning of each event is at the same point in the observation timeline; risk periods for different events for the same player overlap. Total time models are marginal since the player is at risk from the start of play, independent of any previous injury. Irrespective of the definition, the risk interval for the first injury is the same.

Illustrations of the risk interval formulations: (A) three players with recurrent injuries; (B) gap time; (C) calendar time; (D) total time. A circle (•) indicates an injury event and a solid square (▪) indicates censoring. Each time to an event or censoring is a separate risk interval.

The Kaplan-Meier (K-M) method is used to estimate the survival function non-parametrically from observed (censored and uncensored) survival times.

The A-G model is a simple extension of the CoxPH model where players contribute to the risk set for an event as long as they are under observation at the time the injury occurs and share the same baseline hazard function. However, the A-G model requires the strongest statistical assumptions including that of an independent increment in which any given injury occurrence is not affected by previous injuries, that is within players, injuries are independent. This restriction means that injury dependence cannot be included and the A-G model inherently assumes that injuries do not change the player and that the player does not learn from previous injuries. Moreover, this model does not allow investigation of effects that might change based on injury-specific covariate effects, but there is the possibility of incorporating injury dependence via time-dependent covariates. Given these limitations, the A-G model is recommended when there is no injury dependence and no covariate/injury effects.

Analysis of recurrent injury data frequently assumes that the player injury histories are all statistically independent (at least conditionally on observed time-fixed covariates) so that the interoccurrence times appear in an independent manner. However, some players are intrinsically more or less susceptible to experiencing an injury than others. The frailty model is characterised by its inclusion of a random effect, or frailty, term can account for the within-player correlation between injuries. If the frailty is less than 1, a player tends to experience the injury at a later time than another player, whereas the opposite occurs if the frailty is greater than 1.

The WLW-TT model is a marginal model and assumes a common baseline hazard for all injuries within a player. Marginal models consider the marginal distribution of each failure time and impose no particular structure of dependence among distinct failure times on each player. Each recurrence is modelled as a different stratum and each stratum is treated as marginal data. This model is marginal with respect to the risk set since each player is at risk from the beginning of the study and can be at risk for several events simultaneously.

The PWP-GT model is a conditional model which allows for event dependence via stratification by event number so that different events can have different baseline hazards. The main difference to the marginal model is that a player cannot be at risk for the later injury until a prior event occurs. This conditional model preserves the order of sequential injuries in the creation of the risk set and therefore incorporates injury dependence. The PWP-GT model is estimated with the data organised in interoccurrence/gap time (ie, gap time risk set or time since the previous injury).

The outcome being modelled is the probability of remaining injury-free over the 29 matches. As shown in

K-M curve representations of the observed probability of remaining free of injury were used to provide a visual comparison of each model fit. The log likelihood (LL), Akaike information criterion (AIC) and Bayesian information criterion (BIC) were used to compare the goodness of fit of the fitted models in terms of fitting the observed data.

The most common criterion for evaluating the performance of a statistical model is its accuracy in terms of data fit. In this sense, the model accuracy is an assessment of the closeness of estimates to the exact (or observed) value and can be computed on a point-by-point basis. The most widely used measures of accuracy are the mean-squared error (MSE), the root MSE, the mean absolute error and the mean absolute percentage error.

Three test criteria were used to compare the fitted models: likelihood ratio (LRT), F

As an example, the LRT begins with a direct comparison of the likelihood scores of the two models and tests whether the frailty is necessary for analysing recurrent sports injury events. A significant LRT suggests that a random effect (frailty) accounts for the within-player correlation between injuries. A similar approach is used for the F-ratio test.

In the bootstrapping procedure, a large number of random samples are generated.

Finally, a simulation approach for calculating the size and power of the models was undertaken.

Overall, 47 contact injuries were sustained by the 35 players during a total of 557 player appearances. The median follow-up time was 18 matches (range 1–29 matches). More than half of the players (54.3%) sustained 1–6 injuries, with 40% sustaining >1 injury over the 29 matches (

The distribution of number of injuries sustained by 35 National Rugby League players, the respective number of matches with a number of injuries and the injury incidence rates per 1000 matches

Number of injuries | Number of players | Total number of injuries | Proportion of players | Total number of matches | Injury incidence rates |
---|---|---|---|---|---|

0 | 16 | – | 45.7 | 134 | – |

1 | 5 | 5 | 14.3 | 107 | 46.7 |

2 | 7 | 14 | 20.0 | 133 | 105.3 |

3 | 2 | 6 | 5.7 | 55 | 109.1 |

4 | 4 | 16 | 11.4 | 108 | 148.1 |

5 | – | – | – | – | – |

6 | 1 | 6 | 2.9 | 20 | 300.0 |

Total | 35 | 47 | 557 |

Recurrent injury history of 35 professional rugby league players. The event of interest is any contact-injury sustained by a player, which is denoted by a circle (○). Censored data which arise when the outcome injury status is either not-injured or unknown is denoted by solid squares (▪).

Standard Kaplan-Meier (K-M) curves for probability of remaining free of injury for 35 professional rugby league players. Actual and fitted survival curves from (A) CoxPH model, (B) A-G model, (C) frailty model, (D) WLW-TT model and (E) PWP-GT model. The grey shaded regions are 95% CIs for the fitted survival curves. Models were adjusted by age, match experience and body mass of the players.

Model selection criteria (log likelihood (LL), Akaike information criterion (AIC) and Bayesian information criterion (BIC)) of the fitted models for sports injury recurrent data*

Model | Model selection criteria | ||
---|---|---|---|

LL | AIC | BIC | |

Andersen-Gill (A-G) | 135.0 | 275.9 | 355.6 |

Frailty | 134.9 | 277.9 | 378.0 |

Wei-Lin-Weissfeld total time (WLW-TT) marginal | 158.1 | 334.2 | 487.6 |

Prentice-Williams-Petersen gap time (PWP-GT) conditional | 154.8 | 327.7 | 481.1 |

*The LL, AIC and BIC were not reported due to the small estimated likelihood for the CoxPH model for only the first event.

Mean square error (MSE), root mean-squared error (RMSE), mean absolute error (MAE) and mean absolute percentage error (MAPE) of the fitted models for sports injury recurrent data

Model | Model accuracy measures | |||
---|---|---|---|---|

MSE | RMSE | MAE | MAPE | |

Cox proportional hazards (CoxPH) | 0.04 | 0.19 | 0.15 | 0.64 |

Andersen-Gill (A-G) | 0.001 | 0.04 | 0.03 | 0.13 |

Frailty | 0.001 | 0.03 | 0.03 | 0.12 |

Wei-Lin-Weissfeld total time (WLW-TT) marginal | 0.03 | 0.18 | 0.15 | 0.64 |

Prentice-Williams-Petersen gap time (PWP-GT) conditional | 0.01 | 0.10 | 0.09 | 0.47 |

The p-values from the pairwise LRT, F and bootstrap model comparison tests are shown in

Pairwise goodness-of-fit (likelihood ratio test (LRT), F-ratio (F) and bootstrap (BS)) p-values for comparing the Cox proportional hazards (CoxPH) model, Andersen and Gill (A-G) model, frailty model, Wei-Lin-Weissfeld total time (WLW-TT) marginal model and Prentice-Williams-Petersen gap time (PWP-GT) conditional model for sports injury recurrent data

Comparison of models | Goodness-of-fit p values | ||
---|---|---|---|

LRT* | F | BS† | |

CoxPH vs A-G‡ | – | – | – |

CoxPH vs frailty | – | <0.001 | – |

CoxPH vs WLW-TT | – | 0.67 | – |

CoxPH vs PWP-GT | – | 0.99 | – |

A-G vs frailty | 0.84 | 0.50 | 0.85 |

A-G vs WLW-TT | 0.03 | 0.03 | 0.02 |

A-G vs PWP-GT | 0.20 | 0.08 | 0.14 |

Frailty vs WLW-TT | 0.02 | 0.02 | <0.001 |

Frailty vs PWP-GT | 0.03 | 0.02 | 0.01 |

WLW-TT vs PWP-GT‡ | – | – | 0.78 |

*LRT test is based on log likelihood and is not appropriate for comparing first event model (CoxPH model) and recurrent events models (Cox extension models).

†The resampling procedure was based on the CoxPH model in the BS test and hence the extended models were not fitted for first event only when compared with the CoxPH model.

‡Models are not nested.

Simulated estimates (based on 100 simulation replications) of the size and power of the test to compare Andersen and Gill (A-G) model, frailty model, Wei-Lin-Weissfeld total time (WLW-TT) marginal model and Prentice-Williams-Petersen gap time (PWP-GT) conditional model fitted to sports injury recurrent data*

Comparison ofmodels | Simulated model size | Simulated model power | ||||
---|---|---|---|---|---|---|

Pr(P>α)=α | Pr(P>β)=1−β | |||||

α=0.01 | α=0.05 | α=0.10 | 1−β=0.99 | 1−β=0.95 | 1−β=0.90 | |

A-G vs frailty | 0.02 | 0.04 | 0.08 | 0.99 | 0.99 | 0.92 |

A-G vs WLW-TT | 0.03 | 0.08 | 0.13 | 0.97 | 0.93 | 0.88 |

A-G vs PWP-GT | 0.03 | 0.05 | 0.11 | 0.99 | 0.96 | 0.93 |

Frailty vs WLW-TT | 0.02 | 0.05 | 0.10 | 0.96 | 0.90 | 0.86 |

Frailty vs PWP-GT | 0.01 | 0.04 | 0.09 | 0.98 | 0.93 | 0.89 |

WLW-TT vs PWP-GT | 0.01 | 0.04 | 0.18 | 0.98 | 0.90 | 0.80 |

*The re-sampling procedure was based on the Cox model in the bootstrap test and hence the extended models were not fitted for first event only when compared with the Cox regression model.

Knowing how to choose the best model for analysing recurrent events in sports injury settings is important for the generation of accurate and reliable information to guide priority setting for targeting of intervention investments to tackle the sports injury problem. Although there are some guidelines on how to appropriately model injury count data,

The need to correctly statistically model recurrent events occurs in many clinical trials, longitudinal epidemiological studies and sociological research.

In the sports injury literature to date, recurrent injuries have been considered from a clinical management and return-to-play (or time away from sport to recover from injury) perspective.

In the case of injury count data, sports injury counts have been most commonly analysed in the literature as Poisson counts. When players would reasonably be expected to sustain more than one injury, it would be more correct to apply negative binomial models, as we have shown when modelling falls in older people.

Although the CoxPH model is the most commonly used approach for analysing time-to-event data, it fails to take into account the extra variability of the recurrent events and, as this paper has shown, provides only poor fit to recurrent sports injury data. This is perhaps not surprising given that it only considers the time to the first injury and discards the remaining injuries. This is a critical limitation because it means that important information about injury occurrence and associated risk factors is potentially excluded from current models which only consider the time to first injury.

Each of the four tested generalisations of the CoxPH model (A-G, frailty, WLW-TT and PWP-GT models) provided a substantial model improvement over the CoxPH model. In general, the A-G and frailty models performed best and provided better data fits to the recurrent sports injury data when compared to both WLW-TT and PWP-GT models.

There was no statistical difference between the A-G and frailty models when applied to the NRL recurrent injury data analysed in this study in terms of model selection, goodness of fit or accuracy. This was confirmed with the simulation substudy. Nonetheless, as the frailty model requires fewer data assumptions than the A-G model and it does allow investigation of effects that might change based on injury-specific covariate effects (which the A-G model does not), it is recommended that the frailty model be adopted when analysing recurrent sports injury data in the future, when this is consistent with the research question.

The A-G model is the most simple variance-corrected model and incorporates robust variance estimators, which have good statistical properties under some circumstances. This model can also be used to adjust for covariate effects. The frailty model includes a random effect (frailty) to account for the within-subject correlation between injuries and so is a more general model, with fewer assumptions. In the case where there is significant within-person correlation (as applies to our injury data), Kelly and Lim

The statistical model comparison was only conducted on a small injury sample, and it is possible that different conclusions may arise when applied to other injury contexts. We have recently applied the frailty model to other rugby league injury data, including for the purposes of risk factor identification, indicating its likely robustness for this sort of recurrent injury data.

Although the frailty model has offered the best fit to the rugby league recurrent injury count data in this study, this does not guarantee that this model would offer the best fit for other sports injury data sets, and this would need further exploration. Nevertheless, the fitting procedures presented in this paper, and the various model selection criteria, may be used as guidelines for modelling recurrent injury data in other sports injury contexts.

In conclusion, sports injury data characterised by recurrent events due to repeat or subsequent injuries over a period of time need to be appropriately analysed to take into account the different likely dependences within the data. Such data can be appropriately analysed by either the A-G or frailty model, with the frailty model representing a marginally better fit than A-G model. The strength of the frailty model is that it considers individual baseline injury risks for different players, makes fewer statistical assumptions and also is able to model time-varying covariates.

A summary of the important statistical considerations when analysing recurrent injury data.

Guidance on the best statistical model to use for analysing recurrent sports injuries.

The Australian Centre for Research into Injury in Sport and its Prevention (ACRISP) is one of the International Research Centres for Prevention of Injury and Protection of Athlete Health supported by the International Olympic Committee (IOC).

All authors contributed to the concept of the paper and to subsequent drafting of all versions of the manuscript. SU undertook the statistical modelling. TJG coordinated the data collection. CFF assisted with the interpretation of the statistical models.

Dr Shahid Ullah was supported by an Injury Prevention and Safety Promotion (IPSP) Research Fellowship funded through the University of Ballarat. Professor Caroline Finch was supported by a National Health and Medical Research Council Principal Research Fellowship (ID: 565900).

None.

Obtained.

Not commissioned; externally peer reviewed.