Article Text

Download PDFPDF

A higher sport-related reinjury risk does not mean inadequate rehabilitation: the methodological challenge of choosing the correct comparison group
  1. Ian Shrier1,
  2. Meng Zhao2,
  3. Alexandre Piché2,
  4. Pavel Slavchev2,
  5. Russell J Steele2
  1. 1 Centre for Clinical Epidemiology, Lady Davis Institute, Jewish General Hospital, McGill University, Montreal, Québec, Canada
  2. 2 Department of Mathematics and Statistics, McGill University, Montreal, Quebec, Canada
  1. Correspondence to Dr Ian Shrier, Centre for Clinical Epidemiology, Lady Davis Institute, Jewish General Hospital, 3755 Cote Ste-Catherine Road, Montreal, QC H3T 1E2, Canada; ian.shrier{at}mcgill.ca

Abstract

Previous injury is a well-established predictor of subsequent injury in sports medicine. Some have interpreted this to mean that either our current methods of rehabilitation are inadequate or there is some permanent damage to the tissue and 100% rehabilitation is not possible. In 2011, we illustrated that these analyses and interpretations failed to account for the fact that some athletes are more prone to get injured, either physiologically, or because of their role/type of play. We suggested that the appropriate analysis would simply require using statistical methods that measured how each individual athlete’s risk changed from preinjury to postinjury.

In this paper, we revisit our recommendation and illustrate that it too would be flawed if the risk of injury changed over time independent of an injury ever occurring. This might be expected if general fitness were to decline over the season, or if the style of play changed between early season games and postseason championship games. Acknowledging that risk may change regardless of whether an injury occurred or not leads to three different general definitions of 100% rehabilitation: (1) a return to the baseline state, (2) a return to the immediate preinjury state and (3) a return to the state that would have been present had the initial injury never occurred. We guide the reader on how to estimate the risks for each definition and the assumptions that must be acknowledged.

  • recurrent injury
  • rehabilitation
  • analysis
  • epidemiololgy

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Introduction

Previous sports injury is associated with an increase in the risk of subsequent injury.1–3 This is based on prospective studies that compared the risk of a subsequent injury among all injured participants with either the risk of first injury among all participants or the risk of first injury among uninjured participants. Some have interpreted these results to mean that current rehabilitation techniques are insufficient or that any injury results in permanent damage that increases the risk of future of injury.4 5 In this paper, we explain why these interpretations are not appropriate.

One of the key methodological assumptions for determining causal effects is that the two groups being compared have similar prognoses6—the probability of injury in this context. In 2011, we explained that the traditional analyses used in previous studies failed in this regard, and that the previous studies should not be interpreted as supporting causal links between previous injury and subsequent injury.7 In brief, consider that in every sport, different participants have different inherent risks of injury before the start of the season due to their position, fitness, style of play and so on. At the start of the season everyone is uninjured. The participants who become injured are quite possibly the participants who had a higher preseason risk of injury. When we estimate the risk of reinjury among those who suffer their first injury (‘index case’), we eliminate the vast majority of the participants who had the lowest risk of injury at the beginning of the season. Therefore, the ‘average’ risk of reinjury for participants who were injured at least once will in most cases be higher than the group at the start of the season (which included those with the lowest risk of injury) regardless of whether rehabilitation was 100% successful or not. When we examine players who have had two previous injuries, we have likely eliminated the majority of the participants with the lowest risk of a second injury, among all those who had only one injury. Again, the risk of reinjury will always appear increased even if rehabilitation were 100% successful.

Our 2011 paper also provided some simple analyses (and explanations of their underlying assumptions) that could help establish if previous injury caused a subsequent injury in order to allow more appropriate analyses for those investigators who do not have access to the advanced statistical expertise necessary for more complex statistical methods.8–11 In brief, one should compare the risk of injury in the same participants before and after their first (or second, etc) injury. In the real-world data we analysed in our previous paper, a previous injury did not increase the risk of injury, which is consistent with 100% rehabilitation.7

All of these previous studies, including ours, equated risk of injury and state of health. However, we have recently argued that a better approach for causal inference is to separate these concepts and operate within a multistate framework in which participants move from one state to another, and we measure the risk associated with transitions between particular states.12 In brief, in the multi-state framework for the analysis of subsequent injury in sport (M-FASIS) approach, the athlete’s state has several different components. There is a component related to (1) the athlete’s health, which is the effect of injury on the damaged tissue, functional capacity (eg, proprioception) and general fitness, (2) activity level and (3) whether they are having treatment. The combination of these factors leads to a particular athlete state, and each state has an associated risk of reinjury.

This may appear to be only a slight nuance, but it has important implications for subsequent injury analyses. Most relevant here, it highlights the challenges that occur when states change over calendar time. For example, the question addressed in this paper is: what is the appropriate analysis to determine if rehabilitation was sufficient? By definition, the athlete is no longer receiving treatment when the clinician believes rehabilitation is complete, and we can ignore component 3. Our question really is focused on whether the damaged tissue and functional capacity (parts of component 1) have returned to preinjury levels. In the multistate model, it is obvious that if the activity level has changed, the risk will be different even if rehabilitation has restored the health state to normal function. This might be expected to occur over the course of a season as one moves through the regular season due to changes in style of play (our definition of state included activity level) as teams vie for a ranking that allows them to compete in play-offs and championship titles.

The purpose of this paper is to illustrate how previous analyses on the causal effects of previous injury, including our own, did not account for these additional challenges and are thus prone to bias in three particular contexts. After outlining those contexts, we illustrate some conceptual methods for best practice for future analyses. Although not perfect, the practices we recommend represent a substantial improvement over current methods. For all of these examples, we consider 100% rehabilitation to mean that the athlete’s health state has returned to its preinjury level.

Three contexts

1. Bias due to changes in activity level

Figure 1 illustrates an example of the first mechanism of bias, which applies to all injuries regardless of whether there is associated time loss or not. The top panel illustrates how the activity state and health state change over time for all uninjured athletes from the beginning of season in September until an injury occurs in November. The corresponding change in risk is shown in the bottom panel. Even though there is no injury before November, the change in activity over time leads to a change in risk over time.

Figure 1

The top panel is a schematic example of how activity state (style of play) may change over time from preseason (September) to mid-season (November) when an athlete has their first injury. In this example, the health state of the athlete who is injured remains constant between September and November. The bottom panel illustrates how the calculated risk of injury would be expected to show a corresponding change in injury risk over the same time period. The difference between the risk immediately prior to the injury, and the baseline risk, represents an increased risk due to the changing style of activity. Any inference about the effectiveness of rehabilitation following injury that is based on calculations of injury risk needs to account for this non-injury-related increase in risk.

Now, consider three identical athletes with identical health states at the beginning of the season, and in November. One athlete plays and is never injured. Consider further that a second athlete scrapes her skin, and we will define this as an injury where the risk of subsequent injury is not increased even without rehabilitation. Therefore, the ‘injured athlete’s health state’ remains identical to the non-injured athlete’s health state. The third athlete suffers an injury where the risk of subsequent injury would be increased without rehabilitation. However, the athlete receives one rehabilitation treatment immediately and is miraculously cured, that is, 100% rehabilitation and no time loss. In previous sports medicine analyses, as mentioned in the Introduction section, the reference risk was the risk of injury at baseline; the traditional approach used the injury risk for non-injured athletes. In this example, the injured athletes have a higher risk of reinjury in November compared with the injury risk for non-injured athletes at baseline. Further, we previously recommended comparing the reinjury risk of injured athletes to the baseline risk restricted to the same injured athletes. In the current example, we would still observe an increase in injury risk for both of the injured athletes (injury not requiring rehabilitation and injury requiring rehabilitation). In the previous injury analyses, the results from both of these examples would be interpreted as inappropriate rehabilitation. However, because the injury risk after November is identical in the injured and non-injured athletes, it is clear that the increased risk is not due to the injury nor to inappropriate rehabilitation.

2. Bias due to changing health state

Figure 2 illustrates a second mechanism of bias by extending the lines from figure 1 to cover the period from November to January. This bias also applies to all injuries regardless of whether there is associated time loss or not. In this pedagogical example, we have kept the activity state constant between November and January. We did this because we want the readers to focus their attention only on the changing health state, and therefore we hold everything else constant.

Figure 2

Figure 2 extends figure 1 to cover the time period from mid-season (November) to late season (January). In this pedagogical example, the activity state is constant during this period and the health state deteriorates due to a general decrease in fitness. The bottom panel also builds on figure 1 and illustrates the change in risk due to deteriorating health status from November to January.

We now introduce the concept that athletes often start the season after months of rest and in peak general fitness. During a season with frequent games and travel, their general fitness may deteriorate and decrease their health state. To be consistent with figure 1, we kept fitness constant from September to November. After November, we assume a decrease in general fitness. Once again, all three of our athletes (non-injured athlete, injured athlete not requiring any rehabilitation and injured athlete who has 100% rehabilitation with one treatment on the day of the injury) will have an increased risk in January, above the already (biased) increased risk of injury due to the changing activity state that occurred between baseline and November. Any conclusion that this additional increased risk is due to inappropriate rehabilitation is clearly a misinterpretation, as it occurs in injured and non-injured athletes.

3. Bias due to time missed

In figures 1 and 2, we assumed that the athletes never missed any games. We now consider a more complex but real-life context. We will assume that the activity state changes throughout the season and that fitness deteriorates throughout the season in competing athletes, meaning the health state deteriorates throughout the season.

In figure 3, we illustrate the changes in health state for three athletes who are otherwise identical at baseline: athlete #1 plays the season uninjured (solid line), and their health state deteriorates over time due to changes in fitness. Athlete #2 stops playing games between November and January for reasons not related to injury (dashed line), and their health status remains constant between November and January since they are able to continue appropriate fitness training exercises. Athlete #3 has a significant injury in November that reduces their health state dramatically (dotted line). This athlete stops playing in November, has rehabilitation therapy from November to January and returns to the identical state of the non-injured athlete who simply stopped playing between November and January without an injury. Finally, the grey dash–dot line in figure 3 illustrates that the activity level of all non-injured playing athletes increases over time.

Figure 3

The top panel of figure 3 is similar to figures 1 and 2. The grey dashed–dot line indicates the changing activity state of athletes playing the game, which now increases gradually from the beginning of the season (September) through to late season (January). The solid line (athlete #1: uninjured and always playing) illustrates the changing health state of an athlete who is never injured and plays throughout the season. The dashed line (athlete #2: uninjured but missed playing time) illustrates the changing health state of an athlete who stops playing in November for reasons unrelated to injury. Therefore, the health state remains constant until January. The dotted line (athlete #3: injured with 100% rehabilitation) illustrates the changing health state of an athlete who is injured in November. It mimics the uninjured athlete until injury, then deteriorates due to injury and gradually returns to the same state as the uninjured athlete who stopped playing in November for reasons other than injury. The middle panel refers to risk of injury for athlete #1. It shows how the different corresponding components for the non-injury-related risk change over time between September and January given the changing activity and health state of this uninjured athlete described in the top panel (see text for details). The bottom panel refers to risk of injury for athletes #2 and #3. The change in injury risk from September to November is the same as athlete #1. There is no risk from November to January because these athletes are not playing. In January, the corresponding risk for changing health is less than athlete #1 because athlete #2 rested and athlete #3 had appropriate rehabilitation and rest. The corresponding risk for change in activity is the same as athlete #1 (see text for details).

In this context, we would observe that the two athletes who missed games (athletes #2 and #3) have the same health status and activity on returning to play in January. Therefore, as in figures 1 and 2, any increased risk observed for these two athletes (one injured and one not injured) versus a comparison group could not be due to injury. The middle panel of figure 3 illustrates the different components for an observed increased injury risk if we compared athlete #1 to non-injured athletes at baseline, and the bottom panel shows the corresponding increased injury risks if we compared athletes #2 and #3. First, between September and November, there is an increased risk due to changing activity and decreased fitness before the injury occurred (preinjury) in all three athletes. Second, there is an increased risk during November and December for continuously playing athletes (athlete #1) due to lack of recovery time and a further decrease in fitness. Athletes #2 and #3 are not playing during this time so there is actually no risk of injury at all.

We can also compare the risk of injury in January for athletes #2 and #3 that ‘Missed Games’, to the risk of injury in athlete #1 that never got injured (‘Always Playing’); all three athletes had the same baseline risk at the beginning of the season, and the same risk due to activity change (because all are playing at the same level in January). However, athletes #2 and #3 have a lower risk of injury due to health status compared with athlete #1 because they were able to rest and appropriately train while athlete #1 had to compete regularly. If one compared a group of athletes resembling the injured athlete #3 with a group of athletes resembling uninjured but competing athlete #1, the lower risk of total injury in athlete #3 could lead to the erroneous conclusion that rehabilitation for the injured athlete #3 was more than 100%.

We can expand this concept further and suggest that 100% rehabilitation should also include a return to the same general fitness level that existed at baseline. This is logical if the athlete cannot play for an extended period of time and can do fitness training despite their injury. However, fitness training is not always possible (eg, knee injury) and it highlights the complexity of return to play decision making. Should we restrict an athlete from playing if their injury is healed, but their general fitness has not returned to preinjury levels, even if their fitness level is similar to those who were never injured (athlete #1, ‘Always Playing’ risk in figure 3)? Or do we require their fitness to be at the level it was at the time just before the injury or at baseline? If one followed such criteria, to be internally consistent, we would have to also restrict any athlete from playing for poor fitness near the end of the season even if they were never injured.

Mitigating bias where possible: what  is the correct comparison group?

As outlined in our above discussion and illustrated in figures 1–3, it is essential to be more precise about what 100% rehabilitation means. According to the M-FASIS approach,12 there are three logical options:

  1. a return to baseline health state

  2. a return to preinjury health state

  3. a return to the state that would have occurred had there never been an injury.

Because each of these options is different, an appropriate analysis would need to use a different control group for each one.

Return to baseline health state

For this definition, one would simply use our previously recommended analysis,7 where time to subsequent injury in injured athletes is compared with time to first injury from these same athletes.

Return to preinjury health state

From the data, we can measure total injury risk. Total injury risk is the sum of the risk due to health state and the risk due to activity state (eg, playing style). If we can estimate the increased risk due to changing activity since the injury, we can subtract it from the measured total increased risk to obtain an estimate of the increased risk due to the change in the athlete’s health state (ie, did rehabilitation return the health state to preinjury levels?).

Any estimate of the increased risk due to changing activity requires important but hard to verify assumptions. Failing to acknowledge these assumptions means the past inappropriate inferences are likely to persist. Below, we outline two possible approaches to estimate the increased risk due to changing activity.

One approach is to obtain an estimate of the minimum change in risk over time by examining how risk of first injury changes over the course of the season. If we assume that different athletes have different risks of injury, the athletes that get injured early in the season are those that are most likely to get injured. As these high-risk athletes are removed from the population, the average risk of injury for the remaining athletes is expected to decline. Therefore, if the entire population’s risk of first injury was truly constant over the season (participants’ risks may vary between each other as long as each participant’s risk is constant over time), we would expect our calculation for the risk of first injury to be lower in November than it was at the start of the season. Therefore, if we observed an increased risk over time, it is likely due to actual increased risk due to activity or changing fitness. Further, this increased injury risk over time is likely an underestimate because we are comparing the risk of less injury-prone athletes mid-season to the risk of more injury-prone athletes early in the season, instead of their own risk early in the season. Although we might be tempted to conclude the estimated risk represents the minimum that injury risk changed over time, it is not entirely correct. New athletes replace the injured athletes, and so there must be additional assumptions about how the risk of first injury in the replacement athletes compares with the risk of first injury in those who start the season.

A second approach would be to assume that the injury risk for the same activity state among replacement athletes is the same as the injury risk for the season starters (because the risk associated with the position is the same) after adjusting for other known confounders of injury. Under this assumption, any difference between the time to first injury among all starting athletes that got injured and the time to first injury among all the replacement athletes that started to play later in the season must be due to changing activity style, that is, it represents the non-injury-related risk.

In the final step, the total risk among injured athletes can be divided into non-previous injury-related risk and previous injury inadequate rehabilitation-related risk. To obtain the previous injury inadequate rehabilitation-related risk, one simply subtracts the non-injury-related risk (estimated in part using replacement athletes) from the total risk in injured athletes. To determine if the risk of subsequent injury is increased, one simply compares this previous-injury-inadequate-rehabilitation risk to the baseline injury risk of these same athletes.

Return to health state if no injury had occurred

An analysis to address the risk associated with the state that would have occurred had there never been an injury requires all the assumptions necessary to address changing activity state over time, plus additional assumptions to account for changing general fitness over time. To assess changes in general fitness over time, one could compare injury risk among different athletes with different values on general fitness tests at baseline, and at different times of the seasons, always adjusting for activity state and other known confounders. The challenge in this analysis is significant because the confounding factors change over time, and specialised statistical methods would be required.13

In the final step, the total subsequent injury risk among injured athletes is the sum of the increased injury risk due to (1) baseline risk, (2) changes in activity style (estimated from Return to preinjury health state section), (3) expected changes in general fitness had no injury occurred and (4) changes in health state due to previous injury inadequate rehabilitation. To determine if the health state returned to the appropriate level with rehabilitation, we simply compare this estimated risk with the observed risk in athletes who returned from injury. If rehabilitation were 100% (ie, point (4)=0), the two values should be the same.

Other considerations

Real-world data originate from contexts that are much more complex than our idealised vignettes. A return to preinjury state may include factors beyond strength, range of motion, balance and psychology. Some have suggested that the risk of injury may be increased if the activity level of the sport represents a large increase over the recent workload during rehabilitation.14 In our M-FASIS model12 that serves as the foundation for our analyses, this concept would be incorporated because the health state includes the level of activity as one of the components; a large increase in activity represents a different health state. This is important because ‘tests of physical capacity’ will never be fully representative of the physical stresses the athlete will incur once returning to the actual sport and return to play decision making should include concepts related to work overload.

In summary, we have illustrated why previous analyses suggesting that increased risk of subsequent injury compared with risk of first injury should not be interpreted to mean that rehabilitation was less than 100%. Further, investigators need to clearly define what 100% rehabilitation means in each paper. We believe that each of the definitions, (1) return to baseline, (2) return to preinjury risk or (3) return to injury risk that would have occurred had the injury never happened, are all potentially appropriate for particular contexts, and each would require a different analysis.

Key messages

What are the new findings?

  • Research questions that use injury risk pre- and postinjury to evaluate the success of rehabilitation should account for the fact that injury risk may change over time independent of injury.

  • One hundred per cent rehabilitation could meaningfully be defined as (1) a return to the baseline state (in many cases ‘preseason’), (2) a return to the immediate preinjury state or (3) a return to the state that would have existed had an injury not occurred.

  • Evaluating whether 100% rehabilitation occurred using any of the three definitions provided requires important but hard to verify assumptions.

How might it impact on clinical practice in the near future?

  • Some clinicians should become reassured that the research data do not mean they have not been treating appropriately.

  • Return to play decisions should become more transparent as clinicians realise that the definition of 100% rehabilitation requires judgements that need to be explicitly discussed.

References

Footnotes

  • Funding This study was funded by the Collaborative Health Research Program, which is a joint programme between the Canadian Institutes of Health Research and the National Science and Engineering Research Council of Canada.

  • Competing interests None declared.

  • Provenance and peer review Not commissioned; externally peer reviewed.