Article Text
PDF
Abstract
Background In randomised controlled trials (RCTs) of interventions that aim to prevent sports injuries, the intention-to-treat principle is a recommended analysis method and one emphasised in the Consolidated Standards of Reporting Trials (CONSORT) statement that guides quality reporting of such trials. However, an important element of injury prevention trials—compliance with the intervention—is not always well-reported. The purpose of the present educational review was to describe the compliance during follow-up in eight large-scale sports injury trials and address compliance issues that surfaced. Then, we discuss how readers and researchers might consider interpreting results from intention-to-treat analyses depending on the observed compliance with the intervention.
Methods Data from seven different randomised trials and one experimental study were included in the present educational review. In the trials that used training programme as an intervention, we defined full compliance as having completed the programme within ±10% of the prescribed running distance (ProjectRun21 (PR21), RUNCLEVER, Start 2 Run) or time-spent-running in minutes (Groningen Novice Running (GRONORUN)) for each planned training session. In the trials using running shoes as the intervention, full compliance was defined as wearing the prescribed running shoe in all running sessions the participants completed during follow-up.
Results In the trials that used a running programme intervention, the number of participants who had been fully compliant was 0 of 839 (0%) at 24-week follow-up in RUNCLEVER, 0 of 612 (0%) at 14-week follow-up in PR21, 12 of 56 (21%) at 4-week follow-up in Start 2 Run and 8 of 532 (1%) at 8-week follow-up in GRONORUN. In the trials using a shoe-related intervention, the numbers of participants who had been fully compliant at the end of follow-up were 207 of 304 (68%) in the 21 week trial, and 322 of 423 (76%), 521 of 577 (90%), 753 of 874 (86%) after 24-week follow-up in the other three trials, respectively.
Conclusion The proportion of runners compliant at the end of follow-up ranged from 0% to 21% in the trials using running programme as intervention and from 68% to 90% in the trials using running shoes as intervention. We encourage sports injury researchers to carefully assess and report the compliance with intervention in their articles, use appropriate analytical approaches and take compliance into account when drawing study conclusions. In studies with low compliance, G-estimation may be a useful analytical tool provided certain assumptions are met.
- Injury
- Randomised controlled trial
- Methodology
Statistics from Altmetric.com
Introduction
Randomised trials are used to investigate the association between training-related interventions (eg, two training programmes) or non-training-related interventions (eg, the use of different equipment) and sports injury outcomes.1–3 Given a truly random allocation procedure and a sufficiently large study sample, the major advantage of the randomised trial design is the equal distribution of covariates between groups even if these covariates are unobserved or unknown.4 5 In theory, all confounding is then eliminated. For this reason, randomised trials are considered one of the most robust study designs for testing preventive measures, and the observed difference between two groups is simply reported as the so-called effect of the intervention.
Analytically, sports injury researchers usually consider the following three approaches to analysing data from randomised trials; (i) intention-to-treat (ITT) analysis, which is used where data are analysed according to assigned intervention regardless of the participants’ compliance with the assigned intervention; (ii) per protocol (PP) analysis, which is used where data are analysed based on the received interventions among those who complied with the assigned intervention; and (iii) as treated analysis, which is used where data are analysed according to the intervention the study participants actually performed (as opposed to ‘received’).4
In the present educational review, we shed light on the ITT analysis because it is generally preferred to the other two. The advantage of the ITT approach is that, in theory, the groups remain comparable on all other aspects (ie, exposures) apart from the intervention, while with PP analysis, the advantage of randomisation is lost and the risk of bias because of confounding is increased. As the ITT principle requires sports injury researchers to analyse their data in accordance with athlete allocation, the researcher needs to draw assumptions regarding compliance with the intervention when interpreting the results. At times, the researcher assumes that athletes/players enrolled in the trial comply with the intervention to which they were allocated. As an example, Buist et al conclude ‘This randomised controlled trial showed no effect of a graded training program (13 weeks) in novice runners, applying the 10% rule, on the incidence of RRI (running-related injury) compared with a standard 8 week training program’. 6 Here, the assumption was that all athletes/players complied with the intervention and there was no drop-out. In this case, the results from an ITT analysis can (in the absence of bias) be interpreted as the effect (by some known as the ‘real-world efficacy’)7 8 of the intervention.
Notably, if the compliance is low (ie, drop-outs were observed or that those who remained in the study did not adhere/execute the intervention), the effect of the intervention cannot be assessed via ITT. Hence, any conclusion on the effect of the intervention, as the earlier quote by Buist et al describing the effect of the training programme, might be flawed.9 There has been informed methods papers and systematic reviews about both the definitions of compliance and level of compliance in randomised trials of sports injury prevention.10 11 In a systematic review, about 72% of the intervention studies mentioned compliance (or a related term), whereas only 51% provided compliance data and fewer than 20% of all included studies considered compliance levels in the analysis.10 To highlight the importance of reporting compliance, we carefully evaluated eight of our own previous trials with the purpose to describe the compliance during follow-up and address compliance issues that surfaced. Based on these observations, we discuss how results from ITT analyses can be interpreted according to the observed compliance.
Methods
Data from seven different randomised trials and one experimental study were included in this article as the authors have full access to all data gathered throughout the follow-up period (see table 1). In four trials/studies, the association between a training programme and running-related injury (RRI) was investigated. These four trials were: Groningen Novice Running (GRONORUN) study,2 6 RUNCLEVER,1 12 ProjectRun21 (PR21)13 and Start 2 Run (S2R).14 In four other trials,3 15–17 the association between running shoe technology and RRI was examined. In addition, all runners provided signed informed consent prior to inclusion.
Overview of the trials included in this methodological article
Injury definition in the eight trials
The outcome of interest was RRI in all studies analysed. Across trials, there were slight differences in injury definitions although all were based on a time-loss approach. For instance, PR21 used the consensus-based definition by Yamato et al 18: ‘Running-related (training or competition) musculoskeletal pain in the lower limbs that causes a restriction on or stoppage of running (distance, speed, duration, or training) for at least 7 days or 3 consecutive scheduled training sessions, or that requires the runner to consult a physician or other health professional’ whereas the RUNCLEVER trial used the following definition: ‘An injury sustained on muscles, joints, tendons and/or bones during or after running and attributed to running. The injury must have caused a training reduction (reduced distance, intensity, frequency, etc) for at least 7 days’.
Description of the trials using training program as intervention
The GRONORUN trial included 532 novice runners. At baseline, the participants were allocated into a standard 13-week start to run programme or into a progressive 8-week running programme. Detailed information of the intervention has been presented by Buist et al. 2
The RUNCLEVER study included 839 recreational runners into a 24-week follow-up. During the first 8 weeks, all runners were asked to follow the same running programme (pre-conditioning). At 8-week follow-up, the runners were randomised into a programme focusing on increasing the amount of kilometres without increasing intensity or into a programme focusing on increasing the weekly amount of kilometres run at a high intensity. The runners had to follow the programme they were allocated to during the last 16 weeks of follow-up. Ramskov et al have published a design paper including a description of the interventions elsewhere.1
PR21 was a 612-person 14-week study including all types of runners willing to follow a ½-marathon running programme. At baseline, the runners were not randomised into a certain programme. Instead, they had to self-select between three pre-defined programme: a distance-based programme, an intensity-based programme and a mixed programme. Based on this, the study design is described as experimental in this article. Although the study design was not a randomised trial, it was included in this article to visualise if the compliance with a training programme would differ between a scenario where study participants were randomised to a certain programme in the traditional way versus a scenario where study participants were able to self-select between different pre-defined programme. Damsted et al have described the study design elsewhere.13
S2R was a 4-week trial including 56 obese novice runners. At baseline, novice runners were allocated into one of the following two running programmes: a short-distance programme (3 km) or a long-distance programme (6 km). No study protocol has been published, but the results have been presented elsewhere.14
Description of the trials using running shoes as intervention
Four randomised controlled trials were designed to systematically investigate the influence of different running shoe features on the risk of RRI in leisure-time runners.
The first trial included 304 leisure-time runners in a 5-month follow-up. A stratified random allocation was performed based on age, sex, body mass index and recent regular running practice. In one group, the participants received a pair of shoes with a soft midsole, while in the second group, they received a pair of shoes with a hard midsole. Both the participants and the investigators were blinded regarding the shoe version distributed. Anonymised running shoes were provided by a renowned sport equipment manufacturer. The difference in midsole hardness between the two shoe versions was about 13%.3
In the second trial, the foot morphology of 423 leisure-time runners was analysed using the foot posture index method. A stratified random allocation was subsequently performed based on foot posture. Participants were given either the motion control or the standard version of a regular running shoe model (de-identified but commercially available) and were followed up for 6 months. Participants and assessors involved in the shoe distribution and participant follow-up were both blinded regarding the shoe allocation.16
The third trial focused on the heel-to-toe drop of standard cushioned running shoes. A total of 577 leisure-time runners were stratified according to their experience with shoes characterised by a low heel-to-toe drop (<10 mm). Based on two pre-established randomisation lists, they received a pair of shoes with a drop of 10 mm, 6 mm or 0 mm and were then followed for 6 months. Both the participants and the investigators were blinded regarding the shoe version distributed.15
In the fourth trial, 874 leisure-time runners were followed over 6 months after having received at random a pair of shoes with either a soft or a hard midsole. The difference in cushioning (global stiffness) between the two shoe types was >35%. The shoes were de-identified and both the participants and investigators were blinded regarding the shoe version distributed. The running technique of each participant was analysed on an instrumented treadmill at baseline in the study shoes. The study design was described elsewhere.17
For each of these trials, the participants were asked to use exclusively the study shoes during the whole follow-up period, and they had to specify which pair of shoes they used for every single session uploaded into the electronic system.19
Definition of compliance
The so-called ‘10% rule’ is commonly used as a guideline for a maximum training progression by runners, coaches and clinicians.20 In the trials using training programme as an intervention, we therefore defined full compliance (100%) as having completed the programme within ±10% of the prescribed running distance (PR21, RUNCLEVER, S2R) or time-spent-running in minutes (GRONORUN) for each planned training session. For instance, if the prescribed running distance was 5 km, 5 km and 7 km in the three first sessions and the running distance actually performed was 5.2 km, 4.9 km and 10 km, respectively, the runner was considered compliant in the first two sessions, but non-compliant in the third session where the distance covered was 43% longer than the prescribed distance. In addition, the participants in the PR21, RUNCLEVER and S2R trials had to run the exact number of sessions prescribed for a particular week. For instance, if three training sessions was scheduled in a week, the runners was considered non-compliant if they completed more or less than three sessions that week even if the distance covered was within the ±10%-margin of the prescribed running distance. In GRONORUN, the date at which the participant registered their running activity was not monitored. Consequently, we were unable to assess whether the prescribed weekly training was actually performed within a week or over multiple weeks. Thus, the compliance requirements were less restrictive in the GRONORUN-analysis compared with the PR21, RUNCLEVER and S2R analyses. Essentially, a GRONORUN participant could have completed the ‘1 week sessions’ over 3 weeks and be classified as 100% compliant in week 1. Based on this, there is a slight discrepancy between the definitions of compliance across trials.
In the trials using shoes as intervention, compliance was a percentage of running sessions (training and competitions) performed in the study shoes provided by the research team. Full compliance (100%) was defined as using the prescribed running shoe for all the running sessions they completed during follow-up. During follow-up, the participants were asked to practice running at least once a week, and to upload all their sports activities on an electronic system (www.tipps.lu) at least once a week. However, the recommendation regarding one weekly running session was not included as a part of the definition of compliance. In other words, this means that a runner is considered 100% compliant if he/she, for example, runs only two times during a 24-week follow-up, although at least 24 running sessions were recommended, provided the two sessions were performed in the prescribed running shoe.
Statistics
Full access to the data of each study was provided by the respective principal investigator. MLB analysed data from the trials using training programme as intervention, whereas LM analysed data from the running shoe trials. All data management and statistical analyses were performed in STATA V.15.1.
Results
The characteristics (eg, duration of follow-up, population, intervention and number of randomised runners) of each trial are presented in table 1. In table 2, the results from the four trials (GRONORUN, PR21, RUNCLEVER, S2R) on training intervention are presented. For each trial, the number of participants still fully compliant with the study protocol is visualised on a weekly basis during follow-up. In the trials using a running programme intervention, the number of participants being fully compliant was 0 of 839 (0%) after 24-week follow-up in RUNCLEVER, 0 of 612 (0%) after 14-week follow-up in PR21, 12 of 56 (21%) after 4-week follow-up in S2R and 8 of 532 (1%) after 8-week follow-up in GRONORUN. In the trials using a shoe-related intervention (table 3), the proportions being compliant at the end of follow-up were 207 of 304 (68%) in the first 21-week trial, and 322 of 423 (76%), 521 of 577 (90%), 753 of 874 (86%) after 24-week follow-up in the other three trials, respectively.
Number of participants still fully compliant with the protocol during follow-up
Compliance in the shoe-related trials. Values are cumulative number (and percentage) of participants compliant for shoe use during follow-up
Discussion
The findings in this educational review reveal that the proportion of runners being fully compliant at the end of follow-up ranged from 0% to 21% in the trials using training programme as intervention and from 68% to 90% in the trials using running shoes as intervention.
Interpreting results from ITT analysis
Based on the findings in this article, researchers and readers of scientific articles should be careful when interpreting the results from randomised trials when compliance is not reported. Returning to the conclusion from the trial by Buist et al,6 which read: ‘This randomised controlled trial showed no effect of a graded training program (13 weeks) in novice runners, applying the 10% rule, on the incidence of RRI compared with a standard 8 week training program’, we now know, based on the observed compliance in the GRONORUN trial in table 2, that the conclusion is inappropriate because of low compliance. The authors were unable to compare the effect of the two training programmes using ITT analysis because a majority of the runners did not follow the programme. A better conclusion based on the ITT analysis would be that ‘there was no difference in injury occurrence between runners randomised to two different running programs’. Here, the conclusion is targeted towards the effect of the randomisation, not the effect of the intervention.
As compliance problems also exist in other trials,10 we allow ourselves to express concerns when the authors complete an ITT analysis, conclude on the effect of the intervention, but fail to report results on the compliance with the interventions. This is underlined by the findings by Hislop et al 21 and Attwood et al 22 who reported non-compliance with different exercise programmes (eg, balance training) as high as 69% and 85% in the intervention and control groups, respectively. This illustrates that the findings from the present educational review may be transferable to other sports (eg, football, handball) and other types of interventions (eg, strength training, taping).
Reasons for noncompliance
We believe the range of compliance reported in the earlier trials may be influenced by (i) the definition of compliance used, (ii) the population studied and (iii) the nature of the intervention.
The definition of compliance will affect the proportion being considered as compliant at a certain time during follow-up. Table 2 suggests that the proportion of runners who complied with the training programme differed across the GRONORUN trial, the RUNCLEVER and the PR21 projects but the definition of compliance was different in the three trials. As the data collection on training sessions was not registered on a weekly basis in GRONORUN, the definition was less strict than the one used in RUNCLEVER and PR21. If a similar definition had been used in the GRONORUN trial, the compliance might have been comparable to those of RUNCLEVER and PR21. Still, the proportion being compliant during follow-up in the S2R also seem better than those presented for RUNCLEVER and PR21. Across these trials, similar definitions of compliance were used, which highlights that other aspects than just definition of compliance could influence the results.
The type of runner eligible for inclusion in the studies is a major difference between the GRONORUN and S2R to PR21 and RUNCLEVER. In the first two, novice runners were included, whereas the other two included recreational runners (ie, more experienced). More experienced athletes with already established training routines may be less willing to change training plan than novice runners.
It is easy to imagine that the nature of an intervention would greatly influence the compliance; being asked to wear a certain type of shoe is easier to do than to run specific distances at a specified pace on scheduled days. We report a substantial difference between the training-intervention trials (proportion who were compliant=0%–21%) and the equipment-related trials (76%–90%).
Freedom to choose is not the only prerequisite for high compliance. In PR21, the participants had the freedom to choose their running programme but still the compliance was poor.
The cavalry coming over the hill for sport and exercise medicine researchers: G-estimation
What should sports injury researchers do in response to the problems with ITT analyses? The good news is that there are other analytical approaches, such as G-estimation, which includes instrumental variables (IV) analysis and can provide valid estimates under certain assumptions. In fact, Greenland et al 23 argued in a methodological article that G-estimation should become standard procedure instead of ITT and PP analysis in trials with low compliance. Possibly, IV analysis is the best known technique to address compliance problems in clinical trials.24 Interestingly, the IV assumptions lead to corrections for confounding by non-compliance in randomised trials. This application is especially important because treatment assignment can provide a perfect IV for confounding control, and IV methods provide an alternative to ITT analysis.24 The IV method, as well as other G-estimating methods like structural nested modelling,23 is complex. Based on this, we recommend sports injury researchers to collaborate with epidemiologists and statisticians when analysing data,25 even though codes to complete the analysis is available in most statistical software packages.23
Besides compliance, censoring (athletes leaving the study prior to end of follow-up) is an important aspect to consider when analysing data from a randomised trial.26 As it is very common that athletes drop out during the course of the study,26 it is necessary to extend the analysis for instance by using inverse probability weighing (IPW).27 This is necessary as the G-estimations rely on the assumption that all study participants are followed to the end of the trial, regardless of whether they comply with the protocol.23 We encourage the reader to consult an introduction to IPW provided by Mansournia and Altman .27 In addition, it may be necessary to handle time-varying confounding as it is likely that athletes change some of their characteristics, training habits and equipment use during follow-up.28 Time-varying confounding is discussed in detail elsewhere.28
Future recommendations
A call for more RCTs was made in a recent BJSM editorial.29 Sports injury clinicians seek evidence-based knowledge on treatment options that ensure a fast recovery. However, in sports injury research compliance problems are considerable and it is questionable if a difference observed is a fair representation of the effect of the intervention.
To shed light on the injury occurrences among different types of athletes following different programmes, we recommend authors to: (i) think about strategies that will improve compliance during study planning; (ii) define compliance a priori, calculate and report it, and define an ‘acceptable threshold’; and (iii) analyse data according to the ITT and/or PP principle, report observed compliance and formulate conclusions appropriately. If sports injury researchers observe high proportion of athletes being non-compliant, they should consider using alternative analytical approaches such a G-estimation or IPW. Another option is to design prospective cohort studies rather than randomised trials allowing people to exercise in the way they believe is most appropriate. This avoid the strong contrast between what participants currently do in terms of training habits and what the trial demands. Still, this comes at a price of large sample sizes.
As compliance with the intervention is rarely reported in original articles in the sports injury context,10 it’s time authors do a better job (eg, include compliance data as supplementary material) and reviewers at the sports medicine journals recommend editors to reject papers with poor reporting as they would with any major trial reporting failure.9
Conclusion
If the low compliance reported in the eight trials included in the presented educational review is transferable to other sports (eg, football, handball, rugby) and to other types of interventions (eg, strength training), sports injury researchers need to carefully assess and report the compliance with the intervention in their articles and carefully consider how to formulate their conclusion.
If an ITT analysis is performed and athletes/players do not comply with the intervention, a conclusion about the effect of the intervention is inappropriate. Instead, researchers can draw conclusions about the ‘real world’ effect of being allocated to a group, which is equivalent to the effect of the randomisation, or analyse data using other analytical techniques such as G-estimation or IPW provided certain assumptions are met.
What is already known
In accordance with the CONSORT reporting guideline for randomised trials, sports injury researchers should report compliance with the interventions in RCTs to allow the informed reader to evaluate whether any conclusion regarding effect of the intervention is appropriate.
Based on an intention-to-treat analysis, researchers may conclude as to whether or not the intervention was effective, provided the sample size is sufficiently large and athletes/players comply with the intervention.
What are the new findings?
As low compliance may exist in many trials in our field, we encourage sports injury researchers and readers of sports injury literature to carefully consider if it is fair to conclude that one intervention/treatment is better than the other if the intention-to-treat principle is applied.
We recommend authors: (i) think about strategies that will improve compliance during study planning; (ii) define compliance a priori, calculate and report it, and define an ‘acceptable threshold’; and (iii) analyse data according to the intention-to-treat and/or per protocol principle, report observed compliance and formulate conclusions appropriately.
We also recommend sports injury researchers explore the method of G-estimation. This is now a standard analytical approach (provided assumptions are fulfilled) in trials with low compliance. Inverse probability weighting is important in trials with large drop-out (censoring) before the end of follow-up.
References
Footnotes
Contributors DR collected data for the RUNCLEVER, CD for PR21, MLB for Start 2 Run, SWB for GRONORUN and LM and DT for the shoe-related trials. LM and MLB analysed the data. RON drafted the manuscript, while the remaining co-authors revised it for important intellectual content.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient consent for publication Not required.
Ethics approval The study protocols for the running programme trials were approved by the local ethics committees: Groningen Novice Running (Medical Ethics Committee of the University Medical Center Groningen, The Netherlands ISRCTN37259753), RUNCLEVER (Northern Denmark Region N-20140069), ProjectRun21 (Central Denmark Region 187/2015), and Start 2 Run (Northern Denmark Region N-20160031). The study protocols for the shoe-related trials have previously been approved by the National Ethics Committee for Research in Luxembourg (ref.: 201201/02, 201211/04, 201407/09 V1.1, 201701/02 V1.1, respectively, for study 1–4). Prior to data collections, all trials were approved from the local ethics committees and from the data protection agencies.
Provenance and peer review Not commissioned; externally peer reviewed.