Article Text
PDF
Abstract
Objective Exercise-training programmes have reduced lower limb injuries in trials, but their population-level effectiveness has not been reported in implementation trials. This study aimed to demonstrate that routinely collected hospital data can be used to evaluate population-level programme effectiveness.
Method A controlled ecological design was used to evaluate the effect of FootyFirst, an exercise-training programme, on the number of hospital-treated lower limb injuries sustained by males aged 16–50 years while participating in community-level Australian Football. FootyFirst was implemented with ‘support’ (FootyFirst+S) or ‘without support’ (FootyFirst+NS) in different geographic regions of Victoria, Australia: 22 clubs in region 1: FootyFirst+S in 2012/2013; 25 clubs in region 2: FootyFirst+NS in 2012/2013; 31 clubs region 3: control in 2012, FootyFirst+S in 2013. Interrupted time-series analysis compared injury counts across regions and against trends in the rest of Victoria.
Results After 1 year of FootyFirst+S, there was a non-statistically significant decline in the number of lower limb injuries in region 1 (2012) and region 3 (2013); this was not maintained after 2 years in region 1. Compared with before FootyFirst in 2006–2011, injury count changes at the end of 2013 were: region 1: 20.0% reduction (after 2 years support); region 2: 21.5% increase (after 2 years without support); region 3: 21.8% increase (after first year no programme, second year programme with support); rest of Victoria: 12.6% increase.
Conclusion Ecological analyses using routinely collected hospital data show promise as the basis of population-level programme evaluation. The implementation and sustainability of sports injury prevention programmes at the population-level remains challenging.
- injury prevention
- exercise training
- implementation
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
What are the findings?
The efficacy of exercise-training programmes has been shown in highly controlled randomised controlled trials (RCTs) (up to 50% of lower limb injuries can be prevented).
The available RCT evidence is of limited practical value because trial conditions and inherent design controls do not reflect the real-world contexts of sports delivery.
In addition, there have been low levels of compliance with the interventions, and the support needed to ensure wider implementation has not been assessed.
How might it impact on clinical practice in the future?
This is one of the first reports of the effectiveness of exercise-training programmes in a real-world sport delivery context, using a controlled ecological study design.
Routinely collected hospital data can contribute to evaluations of the effectiveness of sports injury prevention interventions that are implemented in geographically defined areas.
Exercise-training programmes that are supported by context-specific and evidence-informed, context-specific implementation plans are associated with proportionally larger injury reductions than the same programmes without such implementation support.
Background
Injuries during participation in sport are relatively common and have significant public health impacts in terms of injury treatment costs, quality of life and future participation.1–3 Several large epidemiological studies have identified that lower limb muscle and joint injuries are the most common type of injury across a wide range of sports.2 4 5 Lower limb injuries have been identified as a priority for injury prevention activities in both elite/professional6 and community/recreational7 Australian Football (AF).
Since the late 1990s, sports injury prevention research has focused on developing and investigating the efficacy of exercise-training interventions (including all forms of physical preparation for sport and exercise), primarily to prevent lower limb injuries. Systematic reviews and meta-analyses have demonstrated that exercise-training programmes are effective in preventing all injuries, lower limb injuries generally and ankle/knee injuries specifically.8–10 These preventive effects are of the order of 20%–50% reductions as measured in randomised controlled efficacy trials (RCTs).10 11
Demonstrating that injury prevention programmes actually prevent injuries in the real world is an important part of their evaluation. While efficacy evidence from RCTs is important, it rarely takes into account the intervention’s implementation context. Effectiveness trials aim to determine the extent to which an intervention actually prevents injuries when delivered as it would be used in real-world practice.12 There have been few sports injury prevention programmes evaluated through ecological designs, with some notable exceptions being nationwide programmes implemented and evaluated by New Zealand’s Accident Compensation Commission using national injury compensation data,13 14 a squash protective eyewear promotion campaign in Australia15 and a safe communities evaluation in Sweden.16
Evaluation study designs to assess the effectiveness (rather than efficacy) of injury prevention programmes have been discussed.12 17–19 Approaches based on quasi-experimental designs and interrupted time-series analyses have been shown to be useful, particularly for population-wide evaluations when RCTs are neither practical nor desirable.12 20 Importantly, the stringent design controls inherent within RCTs and their inability to mimic real-world settings means that it is often impossible to assess determinants of intervention effectiveness and associated implementation factors in RCTs. Nonetheless, implementation studies should still aim to include some control conditions to compare different implementation strategies.
The National Guidance for Australian Football Partnerships for Safety (NoGAPS) study is one of the first large-scale sports injury prevention implementation studies to use a controlled, ecological design to assess the effectiveness of a sports injury prevention intervention at the population-level. Its associated injury prevention programme, known as FootyFirst, is an exercise-training programme developed to prevent lower limb injuries in community AF (community-AF).21 An evidence-informed and context-specific implementation plan was also developed to support the implementation of FootyFirst in specific geographic regions.22 Given the varying intervention strategies in different geographical regions, this analysis hypothesised that injury reductions would be greater in regions that received FootyFirst with implementation support than in regions that received FootyFirst without implementation support. The aim of this paper was to demonstrate how routinely collected hospital data can be used for programme evaluation, as applied in an ecological evaluation of the population-level impact of FootyFirst (±implementation support).
Methods
The NoGAPS study protocol, and the design and content of FootyFirst and its associated implementation plan, have been published elsewhere.21–24 FootyFirst was implemented over 2 years from the start of the 2012 and 2013 football seasons (April–September). Evaluation of FootyFirst was based on components of the RE-AIM Sports Setting Matrix25 and used a controlled ecological design over 2 years.23 Trends in hospital-treated injury counts were one outcome used to evaluate the effectiveness of FootyFirst. The implementation activity outcomes are described elsewhere.24
The ecological design
FootyFirst was implemented in a controlled ecological study applied to three distinct (defined by non-overlapping, non-contiguous, well-defined (according to postcode) local government areas (LGAs)) geographical regions in Victoria, Australia (figure 1). These regions were chosen based on each having (a) a representative and strong microcosm of community-AF of comparable size, playing standard and administrative capacity; (b) close geographical location to, and previous links with, the research team; (c) a mix of metropolitan and regional LGAs and football competitions; (d) large numbers of relevant teams and registered players, across a range of competition levels and (e) similar levels of football league professionalism and ability to pay players and coaches. For evaluation purposes, all regions were also required to have a large hospital with a 24 hours emergency department (ED) servicing the relevant LGAs.
Geographic location of the three FootyFirst regions in Victoria, Australia.
Each region received a different combination of programme and delivery mode:
Region 1 (n=22 clubs): FootyFirst and implementation support (FootyFirst+S) in both 2012 and 2013.
Region 2 (n=25 clubs): FootyFirst and no implementation support (FootyFirst+NS) in both 2012 and 2013.
Region 3 (n=31 clubs): no FootyFirst and no implementation support in 2012 (control); FootyFirst and implementation support (FootyFirst+S) in 2013.
Rest of Victoria: external control with no access to FootyFirst.
Full details of this implementation support are published elsewhere, and included such things as coach education, resource provision, mentoring to coaches and sport administrators and a local promotion strategy.24
Comparison of region 1 and region 2 results assessed the effects of having or not having implementation support, in addition to FootyFirst, evaluated over 2 years (ie, FootyFirst+S vs FootyFirst+NS). Comparison of regions 1 and 3 in year 2 enable some conclusions to be made about the first year of FootyFirst+S uptake and its sustainability into a second year.
Injury data
The number of injury admissions and ED presentations to the hospitals serving the three FootyFirst regions, and for the rest of Victoria, was obtained from routinely collected health sector data for the state of Victoria, Australia. Data were extracted for all injuries and lower limb injuries sustained during participation in AF specifically in males aged 16–50 years.
The Victorian Admitted Episodes Dataset (VAED) provides a complete dataset of all injury-related hospital episodes of care to public and private hospitals in Victoria. The VAED is coded to the International Classification of Diseases and Related Health Problems, Tenth Revision, Australian Modifications (ICD-10-AM),26 which includes 200 ‘activity codes’ for identifying specific types of sport/leisure activity in which the person was participating at the time of injury.27 28 AF-related injury cases were identified if the activity when injured was recorded as AF (ICD-10-AM activity code U50.00) or U50.09 (Football, unspecified) since AF is the major football code in the state of Victoria. Extraction was restricted to cases with a principal diagnosis of lower limb injury (ICD-10-AM diagnosis code in the ranges S30–S39 or S70–S99 ranges, online supplementary file 1) with readmissions excluded.
Supplemental material
The Victorian Emergency Minimum Dataset (VEMD) includes ED presentations to 39 Victorian public hospitals with 24 hours EDs, covering about 90% of all ED visits. The VEMD data codes are based on the National Health Data Dictionary.29 Cases were selected if the text narrative indicated AF and if they occurred at a ‘sports and athletic area’/‘other’/‘unspecified’ location. Cases were restricted to initial visits and did not include those subsequently admitted to hospital, to avoid double counting of incident cases with the VAED. The lower limb injuries were identified when any of the diagnosis codes involved the lower limb (pelvis, hip, thigh, knee, lower leg, ankle, foot or multiple lower limb) (see online supplementary file 1).
FootyFirst was implemented in the community-AF clubs in each region. Allocation of LGA codes in the hospital-treated datasets was based on the patient’s LGA of residence rather than the LGA where the injury occurred or the LGA of the hospital (where they were treated). This assumed that players participated in a community-AF club close to where they lived, were injured while playing football in that club’s region and sought any hospital treatment in the same LGA.
Data analysis
Data on hospital admissions and ED presentations were combined, to provide the total number of adult males aged 16–50 years who received hospital treatment for AF-related injuries (ie, incident cases). Small cell counts (one to four cases) were suppressed as a privacy protection measure in the provided data. For analysis purposes, these missing values were uniformly replaced with an injury count of three, as the expected average based on conservative trends.
An interrupted time-series approach was used to analyse the data (see online supplementary file 2), with the point of interruption relating to the timing of FootyFirst implementation in 2012 and 2013,30 because of the method’s value in public health intervention evaluations.18 20 30 Data were analysed for the period 2006–2013, inclusive, to cover 6 years pre-FootyFirst and the 2 years of FootyFirst implementation. It was also restricted to the months of April–September to coincide with the community-AF season.
Supplemental material
A generalised least squares model was fitted to the seasonally adjusted monthly time-series data.31 32 This incorporated a base level (intercept), trend and change in trend due to FootyFirst in each of 2012 and 2013.30 Seasonal adjustment was through the ‘Seasonal and Trend decomposition using the Loess non-parametric seasonal decomposition procedure.33 Comparison of the change in trend (slope) parameters was used to assess trends before and after each FootyFirst implementation phase (2012 and 2013).
Results
The number of both AF-related injuries overall and lower limb injury-only cases generally increased over years in each region, with the exception of 2012 in region 1, corresponding to the years when FootyFirst+S was first introduced in that region (table 1, figure 2). The proportion of all cases that were lower limb injury, was consistently higher in region 2. From a medical treatment point of view, 66%–80% of the injuries were ED presentations only, suggesting that many cases were not severe enough to require hospital admission. Those admitted were mostly 1 day only admissions, indicating that most of the hospital-treated cases were also not severe. Emergency admissions to hospitals, presumably for acute injuries requiring immediate treatment, accounted for 42%–62% of all admissions for most years and regions. The nature of the injuries is summarised in online supplementary file 1.
Trends in the monthly numbers of hospital-treated AF-related lower limb injury combined over time in each of the three FootyFirst regions and the rest of Victoria. In each component, the broken line is the fitted trend; vertical grey lines indicate the points at which FootyFirst was implemented in both 2012 and 2013. Region 1: FootyFirst+with ‘support’ (S) in both 2012 and 2013; region 2: FootyFirst+without ‘support’ (NS) in both 2012 and 2013; region 3: no FootyFirst in 2012, FootyFirst+S in 2013; rest of Victoria: no FootyFirst in either year.
Numbers of hospital-treated AF-related injuries in the FootyFirst regions both pre-FootyFirst and post-FootyFirst implementation (males aged 16–50 years only; counts are for the months from April to September combined)
There was a non-significant decline in the number of hospital-treated AF-related lower limb injury-only cases in both region 1 (2012) and region 3 (2013) after the first year of introduction of FootyFirst+S (table 2). This non-significant decline was not maintained after 2 years of FootyFirst+S in region 1. There were no reductions in lower limb injury-only cases in region 2, which received FootyFirst+NS in both years. As an ecological control, trends in numbers of lower limb injury-only cases in the rest of Victoria were also analysed. These showed a significant decline in the first year, but a non-significant decline in cases, after 2 years.
Trends in the numbers of hospital-treated* AF-related LLI-only cases in each FootyFirst region compared with the rest of Victoria
Region 1 was the only region to show a proportionate decline in the mean monthly number of lower limb injury-only cases when the 6 years preimplementation and 2 years postfirst implementation of FootyFirst were compared (table 3).
The mean monthly number of hospital-treated AF-related lower limb injury-only cases in each FootyFirst region and across the rest of Victoria*
Discussion
This study demonstrates that routinely collected hospital data can be used as the basis of ecological injury prevention programme evaluations. The adopted controlled ecological design evaluation assessed trends in hospitalised lower limb injuries in regions that were exposed to an lower limb injury exercise-training programme that were/were not accompanied by appropriate implementation support. Although the reported declines were not statistically significant, the proportionate declines were of a magnitude considered to be practically meaningful by the partner agencies involved in the project.
Therefore, this controlled ecological design evaluation of an exercise-training programme to prevent lower limb injuries in community-AF provides evidence that such programmes can be effective at the population-level in reducing injuries when they are accompanied by appropriate implementation support. The only regions that showed a reduction in hospital-treated injuries were those that received FootyFirst+S for an initial 1 year (region 1 in 2012, region 3 in 2013); regions that received FootyFirst+NS or did not receive FootyFirst at all (region 3 in 2012, rest of Victoria) showed no reductions. This was achieved because the FootyFirst programme and its delivery support were based on strong scientific and epidemiological evidence, and expert-informed and end user-informed processes underpinned their development and strong focus on maximising its reach, adoption and implementation.11 21 22
Methodology considerations
As with any ecological study, there are important consideration relating to the study design used in this evaluation. It is possible that other external factors, which were unable to be assessed, may have affected the hospital-treated injury counts, and hence the cross-region comparisons. One such factor might have been changes in football participation, but this could not be assessed as data were unavailable at the LGA level. Inclusion of an external control group (the rest of Victoria) assessed this to some extent. Nonetheless, it is known that a sizeable proportion of lower limb injuries in community-AF players receive medical treatment in hospital settings,34 so there is a plausible rationale for expecting that FootyFirst+S would have a direct impact on at least some hospital-treated AF injuries, especially in regions with limited access to other health service providers to treat sports injuries.
The pre-post FootyFirst evaluation was limited by the amount of data available, especially the number of injury counts in region 1. Conducting a similar study using more postintervention data is recommended in the future. The statistical model assumed a linear trend in the counts of hospital-treated cases preinterruption and postinterruption, and it is possible that this assumption may hold only over short intervals.
To minimise these limitations, we ensured methodological rigour in our data analysis approach. Time-series designs are the strongest quasi-experimental designs for estimating intervention effects in non-randomised settings and provide statistical control for prior outcome trends and the study of dynamics of change in response to an intervention.20 30 The adopted interrupted time-series approach estimated the effect of the intervention while taking into account both time trends and autocorrelation among the observations.30
Importantly, a related comparison of the implementation activities across the regions inherent in this ecological evaluation also found that the awareness and uptake of the programme were higher in the regions with the implementation support.24 These implementation activity changes are necessary precursors to any injury reductions. The results presented in that paper add weight to our conclusion that population-level hospital-treated sports injury rates can be reduced in regions that receive targeted evidence-based injury prevention programmes accompanied by evidence-informed and context-specific implementation support, compared with those which receive the programme without support. It was not possible to assess awareness of FootyFirst across the rest of Victoria, and it is conceivable that some clubs/teams outside the intervention regions may have been aware of it.
Hospital data limitations
The hospital-treated injury counts for a particular region may have been affected by differential case capture, completeness and coding issues at the hospital level (particularly related to the activity at the time of injury), and the variable quality of ED case narrative data. Differences in data completeness could have affected the ability to accurately compare time-series trends across regions, especially if not all relevant injury cases were captured.
The evaluation regions were defined based on the LGA of the community-AF clubs that participated in each league because each of the football league’s activities were conducted fully independently from those in other regions. Because the injury case selection was based on the postcode of residence of the injured person, the ED presentations data may be affected by their proximity to one of the 39 major hospitals contributing to the VEMD. It was assumed that hospital-treated AF players lived in the same LGAs as the clubs that made up the league in which they participated, and attended a hospital close to where they lived/played. It is possible that injury counts might have been overestimated or underestimated if some people treated in the region-specific hospitals lived outside those regions, or if players from the FootyFirst leagues received hospital treatment elsewhere. Furthermore, changing injury referral patterns may have influenced injury counts over the study period. However, since sports medicine is a stable medical specialty in Australia there is unlikely to have been any change in its hospital referrals over this time.
In July 2012, the Victorian Hospital Admission Policy changed significantly, and episodes of care delivered entirely within a designated ED could no longer be categorised as an admission, regardless of the amount of time spent in the hospital. The influence of this change in hospital admission policy was minimised by excluding cases recorded as spending their entire episode of care in the ED throughout the VAED data.
Finally, our study is also limited by the amount of data available post-FootyFirst. Conducting a similar study using more postintervention data is recommended in the future. The statistical model assumed a linear trend in the counts of hospital-treated cases preinterruption and postinterruption, and it is possible that this assumption often may hold only over short intervals.
Conclusion
Despite these limitations, this study demonstrates the feasibility of using routinely collected hospital data to evaluate the impact of a community-based injury prevention programme in an ecological study. Evidence of population-level effects is required to convince policy makers and peak sports bodies of the potential value of investing in population-wide injury prevention initiatives.35 Trends based on routine data collections, such insurance claims or hospital treatment, can be powerful drivers of public health investment.3 13 14 35 Although not statistically significant, the magnitude of the proportionate injury reductions associated with the first year of FootyFirst+S were considered to be practically meaningful to the partner agencies interested in supporting the implementation of the programme into the future.
Implications for sports injury prevention
This is one of the first studies to evaluate the population-level impacts of a sports injury prevention programme using routinely collected hospital data. It is therefore a first step towards their upscale from smaller local-level cohorts of sports teams to broader groups of participants. Most previous sports injury prevention evaluations have had limited generalisability to larger populations because they have been based solely on club-level injury surveillance activities, in well-defined small groups of sports participants. Importantly, this study also links trends towards population-level evidence of intervention effectiveness to demonstrable improvements in implementation activity.24
Although there continues to be much interest in sports injury prevention programmes, challenges with their implementation are also recognised.36 The RE-AIM framework identifies that addressing and ensuring aspects such as programme reach, adoption and implementation are all necessary for population-level impact.37 How these factors manifest and need to be addressed will differ across the sports delivery system.25 This study provides evidence that practically meaningful proportionate reductions in injury counts can be achieved through changes in these parameters, but only when evidence-informed and expert-informed preventive programmes are accompanied by stakeholder-relevant delivery processes and appropriate setting-specific infrastructure support to ensure their roll-out. In other words, for population-level injury reductions, it is not enough to just have a programme that should work.
Acknowledgments
The routine hospital data were provided to the authors by the Victorian Injury Surveillance Unit, at Monash University. Aspects of this project were conducted when AD, CFF and JLC worked at Monash University (before 2013) and when AD, AM and CFF worked at Federation University Australia (2013–2016). The Chief Investigator team for the NoGAPS project was CFF, AD, Belinda Gabbe, DGL and JLC.
References
Footnotes
Contributors CFF designed the study and led the evaluation as the lead Chief Investigator of the NoGAPS project. AD, DGL and JLC all contributed to the design of FootyFirst and its implementation strategy. SEG extracted all of the data from the VAED and VEMD datasets and assisted with data analysis and presentation. AM undertook the statistical modelling of the data. All authors contributed to the writing and/or editing of the paper before submission and approved its submission.
Funding This study was funded by a National Health and Medical Research Council (NHMRC) Partnership Project Grant (ID 565907) with additional support (both cash and in kind) from the project partner agencies: the Australian Football League (AFL); Victorian Health Promotion Foundation (VicHealth); NSW Sporting Injuries Committee (NSWSIC); JLT Sport, a division of Jardine Lloyd Thompson Australia Pty Ltd; the Department of Planning and Community Development; Sport and Recreation Victoria Division (SRV) and Sports Medicine Australia National and Victorian Branches (SMA). CFF was partially supported by an NHMRC Principal Research Fellowship (ID1058737). JLC was supported by a NHMRC practitioner fellowship (ID1058493). AD received full salary support from the NHMRC Partnership Project Grant. Aspects of this research are related to the work of the Australian Centre for Research into Injury in Sport and its Prevention (ACRISP), which is one of the IOC centres of research excellence for the prevention of injuries and promotion of health in athletes.
Competing interests None declared.
Patient consent Not required.
Ethics approval Monash University and Federation University Australia Committees.
Provenance and peer review Not commissioned; externally peer reviewed.