Article Text
PDF
Abstract
Objectives To assess if a multi-strategy intervention effectively increased weekly minutes of structured physical activity (PA) implemented by classroom teachers at 12 months and 18 months.
Methods A cluster randomised controlled trial with 61 primary schools in New South Wales Australia. The 12-month multi-strategy intervention included; centralised technical assistance, ongoing consultation, principal’s mandated change, identifying and preparing school champions, development of implementation plans, educational outreach visits and provision of educational materials. Control schools received usual support (guidelines for policy development via education department website and telephone support). Weekly minutes of structured PA implemented by classroom teachers (primary outcome) was measured via teacher completion of a daily log-book at baseline (October–December 2017), 12-month (October–December 2018) and 18-month (April–June 2019). Data were analysed using linear mixed effects regression models.
Results Overall, 400 class teachers at baseline, 403 at 12 months follow-up and 391 at 18 months follow-up provided valid primary outcome data. From baseline to 12-month follow-up, teachers at intervention schools recorded a greater increase in weekly minutes of PA implemented than teachers assigned to the control schools by approximately 44.2 min (95% CI 32.8 to 55.7; p<0.001) which remained at 18 months, however, the effect size was smaller at 27.1 min (95% CI 15.5 to 38.6; p≤0.001).
Conclusion A multi-strategy intervention increased mandatory PA policy implementation. Some, but not all of this improvement was maintained after implementation support concluded. Further research should assess the impact of scale-up strategies on the sustainability of PA policy implementation over longer time periods.
Trial registration number Australian New Zealand Clinical Trials Registry (ACTRN12617001265369).
- school
- physical activity
- children
- implementation
- intervention effectiveness
Data availability statement
The data that support the findings of this study are available from the corresponding author upon reasonable request.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Background
To improve child physical activity (PA) levels, the WHO recommended schools adopt policies that support children’s daily PA.1 Interventions that increase opportunities for regular PA during the school day effectively increased children’s moderate-vigorous physical activity (MVPA).2 In addition to teaching physical education (PE), a number of countries including Australia,3 China,4 Denmark,5 England6 and several Canadian provinces7 and US states8 9 have policies or guidelines regarding the minimum amount of time that primary schools schedule structured PA each week. Despite their existence, most schools fail to implement such policies.9–11 For example, in a study of Canadian elementary school teachers, only 43% implemented the mandatory 30 min/day PA policy that required organised in-class opportunities for children to be active.12 An Australian study (2017) found that only 24% were meeting the recommended 150 min of weekly PA.13 To enhance the potential to achieve broad public health benefits, school PA policies and strategies are needed to assist schools overcome barriers to their implementation and scale-up. We also need to identify whether schools’ continue to implement policies (implementation maintenance) once support is removed, as this encourages implementation in the first place and maximises benefits at scale-up.
There is limited research of strategies that facilitate schools’ implementation of health innovations.14 A Cochrane review14 identified only one controlled trial in primary schools that aimed to implement PA guidelines.15 This quasi-experimental study in seven US schools provided: on-site training, ongoing technical assistance, modelling, audit and feedback, resources and coalition building support.15 Improvements in the implementation of PE congruent with national guidelines were found, but effects were not sustained at 2 years. In 2017, we undertook a pilot cluster randomised controlled trial (RCT) in 12 Catholic primary schools. We aimed to determine the efficacy of a 9-month strategy to improve teachers’ implementation of the New South Wales (NSW) Sport and Physical Activity Policy, which requires schools to schedule 150 min of moderate, with some vigorous, PA per week for students in kindergarten to grade 10.3 The 150 min may include: PE (which in Australia is typically taught by generalist classroom teachers), sport and other structured activities such as energisers16 (ie, 3–5 min structured classroom PA breaks) or active lessons (eg, integrating PA into maths lessons).17 Intervention schools received: executive support, training for in-school champions, ongoing support, tools and resources.13 Immediately following the intervention, teachers in intervention schools scheduled significantly more minutes of PA per week than teachers in control schools (36.6 min, 95% CI 2.7 to 70.5, p=0.04).13 The extent to which these effects were maintained following cessation of implementation support or factors important for interpreting implementation findings (eg, a description of implementation context and processes) were not assessed.
The primary objective of this study was to assess whether a multi-strategy intervention effectively increased weekly minutes of structured PA implemented by classroom teachers at 12 months and 18 months. Our secondary objective was to describe the types of activities teachers implemented to achieve PA policy adherence (eg, PE, energisers, sport and integrated lessons).
Methods
A trial protocol has been published.18 This paper reports primary trial outcomes only. The study adheres to the Consolidated Standards of Reporting Trials19 and Standards for Reporting Implementation Studies (STARI)20 guidelines.
Study design and setting
An RCT was undertaken in 62 primary schools (31 per group), in the Hunter New England (HNE) region, of NSW Australia. The HNE is geographically large (130 000 km2) with a demographically and socioeconomically diverse population residing in metropolitan, urban and suburban areas, regional centres and rural and isolated remote communities.21 There are approximately 427 primary schools in this region of which 324 (76%) are government and 65 (15%) are Catholic.
Participants, recruitment, randomisation and blinding
Government and Catholic schools in the HNE were eligible if they were not participating in another PA trial and only enrolled primary school students who did not require specialist care. Following baseline data collection, schools were randomised to intervention or control by an independent statistician using a computer-based random number generator. Allocation was stratified by the schools’ geographic location (rural vs urban) and type (government, Catholic).7 Data collectors were blinded to group allocation. All surveys were deidentified prior to data entry. Due to the nature of the intervention, school and programme staff were not blinded.
Multi-strategy implementation intervention
The protocol includes a detailed description of the development of the intervention.18 The intervention was designed, using Behaviour Change Wheel (BCW)22 and Theoretical Domains Framework (TDF).23 Following extensive formative research which included (i) literature reviews; (ii) interviews with 76 primary school teachers using an adapted TDF survey and (iii) observations of teachers’ delivery of PE, sport and the school environment, the recommended process described by Michie et al23 was undertaken to map the identified barriers to the BCW and TDF. In consultation with an advisory group, strategies were purposefully selected to address known barriers to policy implementation.11 The intervention, described in table 1, was delivered over one school year (ie, four school terms) November 2017–November 2018.
Description of multi-strategy implementation intervention* and fidelity to and satisfaction with the intervention
Control group
Control schools had access to ‘usual’ implementation support from the NSW government which included: access to information and resources such as example policies and templates via a website as well as telephone support if requested by the school. The delivery of the multi-strategy intervention was under the control of the research team and not provided to control schools during the study period.
Data collection and measures
Baseline data (0 months) were collected between October 2017 and February 2018 and final data collection (12 months post baseline) were collected October–November 2018. Maintenance data were collected approximately 18 months post baseline (April–June 2019) that is, 6 months with no active implementation support.
Primary trial outcome: weekly minutes of structured PA implemented by classroom teachers at 12 and 18 months
As per the pilot study,13 the mean weekly minutes of PA implemented by teachers was measured via a daily log-book that teachers completed during a 1-week period at baseline, 12 and 18 months. The log-book included the time and type (ie, PE, sport, energisers or active lessons) of PA implemented. As we aimed to assess weekly PA implementation, teacher data were valid if they provided responses across the entire school week (ie, 5 days) and did not exceed 250 min. Values above 250 min were deemed by the project partners unlikely given Department of Education’s (DoE’s) guidance of minimum time required for scheduling other subjects.24 Only teachers with valid data were included in the analysis sample. Teacher log-books are successfully used in classroom-based obesity prevention interventions13 25 26 with high response rates (ie, >80%)25 and established reliability.8 26
Secondary outcome: weekly minutes of PE, energisers, sport and integrated lessons implemented by classroom teachers at 12 and 18 months
The mean weekly minutes of PE, sport, energisers and active lessons implemented were also collected from teacher log-books (as per the primary outcome).
School and participant characteristics
Detail regarding school type, postcode and school size was obtained from websites. Principals and teachers were invited to complete a paper survey which asked their; sex, age (years), years teaching experience, grade level taught, employment status and if they were a specialist PE teacher.
Process measures
To contextualise the study findings measures, recommended by Proctor et al,27 were assessed within intervention schools at follow-up.
Acceptability, appropriateness and feasibility of the policy
Validated self-report measures28 were included in the teacher’s pen-and-paper surveys. They were asked to report (using a five-point Likert scale from 1=strongly disagree to 5=strongly agree), their perceptions as to whether the policy was: (i) welcomed, appealing, liked and met their approval (Acceptability of Intervention Measure); (ii) a good fit, suitable, applicable and compatible within the context of their school (Intervention Appropriateness Measure) and (iii) possible, easy, do-able and implementable (Feasibility of Intervention Measure). A total score for each domain was calculated by averaging the item responses.29
Fidelity to and satisfaction with the multi-strategy implementation intervention
Project records as well as postintervention surveys completed by school champions were used to determine the proportion of schools that received and engaged with each of the implementation strategies. School champions and teachers were asked how satisfied they were with each of the implementation strategies.
Sample size
The average primary school had 13 classrooms. Using a conservative 70% response rate estimate and assuming 20% loss-to-follow-up, a sample of 31 schools per group would provide a sample of approximately 450 classes (225 per group) at follow-up. Based on pilot data an SD of 45 min, and a conservative Intraclass Correlation Coefficient (ICC) of 0.2, the sample was sufficient to detect an absolute difference of 18.0 min of weekly minutes of PA, with 80% power and alpha 0.05.
Statistical analysis
Analyses of the study outcomes were performed under an intention to treat framework, with teacher responses analysed according to the experimental group their school was originally randomised to. Class (nested within a school) was the unit of analysis. Differences between the intervention and control group with regards to changes in the primary outcome and types of PA implemented (ie, PE, energisers, sport and integrated lessons) from baseline to each of the follow-up time-points, were assessed using linear mixed effects regression models. Linear mixed models estimate and account for the correlation of data within clusters (ie, schools) through the inclusion of random effects, thus accounting for the lack of independence of observations from cluster trials such as this one. Linear mixed models also use all available data, regardless of missing outcome data, assuming data are missing at random. A separate model was conducted for each outcome, and included fixed effects for treatment group (intervention vs control), time (baseline, 12-month and 18-month follow-up), a time by group interaction term and variables prognostic of the outcome (school type, geographic and socioeconomic location of the school).7 The model included a random intercept for school to allow for the clustered design, a random intercept for teacher (nested within school) to account for repeated measurement of some teachers, as well as a random slope. Descriptive statistics described the process measures reported by the intervention group.
Partner and end-user involvement
The DoE and Catholic Schools Office (CSO) (authors JB and BD) identified the research question. The DoE were partner investigators on the grant. The intervention and study materials were designed following extensive formative research and consultation with principals, teachers, DoE and CSO representatives. Participant burden was assessed during school ethical approvals. An Advisory Group, which included DoE and CSO, oversaw all aspects of the study. Data have been shared with DoE and CSO and will be presented at their principal and teacher forums.
Deviations from registered protocol
None.
Results
School and participant characteristics
Figure 1 presents a flowchart of the eligible and participating schools in the study. Four hundred and thirty-two schools were assessed for eligibility, with 62 schools meeting inclusion criteria and consenting to participate. One school was excluded prior to randomisation because it was participating in another PA intervention. There were no differences in the baseline characteristics of schools (table 2), with 42% of schools from both intervention and 39% of control groups from inner/outer regional areas and 58% from major cities (table 2). Overall, 44% of schools from major city areas were classified as most disadvantaged, compared with 88% of schools from inner/outer regional areas. School size (data not shown) ranged from 40 to 900 students, with the mean size slightly higher in the intervention group compared with the control group (300.3 vs 261.6, respectively). Of the remaining 61 schools, 3 provided invalid data (ie, no surveys with 5 days of data ≤250 min), leaving a total of 58 schools contributing valid data at 12-month and 18-month follow-up, from a total of 403 and 391 teachers, respectively. Across all three time points loss of data due to reporting of PA above 250 min represented a 4% loss of data. The characteristics of all teachers providing valid data across each of the three time points was similar across both intervention and control groups (see table 3).
Time schedule of participant enrolment, data collection and intervention delivery. PA, physical activity.
Baseline school characteristics by experimental group
Teacher characteristics by experimental group
Primary outcome: weekly minutes of structured PA implemented by classroom teachers at 12 months and 18 months
At 12-month intervention teachers increased their overall implementation of PA per week by an average of 44.2 min (95% CI 32.8 to 55.7; p<0.001) more than the control group (table 4). This was maintained at 18 months, with the intervention group increasing their implementation from baseline to 18 months by an average of 27.1 min (95% CI 15.5 to 38.6; p≤0.001) more than the control. The difference in the change from 12-month to 18-month follow-up between the two experimental groups was statistically significant (−17.2 min (95% CI –28.8 to –5.64; p=0.004)), with the intervention group recording a within group change of only 1.3 min (95% CI −6.6 to 9.3; p=0.74), compared with the usual care group which recorded a reduction of −15.9 min (95% CI –24.3 to –7.4; p<0.001).
Changes in weekly minutes of physical activity implemented from baseline to 12-month and 18-month follow-up
The proportion of teachers in the intervention group meeting the mandated 150 min of PA per week was 61.9% (n=138) at 12 months and 59.4% (n=117) at 18 months compared with the control group which had 17.2% (n=31) and 29.9% (n=58) at 12 months and 18 months, respectively. The difference in the change in proportion of teachers scheduling 150 min of PA per week between intervention and control was significantly different from baseline to 12 months (OR: 7.56; 95% CI 3.88 to 14.7, p<0.001) and from baseline to 18 months (OR: 3.62; 95% CI 1.93 to 6.79, p≤0.001).
Secondary outcome: types of activities teachers implemented to achieve PA policy adherence (eg, PE, energisers, sport and integrated lessons)
At 12 months teachers in the intervention group had a significantly greater increase from baseline, in implementation of energisers (23.1 min; 95% CI, 16.5 to 29.6; p<0.001), PE (10.4 min; 95% CI 1.89 to 18.8; p=0.017) and integrated lessons (6.96 min; 95% CI 3.15 to 10.8; p≤0.001) (table 4). There were no differences between groups in the change in implementation of sport from baseline to 12 months. The significant between group difference was only maintained for energisers at 18 months, with the intervention group increasing their implementation from baseline, by an average of 23.4 min (95% CI 16.9 to 30.0; p≤0.001) more than the control.
Process measures
Perceived acceptability, appropriateness and feasibility of the policy
Teacher’s mean scores (out of a total score of 5) for the perceived acceptability, appropriateness and feasibility of the policy were; acceptability (mean 3.81, SD 0.70), appropriateness (mean 3.81, SD 0.67) and feasibility (mean 3.59, SD 0.82) indicating an overall moderate approval29 of the PA policy.
Fidelity to and satisfaction with the multi-strategy implementation intervention
Table 1 outlines the proportion of schools that received, engaged with and were satisfied with each of the implementation strategies. Most strategies were delivered to all schools except one school did not attend the school champion training workshop, one school did not attend the educational outreach meeting, and 10 schools advised that their school had adequate equipment and declined the equipment packs. Overall school champions and teachers were very satisfied with the multi-strategy implementation intervention with the proportion of school personnel very satisfied ranging from 68% to 100%.
Discussion
Why this study is important?
This is one of few implementation trials internationally to examine the impact of strategies to improve the implementation of school PA policies and is the largest to do so. The study used a comprehensive evaluation framework to report the effects of an implementation strategy that was developed using a theoretically guided process, undertaken in partnership with end-users and drew on considerable formative research. The study found that the strategy was effective in improving initial policy implementation, and that such improvements were maintained in part, at longer term follow-up. The findings have important implications for policy makers and practitioners interested in improving student PA in this setting.
How effective was the intervention?
The size of the intervention effect (47 min) on the mean minutes of PA implemented was larger than a quasi-experimental study by Cradock et al8 in the USA (18 min) and a randomised trial by Naylor et al30 31 Action Schools! BC (AS!BC) in Canada (10 min) that also sought to support schools implementation of a 150 min MVPA policy through scheduling PE, recess and integrated classroom PA. The absolute change in minutes scheduled by the intervention groups in these studies was, however, comparable (44.2 min vs 46.5 min vs 55.2 min/week). All three studies used similar implementation strategies, including: to identify and train school champions, provide equipment and curricular materials. Similar to others32 we trained generalist classroom teachers to deliver PA, as compared with other studies that trained PE teachers and school staff wellness champions to implement the policy.8 Given the well-documented barriers generalist classroom teachers report in implementing PA11 these findings are promising given the potential population reach classroom teachers have.
Characterising the effect of the intervention
The intervention effectively increased teacher’s willingness to deliver energiser breaks. Teacher’s initial and sustained implementation of energisers contributed to 52% and 85% of the intervention effect at 12 and 18 months, respectively. This is consistent with both AS!BC32 and a 3-year RCT which aimed to increase the adoption of energisers by classroom teachers as part of the US CATCH programme.33 Undertaken in 30 Texas middle-schools the study found at the end of year 1 approximately 40% of teachers had implemented energisers which increased to approximately 48% of teachers by the end of year 2. These findings and ours suggest that energisers are acceptable, and possibly sustainable, PA strategies for teachers. This may be because energisers are characteristically short, easily embedded within or between lessons and require minimal to no equipment. However, evidence from our studies13 and others34 suggest that despite their simplicity, teachers still require some support to implement energisers. While similar implementation strategies were employed in both CATCH and our study, the intensity of ongoing support and the resources provided to teachers differed. Compared with CATCH, which provided printed resources to teachers, we promoted teacher’s use of existing online energisers. In doing so we helped teachers overcome barriers related to confidence and competence to deliver PA.11 In turn, this may reduce the need for ongoing intensive implementation support to upskill teachers, thereby potentially providing a more cost-effective, scalable and sustainable intervention.
Maintaining intervention effects
In contrast, our findings suggest that once implementation support ended, the intervention was not effective at maintaining the modest improvements in teachers’ implementation of PE (despite this being a mandatory subject) and integrated lessons. While there is limited empirical evidence, sustainability frameworks suggest that organisational factors such as funding and leadership support, staff turnover, training and programme fit, are associated with the continued delivery of health programmes in schools.35 To ensure that such interventions are resilient to attenuation over time, prior to withdrawing implementation support, future studies may consider supporting schools to: identify ongoing funding sources, establish processes that enable the handover of programme knowledge to new staff and develop plans for how the programme may be able to adapt overtime while still retaining core components.
Strengths and limitations
This is the largest cluster RCT to assess the effectiveness of a multi-strategy implementation intervention on schools implementation of a PA policy. We specifically selected implementation strategies and behaviour change techniques that addressed known barriers and were mapped against a robust theoretical framework. We assessed implementation processes and conducted a follow-up which is rarely done in school-based studies. Our study also had a number of limitations. The primary outcome relied on self-report via a log-book, a method selected on the basis of use in previous trials8 32 36 analogous evidence suggests such measures may represent a valid measure of implementation in this setting, and the pragmatics of undertaking research at such a large scale. However, such measures are at risk of social desirability and recall bias which likely lead to overestimates in the reported. Nonetheless, the use of more objective measures, that capture the fidelity to which strategies were implemented, may improve the internal validity of the trial and its findings. In addition, increasing the frequency that such data is collected throughout the study period could identify any seasonal impact on scheduling. Further, increased scheduling of PA does not guarantee that increased activity is delivered, delivered to a standard that increases students MVPA or that all students participate. For example, in our pilot study, despite an increase of 36 min in teachers weekly scheduling of PA, we saw only an approximately increase of 15 min in student weekly MVPA. The implementation strategy was developed using a theoretically guided process and drawing on considerable formative evaluation undertaken in the setting. However, the process may not have considered in sufficient detail the extent to which characteristics of schools may interact with core components of implementation intervention components and other contextual factors to enhance or impede implementation success. A more nuanced strategy development process articulating, and then assessing and reporting these interactions may have provided useful insights to guide future implementation efforts. Finally, a deeper understanding of what helped drive the intervention effect could have been explored more rigorously through a comprehensive approach such as that recommended by McKay et al37 using both qualitative and quantitative measures.
Conclusion
School PA interventions must be effectively implemented at scale if we are to achieve public health benefit.37 However, a recent systematic review reported that scaled-up PA interventions lose up to 60% of their prescale effect.38 A primary impediment to the successful implementation at scale is the selection of interventions that are not amenable to scale-up. This trial exceeded the intervention effect from the pilot study suggesting that both the PA practices and the implementation intervention is amenable to scale. However, future studies are needed to determine the minimal intervention ‘dose’ required to sustain schools delivery of all intervention components and the cost to do so.
Key messages
What are the findings?
The 12-month multi-strategy implementation intervention significantly increased teachers’ implementation of weekly minutes of physical activity (PA) and the proportion of teachers complying with a mandatory PA policy.
Teachers’ implementation of energisers contributed the most time to the intervention effect at 12 months and 18 months, suggesting they are amenable school PA practices for scale and sustainability.
The intervention had very little effect on teacher implementation or maintenance of other PA practices (ie, physical education, sport and integrated lessons).
How might it impact on clinical practice in the future?
Policy makers and researchers looking to support schools implement efficacious PA policies or programmes should consider the use of a theoretically designed, multi-strategy implementation intervention, targeting known barriers to implementation. This may help overcome the limited effects found in school-based PA programmes once they move from efficacy to scale.
Data availability statement
The data that support the findings of this study are available from the corresponding author upon reasonable request.
Ethics statements
Ethics approval
Ethical approval was obtained from the Hunter New England Human Research Ethics Committee (no. 06/07/26/4.04), The University of Newcastle Human Research Ethics Committee (H-2008-0343) and relevant school bodies.
References
Footnotes
Twitter @adamjsmith_92, @luke.wolfenden1
Contributors Thankyou this shoudl be
CO, AH and CLecathelinais developed the analysis plan.
Funding LW was supported by an NHMRC Career Development Fellowship (APP1128348), Heart Foundation Future Leader Fellowship (101175) and a Hunter New England Clinical Research Fellowship; RS was supported by an NHMRC TRIP Fellowship (APP1150661).
Competing interests Authors NN, RS, KG, NM, MP, RJ, VA, JW and LW receive salary support from Hunter New England Local Health District, which contributes funding to the project outlined in this study. Similarly, author CR and receive salary support from the New South Wales Health Office of Preventive Health which also contributed funding to this project. All other authors declare that they have no competing interests. The project is funded by the National Health and Medical Research Council (NHMRC) Partnership Project grant (APP1133013). The NHMRC has not had any role in the design of the study as outlined in this protocol and will not have a role in data collection, analysis of data, interpretation of data and dissemination of findings. As part of the NHMRC Partnership Grant funding arrangement, the following partner organisations also contribute fund: Hunter New England Local Health District and the NSW Health Office of Preventive Health. Individuals in positions that are fully or partly funded by these partner organisations (as described in the Competing interests section) had a role in the study design, data collection, analysis of data, interpretation of data and dissemination of findings. At the time of this study NN was supported by an NHMRC TRIP Fellowship (APP1132450) and a Hunter New England Clinical Research Fellowship; LW was supported by an NHMRC Career Development Fellowship (APP1128348), Heart Foundation Future Leader Fellowship (101175) and a Hunter New England Clinical Research Fellowship; RS was supported by an NHMRC TRIP Fellowship (APP1150661).
Patient and public involvement Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.
Provenance and peer review Not commissioned; externally peer reviewed.