Background There is a lack of effective intervention strategies that promote physical activity (PA) in school children. Furthermore, there is a gap between PA intervention research and the delivery of programmes in practice. Evaluation studies seldom lead to adaptations in interventions that are subsequently evaluated by implementation on a wider scale. The stepwise development and study of JUMP-in aims to add knowledge to better understand how, when and for whom intervention effects (or lack of effects) occur.
This paper describes the stepwise development of JUMP-in, a Dutch school-based multi-level intervention programme, aimed at the promotion of PA behaviour in 6–12-year-old children. JUMP-in incorporates education, sports, care and policy components. JUMP-in consists of six programme components:
Pupil Follow-up Monitoring System;
School sports clubs;
In-class exercises with “The Class Moves!”;
Personal workbook “This is the way you move!”;
Parental Information Services;
Extra lessons in physical education, Motor Remedial
Teaching and extra care.
The process and effect outcomes of a pilot study were translated into an improved programme and intervention organisation, using the RE-AIM framework (Reach, Efficacy, Adoption, Implementation and Maintenance).
This paper presents the process and results of the application of this framework, which resulted in a widescale implementation of JUMP-in.
Results The application of the RE-AIM framework resulted in challenges and remedies for an improved JUMP-in intervention. The remedies required changes at three different levels: (1) the content of the programme components; (2) the organisation and programme management; and (3) the evaluation design.
Conclusions Considering factors that determine the impact of PA interventions in ‘real life’ is of great importance. The RE-AIM framework appeared to be a useful guide by which process and effect outcomes could be translated into an improved programme content and organisation.
Statistics from Altmetric.com
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.
Strong evidence supports the need for urgent action to increase physical activity (PA) among youth.1 2 The benefits of regular PA in children are well documented. PA is important in maintaining physical and mental well-being and can prevent overweight and obesity.1,–,8 Many children, however, do not engage in sufficient PA to gain health benefits, including in The Netherlands.9 10 De Vries and colleagues11 showed that in deprived city areas in The Netherlands only 3% of the children met the PA Public Health recommendation. Promoting PA among children is a complex challenge. Studies that incorporated whole-of-school approaches, including curriculum, policy and environmental strategies, appeared to be more effective than those that incorporated curriculum-only approaches.12,–,14 In 2002 the Municipal Health Service and the Municipal Sports Service of the city of Amsterdam started the development and implementation of JUMP-in, a Dutch intervention that aims to prevent overweight by stimulating PA in schoolaged children in socially and economically deprived areas in Amsterdam. The intervention incorporates policy, environmental, and individual components and involves municipal authorities and agencies, primary schools, local sport services, sports clubs and youth health care.
The stepwise development of JUMP-in, a schoolbased multi-strategy approach
Step I (2002–4): Development and pilot testing of the first version of JUMP-in
The development process of JUMP-in started in 2002, based on the Precede Proceed model.15 The Intervention Mapping protocol16 was applied in order to systematically design the intervention. A complete description of the development of the pilot programme can be found in Jurg et al.17 The pilot programme entailed six components: (1) extracurricular school sports activities; (2) a Pupil Follow-up System, monitoring children's PA behaviour; (3) regular breaks for PA by means of calendars with in-class exercises (“The class moves!”); (4) game cards with assignments, aiming at increasing awareness and self-efficacy with regard to PA (“Choose your card!”); (5) Parental Information Services aimed at increasing knowledge and awareness with regard to children's PA; and (6) an activity week with a variety of sports for children and parents, demonstrations, a sports market, etc. The feasibility and quality of the JUMP-in intervention components were tested in a pilot study in the period 2002–4. This pilot study provided information about the effects on PA and the social cognitive determinants in a population of school children aged 9–12 years from six primary schools. The pilot evaluation also provided process measurements, which were used to illuminate the effect evaluation. The results of the pilot evaluation have been reported previously.18,–,20
Step II (2005): Translation of pilot outcomes into an adapted programme
The process and effect outcomes of the pilot study were translated into an adapted programme and organisation, using the RE-AIM framework (Reach, Efficacy, Adoption, Implementation and Maintenance; table 1).21,–,23
Step III (2006–9): wide-scale implementation
In 2006 the implementation of the revised JUMP-in programme, including a detailed formative, process and effect evaluation, started in 60 schools in deprived city areas in Amsterdam. This paper presents the results of the translation of the pilot outcomes into an adapted programme, conducted in step II.
Methods Step II: the application of re-aim
This paragraph describes the translation of the pilot evaluation outcomes into an adapted programme and organisation. The RE-AIM framework provides a tool to evaluate the impact of interventions not solely by its efficacy, but also by the process of delivery, and by its institutionalisation.22 24,–,26 Successful delivery, high implementation fidelity and incorporation of the intervention method into the daily routine contribute to the effectiveness of interventions. The RE-AIM framework operationalises this process in the dimensions Reach, Adoption, Implementation and Maintenance. The integration of these dimensions is important in the translation of research into practice.27,–,30 By considering effect and process outcomes across the RE-AIM dimensions, the effectiveness of intervention methods, delivery strategies and planning procedures can be increased. Below, the JUMP-in pilot results are addressed across the five RE-AIM dimensions. In addition, a summary of the encountered key challenges and remedies for an increased overall impact of the programme is provided (table 2).
Results for the dimension Reach
The primary target group of JUMP-in is children aged 6–12 years (1). Important intermediate target groups are (2) parents, (3) schools — school directors, school staff and physical education (PE) teachers, (4) local municipalities — city districts’ policymakers, sports coordinators — and (5) local sports clubs.
Ad (1). Children
A major advantage of the JUMP-in intervention is the relatively easy access to children through the school. However, the number of children reached differed for the several programme components. All children were reached by the programme components that were part of the in-school curriculum, but after-school sports activities reached mainly children who were already active. Based on qualitative data, it can be concluded that children who would benefit the most from participating — because of their overweight, physical inactivity or motor disabilities — were the hardest to reach.
Ad (2). Parents
Parents were relatively easy to reach through the school as well; written information reached, in principle, all parents. Nonetheless, information meetings organised at school reached predominantly parents who were already aware of the importance of PA for their children and who were ready to support their children. Again, the parents who needed information the most — because of their lack of knowledge and their own unhealthy lifestyle — proved to be the hardest to reach.
Ad (3–6) Schools, local municipalities, sports clubs
It was not difficult to reach the schools and the local municipalities, because they are part of existing networks and easy to approach. Sports clubs are part of existing networks too, but not all were easy to reach, because some trainers are volunteers. Furthermore, in most of the deprived city areas there is a lack of sports clubs. Instead, city districts organise a variety of easy accessible sports offers for the local youth, organised after school time in public playgrounds and local sports locations. These activities, however, mainly consist of shortterm sports courses, sports competitions and PA games, all without further engagement. Because the coordinators and trainers of these local sports activities are employed by the city districts, they were easy to approach.
Results for the dimension Efficacy
The pilot JUMP-in succeeded in influencing PA among children; for children of control schools the level of PA decreased considerably, corresponding with the trends in this age group,31 32 whereas the children of intervention schools became only marginally less active. After one school year JUMP-in appeared to have succeeded in preventing children from becoming less active. A common weakness in the assessment of effectiveness regarding PA behaviour in youth is the lack of adequate assessment procedures to measure PA among youth33,–,35). A gold standard is not available, which hampers the assessment of changes in PA, or in hypothesised causal determinants. In the pilot study, data on PA were based on self-report by questionnaires. Self-report data suffer from memory bias, problems with concentration and comprehensibility. Furthermore, the cognitive requirements for completion of a questionnaire about PA are high for young children, which influences reliability and validity of the results.36 Therefore, in the evaluation study of the pilot, only the older groups (aged 10–12) participated.
The pilot success in preventing children of intervention schools from becoming less active could not be explained by changes in the hypothesised determinants. Only in Grade 4 were some effects on potential determinants identified (ie attitude and habit strength). Previous studies have shown that, in contrast to hypotheses derived from various social cognitive theories,37 38 changes in behaviour can be realised without first changing the underlying cognitive determinants. These findings suggest that environmental changes (eg in-class exercises and school sport activities) may suffice for behavioural change in children.35 39,–,42
Results for the dimension Adoption
The willingness to participate among school directors, PE teachers, city districts’ policymakers and sports clubs was high. All the city districts, and almost all the eligible schools that were offered participation, were willing to participate. Encouraging the school staff to implement the programme components, however, was rather difficult and had to be repeated for every new component. One aim of the process evaluation was to ascertain how PE teachers and the school staff perceived the programme, in terms of characteristics that were hypothesised to influence adoption and implementation, such as outcome beliefs, subjective norms and self-efficacy.18 20 43 44 In the pilot, linkage systems were used as an interactive dissemination approach, to improve the fit between programme planners and users.16 These linkage systems, in which programme leaders and the programme implementers (at school, city district and sport level) participated, created a means to exchange information and ideas. This joint involvement supported the development of needs-based, tailored programmes and implementation strategies. Nevertheless, there were factors hampering adoption among the school staff in the pilot, mainly related to three factors. First, inefficient and insufficient communication about the project led to differences in expectations and lack of clarity about tasks and responsibilities. Second, the school teachers felt they had had insufficient preparation time, which led to resistance against joining the project. Finally, during the pilot period the programme developed continuously and new networks were built. Despite the linkage systems, the lack of formal guidelines about how to fulfil new tasks was found to hamper the cooperation of participants from time to time.
Results for the dimension Implementation
The pilot schools implemented most of the components of the programme. The effort and commitment of the city districts, PE teachers and school directors appeared to be important factors in the prediction of successful implementation. Nonetheless, there were some impeding factors that hampered the implementation process18 20:
▶. The content of some new components, such as the Pupil Follow-up System, and ‘Choose Your Card!’ was still in a developmental phase;
▶. Differences in and lack of clarity with regard to outcome expectations, responsibilities and sets of tasks;
▶. The lack of behavioural capabilities of PE teachers and the school staff, especially regarding components that required relatively new skills, such as the Pupil Follow-up system, parental information, “Choose Your Card!” and “The Class Moves!”;
▶. The lack of skills among PE teachers to coordinate the implementation and to coach sports trainers;
▶. The cooperation between the city districts, schools and sports clubs was not always facilitated by existing networks or past contacts;
▶. Practical factors sometimes hampered the implementation (eg a fully booked school year, the workload of teachers).
Results for the dimension Maintenance
The pilot scored positive at the maintenance level; all schools and city districts decided to continue the programme and to embed JUMP-in into their policy. City districts allocated money and manpower to ascertain continuation of JUMP-in, by structural finances for sports facilities and by structural employment of PA managers in the city district, and PE teachers, to fulfil the management and coordination tasks connected with JUMP-in. Moreover, the city districts extended the programme to many other schools in their local area. Examples of factors that facilitated embedding were the effort and commitment of project members and project targets that were in line with the policy of the involved organisation.
The process evaluation showed that 2 years were needed to achieve full embedding of JUMP-in into policy. Factors that hampered embedding the most were absence of guidelines about the tasks of the implementers and the lack of formalised contracts and agreements before the programme started. Furthermore, it can be concluded that, to reach continuity, professionalism (quality) and uniformity, effective formalised partnerships in the field of sports, health care and education have to be built, with shared vision, clear strategic and operational objectives and a whole-system approach to tackle overweight and physical inactivity.
Towards an improved JUMP-in programme
The application of the RE-AIM framework on the results of the pilot evaluation resulted in challenges and remedies for an improved JUMP-in programme (see table 2). The remedies required changes at three different levels: first, the content level of the programme components; the methods and strategies selected to achieve an increase in PA behaviour. Second, remedies were needed at the level of organisation and programme management. This second level is crucial for the quality of putting together effective local planning and action. Third, the evaluation design was adapted regarding key internal and external validity factors.45 46
A. Programme components
The most important weaknesses concerning the pilot components were: (1) the uni-dimensional focus on sports participation, instead of daily PA behaviour; (2) the lack of attention to hampering factors at individual level, such as overweight and being delayed in motor development; (3) the fact that school sports clubs especially reached already active children; and that (4) the game approach of “Choose your card” did not reach parents and did not affect awareness among children. Furthermore: (5) the information meetings did not reach all parents; (6) the messages were not tailored to the individual child/family; and (7) the activity week ended up in an arbitrary set of unstructured fun activities, due to a lack of specified goals.
While the complete programme is implemented for all children, certain programme components specifically target inactive and overweight children and parents, such as Parental Information Services, extra care and Club Extra. Other programme components give priority to inactive and overweight children, such as school sports clubs.
Table 3 presents an overview of the revised JUMP-in programme components, after the application of the RE-AIM framework, including the aims, change objectives and underlying theories.
I. The Pupil Follow-up System (PFS)
The PFS is a monitoring instrument that identifies (changes in) risk factors and care needs by annually assessing and registering children's level of PA, BMI and motor skills. The PFS also contains an attendance list that registers the presence of children at after-school activities. PFS facilitates tailored solutions for individuals in a structured way, such as motor remedial teaching, physiotherapy, referral to youth health care. In cases in which additional support or care is required, existing school network channels are used. Furthermore, the PFS creates the opportunity to tailor the other programme components to the characteristics of the school and environment. The PE teacher is trained to use the system via a password-protected area on the JUMP-in website.
II. Daily exercises with “The Class Moves!”
This programme offers recurrent breaks for PA, relaxation and posture exercises, during regular lessons, by means of calendars. For each grade the materials are adapted to the level of sensory motor development. An instruction book for the school teacher is available. The aim is to make PA a daily habit, to increase enjoyment in PA and to contribute to a healthy sensory motor development. Although the materials were not revised after the pilot, the refined instruction for the school staff and the implementation are professionalised by certified implementation supervisors from a school advisory service. In future implementation, agreements are made with the director and school staff about the extent to which the programme will be used, and the implementation is accompanied by a process evaluation and follow-up instructions if needed.
III. Personal workbook: “This is how you move!”
The game approach “Choose your Card!”, carried out in the pilot period, is replaced by the method “This is how you Move!” This method consists of personal workbooks for children and their parents, with assignments to perform in class and at home, and an instruction book for the school staff. The method is especially aimed at raising awareness of the importance of PA for health and one's own PA behaviour and at improving self-efficacy, social support, self-regulation and planning skills of both children and parents. The assignments vary from “design and perform your own favourite exercise for inside the house/with the family” to “make a plan for a week's schedule for daily PA”. The children regularly bring the book home to make assignments together with the parents. Informational messages for parents are embedded in the workbook. The instruction for the school staff is provided by the School Supervisory Service.
IV. School sports activities
In or near to the school premises, continuously easily accessible school sports activities are offered on a daily basis. During school hours children will get acquainted with a variety of sports. Subsequently they can join the club during out-of-school hours. Existing local offers of physical activities and sports clubs are involved. In case of waiting lists for a school sports club, priority is given to children not yet participating in organised sports. Children who are not yet ready to participate in regular school sports, such as children who are overweight or have low self-esteem or low (perceived) sport competence, receive adapted sports. These adapted sports provide a safe social environment that allows children to enjoy physical activity and become competent in sport-related skills.
V. Parental Information Service
Parents are important and responsible for the PA behaviour of children, and therefore the Parental Information Service in JUMP-in is intensified.47 The adjusted instruments are embedded in the entire JUMP-in programme and contain several options for tailor-made information (eg information meetings, personal consultations, courses, sports activities for parents). To reach parents, multimedia instruments and a JUMP-in information film for parents are developed, because written information reaches only marginal groups of parents. Parents are asked to support sports activities for children and for themselves, and they are involved in the assignments in “This is the way you move!” This workbook also includes informational messages for parents. Parents are also contacted by the school if their child has deficits in his or her motor development, is overweight or needs stimulation in sports. Financial support is provided for sports activities and materials, if needed. The Parental Information Service is carried out by professional information officers. Information can be given in the parents’ own language if necessary.
VI. Club Extra and extra care
Children are monitored by the PFS and those who are delayed in their motor development or who experience hampering factors in their PA behaviour (such as overweight) receive additional physical education lessons (Club Extra) or motor remedial teaching (MRT). These lessons start in small groups during school hours, given by a qualified MRT or PE teacher. After several weeks the lessons are continued after school time, if needed. If necessary, parents are involved as well. Parents are referred to the school nurse, who carries out a consultation or refers to the hospital or a dietician.
B. Organisation and programme management
Our experience confirms that a complex intervention such as JUMP-in demands a coordinated response across a number of different sectors. Organisational innovations such as formalised agreements between schools, sports and health care are needed, both in the preparation and adoption phase and in the implementation and maintenance phase.
Tools to optimise the preparation and adoption phase
Thorough adoption and preparation are needed before implementation can take place. Such implementation should be tailored to the characteristics of the school and the environment. To optimize the adoption, preparation and implementation phases, a “flow diagram” is developed, which is divided into steps. Each step needs to be successfully completed before proceeding to the next step. A school scan and environment scan are carried out, which provide information about:
▶. Policy and practice with regard to the local sports infrastructure;
▶. School and city district policy with regard to sports, education, extracurricular education programmes, afterschool childcare centres and youth health care;
▶. Existing local programmes targeting overweight and inactivity among youth, including primary and secondary prevention;
▶. Practice and policy with regard to parental participation at school;
▶. Lack of existing networks in the field of sports, extracurricular education, prevention programmes and youth health care.
A city district officer (usually the sports coordinator), school directors and PE teachers perform the scans. Subsequently, networks are created and practice and policy with regard to sports, health care and education are prepared, in order to implement the JUMP-in programme.
Keys to optimise the implementation and maintenance phase
It is clear that, to implement and embed the programme in daily practice and in policy, highly structured cooperation is required between municipal authorities, local city districts, schools, youth health care, welfare organisations, school supervisory services, local municipal sport services and local sports clubs, aiming at effective local planning and action. However, as Tones and Green48 noted, “a rational planning process can not tackle all organisation and coordination problems.” There is no single organisation controlling and coordinating all facets of the whole-system approach such as JUMP-in. Nonetheless, four main keys have been identified that improve programme management and organisation: (1) good communication; (2) clear strategic planning; (3) realistic operational objectives; and (4) the building of effective partnerships. To achieve long-term maintenance of the programme, participation is also needed in the public health policy debate (both at a local and at a national level), in order to generate attention and finances for the intervention.
C. Research design
The revised JUMP-in evaluation framework (presented in fig 1) is adapted from the Environmental Research framework for weight Gain prevention (EnRG).42 An important factor that influenced the development of the framework is the recent increased attention to social–ecological theories, which highlight the importance of environmental influences.35 49 50 Ecological models are distinct from most social cognitive theories in that they hypothesise a direct influence of the environment on behaviour, unmediated by cognitive factors.49,–,51 It has even been argued that reductions in levels of obesity and sedentary behaviour seem unlikely without modifying the environment.52 The results from the pilot study JUMP-in and previous studies39,–,41 53 appear to underline this hypothesis. Environmental changes (eg the creation of school sports clubs) appear to have led to behavioural changes, without influencing the related cognitions.19 Only few papers report analyses of potential moderators, mediators and differential environment– behaviour relationships in distinct subgroups.54 Evaluation of the revised JUMP-in intervention gains insight into the causal mechanisms by which PA behavioural change is likely to occur.
The evaluation study assessing the effectiveness of the revised JUMP-in intervention is a quasi-experimental controlled trial. The study was carried out in 19 primary schools among 2700 boys and girls in group 3–8 (aged 6–12 years) and their parents. The population mainly consists of people from socially and economically deprived areas. Baseline measurements were conducted in Autumn 2006 (T0). In Spring 2007 the first post-test (T1) was carried out, and in Spring 2008 the second post-test (T2).
Primary outcome measures include:
Total daily PA (a subgroup of 349 children), measured by Actigraph AM-7164 accelerometers, Fort Walton Beach, Florida, USA.
Sports participation by interview (all children) and questionnaires (9–12 years, all parents).
Secondary outcome measures include:
Anthropometrics: body weight, body height, waist and hip circumferences (all children);
Social environmental influences and cognitions of PA behaviour (ie attitude, perceived sport competence, habit strength, social influence, perceived behavioural control, planning skills), measured by questionnaire (9–12 years). The questionnaire is developed based on literature study,55 social–cognitive theories,37 social–ecological theories35 49 50 and the model of physical exercise and habit formation.56 The items were also developed using existing validated questionnaires57,–,60 and were pretested.
Parental determinants of supporting children's PA behaviour, by questionnaire (all parents). The parental questionnaire is developed based on literature study and focus group interviews,61 social–cognitive theories,37 social–ecological theories,35 49 50 60 and the model of physical exercise and habit formation.56 The questionnaire was also pretested.
The effect evaluation aims to determine the effect of the JUMP-in programme on social cognitive determinants, as well as PA behaviour. A mediation analysis will be performed to assess the mediating effect of hypothesised mediators. In addition, an analysis of moderators will be performed to analyse whether effects are more prominent among or restricted to certain subgroups.
The process evaluation concerns the constraints and the success and failure factors at all organisational and operational levels, linked to the implementation of JUMP-in. The process evaluation will also provide information on “health promotion outcomes”: (1) health literacy; (2) social action and influence; and (3) healthy public policy and organisational practice.64,–,67 Insight into the health promotion outcomes will explain the relation between implementation, capacity and efficacy of JUMP-in. Process data will be collected during the first and second school years by semi-structured interviews, questionnaire and document analysis, minutes and documentation of programme activities.
This paper aims to give an overview of the stepwise development of the JUMP-in intervention, using the application of the RE-AIM framework. The framework appeared to be a useful guide in combining process and effect outcomes and translating them into an improved programme content and organisation. However, despite the concrete level of the RE-AIM dimensions, it is difficult to gain insight into mutual causal relationships between outcomes on the different RE-AIM dimensions: outcomes are strongly associated with each other. For example, the game approach of “Choose Your Card!” had weak results on Adoption and Implementation and subsequently did not affect the awareness of children (Efficacy). Further, the results on the dimension Reach depended for an important part on the extent to which the programme was adopted by the implementers (Adoption). For example, the in-school programme “The Class Moves!” only reached children in classes where schoolteachers adopted the programme. However, the RE-AIM metrics provide a broad perspective of impact indices and issues important in developing evidence-based and practice-based interventions promoting PA.22 68 69
There is a lack of knowledge about effective intervention strategies to increase PA among school children.70 Furthermore, there is a gap between PA intervention research and the delivery of evidence-based programmes in practice. Evaluation studies seldom report on external validity and they seldom lead to adaptations in interventions implemented and evaluated on a wider scale.71 46 Translating research evidence into programme change is challenging, and the evidence around how to effectively promote and facilitate this process is still relatively limited.30 A common problem is that it may take years to find improvements in PA, or even in causal determinants. It has also been recognised that creating collaboration and intensive participation may take years.72,–,74 Nevertheless, considering factors that will determine the “real life” impact of evidencebased interventions is of great importance. The continuing study of JUMP-in aims to add knowledge about potential moderating and mediating variables, as well as process measures, that can help in better understanding how, when and for whom intervention effects (or lack of effect) occur.
Funding The pilot study was supported financially by the Public Health Fund (Fonds OGZ) and the Ministry of Health, Welfare and Sport. JUMP-in receives financial support from the Larger City Policy funds of Amsterdam.
Competing interests None.