Background With the heightened awareness of concussions in all sports, the development and implementation of effective prevention strategies are necessary. Education has been advocated as an effective injury prevention intervention.
Purpose To examine the effectiveness of the ‘Smart Hockey: More Safety, More Fun’ video on knowledge transfer among minor league hockey players.
Study Design Cluster-randomised controlled trial.
Methods A total of 267 participants from two age divisions and competitive levels were assigned to either a video or no-video group. The video was shown (or not shown) to the entire team as a result of random assignment. To evaluate the effectiveness of the educational video, questionnaires specific to concussion knowledge and players’ attitudes and behaviours were completed.
Results There was a significant increase in the players’ concussion knowledge scores immediately following exposure to the video (F(1,103)=27.00, p<0.001). However, concussion knowledge at 2 months was not significantly different between the video and no-video groups, after controlling for prior knowledge level, age and competitive level (F(1,115)=0.41, p=0.523). Similarly, players’ attitudes and behaviour scores at 2 months did not differ between groups (F(1,115)=0.41, p=0.507).
Conclusions We were able to show that a single viewing of an educational video in hockey could immediately improve knowledge about concussion but that this effect was transient and lost at 2-month follow-up. Future prevention endeavours in hockey and other sports should attempt to incorporate strategies and modalities to enhance knowledge retention.
- Ice hockey
- Injury Prevention
- Head injuries
- Intervention effectiveness
Statistics from Altmetric.com
Ice hockey is one of the most popular of all winter sports. Canada has over 500 000 youth players registered on an annual basis1 and the sport is growing rapidly in the USA and other northern countries. At the youth level, concussions are one of the most frequent types of injuries.2–4 The inherent characteristics of hockey (eg, high acceleration/deceleration, body checking and limited space) predispose players to a higher likelihood of injury.5 Body checking has been associated with a large number of these injuries.6 ,7
Body checking is a common cause of many ice hockey injuries, including concussions.7–10 The age and competition level when body checking should be introduced in hockey has been debated for many years. Recently, Hockey Canada approved a body checking rule change to remove body-checking from Peewee levels (under 13 years old) and legal body checking will be introduced at the Bantam level (13 year-olds) starting in 2013–14.11 However, in 1998/1999, Hockey Canada introduced a 5-year voluntary pilot programme where the legal age for body contact was lowered to the Atom division (10-year-olds and 11-year-olds). Cusimano et al8 found that there was a 10-fold increase in the odds of sustaining body checking-related injuries for Atom players in the 5 years following this rule change to allow body checking at the Atom level. In addition, the incidence rate of concussion among Ontario minor league players jumped from 4.81% among novice players to 21.63% among Atom players.
Despite rule changes and protective equipment, traumatic brain injury (TBI) and other injuries continue to be prevalent among hockey players and remain a primary concern. Education has been advocated as an effective injury prevention intervention.12 A number of educational programmes have been developed to teach athletes and coaches about concussion prevention safety in hockey (eg, Centers for Disease Control—‘Heads Up: Concussions in High School Sports’).13 Video is an attractive teaching medium because it can be widely distributed and the format is familiar to all age groups. Recently, more novel approaches have been used, such as using video games in an educational setting as a means of promoting concussion knowledge acquisition in youth hockey players.14 Although a general attitude exists among the public that education is an effective means of injury prevention and knowledge translation, the empirical evidence in support of this position is scant.15–19 A recent systematic review on the effectiveness of interventions in preventing injuries in hockey failed to show definitive evidence supporting the effectiveness of educational interventions.20 Models of health behaviour suggest that awareness is a necessary step in changing health-related behaviour21 and in fact, when education is combined with other interventions, it may be effective in reducing injury.22 More evidence is required in understanding what sorts of educational interventions are useful in this continuum.
The purpose of this evaluation was to assess the effectiveness of the ‘Smart Hockey: More Safety, More Fun’ video on knowledge transfer among minor league hockey players. Specifically, this paper will focus on concussion knowledge transfer related to the Smart Hockey video and provide initial insight into the attitudes and behaviours of minor hockey league players.
Participants were recruited from two age divisions, Atom (10-year-olds) and Bantam (14-year-olds), as well as from two competitive levels, competitive (AA) and recreational (house league (HL)) within the Greater Toronto Area (GTA). A total of 267 players with a mean age of 11.6 years from a total of 8 Atom AA (n=106 players), 5 Bantam AA (n=60 players), 9 Atom HL (n=54 players) and 9 Bantam HL (n=47 players) teams initially consented to participate in the study (see figure 1 for summary of study participation). Consent was obtained at two levels. First, coaches consented to their team's participation. Once the overall team was a part of the study, individual players gave assent and the legal guardian/parent gave written consent to completing the questionnaires. St Michael's Hospital research ethics board approved all methods.
The Think First Injury Prevention Foundation and representatives from the National Hockey League Players Association, Hockey Canada, the Canadian Academy of Sport and Exercise Medicine and JOFA sporting equipment developed a video called ‘Smart Hockey: More Safety, More Fun’ (Smart Hockey video), which could be widely distributed to coaches and parents in hockey leagues. Professional hockey players whom children would immediately recognise and respect, as well as noted experts, delivered messages about injury prevention aimed at younger audiences. The video included information on the mechanism of concussion, in-game tactics to reduce high-risk manoeuvres and guidelines for leaving and returning to play.
The concussion knowledge (CK) questionnaire was developed through expert review and initially pilot tested on 75 10-year-old and 14-year-old players from HL and AA competitive levels. No substantial changes were made to the questionnaire following pilot testing, only minor grammatical changes. The questionnaire consisted of 11 multiple choice questions and two open-ended questions, designed to assess concussion knowledge, such as mechanism of concussion, signs and symptoms of concussion and return to play guidelines (figure 2). For the purposes of this study, we calculated the concussion knowledge score, as the sum of questions 1–11 where each correct answer was given a value of 1 and all others 0. The qualitative responses to the two open-ended questions were not included in the analyses.
Attitude and behaviour questionnaire
An attitudes and behaviour (AB) questionnaire was also developed in a parallel process to the concussion questionnaire through expert review and consisted of 11 questions; questions 7–10 were about players’ attitude while questions 1–6 and 11 were about their behaviour (figure 3). Answers corresponding to a more aggressive or unsafe attitude were given a value of 1 and others, 0. Thus, higher scores on AB were indicative of a more aggressive or a more unsafe attitude or behaviour.
Once teams agreed to participate, a research assistant randomly assigned teams by a coin flip within each age and competitive group to either a video or no-video group. The video was shown (or not shown) to the entire team as a result of random assignment. Appointments were scheduled for a member of the research team to meet with each hockey team individually after a practice or game at the team's hockey arena in order to administer the questionnaires to all consenting players. Both groups completed the CK and AB questionnaires at the beginning of the meeting (‘baseline’). The video group watched the Smart Hockey video and completed the CK questionnaire immediately after the viewing (‘post-video’), while the no-video group received just a discussion about injuries in hockey and a question and answer opportunity. These discussions were consistent across groups and research assistants presented information about concussion diagnosis and management. Two months after the initial meeting, both the video group and no-video groups completed the CK and AB questionnaires (‘2-month follow-up’).
Statistical Analysis Software (V.9.2) for Windows was used for analyses in the current study. The primary outcome measures were concussion knowledge and attitude and behaviour at 2-month follow-up. To examine differences between the video group and no-video group at 2 months, we used a mixed-effects analysis of covariance (ANCOVA), allowing for random effects of team cluster and controlling for baseline scores, division and level as covariates. In addition, we also examined the CK and AB scores immediately following presentation of the Smart Hockey video. Since the unit of randomisation was the team, we also carried out a parallel analysis using the team's average scores on each of the questionnaires as the outcome of interest at each session and examined the changed scores at the team level for each group.
Group and player characteristics are summarised in table 1. Eighty-seven players from eight teams (5 Atom; 3 Bantam) completed the questionnaires prior to and following the viewing of the Smart Hockey video. Of the 87 players who watched the Smart Hockey video, 61 players completed the questionnaires at the 2-month follow-up. Seventy-four players from nine teams (6 Atom; 3 Bantam) assigned to the no-video group completed the questionnaires at the initial meeting and at the 2-month follow-up. Since our primary objective was to determine concussion knowledge and attitude and behaviour at 2-month follow-up, 132 players were lost due to incomplete data. The average numbers of years played by all players was 6.00 years (SD=0.77), and there was no significant difference in years played among the video group group, no-video group, and those lost due to incomplete data (p=0.77).
Means and SD of the CK score for each group are shown in table 2. With regard to the primary objective of the study, concussion knowledge initially and scores at 2 months were not significantly different between the video and no-video groups, after controlling for prior knowledge level, age and competitive level (F(1,115)=0.41, p=.52). See table 3 for a summary of results of the ANCOVA analyses.
There was a significant increase in the players’ CK scores immediately following exposure to the video (F(1,103)=27.00, p<0.001). There were also main effects for division (F(1,103)=17.81, p<0.0001) and level F(1,103)=10.70, p=0.002). Specifically, the older players (Bantam) scored on average 1.84 points (95% CI 0.97 to 2.70) higher than did Atom players. The group of players categorised as competitive (‘AA’) recorded significantly higher CK scores than recreational level players (mean difference=1.42, 95% CI 0.56 to 2.28). Overall, regardless of the players’ division and level, there was a significant increase in players’ CK scores immediately following presentation of the video.
Results of the parallel analysis at the team level found that the average CK change score for the video group was 1.89 points higher than the baseline values immediately following the video. The increase appeared to be transient CK scores for the players at the Atom level with an average CK change score of −0.32 when baseline and 2-month follow-up scores were compared. On the other hand, the CK scores of the Bantam level teams in the video group remained elevated at the 2-month follow-up as the mean CK change score was 1.86 points higher compared to baseline. The mean change score for the teams for the no-video group were similar to baseline values, with an increase of 0.39 at the 2-month follow-up (see table 4 for a summary of the team level analysis).
Attitude and behaviours
An AB score is an indication of a player's aggressiveness in attitude and behaviour. Accordingly, the higher the AB score, the more aggressive a player performs. For AB scores at 2 months following a viewing of the Smart Hockey video, there were no differences between the groups, after controlling for baseline AB score, age and competitive level (F(1,115)=0.41, p=.507). See table 3 for a summary of results of the ANCOVA analyses. Also, examining the team's average score revealed similar results (see table 4); the average AB scores were seen to be quite low and the change scores comparing 2-month follow-up with baseline did not differ between groups (video group=−0.27; no-video group=−0.06).
We were able to show that a single viewing of an educational video in ice hockey could immediately improve knowledge about concussion, but this effect was transient and lost at 2-month follow-up. These results are consistent with prior research that has shown that educational videos can be effective in the short term.23–26 However, the main finding that a single viewing of a video did not lead to long-term knowledge retention is not surprising. Research in knowledge transfer indicates that individuals vary in the optimal learning methods—different learners learn differently—some are visual learners while others are more oral while others are more experiential in their learning preferences.27 Thus, there is no evidence to indicate that one specific modality (eg, educational videos, websites, seminars) is the best technique to employ. Future prevention endeavours in hockey and other sports should incorporate a variety of educational modalities.
The failure of our study to show sustained results is also consistent with what is known about knowledge transfer. Knowledge transfer quite likely occurs through a variety of mechanisms; thus, it is not surprising that a single modality without reinforcement did not result in long-term change in knowledge, behaviour or attitude. Although it was possible that the consistent reinforcement for the principles espoused in the video did occur in the teams that showed sustained change, we did not measure this intrateam or league reinforcement of such concepts. Reminders have been shown to be effective;27 ,28 therefore, it may be valuable to incorporate dressing room reminders such as posters and game plan sheets for coaches in the future. Having strategic rule changes such as no head hitting rules that are consistently and firmly enforced could have the effect of this constant reinforcement. Other rules such as fair play concepts and rules could also provide this consistent level of reinforcement.29 ,30 Legal, financial and other incentives and disincentives provided to leagues and coaches could also be a form of such reinforcement. Given the ubiquitous nature of social media particularly among youth, using platforms such as Facebook and Twitter to disseminate concussion knowledge may be promising avenues to explore as well. Thus, combining strategies with educational interventions such as the ThinkFirst Smart Hockey video will quite likely have better results. Future research will be required to test this hypothesis.
One of the expected strengths of the research design was an attempt to capture different age groups and competitive levels of play. Specific to the concussion knowledge scores, there were no differences between levels of competition; however, we observed significantly higher scores for players in the older age group. These differences may be a reflection of a longer period of exposure to educational opportunities (coaches, parents, media, etc).31 The exploratory analysis of averaged team scores for CK also provided important insight regarding potential differences between the Atom level and Bantam level players. Despite the fact that the magnitude of change scores for the video group were similar for both the Atom level and Bantam level teams immediately postvideo, the scores differed at the 2-month follow-up. Specifically, the Bantam level teams’ averages remained elevated at the 2-month follow-up and the Atom level teams’ averages returned to baseline levels. This exploratory analysis suggests that the older age groups retained the concussion knowledge compared to the younger age group. Given these differences between the Atom and Bantam players, it may be appropriate to develop age-specific educational initiatives or reminders, which may result in more effective long-term knowledge transfer for hockey players of younger age.
No significant differences were observed between groups, age of players and competition level on scores for attitude and behaviour. We also explored the interaction between level of play and age of players—which was also not significant in the study sample—since previous research had suggested that players aggress because they have learnt that aggression is rewarding.32 ,33 This may be due to the fact that players at all levels and in all age groups receive consistent cultural messaging that quite likely emanates from potentially aggressive parents, ‘hard line’ coaching styles, the media, television personalities and negative professional role models. Nevertheless, the influences that define the culture of ice hockey and sport more generally need to be considered in prevention strategies because ignoring them will undermine and negate any attempt at prevention.
Our study has numerous limitations. First, there was no universal acceptance of the study by all teams. It could be that the teams that consented were different from those that did not consent. Although we randomised the teams, participation from individuals had to be voluntary, and so only a subset of those who consented provided a full set of data at all three data collection times. Different teams who received the intervention or discussion could have received the messages differently and this may have confounded the results; however, a single research assistant delivered all the sessions in both groups. Measuring the confounding influences or constant reinforcement or discouragement of messaging was not feasible with the budget for the study. Although we measured the injuries in our study, this was difficult without a standardised surveillance system in place. Future studies should explore these outcomes in both sexes, explore the multifaceted approaches and incorporate the injury outcomes measured by a standardised surveillance system.
We were able to demonstrate that a single viewing of a sports safety video was able to show transient effects in knowledge and attitudes and behaviours. How to achieve sustained beneficial effects will require larger scale, multifaceted approaches that include injury outcomes.
What this study adds
We were able to show that a single viewing of an educational video in ice hockey could immediately improve knowledge about concussion but that this effect was transient and lost at 2-month follow-up. The present study indicated age-related differences in concussion knowledge and retention; therefore, it may be appropriate to develop age-specific educational initiatives or reminders, which may result in more effective long-term knowledge transfer for hockey players.
Contributors MDC, MC and PD designed the study. MDC, MC and MH analysed the data. All authors interpreted the results, drafted the manuscript and decided to submit the manuscript. MDC is the guarantor.
Funding This study was funded by the Ontario Neurotrauma Foundation (# ONF-02006-O) and Canadian Institutes of Health Research Strategic Team Grant in Applied Injury Research (# TIR-103946). The funding organisations had no role in the design of the study and the collection, analysis and interpretation of the data.
Competing interests None.
Ethics approval St Michael's Hospital.
Provenance and peer review Not commissioned; externally peer reviewed.
▸ References to this paper are available online at http://bjsm.bmj.com
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.