Concussion, defined as a “complex pathophysiological process affecting the brain, induced by traumatic biomechanical forces”,1 is one of the most common and potentially dangerous types of injuries sustained in collision sports.2^–5 Specific risk factors are poorly delineated and rarely studied which makes primary prevention a difficult task as there is little evidence to base preventive strategies.

Equipment use such as properly fitted and maintained helmets/headgear, mouthguards and face shields are commonly purported concussion prevention strategies. Their use in some sports is obligatory (eg, football, lacrosse, ice hockey), whereas in others they are optional (eg, rugby, soccer). The benefits of helmet use has been established with empirical evidence in some sports that do not require its use (ie, bicycling),6 but there is much debate on the use of headgear in other sports (ie, rugby, soccer, rodeo, skiing, snowboarding, Australian rules football, etc). During the 1960s and 1970s, mouthguard use was mandated by several US and Canadian organised sports including football, ice hockey, lacrosse, field hockey and boxing. The premise for mandating use was not only to decrease dental injury rates, but also reduce players’ risk of concussion.7^–15 The benefits of mouthguard use in protecting athletes from dental injury is supported in the literature,7^–9 16^–23 however, controversy exists as to whether mouthguard use can reduce athletes’ risk of concussive head injuries.24^–26 Face shields attached to helmets are another piece of equipment that has been hypothesised to decrease the frequency of concussive head injuries in sport. During the l970s, full facial protection was mandated by all organised youth ice hockey associations worldwide.27 These rules, combined with the improvement of face shield standards, have been shown to reduce the frequency of facial, dental and eye injuries.8^–15 28^–31

To date, we are not aware of any studies that have systematically reviewed the evidence on the effects of all types of equipment use on concussion prevention. The objective of this manuscript is to systematically review the literature and determine if there is scientific evidence that mouthguard, helmet/headgear and face shield use, compared with non-use, reduces sport-related concussion rates and/or severity of injury among male and female athletes of all ages and levels of competition. Study limitations will be discussed and specific recommendations will be made for future research.

METHODS

Data sources

Twelve electronic databases were searched using a combination of Medical Subject Headings and text words by means of “wild cards” and Boolean operators to identify potentially relevant English and non-English articles. The search strategies administered were as follows:

1. mouthguard$ OR (mouth guard$) OR (mouth protect$) OR (gum shield) [all fields] AND (concussion$ OR (brain injur$) OR (head injur$) OR injur$) [all fields];

2. (helmet$ OR headgear$ OR (head protect$)) [all fields] AND (concussion$ OR (brain injur$) OR (head injur$) OR injur$) [all fields] AND sport$;

3. ((face shield$) OR visor$ OR (face protect$)) [all fields] AND (concussion$ OR (brain injur$) OR (head injur$) OR injur$) [all fields] AND sport$.

For mouthguard articles, the Science Citation Index Expanded (1900–2008), Social Sciences Citation Index (1956–2008) and Arts & Humanities Citation Index (1975–2008) databases (ISI Web of Science, 1900–Sept 2008) were also searched by entering a commonly referenced original study,13 revealing 43 citations. Furthermore, the bibliographies of the selected articles were hand-searched to identify additional articles not revealed by the above search strategies.

Study selection

The following inclusion criteria were used to select relevant studies: (1) original data; (2) case report, case series, cross-sectional, case control, cohort, quasi-experimental or experimental study designs; (3) published peer-reviewed abstracts; (4) outcome includes a measure of concussion sustained in sport or biomechanical study of concussion; (5) studied concussions associated with mouthguard, helmet/headgear or face shield use; (6) exposure included some measurement of mouthguard, helmet/headgear or face shield use; (7) male or female subjects; (8) all ages; (9) all levels of competition; (10) all types of sports (contact and non-contact); (11) all languages; and (12) peer-reviewed and non-peer reviewed reports. Exclusion criteria were as follows: (1) review articles; (2) commentaries; (3) letters to the editor; (4) no measure of mouthguard, helmet/headgear or face shield exposure; and (5) studies exclusively examining head injuries other than concussion associated with mouthguard, helmet or face shield use. The titles and abstracts of the identified citations were screened by two reviewers to identify all potentially relevant articles. If there was insufficient information available (eg, no abstract), the full papers were reviewed. Potentially relevant articles were obtained and the methods section reviewed to identify studies that met the pre-determined inclusion criteria.

Data extraction, validity assessment and study characteristics

The following characteristics were extracted and synthesised to assess the quality of evidence of studies selected for review: study design, study population, sample size, exposure and outcome measures, type of mouthguard, helmet/headgear or face shield studied and results. The quality of evidence was assessed based on epidemiologic criteria regarding internal and external validity (ie, strength of study design, sample size/power calculation, selection bias, misclassification bias, control of potential confounding and effect modification).

RESULTS

The electronic databases and number of potentially relevant citations identified using the above search strategies are shown in table 1.

Mouthguards

Inclusion criteria were applied to the articles identified in the search strategy which resulted in the selection of 17 studies for review (15 epidemiological studies (8 descriptive observational, 7 analytical) and 2 biomechanical studies). Table 2 highlights the individual study characteristics of these articles. A comparison between studies on this topic area was difficult due to the variability in research designs, definition of concussion, mouthguard types, measurements used to assess mouthguard exposure and concussion and variety of sports assessed.

Of the eight descriptive observational studies, five were cross-sectional surveys and three were case reports or series. Three of the five cross-sectional surveys reported results to suggest a protective effect of mouthguard use on concussions,13 32 33 whereas two suggested mouthguard use did not play a role in concussion prevention.34 35 Only one study reported statistical tests of significance. Specifically, McNutt et al revealed that 56% of all concussions sustained among 2470 football players from 21 Alabama high schools over three consecutive seasons (1984–6) were sustained by players who did not wear mouthguards (p = 0.001).32 The remaining three descriptive observational studies were case series or reports. The case report suggested a protective effect on repeat concussion over one season in a rugby player who wore a custom-fitted, triple-laminated, pressure moulded mouthguard.36 One case series revealed no post concussion differences in neurocognitive deficits (via ImPACT) between mouthguard users and non-users at the time of first follow-up assessment (mean 3 days),37 while the other case series reported no concussions sustained by one University football team over one season for players who wore custom-fitted mouthguards.38

Five prospective cohort studies were conducted to assess the association between mouthguard use (vs non-use) and concussion rates. One study provided evidence to suggest a protective effect of mouthguard use on concussion severity (measured by time loss from unrestricted participation),39 three studies found no significant difference in concussion rates,19 40 41 and one study examined concussion rates between football players wearing custom versus non-custom mouthguards and thus did not examine risk differences between users and non-users.26 The one cohort study suggesting a protective effect on concussion severity did not measure individual mouthguard exposure and therefore could not provide a measure of relative risk.39 The fifth prospective cohort was a large study of 1033 professional ice hockey players revealing that the risk of concussion was not significantly different between players wearing mouthguards versus no mouthguards (relative risk: 1.42, 95% CI: 0.90 to 2.25), however, symptom severity measured subjectively via the modified McGill ACE symptom scale was significantly greater for athletes not wearing mouthguards compared with those who did (p<0.01).41

Two cluster-randomised controlled trials (RCTs) were conducted on this topic area, one of which studied concussion rates between users of a specific type of custom mouthguard (WIPSS) versus all other mouthguards and therefore did not specifically address the primary question of interest (OR: 1.06, 95% CI: 0.51 to 1.61).42 The other RCT assessed head/orofacial injury rates in Australian rules football using two study arms (custom mouthguard vs usual mouthguard behaviours).43 The authors’ stated that the study was underpowered to determine injury risk for orofacial injury and there was no report of concussion rates for each study arm. Thus, the true association between mouthguard use and concussion risk could not be ascertained. Lastly, two biomechanical studies were conducted to address the primary question of interest, both of which provided evidence suggesting a protective role of mouthguard use in reducing impact related forces.12 44 Hickey et al revealed a reduction in the amplitude of bone deformation and intracranial pressure by approximately 50% with the use of a mouthguard,12 whereas Takeda et al showed that mouthguard use significantly decreased the distortion of the mandibular bone and the acceleration of the head compared with no mouthguard use (p<0.01).44

Helmets/headgear

Based on the inclusion and exclusion criteria, 19 epidemiological (six descriptive observational and 13 analytical) and 10 biomechanical studies were selected for review. Table 3 highlights the individual study characteristics of these articles. Again, a comparison between studies on this topic area was difficult due to the variability in research designs, definition of concussion, measurements used to assess helmet exposure and concussion, and variety of sports assessed.

In skiing and snowboarding, three of the four clinical studies reviewed were large case-control designs.45^–47 All three case-control studies reported a reduced risk of head injury among subjects wearing helmets (29%, 60% and 15% reduction, respectively). The response rate in Hagel et al45 was 70.1% whereas in Sulheim et al46 and Mueller et al47 it was 100%, which was attributed to using medical records from the ski patrollers. All studies used individuals with other injury types as controls but Sulheim et al46 also examined the relationship using a second set of non-injured controls. The studies sampled skiers and snowboarders from the hill, but they did not take into account individual skiing distance exposures. They used matching techniques and controlled for potential confounders and effect modifiers. It is important to note that head injury was the primary outcome of interest in these studies, not concussion. However, Hagel et al45 revealed that concussions accounted for 69.7% of the head injuries sustained. Fukuda et al48 conducted a case-series which did not find a significant association between helmet or knit cap wearing and the occurrence of head injuries requiring medical attention.

In rugby, two of the five studies selected for review were RCTs, two were prospective cohorts and one was a cross-sectional survey. The RCT by McIntosh et al49 utilised 16 under-15 rugby union teams and used time loss injuries as the primary outcome of interest. The authors’ found no significant injury risk reduction among teams who wore headgear. This study was limited by the small number of concussive events using a time loss definition, the specific age group used as the study population and the short duration of the study (one season). The second RCT conducted by McIntosh et al50 was much larger in scale and covered a greater number of age groups. They also found no significant differences in concussion rates between the study arms. Kemp et al51 conducted a prospective cohort study that revealed a 57% reduction in concussion risk among players wearing headgear, whereas Marshall et al40 found no significant risk reduction among headgear users in their cohort study. Kemp et al51 used an inclusive definition of concussion, the symptom-based Concussion in Sport Group (CISG) definition,52 which may have overestimated the number of concussions. Fewer concussions (n = 22) were reported in the cohort study by Marshall et al,40 and the study population included both males (n = 240) and females (n = 87), as well as individuals from different leagues and levels of play. This study may therefore not be representative of a specific target population, with the measure of effect diluted towards the null. The cross-sectional study conducted by Kahanov et al53 reported more concussions among players who did not wear headgear, but did not report a measure of effect. Concussions were self-reported in this study and the response rate was 65%. Other limitations of this study are that of under-reporting and lack of denominator data to calculate injury rates.

In American football, two of the three studies were prospective cohorts and one was a cross-sectional survey. Alles et al conducted a 3-year prospective cohort study (1975–7) using the National Athletic Injury/Illness Reporting System (NAIRS).54 Over 16 000 high school and college football athletes were utilised in this study. They found no difference in concussion risk between 13 different helmet brands and types although no statistical analyses were provided in this study to assess the measure of effect. Collins et al55 conducted a prospective cohort study on high school athletes in Western Pennsylvania revealing a 31% decrease in concussion risk for athletes wearing the Riddell Revolution helmet compared with the older traditional Riddell helmet designs. The sample size was large (n = 2141) but there were a small number of concussions reported by team trainers (n = 136). A limitation was that the new helmet design was compared with other helmet designs that may have been several years old, without controlling for age of the helmet in the analysis. Torg et al56 conducted a cross-sectional survey among 155 individuals who purchased a polyurethane football helmet cover. Concussion rates were not significantly different with the use of this cover. Selection bias was a concern in this self-report survey as the outcome and exposure measure may have been associated with their inclusion into the study (helmet purchaser).

In bicycling, several studies were identified in our search that assessed helmet use and head injury rates, including a Cochrane review administered by Thompson et al.6 This review consisted of five case-control studies utilising populations of bicyclists who experienced a crash.57^–61 Helmet use was the exposure of interest and head injury, brain injury and severe brain injury (using an abbreviated injury score, or head AIS) were the primary outcomes of interest, not concussion. Helmet use provided a 63–88% reduction in the risk of head, brain and severe brain injury for all ages of cyclists.6 An additional case-control study57 revealed a significant reduction in head injury risk when wearing a helmet. Cases consisted of patients presenting to a hospital or emergency department, with the controls consisting of individuals who sustained non-head injuries while biking as well as population-based controls.58 Furthermore, two case series62 63 reported similar risk reductions as the case-controls, however there was a small number of head injuries among subjects wearing helmets (n = 4 and n = 1, respectively). Two retrospective cohort studies reported head injury risk reductions,64 65 although Wasserman et al65 revealed that helmets did not significantly decrease the risk of concussive head injury.

Lastly, there were single epidemiological studies conducted in the sports of football (soccer), ice hockey,and rodeo pertaining to helmet/headgear use and head injury. Delaney et al66 conducted a cross-sectional study using headgear as the primary exposure of interest in football (soccer).65 The study was a non-randomised, self-report questionnaire which utilised the CISG symptom-based definition52 as the outcome of interest, where the presence of symptoms was assumed to equate to the diagnosis of concussion. Concussion frequency was significantly higher (265%) among players aged 12–17 years who did not wear headgear compared with those who did.65 The sample size was large (n = 278) and concussion frequency was high (n = 133) in this study. However, the cross-sectional design does not assess the temporal association between risk factors and outcome. In ice hockey, Kraus et al67 conducted a case control study over two seasons using the introduction of a mandatory hockey helmet as the primary exposure of interest. Head injury was the primary outcome of interest, not concussion. The authors’ found that helmet use significantly decreased the risk of a head injury. There were several biases in this study pertaining to outcome measurement and the utilisation of cases and controls from different seasons (pre and post introduction of a rule change). In rodeo, one cross-sectional study was conducted to examine the association between head injury rates and helmet use. Brandenburg and Archer68 revealed that helmet use decreased the risk of head injury by 57%. Head injury was defined as “any event that resulted in being dazed, confused, having brief visual changes, cuts or bruises anywhere above the neck, tooth or mouth injuries, headaches, loss of consciousness or facial injuries as a result of a head or face impact, or stiffness in the neck or a broken neck”. There were several limitations of this study including a low response rate (31%), self-report of concussions (potential recall bias) and the relatively small sample size (n = 81).

Biomechanical studies

There were two laboratory studies conducted using American football helmets. Viano et al69 found that the average reduction in concussion risk with newer helmets was 10.8% (range: 6.9 to 16.7%). They used a National Operating Committee on Standards for Athletic Equipment approved severity index (calculated using the resultant acceleration of the tests), to determine the reduction in concussion risk. In their analysis, all new helmets were grouped together so therefore the results do not reveal which of the new helmets had the greatest reduction in acceleration. Furthermore, the newer helmets used thicker padding than the older helmet which may have been one reason for the acceleration reduction. The authors’ did not administer any statistical tests of significance between the helmets because repeat tests for each reconstruction were not carried out with each new helmet. Lewis et al70 also found that the use of a football helmet significantly reduced mean peak Gs. Linear acceleration was the only outcome measure in this study, with no report of rotational acceleration.

For rugby headgear, McIntosh et al71 found that increased thickness of polyethylene foam improved energy attenuation, but increasing the density of the foam did not significantly alter energy attenuation. They recommended a thickness of 16 mm in headgear but also stated that the reduction in headform acceleration may not be directly related to the reduction in head injury risk in the field. In addition, they found with subsequent impacts to the same sites, the foam headgear was not as effective in reducing impact forces. Based on this biomechanical study, a subsequent RCT was performed to test headgear efficacy in the field.50

For football (soccer) headgear, four studies examined the protective effect of headgear. McIntosh et al72 found that the foam material was completely compressed at impact energy of 20J and postulated that it is unlikely for headgear to reduce concussion or severe head injuries. Hrysomallis73 examined the protective effects of seven different types of headgear and found that there was a significant correlation between headgear thickness and head injury criteria (HIC) scores. Broglio et al74 found that headgear reduced the peak force of impact when compared with the control. Withnall et al75 found that there was no significant reduction in acceleration with headgear use while heading the ball, but both the linear and rotational acceleration was reduced by approximately 33% across all headgears in the head-to-head contacts.

In addition, Caswell76 found that the contemporary helmet designs performed better than the traditional helmet designs in lacrosse, but both designs did not perform well with repetitive impacts. For bicycling helmets, Bishop77 found that helmets with polystyrene liners were much superior at reducing impacts than those helmets with soft foam liners. McIntosh78 compared cricket, baseball and ice hockey helmets. They found that cricket helmet performance was satisfactory at low impact speeds but not at high impact speeds, whereas hockey and baseball helmets offered better reduction in headform acceleration at high ball speeds.

Faceshields

Based on the inclusion and exclusion criteria, 4 clinical studies and 1 biomechanical study were selected for review. Table 4 highlights the individual study characteristics of these articles. Three of the four clinical studies were analytical (ie, prospective cohorts) and one was a large case series analysis. Benson et al2 conducted a large prospective cohort study investigating the risk of injury among 642 Canadian intercollegiate ice hockey players wearing full versus half face shields (or visors). It was revealed that the risk of sustaining a concussion was not significantly different between the two cohorts. A subsequent multivariate analysis using the same cohort revealed players who wore visors missed significantly more practices and games per concussion (2.4 times) than players who wore full facial protection (4.07 sessions (95% CI: 3.48 to 4.74) versus 1.71 sessions (95% CI: 1.32 to 2.18) respectively), and this did not depend on whether the athletes were forwards versus defensemen, rookies versus veterans or whether the concussions were new versus recurrent.39 Stuart et al79 prospectively assessed concussion rates among 282 United States Junior A hockey players wearing different types of facial protection (full, visor and none). No significant difference in concussion rates were found between the three cohorts, however the study does not appear to be appropriately powered to be able to measure the true association due to the small sample size and small number of concussions sustained during the study period (ie, 4 concussions among players wearing no facial protection, 5 concussions among players wearing visors and 2 concussions among players wearing full face shields). Stevens et al80 conducted a large case series study of concussions utilising 787 professional ice hockey players to assess concussion rates associated with visor use versus no facial protection. They found no statistically significant differences in concussion rates (OR: 1.39, 95% CI: 0.71 to 2.71) or time loss (games) (p>0.05) between the two study groups. Lastly, Lemair and Pearsall conducted a biomechanical study to assess peak acceleration (PA) differences following blunt impacts to a surrogate headform between 3 commercial model full face shields (cages) versus 3 visors.79 Facial protection substantially reduced PA during blunt impacts within threshold safety limits (below 300 g), with cages having lower PA than visors.79

DISCUSSION

To date, several concussion prevention strategies have been introduced or proposed in sport, including equipment use (properly fitted and maintained helmets/head gear, mouthguards, face shields), neck musculature conditioning, rule changes (eg, no head checking in ice hockey), strict rule enforcement and stiffer penalties for illegal play (eg, elbowing in football (soccer)), enhanced player respect and improved coaching techniques.39 82 83 None of these strategies, however, has been uniformly implemented by the governing bodies across all sports primarily because of a lack of scientific evidence to support their effectiveness in reducing injury, or philosophical differences among administrators.

Mouthguard use has been mandated in several sports world-wide in an attempt to reduce the incidence of dental and concussive head injuries. Although there is evidence to support its protective role for dental injury, the evidence of a protective effect on concussion has stemmed primarily from a limited number of case series and retrospective cross-sectional surveys. Two papers reviewed in this systematic review have been universally cited in the concussion prevention literature to support mouthguard use as an effective means of preventing concussion.12 13 Some authors have even stated that the most important value of mouthguard use in sport is the concussion saving effect following impact with the mandible.7 9 17 Despite lacking evidence of clinical effectiveness, these notions have inappropriately influenced decision-making regarding mouthguard use in sports for concussion prevention.

Several theories have been published that provide biological plausibility to support the hypothesis that custom-fitted mouthguards may protect athletes from concussive head injuries (eg, reducing force transmission to the brain via absorption and neck musculature stabilisation).7^–9 13 14 17 25 84^–86 However, traditional assumptions and theories are slowly being replaced with evidence.87 In this paper, both biomechanical studies showed a reduction in force transmission following a blow to the surrogate headform with the use of a mouthguard. Five of the eight descriptive observational studies suggested a protective role of mouthguard use on concussion frequency but such studies are fraught with methodological limitations (ie, self-selection bias and thus non-comparable groups, self-diagnosis of concussions retrospectively in time, temporal association, etc). Only two of the seven analytical studies were appropriately designed with adequate statistical power at study onset to be able to answer the primary question of interest; one of which was in basketball19 where impact forces may be very different and concussions less frequent than typically observed in collision sports. The second study was in ice hockey where the clinical significance of a protective effect cannot be discounted with a relative risk of 1.42 (95% CI: 0.90 to 2.25) over one regular season of play.41 A type II error may have been committed in this study (ie, stating there is no difference when in fact there is) with the lower incidence of observed concussions experienced during this year of study than what is typically observed over one season of play.

There are several types of mouthguards on the market today, including: (1) ready-made, or stock (type I), which does not require fitting and are held in place only when the jaw is closed; (2) mouth-formed or “boil-and-bite” (type II), which are softened in boiling water and then molded to the teeth using the fingers and tongue; (3) custom-fitted (type III) which is made by dentists using an impression of the upper dental arch and stone model of the teeth; and (4) bimaxillary which consists of one piece and covers both upper and lower teeth.17 A custom-fitted mouthguard has been reported to be necessary to ensure retention of the mouthguard in collision or contact sports; the simpler designs have not been shown to afford much protection, tend to fit poorly and reportedly interfere with breathing and speech.24 Therefore, future studies must specify exactly what is tested, and in the perfect world, assess pressure laminated, custom-fitted guards so that the occlusal surface thickness can be controlled. It has been shown that the thickness before and after moulding type II, or commercial “boil-and-bite” mouthguards, decreases 70–99% at the occlusal surface, attributed to a lack of control of pressure exerted by the wearer during fitting.17

Clinical evidence from analytical studies is provided in this review to suggest that helmet use reduces head and brain injury risk in bicycling and head injury risk in skiing and snowboarding. No study in skiing, snowboarding or bicycling explicitly studied concussion risk associated with helmet use, although Hagel et al45 did reveal that 70% of the head injuries were concussions in his case-control study of skiers and snowboarders. No analytical studies have been conducted in the sport of football (soccer) and rodeo to assess the protective role of headgear use on concussions. The single observational descriptive studies conducted in these sports suggest a potential protective role of helmet and headgear use on concussions in rodeo and soccer, respectively, but do not provide good evidence. Inconclusive evidence is provided to suggest headgear use reduces concussion risk in rugby. The role of helmet use in concussion prevention could not be assessed in American football and ice hockey as rules are in place at all levels of competition that mandate helmet use in these sports, making comparisons impossible. Biomechanical evidence suggests that newer football helmet models reduce both translational and rotational accelerations of a surrogate headform significantly more than older helmet models, as well as significantly reduce peak Gs (linear acceleration) compared with no helmet use. This may be relevant for diffuse axonal injury, but it is unknown whether it is relevant for concussion. Two biomechanical studies (one using soccer headgear and one using rugby headgear) suggested that there may be a minimum thickness required to improve energy attenuation following impact (16 mm thickness of polyethylene foam in rugby and 15 mm thickness in soccer). Furthermore, there is conflicting evidence on the protective role of headgear use in football (soccer) in reducing impact energy from ball-to-head contacts and some evidence to suggest a reduction in linear and rotational acceleration of the headform in head-to-head impacts. Although there appears to be an increase in impact-related force attenuation with the use of some headgear in the laboratory, the clinical study results highlight that there is not necessarily a relationship between the magnitude of biomechanical variables (ie, headform acceleration, energy attenuation) and the reduction in head injury risk on the playing field.

Until the late 1990s, there was controversy that the added protection of a full face shield to a helmet in ice hockey may increase neck and overall injury rates due to the added weight of the shield and a feeling of invincibility leading athletes to take excessive and unwarranted risks.82 83 88 However, data exists to conclusively support the protective effect of full face shield use on dental injuries, facial injuries and eye injuries, without increasing neck injuries or injury rates overall.2 39 79 All of the clinical studies in this paper provide evidence to suggest that there is no difference in concussion rates between ice hockey players wearing different types of facial protection. That said, there is clinical evidence to suggest full face shield use may protect athletes from sustaining a more severe concussion (increased time loss) and biomechanical evidence to suggest a reduction in peak accelerations of the headform following impact with the use of facial protection (cage > visor), with no evidence of harm. Despite the protective evidence, many ice hockey players do not wear a full face shield or visor if given the choice. Reasons for not wearing a visor have been attributed to fogging during the course of a game, reducing peripheral vision, feeling like a target for the opposition and the belief that no facial protection is a sign of increased masculinity and toughness.79 80 89

Study limitations

There were several limitations of the epidemiological studies reviewed in this manuscript:

1. The majority of mouthguard13 32–36 38 and headgear48 53 61 62 65 68 study designs were descriptive observational. All of these studies have biases such as self-selection of equipment use, so study groups may not be comparable.

2. The definition of concussion varied between studies (ie, time-loss, symptom-based51 65 medical attention based, self-report) which makes the comparison of concussion rates difficult. In addition, questionnaires with no clearly stated concussion definition may underestimate injury rates if athletes were required to self-diagnose concussions retrospectively in time.

3. Individual equipment exposure information was rarely collected, which is critical for the assessment of injury risk.

4. The validity of the recording mechanisms was unknown in all but two of the selected studies.2 39

5. The study populations ranged in size, with very few sample size or power calculations reported.2 49

6. There was no standardisation of injury incidence (eg, injuries per 1000 athlete-exposures vs 1000 player-hours) in several studies.

7. Study populations were predominantly samples of convenience with no random selection of subjects (a source of selection bias).

8. Individual baseline concussion history was not reported in the majority of studies, which may have resulted in an overestimation of concussion rates if the athletes selected were more likely to be concussed based on their previous concussion history (that is, if there was differential use of protective equipment in those more susceptible to concussion).

9. Several studies reported head injury rather than concussion as the outcome variable of interest45^–48 56^–59 61 62 68 and therefore the true effect of helmet use on concussions may not be correct.

10. Some studies45 53 68 had large subject drop-outs, loss to follow-up, or a low response rate, which may have lead to an underestimation of the association between protective equipment use and concussion rates if the reason for drop-out was related to injury. Several other studies did not report this information.12 13 19 26 32^–36 38

11. Concussion severity was rarely studied (ie, time loss, grading scale, symptom based, etc).

12. There was a lack of measurement and control for potentially confounding variables in many of the selected studies.13 19 26 32^–36 38 39 49 55 Failure to take into consideration differences in coaching techniques, musculature and cardiovascular conditioning, warm-up routines, other protective equipment use, rules, rule enforcement, age, experience level, skill level, position, previous injury history, venue type (ie, practice vs game) and field characteristics may all significantly effect the results.

13. External validity of the results in the selected studies was limited due to the lack of internal validity.12 13 19 26 32^–36 38^–40 68

14. Results of the selected studies cannot be generalised beyond the specific sport of study. For example, it was difficult to generalise the results of a concussion study in bicycling58 62 63 or non-contact sports such as basketball19 to collision sports such as football, rugby, Australian rules football and ice hockey where impact forces may be very different and concussions more common.

Perhaps the greatest limitation with all the biomechanical studies is that the surrogate headform/dummy and injury criteria do not necessarily mimic the on-field experience with concussion. Potential reasons include:

1. Game and practice situations (eg, environmental, psychosocial, etc) that occur in the field differ from the laboratory.

2. The impact forces required to produce concussion is not entirely known and therefore these studies can only approximate a force that causes concussion. Therefore, it is not known what reduction in impact force, or increase in energy attenuation from the headgear would result in a clinically significant decrease in the number of concussions seen on the field.

3. Two of the studies did not report the validity or variability of the testing apparatus used.69 74

4. The results of the studies using cadavers12 70 lacked generalisability to the sporting arena because of unknown correlation between cadavers and live humans, as well as the use of an experimental fixed skull model that has been shown to alter the nature of the brain injury sustained.90

5. The specific protective equipment was not tested in different environmental conditions (eg, wet, humid, temperature) in most of the selected laboratory studies, as it is possible that their effectiveness varies in different conditions.

6. Some studies used flat rigid force plates71 74 for impact collection, which would be very different from the spherical nature and composition of human heads.

7. In studies that examined ball-to-head impacts,71 74 75 a specific ball make was used and therefore generalisability to other balls, different designs, sizes or inflation pressure, may be limited.

8. Only two studies measured both linear and rotational acceleration.69 72

9. All of the studies except one72 did not repeat tests on same the headgear to investigate the impact of multiple impacts to the headgear. In a real life situation, the headgear will be reused many times and these studies only test the headgear as a new piece of equipment.

Based on the inconclusive results, combined with the above limitations, it is critical that future well-designed and sport-specific prospective analytical studies of sufficient power are conducted to determine the true risk of concussion associated with specific types of mouthguards, helmet, headgear and facial protection. Authors’ must specify the exact properties of the equipment tested (eg, custom-fitted, pressure laminated mouthguards with 4 mm occlusal thickness), assess products that are (or would be) generally accepted and worn by sport participants and study mechanisms of injury in the laboratory that are consistent with those observed in the sporting arena. Furthermore, an operational concussion definition must be clearly stated with specific report of concussion rates, rather than using head injury as the primary outcome variable of interest which includes structural brain injury, skull fracture, diffuse axonal injury, lacerations, facial fracture, dental injury, eye injury and so on. Until such time, there will not be valid scientific evidence to advocate mouthguard, headgear or face shield use solely for the prevention of concussion in sport.

CONCLUSION

Helmet use has been shown to decrease head and brain injury risk among bicyclists, and head injury among skiers and snowboarders, but the effect of helmet/headgear use specifically on concussion risk is inconclusive in these sports, as well as sports such as rugby, football (soccer) and rodeo. In addition, there is no strong scientific evidence of an association between mouthguard or face shield use and reduced concussion risk. At the same time, there is no evidence of an increased risk of injury, while there is evidence to support the use of mouthguards for dental protection. Furthermore, there is scientific evidence that full facial protection in ice hockey may protect athletes from more severe concussions, as measured by time loss from competition. Such findings justify the use of mouthguards and facial protection in collision sports as a means to reduce injuries, but at this time cannot be advocated specifically for concussion risk reduction. Future prospective, sport-specific, analytical studies of sufficient power are needed to determine the true relative risk of concussion associated with specific types of protective equipment use.