Sports concussion is a common injury1^–4 and is characterised by signs and symptoms which generally resolve spontaneously over time.5 Recent consensus6 and position statements7 recommended that sports medicine professionals use a multi-dimensional approach to evaluate athletes following concussion and that self-reported concussion symptoms are included along with tests of concussion such as neuropsychological and postural tests.

Self-reported symptoms are frequently used as a pragmatic method for screening for concussion as well as monitoring the resolution of the injury and in return to play decisions as they are simple to measure and do not require sophisticated equipment or training.8 The international Concussion in Sport (CIS) group6 proposed that the presence of any symptom in a symptom “checklist” should be considered suggestive of concussion, reinforcing the importance of the measuring symptoms associated with concussion.

The typical signs and symptoms6 following concussion include, but are not limited to: loss of consciousness, amnesia, headache, dizziness, difficulty concentrating, nausea and vomiting. Self-reported symptoms by their very nature are subjective, depend on awareness,9 10 honesty and willingness of the athlete to provide accurate information.8 9 11 A number of self-report symptom scales6 8 12^–14 and checklists3 15^–17 have been proposed in an attempt to provide a structure for sports medicine professionals to objectively document concussion symptoms, and it is important that these scales and checklists are reliable and valid. No published review of the scales/checklists and their measurement characteristics has been undertaken to date.

This review will explore the available self-report scales/checklists used to measure acute sports concussion symptoms. Specifically the purpose of this systematic review is to identify self-report scales or checklists that are used to measure symptoms following a sport concussion, and secondly to describe the psychometric properties of these identified scales/checklists, where available.

METHODS

Search strategy

The databases PubMed, Medline, CINAHL, Scopus, Web of Science, Sport Discus, PsycINFO and AMED were searched from their establishment until December 2008. The medical subject heading terms “brain concussion”, “signs or symptoms” and “athletic injuries” were combined with the Boolean operator “AND”. The search was limited to articles published in English. An additional hand search of the reference lists of the retrieved articles was conducted to identify potentially relevant citations that might have been missed in the electronic search. Only full-text articles were considered for this study and these were retrieved to determine whether they met the inclusion criteria.

Eligibility criteria

To be included, an article had to meet criteria 1 or 2, and criteria 3.

1. Original research reporting on the use of scales or checklists to evaluate the symptoms associated with sports concussion

or

2. Review or review style articles which discuss the use and/or development of concussion symptom scales or checklists

and

3. All items of the scale or checklist must have been included and/or referenced to the source information provided.

All articles were initially examined for inclusion by the principal investigator and where uncertainty arose, they were screened for inclusion by a second member of the research team.

Quality assessment

Although not a primary outcome of this study, the quality of the included studies was assessed using the Downs and Black checklist addressing the issues of reporting, internal validity (bias and confounding) and external validity.18 This instrument was chosen as it was developed for use in the quality assessment of both non-randomised control trials and randomised control trials. For the purpose of this review 16 items19 were chosen from the Downs and Black scale and these were scored independently for each article by two members of the research team. Where disagreement existed in the scoring, a third reviewer assessed the article to facilitate the reaching of a consensus. The overall quality score for each article was recorded for descriptive purposes but was not used to exclude any article from the review.

Data extraction and analysis

The following information was extracted from each article by the primary author: study design, year of publication, characteristics of the participants, reliability of the scale/checklist as reported by the study and sport; details of the self-reported symptom scales or checklists (including; name, number of items, time of measurement, format, mode of report, analysis of the data, scoring and psychometric properties) using a standardised protocol. A further search was conducted from other sources (eg, books, conferences proceedings, theses) to locate information pertinent to the psychometric properties of the included scales or checklists. In addition, web sources were accessed to gain clarification on the scales/checklists for the commercially available products and where necessary authors/developers were contacted to obtain information relevant to the identified scales/checklists.

RESULTS

Selection of studies

The initial search resulted in 421 articles, which were reduced to 290 articles after removing duplicates. The hand search resulted in 17 articles, thus giving a total of 307 articles. Full text was available for 295 articles of which 60 met the criteria for inclusion and were considered for the review and their scientific quality. The excluded 235 articles were case reports, reviews and guidelines on concussion management or studies that had not used a symptom scale or checklist. An overview of the review process is presented in fig 1.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Flow diagram of the search strategy and process.

Methodological quality

The quality assessment scores for the 60 included articles ranged from 9 to 15 (maximum score: 17) indicating a generally satisfactory standard of the papers reviewed. The inter-rater score agreement rate for the analysed 60 articles was 90% (54 articles). The six articles which have a difference in opinion were assessed by the third reviewer and a consensus was reached.

Data extraction

The 60 extracted articles spanned a time period from 1995 to 2008 indicating the recent emergence in the use of these scales/checklists in sport concussion research. The majority of these studies originated in the USA and employed a range of research designs, mainly of a descriptive nature. Collectively they included 10 896 participants (5864 with a sports concussion and 5032 non concussed participants) and covered a wide range of sporting activities with the primary sport being American football. The details of these studies are presented in table 1 in chronological order. The primary purpose of the majority of these studies was the monitoring of the resolution of symptoms post concussion, often with reference to a pre-season baseline measurement.

Scales/checklists identified for concussion symptom assessment

The 60 studies investigated a range of topics relating to sports concussion, primarily the monitoring of the resolution of the symptoms, utilised six core different scales/checklists to evaluate the symptoms of concussion. Eight articles were specifically devoted to the development of the scale/checklist with the majority of articles being used to identify a concussion and its recovery. We have used the terminology of scales/checklists despite the fact that there has been little formal development which would indicate that the instruments possess the key elements associated with a scale/checklist.20 These six core scales/checklists all incorporated symptoms typically associated with a sports concussion. However, there was considerable variability in the naming and number of symptoms included.11 21 In addition, different variants of the “post-concussion scale”13 were published. These variants varied in both the number of symptoms included and in the choice of symptoms. The majority of the scales/checklists recorded the athletes’ responses on a 7-point Likert scale3 4 13 22 while a few used a dichotomous classification15 23^–27 using one of two different formats (pencil and paper or computerised). Almost all of the scales/checklists involved self-reports (eg, the athlete checks an item), one scale26^–30 also included an observer monitored checklist of signs of concussion.

Figure 2 provides an overview of the six core scales/checklists and their variants that appear to be derivatives, or have their origins in previously published scales/checklists. We have presented the identified scales/checklist below in an historical approach based on their date of publication/release. It is acknowledged that many of the instruments were in clinical use before they were formally documented. Likewise there have been subtle changes associated with the introduction of new versions of most computer packages. The six core identified scales/checklists and their variants are described in detail below.

• Download figure
• Open in new tab
• Download powerpoint

Figure 2

Identified concussion symptom scales/checklists and their variants. Symptom item number varies with each version; shaded numbers represent the six identified core scales/checklists.

Description of the core scales/checklists and their variants

As there is a possibility of confusion due to the similar names applied to different scales/checklists, we have made slight modifications to names in order to clearly delineate and identify a potential scale/checklist for the purpose of this review.

Historical precedents

In addition to the published scales/checklists several researchers have incorporated symptom reporting as part of their studies to profile the consequences of a concussion even prior to the development of a formal scale/checklist. Several studies have documented these self-reported symptoms, which were used for preseason and post mild head injury assessments. One of the earlier reports was a retrospective study31 32 conducted on patients of football induced mild head injury was the first study to use a self-report questionnaire to collect information pertinent to symptoms post concussion. The symptom items included in this questionnaire were headache, memory, nausea, dizziness and weakness. Later Maddocks et al33 documented similar symptoms including headache, dizziness, blurred vision, nausea, double vision, noise sensitivity and light sensitivity. McCrory et al34 used a comparable symptom profile (headache, blurred vision, dizziness, confusion/PTA, nausea) to obtain data on Australian football players and recorded them as present or absent. All these studies reported a considerable increase in these symptoms following concussion when compared to the preseason levels and the most commonly reported symptom was headache.

Pittsburgh Steelers post-concussion scale

The Pittsburgh Steelers post-concussion scale12 was initially developed as part of the Pittsburgh Steelers concussion management programme in late 1980s22 and/or early 1990s12 35 to document the concussion symptoms.12 22 36 The original items were selected based on clinical experience and reports from a large number of athletes both at amateur and professional level of sports.22 Variants of this scale were used widely in the National Football League13 22 37 38 and National Hockey League.13 22 37 38

A number of variants of this scale have appeared with different terminology39^–43 and numbers of items.39^–43 The original Pittsburgh Steelers post-concussion scale included 17- items and was used to monitor symptoms at sideline and for subsequent follow-up post concussion. This was the first scale to document the severity of post-concussion symptoms on a 7-point Likert scale. The severity of the symptoms was also classified and documented as none, moderate and severe. The eight variants of this scale are described below.

Post-concussion scale (20-item)

The post-concussion scale (PCS) with 20-items38 was designed to document the symptom severity following a sport concussion, more specifically in the acute phase of recovery. The severity is measured using a classification as none, mild, moderate and severe on a 7-point Likert scale. The 20-item PCS scale was subsequently revised and published as the 21-item PCS scale (revised).13

Post-concussion scale (revised)

Post-concussion scale (revised)13 with 21-items was the first scale published and is widely used in the literature. The scale included the symptoms most commonly reported following a sport concussion in an attempt to clarify confusion with terminology. Since then, this scale has been used in the sport concussion research40 44 45 both pre season and post concussion to document the symptoms (7-point Likert scale) and to monitor symptom resolution following a concussion. This scale was incorporated into a computerised neurocognitive package ImPACT (Immediate post-concussion and cognitive testing, NeuroHealth Systems, Pittsburgh, PA, USA) in 2000.46

Post-concussion scale (ImPACT)

ImPACT is a commercial computerised neuropsychological assessment tool. It incorporated a symptom scale with a 7-point Likert scale. The initial version of ImPACT had 21-items47 identical to the PCS (revised), where as in later versions the symptom item “visual problems” was added making it a 22-item scale.22 This scale since has appeared to have gained popularity and is widely used.14 22 48

Post-concussion symptoms scale (Vienna)

This 20-item post-concussion symptom scale, proposed at the First International Conference on Concussion in Sport49 was an adaptation of the PCS (revised) scale. The items “fatigue” and “sleeping less than usual” were replaced with the item “other”. We could locate only one study that had utilised this scale.50

Graded symptom checklist

The 20-item graded symptom checklist (GSC)3 4 16 17 has its origins from the 21-item PCS (revised) scale.3 4 16 21 51 However the GSC has appeared in the research literature with 173 4 16 21 51 and 18-items.17 A recent study17 using the 18-item GSC categorised these symptoms into 4 domains; cognitive, somatic, emotional and sleep problems. The GSC with 27-items was proposed in the National Athletic Trainer’s Association position statement7 to be used for the presence and duration of the post-concussion symptoms at baseline as well as for monitoring symptom resolution at post injury and subsequent follow-up. The 17-item GSC4 was tested on a sample of 94 concussed athletes and 56 control participants and found to have a sensitivity of 0.89 and specificity of 1.0 at the time of injury. The sensitivity dropped to 0.74 post game and to 0.04 seven days post-concussion, where as the specificity remained the same at all the testing periods.4

Head injury scale

The head injury scale (HIS)8 21 52 first published in 2003, is a theoretical 16-item self-report scale derived from the literature that included the most common symptoms related with sports concussion and post-concussion syndrome.8 The 16-items were in the most part derived from the 20-item graded symptom checklist (GSC).21 The symptoms were grouped into brief duration and long duration based on the symptom severity scores, measured on a scale of 1–6, where 1–3 indicate symptoms of brief duration and 4–6 indicate symptoms of long duration.52 The scale was hypothesised to have three relatively latent constructs namely; somatic, neuropsychological and cognitive.8 These constructs were investigated through confirmatory factor analysis procedures and tested for construct validity8 on a sample of 279 healthy athletes. This analysis showed a good, but not a perfect fit, to the 16-item scale, which was subsequently modified, based on this analysis and empirical evidence, and resulted in a 9-item scale which showed a perfect fit to the proposed model. The 9-item and 16-item scales were strongly correlated (r = 0.953). The internal consistence of the 16-item and 9-item HIS were 0.84 and 0.78 respectively. Construct validity was further tested on a sample of 33 collegiate athletes (16 control and 17 concussed) and resulted in a comparable response for 16-item and 9-item scales in monitoring the resolution of concussion symptoms.

McGill abbreviated concussion evaluation post-concussion symptoms scale

The 20-item McGill abbreviated concussion evaluation (McGill ACE) post-concussion symptoms scale53 is a modified version of the PCS (revised) scale and was included in the McGill ACE test battery that was developed as part of a doctorial thesis project however no scale validation has been published in the scientific literature.54 55The 20-items were grouped into three categories, somatic, cognitive and affective symptoms. The items “fatigue”, “sleeping less than usual” of the PCS (revised) were replaced with an item “other” in the McGill ACE post-concussion symptom scale.

CogState sport symptom checklist

CogState sport symptom checklist2 56 was incorporated as a part of the computerised CogSport neuropsychological test battery (CogSport Ltd, Melbourne, Australia). This checklist was derived from the McGill ACE post-concussion symptoms scale. In later versions (V.5.6 and V.3.3), the Sports Concussion Assessment Tool (SCAT) post-concussion symptom scale was incorporated with 25-items (18 immediate and 7 follow-up items) scored on a 7-point Likert scale. This Likert scale was deliberately included to enhance research applications and thus aligning the package with the evolving international consensus statements on concussion in sport. The number of items included in the earlier versions (Concussion Sentinel-3 and V.2.2) had 21 and 14 symptom items respectively with the responses recorded as yes/no.

Post-concussion symptom assessment questionnaire

Post-concussion symptom assessment questionnaire (PCSQ)57 was developed as part of a standardised protocol for the initial evaluation and documentation of mild head injury. Presence of the symptoms was assessed by asking athletes to circle either yes or no to each symptom presented and to document the symptom severity on a 10 cm visual analogue scale. The responses on this questionnaire were used to make decisions for return to play (RTP) and for further follow up post mild head injury.

Concussion resolution index (CRI) post-concussion questionnaire

This is a web-based questionnaire15 24 58 developed in 2001 as part of a computerised neuropsychological test protocol (HeadMinder). It assesses the presence and severity of 15 common neurophysiological symptoms associated with concussion.15 23 24 58 This questionnaire records immediate post injury symptoms and subsequent follow-ups. The CRI’s development23 was based on a normative sample of 414 individuals and was validated with 26 concussed athletes. This questionnaire incorporated a three light (red, yellow and green) classification system to monitor the resolution of both cognitive and self-reported neurophysiologic symptoms, where the red light indicated the presence of statistically significant cognitive and self-reported symptoms; a yellow light indicated borderline results and a green light indicating the non significant cognitive and self report symptoms.23

Signs and symptom checklist

Signs and symptom checklist (SSC)26^–30 59 first published in 2004, was developed by the mild traumatic brain injury committee members (formed by National Football League in 1994) consisting of medical and research experts in the fields of traumatic brain injury, basic science research, sport neuropsychology, sport neurology and epidemiology as part of a standardised report form.30 The signs and symptoms included in this 34-item checklist were those most commonly reported in the literature to be associated with mild traumatic brain injury and post-concussion syndrome.27 30 59 These symptoms were grouped under six categories: general, cranial nerve, memory, cognitive, somatic and unconsciousness. Most of these symptoms were recorded by spontaneous reports by the players and some by physician observation. This checklist is used for acute and follow-up assessments.

Sport concussion assessment tool post-concussion symptom scale

This scale6 was proposed by the international Concussion In Sport (CIS) group in 2004 as part of a comprehensive concussion screening instrument—SCAT. The SCAT was widely published as part of the group’s “summary and agreement statement” following the second international symposium on concussion in sport.6 This scale contains 18-items that are used to screen for an acute concussion and a further 7-items to gather information at a follow up visit. The items were obtained from published sources and agreed upon by a panel of experts, thus establishing the face validity of the instrument. The instrument, which still needs further formal investigation, has since been adopted by a number of organisations.

Concussion symptom inventory

The concussion symptom inventory (CSI)25 60 is an empirically derived scale for the monitoring of the symptoms following a sports-related concussion. This scale was developed by combining the items from similar symptom inventories used in three large prospective studies.60 A systematic, statistical and decisional strategy was followed in reducing the number of items from 27 to 12, which were subsequently analysed using a Rasch analysis to arrive at a final set of nine items. The scoring of the items using a 7-point Likert was compared with that of a collapsed yes/no strategy and little difference was found when examining the sensitivity and specificity of the instrument to the detection of a concussion between the scoring systems. The data driven approach used to arrive at this reduced set of key symptom items represents a major step in the development of science of symptom measurement, however to our knowledge, the CSI has not been used in any published prospective studies.

Symptom items

The items in the six core scales/checklists were mostly generated using the athletes language and written to avoid ambiguity. However, using the athlete’s language has the potential to be limited to the jargon associated with a specific sport and of the country/region in which the study was conducted. Changes in the naming of various symptom items have resulted, in many instances, with different words being used to describe the same item. This makes comparison between like items on different scales/checklists difficult.

Psychometric properties

The systematic search isolated a limited amount of published information on the psychometric properties of the scales/checklists (table 2). The reliability (internal and test-retest) was the most frequently reported property. In the majority of instances the reporting of the statistical properties appears to have followed the use of the scales rather than driven their development. However, more recently there have been some data driven initiatives to full this void and provide some credibility to the scientific use of the scales/checklists.60

DISCUSSION

This systematic review was conducted to identify the scales/checklists that are used in sports concussion assessment and to explore their origins and psychometric properties. Those studies which have utilised a scale/checklist for the assessment of self reported symptoms were included. The review identified 6 core scales/checklists utilised in the sports concussion symptom assessments with a larger number of “derivative” scales. In general, the scales/checklists have their historical origins from the clinical neuropsychology literature and were used in the assessment of head injury population.

The use of symptoms as a measure of an individual’s status following a head injury has a long history. Meyer61 was among the first to describe the symptoms associated with a closed head injury in 1904. In a seminal paper in 1945, Denny-Brown62 classified the symptoms as those due to structural injury, psychiatric complaints, symptoms of change in personality and complaints of uncertain or variable derivation based on the reports made on 200 closed head injury patients who were monitored until their return to work. Oddy et al63 were the first to construct and administer a formal symptom checklist (37-item) to collect data on patients with severe closed head injury. Although no details of the development of this checklist are provided, it does represent the formalisation of systematic symptom assessment. Subsequent studies64 65 utilised a variety of symptom scales/checklists to document the symptoms following a head injury. A more detailed description on the use of symptoms data is provided by Richardson.66 King et al67 established a reliable symptom questionnaire for the assessment of post-concussion symptoms in patients with head injury from various causes. The Rivermead post-concussion symptoms questionnaire contains 16-items and has been shown to be reliable in either self-administered or a clinician administered format in the assessment of post-concussion symptoms. Subsequently the Rivermead post-concussion symptoms questionnaire has undergone more vigorous investigations.68 This rationalisation appears to be the precursor of the development of a number of specific scales/checklists within the sports domain. Symptom documentation in sport concussion field was introduced by Barth et al in 198931 using a self-report questionnaire. Later the post-concussion symptom scale developed for the Pittsburgh Steelers12 provided the foundation for the development of numerous scales/checklists proposed and used in the sports concussion field.

Currently, a spectrum of scales and checklists are being used in the management of sports concussion. The universal availability of scales/checklists has been enhanced by their inclusion in commercially available neuropsychological software products (eg, ImPACT and CogSport) and their wide distribution through international consensus statements.6 49 69 The clinical use of the majority of symptom scales/checklists has preceded the establishment of their scientific validation and measurement properties.20 Furthermore, the reporting of variants of core scales/checklists that seem to have no justification other that the developers decision to add/delete certain items based on their experience and without a specific process adds to the potential confusion when comparing research findings across the studies. We have used the term scales/checklist as a global descriptor throughout this review as it is not entirely clear if the presented instrument meets the scientific expectations associated with the use of the terms “scale”20 despite their use of the term scale.

Underlining the scientific use of the term scale is the assumption that there has been a systematic establishment of its core measurement properties such as; item selection, validity, reliability and diagnostic potential. In a textbook approach to the development of a scale the developers will have worked through a designated pathway (of studies) establishing the various properties of the instrument before it is made available to the user.20 In reality, researchers may shortcut this process in order to facilitate the demands of their studies or the clinical applications the studies were selected on. Very few of the scales/checklists reported in this review have followed a systematic development process and technically do not meet the classic for recognition as a scale. This is not to say that they do not contribute to the knowledge base concerning the management of a concussion but there is the need for understanding and caution in their use. Although the wide use of symptom scales and checklists by sports medicine professionals might argue for their acceptance as part of a wider testing package in concussion management, this does not supplant the need for solid and systematic enquiry as to the basic assumptions and science underpinning their use.

The symptoms identified in most concussion scales/checklists will vary with the individual who is asked to respond and their “state” independent of any injury and fluctuate naturally over time16 thus challenging the measurement of a true baseline value. This is further compounded by the vary nature of the need for self-report where athletes may selectively report the presence and intensity of symptoms. Symptom scores have been shown to change with intensity of exercise thus making their interpretation in a sports concussion sideline assessment situation problematic at times. A universal set of about 20 key symptoms has evolved from clinical use and appears to be incorporated in most currently used scales/checklists.

Three recent initiatives have begun to contribute, albeit in different ways, toward the building of the scientific foundations for symptom measurement in sports concussion. In 2001, a group of international experts (International Concussion in Sport Group)49 met in Vienna to discuss and formulate a consensus opinion on a range of aspects of sports concussion. This was further endorsed at a similar meetings in 2004 and 2008 where, in addition to a consensus statement on the definition of a sports concussion and its management, the group6 also produced SCAT—a multidimensional approach to concussion assessment which incorporated the post-concussion symptom scale. This “scale” incorporates 25–items (18 baseline+7 follow up) scored on a 7 point-Likert scale. Although the methodology surrounding how the Group reached consensus on the choice of this particular scale, its items and measurement strategy is not well documented it does represent a solid approach to establishing the face and content validity of this scale.

A subsequent and perhaps more rigorous approach to the understanding of the content of scales/checklists is through a determination of their underlying constructs.70 Piland et al8 explored the factorial and construct validity of a 16-item symptom scale and found that the number of items could be reduced to nine while still maintaining a strong correlation (r = 0.95) with the original scale and an acceptable level of internal consistency (α = 0.78). Findings, such as this begin to challenge the general use of clinically driven instruments. A further data-driven analysis of the symptoms associated with a concussion resulted in the proposal of the 12-item concussion symptom inventory (CSI).60 The development of the CSI involved a Rasch analysis of symptom data from three major projects to establish a cohesive set of symptoms which were sensitive to a concussion. Although brief in methodological detail and limited by the initial symptom sets the CSI represents a major contribution in bringing a robust statistical approach to the building of an empirically based symptom set. These initiatives together with the ongoing publication of data on the internal and test-retest reliability are providing the necessary checks to ensure that the field of symptom measurement evolves into a strong clinical science devoid of a guru acolyte mentality.

While there is clearly no evident “gold-standard” scale/checklist which satisfies what might be expected from a psychometric perspective there are a range of instruments which should be considered for use in concussion management. We have documented in table 2 the various data describing the psychometric properties that we have been able to discern from publications as a source for researchers to make their own judgments about whether to adopt an instrument and/or how to interpret the data collected with the instrument. There is the need for researchers to actively explore the fundamental constructs underlying symptom measurements and document their inherent measurement properties.

We acknowledge that numerous studies have used self-reports of symptoms,34 41 71 survey instruments72^–74 or questionnaires75^–79 rather than having a predefined symptom set to investigate aspects of concussion, but we have only reviewed those studies which used a formal scale/checklist in the sports environment for the assessment of acute concussion.

CONCLUSION

This review has highlighted the existence of number of scales/checklists, some of which appear to be used more than others. Many of these scales/checklists however, have not been methodically developed or subjected to scientific scrutiny and do not provide clinicians and researchers with sufficient information to make an informed choice. These scales/checklists obviously make a contribution to the detection, assessment and return to play decisions, however there is a need for the clinical user to be aware that many of these scales/checklists have “evolved” rather than being scientifically developed. In choosing to adopt a specific scale/checklist, a decision should be made by the user on what is appropriate to use in their circumstances. This responsible step will ensure that the user knows the limits of the scale/checklist they have adopted.