Article Text

Advances in neuropsychological assessment of sport-related concussion
  1. Ruben J Echemendia1,2,
  2. Grant L Iverson3,
  3. Michael McCrea4,
  4. Stephen N Macciocchi5,
  5. Gerard A Gioia6,
  6. Margot Putukian7,
  7. Paul Comper8
  1. 1Psychological and Neurobehavioral Associates, Inc., State College, Pennsylvania, USA
  2. 2Department of Psychology, University of Missouri-Kansas City, State College, Pennsylvania, USA
  3. 3Department of Psychiatry, University of British Columbia, Vancouver, British Columbia, Canada
  4. 4Department of Neurosurgery and Neurology, Medical College of Wisconsin, Milwaukee, Wisconsin, USA
  5. 5Rehabilitation Psychology and Neuropsychology, Shepherd Center, Atlanta, Georgia, USA
  6. 6Division of Pediatric Neuropsychology, Children's National Medical Center & Department of Pediatrics and Psychiatry, George Washington University School of Medicine, Rockville, Maryland, USA
  7. 7Department of Athletic Medicine, Princeton University, University Health Services, Princeton, New Jersey, USA
  8. 8Kinesiology and Physical Education & Graduate Department of Rehabilitation Science, University of Toronto, Ontario, Canada
  1. Correspondence to Dr Ruben J Echemendia, Psychological and Neurobehavioral Associates, Inc., 204 East Calder Way, Ste. 205, State College, PA 16801, USA; rechemendia{at}comcast.net

Abstract

Objective To critically review the literature from the past 12 years regarding the following key issues in sports-related neuropsychological assessment: (1) the advantages and disadvantages of different neuropsychological assessment modalities; (2) the evidence for and against the current paradigm of baseline/postinjury testing; (3) the role of psychological factors in the evaluation and management of concussion; (4) advances in the neuropsychological assessment of children; (5) multi-modal assessment paradigms; (6) the role of the neuropsychologist as part of the sports healthcare team and (6) the appropriate administration and interpretation of neuropsychological tests.

Design Targeted computerised literature review (MEDLINE, PubMed, CINAHL and PsychInfo) from 2000 to the present using key words: neuropsychological, neurocognitive, assessment, testing, concussion and sports.

Results More than 2600 articles were identified using key word searches of the databases, including many duplicates. Several books were also reviewed. The articles were pared down for review if they specifically addressed the key areas noted above.

Conclusions Traditional and computerised neuropsychological tests are useful in the evaluation and management of concussion. Brief cognitive evaluation tools are not substitutes for formal neuropsychological assessment. At present, there is insufficient evidence to recommend the widespread routine use of baseline neuropsychological testing. Although scant, research suggests that psychological factors may complicate and prolong recovery from concussion in some athletes. Age-appropriate symptom scales for children have been developed but research into age-appropriate tests of cognitive functions lags behind. Neuropsychologists are uniquely qualified to interpret neuropsychological tests and can play an important role within the context of a multifaceted-multimodal approach to manage sports-related concussions.

  • Concussion
  • Head injuries
  • Measurement

Statistics from Altmetric.com

Introduction

Sports-related concussions are frequently associated with one or more symptoms, impaired balance and cognitive deficits.1–14 These problems can be measured using symptom scales,15–19 balance testing6 ,12–14 and neurocognitive testing. All three assessment modalities can identify significant changes in the first few days following injury, generally with normalisation over 1–3 weeks.2 ,5–7 ,12 ,20 The presentation of symptoms and the rate of recovery can be variable, which reinforces the value of assessing all three areas as part of a comprehensive sport concussion programme.

Neuropsychological assessment has been described as an important ‘cornerstone’ of concussion management.21 Concussion management programmes that use neuropsychological assessment to assist in clinical decision-making have been instituted in professional sports,16 ,22 colleges23 ,24 and high schools.25

In addition to neuropsychological measures, rapid cognitive screening tests have been developed (eg, Standardised Assessment of Concussion, SAC) that are sensitive to the immediate effects of concussion.10 ,26 The SAC can be used on the sideline or later on the day of injury to identify cognitive deficits associated with concussion, and it is embedded in the Sport Concussion Assessment Tool—Second Edition (SCAT227). Although useful on the day of injury and in the few days that follow, brief cognitive screening tests such as the SAC and SCAT2 are not substitutes for more comprehensive neuropsychological assessment.27

Numerous studies have found that both traditional and computerised cognitive tests are sensitive to the acute effects of concussion (eg2 ,6–9 ,12 ,20 ,28–43). Consequently, many position and consensus statements have recommended neuropsychological assessment as a component of concussion management programmes.21 ,27 ,44 ,45

The goal of this paper is to critically review clinical and research advances in neuropsychological assessment over the past 12 years. The seven key questions listed below were addressed.

  1. What are the advantages and disadvantages of different neuropsychological testing modalities (eg, computerised batteries, traditional ‘paper and pencil’ tests or a combination of both)?

  2. What is the evidence for or against the current paradigm of baseline/postinjury testing?

  3. Is the assessment of ‘psychological’ factors important in the evaluation and management of concussion?

  4. Have there been advances in the neuropsychological assessment of children?

  5. What is the role of neuropsychological assessment within the context of multimodal assessment models?

  6. What is the role of the neuropsychologist as part of the sports concussion healthcare team?

  7. Who should administer and interpret neuropsychological tests?

Methods

A targeted computerised literature search was conducted via Medline, PubMed, CINAHL and PsychInfo from 2000 to the present using the key words: neuropsychological, neurocognitive, assessment, testing, concussion and sports. This search identified more than 2600 articles including many duplicates. Several books were also reviewed. The articles were pared down for review if they specifically addressed the key areas noted above. Relevant articles were reviewed by the authors and included as appropriate.

Neuropsychological testing models

Baseline cognitive assessment using traditional neuropsychological tests was introduced in the 1980s.46 Computerised tests were developed in the 1990s to provide an alternative to traditional tests and are now used almost exclusively in many sports settings. Traditional tests have been studied in combination with computerised batteries to assess construct validity.47 ,48 The combined use of traditional and computerised neuropsychological tests in applied settings has been referred to as a ‘hybrid’ neuropsychological testing approach.49 There is no scientific evidence that traditional testing, computerised testing or a hybrid approach is superior; each approach has its strengths and limitations.

Traditional tests are reasonably reliable, valid and sensitive to the effects of sports concussion.7 ,50 ,51 They can be selected to fit the specific needs of the athlete and domains of neuropsychological importance. These tests have a much longer history of being applied in clinical settings, with some having large normative databases. However, traditional tests require face-to-face examination, which may introduce variance in test administration and scoring. These tests are also more labour-intensive, especially in sports settings, while conducting baseline testing in large numbers of athletes.

In contrast, computerised tests such as the Immediate Postconcussion Assessment and Cognitive Testing (ImPACT), Axon Sports, the Automated Neuropsychological Assessment Metrics (ANAM) and Headminder (ImPACT Applications, Inc; Axon Sports, LLC)4 ,52 can be rapidly administered to individuals or groups. Computerised batteries are portable and efficient for the collection, synthesis and storage of large amounts of data. Clinical end-user reports are often immediately available.

Although computerised tests offer certain efficiencies, they also have limitations. First, they are brief and rely on a limited sample of cognitive functioning. Second, although a number of studies have shown that computerised batteries appear to have adequate psychometric properties,29 ,53–59 other studies have raised questions about the reliability, form equivalence and validity of these instruments.60–62 For instance, test–retest reliabilities have been reported to be quite low by some investigators, which increases reliable change (RC) metrics and limits the value of baseline examinations.50 ,60–62 Third, alternate forms may not be equivalent.62 Fourth, while one of the major purported advantages of computerised testing is the application to group testing, recent research indicates that group versus individual test administration may yield different results.63 Based on the combined experience of the present authors, under most circumstances large group baseline testing should be avoided because of an inability to adequately control the testing environment. Fifth, although computers have provided a technological breakthrough in neurocognitive testing, the technology itself is subject to error. Significant variability exists in the accuracy of personal computers’ measurement of response time,64 although they claim to do so with millisecond accuracy. Variability has also been found in the use of computer mice or keyboards, and monitor refresh rates.65 Sixth, computerised batteries have been marketed to clinicians from diverse disciplines, some of whom may have no education or training in cognitive assessment.66 Finally, computerised batteries may be seen as a ‘black box’ approach to neuropsychological assessment, an approach that is partially encouraged by the immediate availability of a clinical report that contains simplified coding for whether a finding is reliable or significant. This may lead some clinicians astray by inaccurately providing a singular or ‘cookbook’ approach to assessment that is complicated by many of the factors discussed above.

Some authors have advocated the hybrid approach because traditional testing may be too labour intensive, and computerised testing may be too brief and may not sample all the domains affected by concussion. Typically, the hybrid approach uses a computerised battery at baseline and a combination of computerised and traditional tests following injury. Although the hybrid method is intuitively appealing, only a few studies to date have examined the clinical utility of this approach (eg47 ,48).

The role and value of baseline testing

Baseline testing is conducted prior to an athlete's participation in a sport or a given season. In the event of a concussion, postinjury scores are compared to baseline scores to determine whether neurocognitive deficits exist.67 Theoretically, baseline scores are thought to increase the diagnostic accuracy when compared to postinjury scores by limiting variance associated with pre-injury confounding variables.

The pre–post injury assessment model may introduce error because test interpretation must take into account test-retest reliability and the inherent error surrounding multiple testing sessions that often occur at intervals that are years apart.49 ,61 ,68–70 Moreover, despite the methodological appeal of pre–post testing, the diagnostic accuracy of this paradigm has not been empirically tested in the sports domain. Consequently, whether this model has greater diagnostic accuracy than postconcussion assessment alone remains to be determined.50 ,71 Recent research suggests that postinjury assessment alone may have promise, obviating the need for widespread and costly baseline assessments.

Echemendia et al72 examined the utility of baseline testing among 266 concussed college athletes who had taken ImPACT at baseline and postinjury. RC from baseline was computed using two different RC methods. These RC methods were then compared to an interpretive method in which athletes scoring 1.5 SD units below expected normative performance were classified as showing a clinically significant change. Using postinjury data alone, a sensitivity of 86% and a specificity of 95% were found when predicting decline on the Verbal Memory Composite. The Visual Motor Speed Composite had a sensitivity of 80% with a specificity of 96%, and on the reaction time composite the sensitivity was 80% and the specificity was 97%. The Visual Memory Composite was not examined due to the differences in ImPACT versions (ie, the original version of the test did not include visual memory). These findings suggest that postinjury neuropsychological test data are robust and may not require baseline testing to identify clinically meaningful postconcussion cognitive decline as long as there exist appropriate, well-developed normative data.

Schmidt et al73 employed similar methodology with 258 concussed college students who were tested with the ANAM battery at baseline and following injury. Two interpretive methods were used: (1) individualised baseline versus postinjury comparison and (2) normative data baseline comparison. The individual baseline method identified 2.6 times more athletes with impairment than the normative method on the Simple Reaction Time subtest, and the normative method identified athletes with impairment 7.6 times more often than the individual baseline method on the mathematic processing subtest. No differences occurred among the other ANAM measures, measures of postural control on the Sensory Organisation Test or a graded symptom checklist.

Although the data are preliminary and limited to a college-age population, these studies suggest that baseline testing may not be as necessary as once thought for identifying postinjury neurocognitive deficits in cognition or balance. Additional studies are needed to further evaluate the utility of baseline testing with different age groups, different neuropsychological test batteries, different instruments used to assess baseline functioning across multiple domains and using different methodologies for examining the clinical usefulness of baseline versus normative comparisons. It is also important to identify those groups of individuals for whom the use of baseline data may be superior to the use of normative data (eg, those athletes who have cognitive abilities in the upper or lower 20% of the normative sample, or those with LD or ADHD), and which normative data should be used with which individuals (eg, mean scores by gender, sport, age and ethnicity).

If baseline data are not used, it is important that robust local normative data are available for comparison to postinjury data. If baseline neurocognitive testing is employed, the integrity of those baselines must be maximised through the use of appropriately trained examiners and well-supervised test administrations in individual or small group settings (<5 : 1 athlete: proctor ratio), creating a proper motivational set and screening for invalid baseline data (eg, sandbagging or purposefully poor performance).

Psychological factors associated with concussion

Athletes with concussion may suffer a broad spectrum of physical and cognitive symptoms. They are also at risk for psychological and emotional disturbance following injury,74–77 including but not limited to depression, anxiety, social isolation and loneliness, frustration, anger and guilt.78 Whether co-occurring psychological symptoms reflect a response to being injured and/or the pathophysiological consequences of concussion is not clear.79

Some of the research related to psychological functioning in athletes has been conducted in the context of establishing a valid cognitive baseline.80 ,81 Screening for psychological disturbance during baseline and postinjury assessments is an important element of concussion management, not only because of the prevalence of psychological difficulties, but also because the early identification and treatment of pre-existing or comorbid psychological issues associated with concussion may prevent the development of persistent postconcussive symptoms in vulnerable individuals.82

For instance, some athletes who have atypical, protracted and perplexing recovery following concussion may also have co-occurring depression or other psychological disorders.74 Many factors may be operative following concussion that can complicate recovery in some individuals. First, removing athletes from participation prevents the athlete from interacting with teammates and disrupts normal social networks. Second, teammates, coaches and friends may view concussed athletes as vulnerable and fragile, a perception that runs counter to the culture of invulnerability of most competitive athletes. Finally, there is usually no outward evidence of injury in concussed athletes, which may lead others to wonder whether the athlete is truly injured.

Concussion assessment of children

The evaluation and management of concussion in children presents a number of unique challenges. There are no published studies that have examined postconcussive impairments in children below the high school level, largely due to the fewer age-appropriate neurocognitive tests to study concussion in this age group.83 ,84 The key assessment domains in children are generally similar to those in adolescents and young adults with concussion (ie, postconcussion symptoms, specific neuropsychological functions and balance), but important differences must be accounted for, including children's cognitive, physical and emotional development as well as their capacity to serve as the primary reporter of their symptoms.85

Three of the computerised batteries in use today for the management of sport concussion have been used with children under the age of 12. A preliminary normative study of children aged 8 and older using CogSport found clear cognitive developmental changes that must be taken into account when evaluating children.86 The Pediatric ImPACT is a developmentally appropriate, computer-administered assessment battery for children in the age range 5–12, adapted from the format of ImPACT. The measure combines both standardised age-specific symptom ratings and six subtests measuring neurocognitive performance (learning and memory, response speed) to assist in multidimensional assessment of concussion.87 Finally, CNS Vital Signs88 has normative data across the lifespan, starting at age 8.

Computerised batteries for preadolescent athletes show early promise but currently lack the necessary evidence to draw any firm conclusions regarding their clinical utility. Similarly, traditional neurocognitive tests have not been applied to children who have sustained sport-related concussions.

In contrast, symptom measures that have been used with children included the Health and Behavior Inventory (HBI89), Acute Concussion Evaluation (ACE90), Postconcussion Symptom Inventory (PCSI91) and Rivermead Postconcussion Symptoms Questionnaire.92 The literature is growing for children aged 8 and older employing both child-report and parent-report measures.90 ,93 Two measures (ACE and PCSI) have been used with children aged 7 and younger. Some of the studies employing these symptom measures document sensitivity to concussion effects through group change (eg94) but few report RC metrics applicable to individual cases. In addition, normative data are not available for most of these measures.

The role of neuropsychological testing in a multidimensional assessment model

The symptoms of concussion are complex and vary widely across individuals. An athlete might experience prominent symptoms in one domain, but subtle problems in another. The heterogeneity of concussion supports the movement towards multimodal concussion assessment, which is premised on the view that single domain-specific outcome measures (eg, cognition or balance) are insufficient for detecting the full spectrum of concussion sequelae. As depicted in figure 1, the multidimensional assessment approach recognises this complexity and integrates outcome measures across multiple domains of functioning.39 ,95 ,96

Figure 1

Multidimensional Model of Concussion Assessment.

Role of the neuropsychologist

The biopsychosocial model of concussion,97–100 the value of neurocognitive testing and the efficacy of psychoeducational interventions to improve outcome101 ,102 all underscore the important roles that neuropsychologists can play as members of a multidisciplinary sports medicine team. Neuropsychologists provide expertise not only in cognitive evaluation and management of the injury, but they also identify and address potential psychological issues and problems.102 These psychological interventions need not be intensive and are most effective when introduced early during the acute or subacute recovery phase.100

Administration and interpretation of neuropsychological tests

While the use of neuropsychological tests for evaluating sport-related concussion has increased exponentially over the past 15 years, little attention has been paid to who should administer and interpret these tests.103 ,104 Differentiating between administering and interpreting neuropsychological tests is important. Although most healthcare professionals can learn to administer tests, significantly more expertise is required when interpreting neuropsychological test data. Interpretation of test scores in injured athletes is complex and requires advanced knowledge in psychometrics (eg, reliability, validity, normative classification, base rates and RC), standardised assessment procedures, the effects of situational factors on test performance, the impact of culture and linguistic differences on test scores and the influence of pre-existing (eg, ADHD or learning disability) or co-occurring conditions (eg, depression or anxiety) on test performance.103 These interpretive issues apply to computerised tests, which may erroneously give the appearance of being easy to interpret because they provide a nearly instantaneous printout of the results105 with scores estimated to be reliably changed from baseline highlighted in different colours. Neuropsychologists are the professionals best trained to interpret neuropsychological tests.103 ,106

Conclusions

  • The role and value of neuropsychological testing as part of a comprehensive concussion evaluation and management programme has been well established. Traditional measures, computerised batteries and hybrid approaches have been studied, each with corresponding strengths and weaknesses. There is no conclusive evidence supporting one approach over another.

  • Brief cognitive screening tests such as the SAC and SCAT2 are not substitutes for more comprehensive neuropsychological assessment.

  • Recent studies have demonstrated the utility of computerised neuropsychological testing in the absence of baseline data. Although popular, there is insufficient published evidence to recommend the widespread universal use of baseline neuropsychological testing within a sport concussion management programme. Additional research is needed to determine the utility of baseline data with different age groups, across different test instruments, among different cultures and languages and those with pre-existing psychological, educational and medical difficulties.

  • Psychological factors can have an important and pervasive impact on an athlete's recovery after concussion. Research examining psychological functioning in athletes and how psychological factors affect postconcussion recovery is imperative.

  • Although advances have been made in symptom scales that are appropriate and useful with children, researches related to psychometrically sound, age-appropriate tests that are sensitive to the effects of concussion in children are needed.

  • Neuropsychological instruments should not be used in a stand-alone manner to make return-to-play decisions.

  • Neuropsychological tests can be administered by properly trained paraprofessionals, but the interpretation of neuropsychological test data is complex and ideally performed by a trained neuropsychologist.

What are the new findings?

  • Neuropsychological testing remains a key element of a comprehensive concussion evaluation and management programme but should not be used in a stand-alone manner to make return-to-play decisions.

  • Brief cognitive screening tests such as the Standardised Assessment of Concussion (SAC) and Sport Concussion Assessment Tool—Second Edition (SCAT2) are not substitutes for comprehensive neuropsychological assessment.

  • Neuropsychological testing is useful in the absence of baseline data as long as adequate normative data are available. There is insufficient published evidence to recommend the widespread universal use of baseline neuropsychological testing within a sport concussion management programme.

  • Research related to psychometrically sound, age-appropriate tests that are sensitive to the effects of concussion in children is needed.

How might it impact clinical practice in the near future?

  • Appropriate use of neuropsychological test data can be very useful in the evaluation and management of athletes diagnosed with concussion. Although neuropsychological testing should be a central element of concussion management programmes, widespread universal baseline testing may not be necessary when appropriate local norms exist.

References

View Abstract

Supplementary materials

  • Supplementary Data

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

    Files in this Data Supplement:

Footnotes

  • Contributors GLI, MM, SAM, GAG, MP and PC all contributed to the design, planning, literature review, manuscript preparation and manuscript review. RJE also contributed to the design, planning, literature review, manuscript preparation and manuscript review. He was also responsible for the overall content as a guarantor.

  • Competing interests See the supplementary online data for competing interests (http://dx.doi.org/10.1136/bjsports-2013-092186).

  • Provenance and peer review Commissioned; internally peer reviewed.

  • ▸ References to this paper are available online at http://bjsm.bmjgroup.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.