rss
Br J Sports Med 35:297-302 doi:10.1136/bjsm.35.5.297
  • Review

Computerised cognitive assessment of athletes with sports related head injury

  1. A Collie1,2,
  2. D Darby1,3,
  3. P Maruff2,4
  1. 1Centre for Neuroscience, University of Melbourne, Parkville, Victoria, Australia
  2. 2Neuropsychology Laboratory, Mental Health Research Institute of Victoria, Parkville, Victoria, Australia
  3. 3Behavioural Neurology Laboratory, Mental Health Research Institute of Victoria
  4. 4School of Psychology, La Trobe University, Bundoora, Victoria, Australia
  1. Correspondence to: Dr Collie, Neuropsychology Laboratory, Mental Health Research Institute of Victoria, Locked Bag 11, Parkville, Victoria, Australia 3052 alex{at}neuro.mhri.edu.au
  • Accepted 4 May 2001

Abstract

Professional and amateur participants in many sports are at risk of brain injury caused by impact with other players or objects. In many cases, mild cognitive deficits may persist after the common neurological signs of brain injury have passed. In recent years, the athlete's cognitive status after concussion has been measured with conventional “paper and pencil” neuropsychological tests. However, such tests are not ideal for sporting settings, as they are designed for the detection of gross cognitive impairments at a single assessment, not for the identification of mild cognitive deficits on repeated assessment. A number of computerised cognitive assessment tests and test batteries have been developed over the past two decades. These batteries offer major scientific and practical advantages over conventional neuropsychological tests which make them ideal for the assessment of cognitive function in sportspeople. This review first describes the problems associated with cognitive assessment of people with sports related cognitive deficits, and then critically examines the utility of conventional neuropsychological and computerised cognitive tests in sporting settings.

Take home message

Careful assessment of cognitive function in athletes with sports related head injury will facilitate clinical strategies for the athletes' recovery and return to play. Computerised cognitive tests and test batteries are designed specifically for the detection of very mild cognitive dysfunction, and offer both practical and scientific advantages over conventional neuropsychological tests.

Modern sport is highly competitive, with the health of elite sports men and women regarded as major assets by themselves, their sporting associations, and the community. Their cognitive health in particular is one of the most important factors in their continuing success, as measured by performance statistics. Head injury is a significant risk whenever athletes perform competitively in sports entailing physical contact with other players or objects. Traumatic brain injury (TBI) can lead to overt problems such as acute headache, nausea, vomiting, dizziness, syncope, confusion, and coma.1 However, in many cases the signs of TBI are covert, manifesting as mild cognitive deficits that may be detected only through careful neuropsychological testing.2 Such subclinical impairment may be difficult to diagnose with certainty using on field or subjective medical assessment techniques.3 This, in turn, complicates decisions about real time return to play, as well as decisions about the longer term health consequences of the injury, including risk of further injury. Sports in which repeated TBIs are common, such as boxing,4 also raise the problem of cumulative cognitive damage,1 the assessment and management of which is currently poorly codified. In addition, people who participate in sports such as scuba diving or high altitude climbing are at risk of brain injury through other mechanisms of brain damage (ischaemia, nitrogen narcosis,5 hypoxia6) that may also contribute to persistent cognitive deficits. These problems are not limited to professional sportspeople. Amateur and recreational participants in boxing,4 some martial arts, and varieties of football including soccer,7 rugby,8 American football,9 and Australian Rules football,10 may exhibit cognitive deficits following head injury incurred in the sporting arena. Although many sports have modified their rules to reduce the incidence of TBI, such injuries still occur regularly.

Conventional neurological and neuropsychological techniques have significant limitations for the accurate evaluation of these conditions during field play.11 For example, neuropsychological tests are expensive in both time and skilled labour requirements. Even if sufficient resources are available to baseline test an entire team or training squad before a season, the measurement properties of most neuropsychological tests are not ideal for repeated testing. New approaches using shorter “paper and pencil” test batteries, as well as computerised tests, have emerged to overcome these problems and facilitate both on field and subsequent decisions about fitness to play. Computerised testing offers the theoretical advantages of infinite randomised forms, standardised self administration, rapid testing, internet based delivery, and centralised data storage, analysis, and reporting. These approaches are relatively new and not yet widely adopted, in part because of continuing evolution. Although computerised testing has definite advantages over conventional neuropsychological testing, there are also a number of limitations that must be considered before they can be applied to identify subtle TBI in sporting contexts. This review will describe the problems associated with cognitive assessment of people with sports related cognitive deficits, and then critically examine the utility of conventional neuropsychological and computerised cognitive tests in sporting settings. As most reports describe athletes with sports related head injury, our discussion will be restricted to the use of cognitive tests in concussed and head injured athletes. This information should form a basis for the wider development and implementation of computerised testing in all sporting arenas.

Research and review articles were considered for discussion in this review if they had been published in international peer reviewed scientific journals in sports medicine, neurology, or neuropsychology and had in our opinion met one or more of the following criteria: (a) the article reported the results of computerised cognitive or conventional neuropsychological assessment of athletes with a sports related brain injury; (b) the article had made an inference about the neurological and/or cognitive consequences of sports related brain injury; (c) the article provided adequate description of a computerised cognitive test or test battery; or (d) the article discussed the limitations of conventional neuropsychological or computerised cognitive tests.

Assessment of cognitive function in sportspeople with head injury

Conventionally, athletes who have received a head injury through sports related incidents are deemed fit to resume participation on the basis of clinical judgment. Such judgments are often made with reference to the athlete's subjective rating of his/her symptoms or other non-standardised assessments of recovery.3 For example, a football player may be allowed to resume competition after a good performance at training. In people who have sustained a head injury, there are potentially serious neurological and cognitive consequences of early re-entry into the sporting arena that judgments made in this way fail to adequately consider. For example, a second concussive episode may exacerbate the effects of an initial concussion disproportionate with its severity, and cause serious and long term neurological and behavioural consequences (“second impact syndrome”12). The effects of concussion have also been shown to be cumulative,1, 13, and repeated exposure to head injury may therefore result in progressively deteriorating cognitive function.4

These findings led to the adoption approximately two decades ago of neuropsychological tests to measure an athlete's cognitive abilities after a concussion. Performance on such tests has since been used to guide decisions about recovery from concussion and resumption of participation. Also, a number of studies have used neuropsychological tests to investigate cognitive function in head injured athletes.2, 8, 10 These studies generally compare the individual athlete's neuropsychological test scores after concussion with those before concussion (baseline test scores collected before the beginning of the sporting season). This approach controls for interindividual differences in performance, including differences occurring as a result of prior head injury, learning difficulties in young athletes, and other sources of individual variability. However, this approach presents a number of methodological and practical problems that are difficult to overcome when conventional neuropsychological tests are used. For example, a typical neuropsychological assessment battery may require two to three hours to administer and requires that a neuropsychologist or trained technician be present to supervise the athlete. These requirements make the baseline assessment of an entire sporting team or squad an unreasonably time consuming and expensive exercise. To overcome this problem, recent studies have used shortened test batteries comprising five or six neuropsychological tests that require about 20–30 minutes to administer.9 These “screening” batteries provide an adequate guide to the athlete's baseline cognitive status, while still allowing comparisons with the status after concussion in a number of cognitive domains. However, this approach still fails to overcome many of the methodological and practical problems that occur when repeated neuropsychological assessments are required (discussed below).

Computerised cognitive tests offer a solution to many of these methodological and practical problems. Computerised tests were designed initially to detect quite severe impairments in patients with neurological and psychiatric illness, in patients with brain lesions, and in people exposed to neurotoxic substances. As such, early computerised batteries comprised neuropsychological tests modified for computer presentation and response recording—for example, Cambridge Neuropsychological Test Automated Battery (CANTAB)14. More recent computerised batteries have used the unique properties of computing hardware to develop tests that are sensitive to very mild changes in cognition, such as those expected to occur in sports related TBI (table 1). To be interpretable according to conventional psychological principles, computerised tests have retained the structure of standard neuropsychological tests. However, the properties of the test may be enhanced such that many of the limitations of neuropsychological tests are minimised. For example, stimulus presentations can be randomised between participants, creating many alternative and equivalent forms of the test and resulting in a reduction in practice effects.

Table 1

Properties of conventional neuropsychological and computerised cognitive tests

In the following sections, we summarise the major limitations of conventional neuropsychological tests when used in sporting settings, and describe how these may be overcome through the use of computerised cognitive tests. As an adjunct to this discussion, table 2 describes some of the properties of some current computerised cognitive test batteries.

Table 2

Analysis of the suitability of some existing computerised cognitive test batteries for use in sporting settings

Detection of mild cognitive dysfunction

Most neuropsychological tests are designed for the assessment of cognitive dysfunction caused by neurological or psychiatric illness or brain lesions, not for the assessment of mild changes in cognitive function over time.15 As a consequence, many conventional neuropsychological tests have poor psychometric properties for serial study, including a limited range of possible scores, floor and ceiling effects, and poor test-retest reliability. We have proposed recently that mild cognitive impairments may only be accurately detected with neuropsychological tests that possess good psychometric properties.16 Thus, although impairments in many cognitive domains may be a consequence of sports related TBI, these impairments will only be observed when reliable, valid, and sensitive neuropsychological tests are used. Analysis of previous sports related concussion studies supports this proposal. The most commonly observed impairments in these studies are on tests of information processing and psychomotor control.9, 10 As a general rule, such measures—for example, Trail Making Test, Digit Symbol Substitution Test—have better psychometric properties for serial testing than neuropsychological tests of more complex cognitive functions.17

One of the consequences of recording response time (RT) as a dependent variable with computerised cognitive tests is that many of these psychometric limitations are overcome—that is, RTs are typically recorded in milliseconds, ensuring that there are thousands of possible levels of performance. Tests of simple RT are also repeatable, as they do not suffer greatly from practice effects,18 which also ensures that they have better test-retest reliability than many neuropsychological tests.17 These advantages were highlighted recently in an article by Bleiberg and colleagues,18 who recorded the performance of six patients with mild TBI on RT measures of psychomotor speed, memory, mathematical, and spatial processing, as well as on a conventional neuropsychological battery. Patients were impaired significantly relative to matched controls on four of the five computerised RT measures encompassing multiple cognitive domains. In contrast, analysis of group performance on conventional neuropsychological tests indicated that the TBI group performed worse than the control group on only two of 12 tests, and actually performed significantly better than controls on another. A number of other studies also show the utility of computerised RT measures in detecting cognitive changes associated with mild TBI.19–21

Detection of performance variability

Computerised cognitive tests are also ideal for measuring variability in performance. Previous studies of people with TBI suggest that performance variability may be a better indicator of cognitive dysfunction associated with a concussive episode than level of performance. For example, Stuss and colleagues19, 20 observed inconsistent performance on tests of RT in patients in hospital with head injury. Similarly, in the study by Bleiberg and colleagues18 described above, patients with TBI displayed erratic performance on computerised cognitive tests both within a day and across four consecutive days of assessment. Conventional neuropsychological tests are unable to give an indication of performance variability because responses on most of these tasks are recorded in a binary fashion—that is, correct or incorrect—or as a single integer. In contrast, computerised tests allow collection of RTs to stimuli, and, if enough responses are collected, the variability in these RTs can be calculated accurately, as in the studies described above.

Sensitivity and specificity

Although it is crucial for a cognitive test or test battery to be relevant to a particular target group—for example, sportspeople—restriction of development to that group raises the potential for the test to lack sensitivity to all potential forms of impairment. Cognitive tests should be developed and then also tested in groups of patients with well defined clinical conditions that are known to interfere with cognitive function. For example, the performance of patients with anxiety disorders or depression can define cognitive profiles or patterns of performance that could indicate a potential reason for poor performance in athletes. Furthermore, an understanding of the nature and severity of cognitive impairments in these serious conditions can provide important clues to the presentation in athletes of the cognitive consequences of more mild conditions that may arise directly or indirectly from head injury such as fatigue and stress. The effect of common medications, such as benzodiazepines and alcohol, on cognitive function can also be determined by such computerised studies.22, 23 These results can then be used to infer differential diagnostic causes in athletes in whom similar patterns of deficiency are found. Cognitive tests should aim to do more than just detect deviation from normal. They should limit the diagnostic possibilities by defining recognisable patterns of abnormal performance.

Alternative forms, practice effects, and test-retest reliability

Practice effects are a particularly important methodological problem for sports related TBI, as the magnitude of these effects may vary depending on the test-retest interval.24 For example, if an athlete is concussed two weeks after a baseline assessment, then practice is likely to have a substantial effect on test performance. In contrast, if an athlete is concussed in the last game of a long season, practice effects will be reduced. Practice effects are assumed to operate only between the first and second administration of a neuropsychological test battery,25 but recent reports suggest that, at least for some tests, they may influence performance on up to four serial assessments.17 When the athlete is assessed at 24 hours, five days, and 14 days after concussion, as is standard, practice effects may be operating at none or all of these assessments. A resolution to this problem is therefore important for accurate decision making.

One method of reducing the magnitude of practice effects is to use alternate forms of the test or test battery. However, many conventional neuropsychological tests do not have alternate forms.17 This means that interpretation of data obtained from a recently concussed athlete will be influenced by the effects of practice and also by the reliability of the test. Also, when a neuropsychological test does have alternate forms, these are not always equivalent, which introduces another systematic source of measurement error. A second method advocated to control for practice effects is to record serial data from an appropriate control group—for example, a non-concussed athlete. This allows the estimation of practice effects and measurement error, which can then be considered jointly with the athlete's performance before and after concussion. However, very few serially recorded data have been published,17 and an adequate control participant may be difficult to recruit. Computerised cognitive tests allow the randomisation of stimulus presentation within a test, between tests, and between subjects if appropriate, and multidimensional stimuli can be used. This may result in the production of many, or indeed infinite, alternate forms of the test. In turn, having many alternate forms will result in reduction of the effects of practice on test performance when the test is administered serially and a consequent reduction in measurement error. This will increase the test-retest reliability of the test.

Practical matters

Perhaps the most attractive properties of computerised cognitive tests to sports administrators and sports physicians is that they offer major practical advantages over conventional neuropsychological assessment techniques. For example, data can be stored and scored automatically by the computer in milliseconds, allowing immediate interpretation by the physician or trainer at the time of assessment. Of particular importance to multicentre research studies is the possibility that data may be transferred between sites, or to a central database, electronically in a matter of seconds. This raises the possibility that an office bound neuropsychologist could collate and interpret the test results of a recently injured athlete and send their interpretation back to the trainer or physician at the sporting arena in a matter of minutes. If enough computing resources are available, baseline assessments may be conducted on an entire sporting team in a single testing session. If the computerised test is internet deliverable and self administered, baseline assessments can be conducted by the athlete in their home environment, removing the need for teams of assessors to visit sporting clubs and interrupt training sessions. Finally, computerised assessment allows standardisation of administration protocols between subjects and between multiple sites. In turn, this results in minimisation of measurement error introduced by the assessor, especially where the test is self administered.

Another problem associated with using paper and pencil neuropsychological tests in sporting settings is that the athlete's test results need to be generated and interpreted by a neuropsychologist. Again, this can be an expensive and time consuming exercise, as most neuropsychologists attached to sporting teams have other professional interests that may take precedence. Finally, and perhaps more importantly, computerised cognitive tests may require less than 30 minutes to complete, while still providing a profile of cognitive performance.26, 27

Limitations of computerised cognitive assessment

There are three major limitations of most computerised cognitive assessment tools. The first is the hardware required to administer the tests. For example, the CANTAB battery requires not only a PC and keyboard, but also a touch sensitive screen.14 Similarly, CogScreen requires a PC, keyboard, mouse, and a stylus to indicate responses.28 This means that the tests are not as portable as conventional neuropsychological tests. The second limitation is the cost involved in setting up a computerised assessment system, which may include purchasing expensive software and hardware—for example, a touch sensitive screen—as well as training neuropsychologists and technicians in administration protocols, data storage, and analysis. In contrast, many paper and pencil neuropsychological tests are available freely in the scientific literature and require no computer expertise to administer or score. Furthermore, many computerised cognitive tests do not have sufficient normative and test-retest reliability data, and have not been validated against conventional neuropsychological measures or for use in different disorders and settings. These limitations may all be overcome by further test development and validation. For example, we have recently developed an internet deliverable cognitive test that runs on any platform and can therefore be administered on any PC, with results returned through an existing internet connection.26 This, and cognitive tests developed by other groups,29 are now being validated for use in many settings.

Despite the many advantages of computerised cognitive assessment, the results from such assessments should not be considered in isolation. In fact, computerised testing should be used mainly to inform decisions on fitness to resume participation when the sports physician or trainer is uncertain of the athlete's status after a conventional neurological and physical examination has been conducted. The cardinal signs of concussive episodes should be considered before the athlete's cognitive status—for example, nausea, headache, retrograde and anterograde amnesia. Further, it may be appropriate to conduct a more detailed neuropsychological examination of the athlete after a concussive episode, in order to gain a greater understanding of the domains of cognition persistently affected by the brain injury.

Summary and conclusions

An increasing awareness of the effects of sports related head injury on cognition has led sports physicians to seek fast and accurate assessments of cognitive function, to facilitate management decisions about time of recovery and resumption of participation. Over the past two decades, this has been accomplished through the use of clinical judgment or conventional paper and pencil neuropsychological tests. However, there are limitations associated with using such tests in repeated measures designs. This has led to the development of computerised cognitive test batteries, which are often specifically designed for the serial assessment of cognitive function in the individual. Such computerised batteries offer both scientific and practical advantages over conventional neuropsychological measures. These include high sensitivity and specificity to mild impairments in cognition as occurs in many athletes with sports related head injuries, and the ability to conduct baseline assessments before the season on entire sporting teams in a matter of hours. Recent work suggests that computerised tests of RT allow the detection of very subtle cognitive changes, return to baseline performance, and also the detection of performance variability. The widespread use of many computerised tests and test batteries is limited currently by their high cost and low accessibility; however, some very recently developed test batteries are designed to be internet deliverable and inexpensive. Another limitation of most computerised test batteries is the lack of normative data; however, this is rapidly being overcome as these test batteries become more commonly used.

It is likely that, in the near future, cognitive testing programs will be more widely implemented using one or other computerised screening test. Such programs would perform baseline testing before the season to establish optimum non-impaired performance. The tool used would preferably be self instructing and brief (10–15 minutes). It should be readily available for athletes to practice before supervised testing by sporting associations or teams. Results would then be stored safely and available for comparison with repeat tests after head injury at appropriate medically based intervals. For example, a mild concussion may be judged during play to be insignificant, which, if backed up by an unchanged computerised test at the time, would allow the player to resume play immediately. A more significant concussion may require testing only when the medical attendant felt the athlete was back to normal. Subtle decrements would indicate persisting impairment, and retesting could continue until return to baseline performance had occurred. If no such return to baseline was achieved, then further neuropsychological and medical assessments would be indicated. Such testing programs are predicted to be available inexpensively via the internet for all levels of sports men and women including amateurs.

In conclusion, physicians and athletes in sports in which there is a risk of concussive head injury may benefit greatly from the use of computerised tests of cognitive function. For the sports physician, this benefit may come in the form of more accurate and informed decision making with regard to the athlete's resumption of participation, and a reduction in time to conduct and interpret baseline assessments of athletes and those made after concussion. For the athlete, dual benefits arise, with a reduced risk of long term deficits in cognitive function caused by early re-entry into the sporting arena, and return to baseline test performance predictive of continued optimal on field accomplishment.

Take home message

Careful assessment of cognitive function in athletes with sports related head injury will facilitate clinical strategies for the athletes' recovery and return to play. Computerised cognitive tests and test batteries are designed specifically for the detection of very mild cognitive dysfunction, and offer both practical and scientific advantages over conventional neuropsychological tests.

References

Free sample

This 2013 issue is free to all users to allow everyone the opportunity to see the full scope and typical content of BJSM.
View free sample issue >>

Email alerts

Don't forget to sign up for content alerts so you keep up to date with all the articles as they are published.