Assessment of cognitive function in sportspeople with head injury

Conventionally, athletes who have received a head injury through sports related incidents are deemed fit to resume participation on the basis of clinical judgment. Such judgments are often made with reference to the athlete's subjective rating of his/her symptoms or other non-standardised assessments of recovery.³ For example, a football player may be allowed to resume competition after a good performance at training. In people who have sustained a head injury, there are potentially serious neurological and cognitive consequences of early re-entry into the sporting arena that judgments made in this way fail to adequately consider. For example, a second concussive episode may exacerbate the effects of an initial concussion disproportionate with its severity, and cause serious and long term neurological and behavioural consequences (“second impact syndrome”¹²). The effects of concussion have also been shown to be cumulative,^{1, 13}, and repeated exposure to head injury may therefore result in progressively deteriorating cognitive function.⁴

These findings led to the adoption approximately two decades ago of neuropsychological tests to measure an athlete's cognitive abilities after a concussion. Performance on such tests has since been used to guide decisions about recovery from concussion and resumption of participation. Also, a number of studies have used neuropsychological tests to investigate cognitive function in head injured athletes.^{2, 8, 10} These studies generally compare the individual athlete's neuropsychological test scores after concussion with those before concussion (baseline test scores collected before the beginning of the sporting season). This approach controls for interindividual differences in performance, including differences occurring as a result of prior head injury, learning difficulties in young athletes, and other sources of individual variability. However, this approach presents a number of methodological and practical problems that are difficult to overcome when conventional neuropsychological tests are used. For example, a typical neuropsychological assessment battery may require two to three hours to administer and requires that a neuropsychologist or trained technician be present to supervise the athlete. These requirements make the baseline assessment of an entire sporting team or squad an unreasonably time consuming and expensive exercise. To overcome this problem, recent studies have used shortened test batteries comprising five or six neuropsychological tests that require about 20–30 minutes to administer.⁹ These “screening” batteries provide an adequate guide to the athlete's baseline cognitive status, while still allowing comparisons with the status after concussion in a number of cognitive domains. However, this approach still fails to overcome many of the methodological and practical problems that occur when repeated neuropsychological assessments are required (discussed below).

Computerised cognitive tests offer a solution to many of these methodological and practical problems. Computerised tests were designed initially to detect quite severe impairments in patients with neurological and psychiatric illness, in patients with brain lesions, and in people exposed to neurotoxic substances. As such, early computerised batteries comprised neuropsychological tests modified for computer presentation and response recording—for example, Cambridge Neuropsychological Test Automated Battery (CANTAB)¹⁴. More recent computerised batteries have used the unique properties of computing hardware to develop tests that are sensitive to very mild changes in cognition, such as those expected to occur in sports related TBI (table 1). To be interpretable according to conventional psychological principles, computerised tests have retained the structure of standard neuropsychological tests. However, the properties of the test may be enhanced such that many of the limitations of neuropsychological tests are minimised. For example, stimulus presentations can be randomised between participants, creating many alternative and equivalent forms of the test and resulting in a reduction in practice effects.

In the following sections, we summarise the major limitations of conventional neuropsychological tests when used in sporting settings, and describe how these may be overcome through the use of computerised cognitive tests. As an adjunct to this discussion, table 2 describes some of the properties of some current computerised cognitive test batteries.

Detection of mild cognitive dysfunction

Most neuropsychological tests are designed for the assessment of cognitive dysfunction caused by neurological or psychiatric illness or brain lesions, not for the assessment of mild changes in cognitive function over time.¹⁵ As a consequence, many conventional neuropsychological tests have poor psychometric properties for serial study, including a limited range of possible scores, floor and ceiling effects, and poor test-retest reliability. We have proposed recently that mild cognitive impairments may only be accurately detected with neuropsychological tests that possess good psychometric properties.¹⁶ Thus, although impairments in many cognitive domains may be a consequence of sports related TBI, these impairments will only be observed when reliable, valid, and sensitive neuropsychological tests are used. Analysis of previous sports related concussion studies supports this proposal. The most commonly observed impairments in these studies are on tests of information processing and psychomotor control.^{9, 10} As a general rule, such measures—for example, Trail Making Test, Digit Symbol Substitution Test—have better psychometric properties for serial testing than neuropsychological tests of more complex cognitive functions.¹⁷

One of the consequences of recording response time (RT) as a dependent variable with computerised cognitive tests is that many of these psychometric limitations are overcome—that is, RTs are typically recorded in milliseconds, ensuring that there are thousands of possible levels of performance. Tests of simple RT are also repeatable, as they do not suffer greatly from practice effects,¹⁸ which also ensures that they have better test-retest reliability than many neuropsychological tests.¹⁷ These advantages were highlighted recently in an article by Bleiberg and colleagues,¹⁸ who recorded the performance of six patients with mild TBI on RT measures of psychomotor speed, memory, mathematical, and spatial processing, as well as on a conventional neuropsychological battery. Patients were impaired significantly relative to matched controls on four of the five computerised RT measures encompassing multiple cognitive domains. In contrast, analysis of group performance on conventional neuropsychological tests indicated that the TBI group performed worse than the control group on only two of 12 tests, and actually performed significantly better than controls on another. A number of other studies also show the utility of computerised RT measures in detecting cognitive changes associated with mild TBI.^19–21

Detection of performance variability

Computerised cognitive tests are also ideal for measuring variability in performance. Previous studies of people with TBI suggest that performance variability may be a better indicator of cognitive dysfunction associated with a concussive episode than level of performance. For example, Stuss and colleagues^{19, 20} observed inconsistent performance on tests of RT in patients in hospital with head injury. Similarly, in the study by Bleiberg and colleagues¹⁸ described above, patients with TBI displayed erratic performance on computerised cognitive tests both within a day and across four consecutive days of assessment. Conventional neuropsychological tests are unable to give an indication of performance variability because responses on most of these tasks are recorded in a binary fashion—that is, correct or incorrect—or as a single integer. In contrast, computerised tests allow collection of RTs to stimuli, and, if enough responses are collected, the variability in these RTs can be calculated accurately, as in the studies described above.

Sensitivity and specificity

Although it is crucial for a cognitive test or test battery to be relevant to a particular target group—for example, sportspeople—restriction of development to that group raises the potential for the test to lack sensitivity to all potential forms of impairment. Cognitive tests should be developed and then also tested in groups of patients with well defined clinical conditions that are known to interfere with cognitive function. For example, the performance of patients with anxiety disorders or depression can define cognitive profiles or patterns of performance that could indicate a potential reason for poor performance in athletes. Furthermore, an understanding of the nature and severity of cognitive impairments in these serious conditions can provide important clues to the presentation in athletes of the cognitive consequences of more mild conditions that may arise directly or indirectly from head injury such as fatigue and stress. The effect of common medications, such as benzodiazepines and alcohol, on cognitive function can also be determined by such computerised studies.^{22, 23} These results can then be used to infer differential diagnostic causes in athletes in whom similar patterns of deficiency are found. Cognitive tests should aim to do more than just detect deviation from normal. They should limit the diagnostic possibilities by defining recognisable patterns of abnormal performance.

Alternative forms, practice effects, and test-retest reliability

Practice effects are a particularly important methodological problem for sports related TBI, as the magnitude of these effects may vary depending on the test-retest interval.²⁴ For example, if an athlete is concussed two weeks after a baseline assessment, then practice is likely to have a substantial effect on test performance. In contrast, if an athlete is concussed in the last game of a long season, practice effects will be reduced. Practice effects are assumed to operate only between the first and second administration of a neuropsychological test battery,²⁵ but recent reports suggest that, at least for some tests, they may influence performance on up to four serial assessments.¹⁷ When the athlete is assessed at 24 hours, five days, and 14 days after concussion, as is standard, practice effects may be operating at none or all of these assessments. A resolution to this problem is therefore important for accurate decision making.

One method of reducing the magnitude of practice effects is to use alternate forms of the test or test battery. However, many conventional neuropsychological tests do not have alternate forms.¹⁷ This means that interpretation of data obtained from a recently concussed athlete will be influenced by the effects of practice and also by the reliability of the test. Also, when a neuropsychological test does have alternate forms, these are not always equivalent, which introduces another systematic source of measurement error. A second method advocated to control for practice effects is to record serial data from an appropriate control group—for example, a non-concussed athlete. This allows the estimation of practice effects and measurement error, which can then be considered jointly with the athlete's performance before and after concussion. However, very few serially recorded data have been published,¹⁷ and an adequate control participant may be difficult to recruit. Computerised cognitive tests allow the randomisation of stimulus presentation within a test, between tests, and between subjects if appropriate, and multidimensional stimuli can be used. This may result in the production of many, or indeed infinite, alternate forms of the test. In turn, having many alternate forms will result in reduction of the effects of practice on test performance when the test is administered serially and a consequent reduction in measurement error. This will increase the test-retest reliability of the test.

Practical matters

Perhaps the most attractive properties of computerised cognitive tests to sports administrators and sports physicians is that they offer major practical advantages over conventional neuropsychological assessment techniques. For example, data can be stored and scored automatically by the computer in milliseconds, allowing immediate interpretation by the physician or trainer at the time of assessment. Of particular importance to multicentre research studies is the possibility that data may be transferred between sites, or to a central database, electronically in a matter of seconds. This raises the possibility that an office bound neuropsychologist could collate and interpret the test results of a recently injured athlete and send their interpretation back to the trainer or physician at the sporting arena in a matter of minutes. If enough computing resources are available, baseline assessments may be conducted on an entire sporting team in a single testing session. If the computerised test is internet deliverable and self administered, baseline assessments can be conducted by the athlete in their home environment, removing the need for teams of assessors to visit sporting clubs and interrupt training sessions. Finally, computerised assessment allows standardisation of administration protocols between subjects and between multiple sites. In turn, this results in minimisation of measurement error introduced by the assessor, especially where the test is self administered.

Another problem associated with using paper and pencil neuropsychological tests in sporting settings is that the athlete's test results need to be generated and interpreted by a neuropsychologist. Again, this can be an expensive and time consuming exercise, as most neuropsychologists attached to sporting teams have other professional interests that may take precedence. Finally, and perhaps more importantly, computerised cognitive tests may require less than 30 minutes to complete, while still providing a profile of cognitive performance.^{26, 27}

Limitations of computerised cognitive assessment

There are three major limitations of most computerised cognitive assessment tools. The first is the hardware required to administer the tests. For example, the CANTAB battery requires not only a PC and keyboard, but also a touch sensitive screen.¹⁴ Similarly, CogScreen requires a PC, keyboard, mouse, and a stylus to indicate responses.²⁸ This means that the tests are not as portable as conventional neuropsychological tests. The second limitation is the cost involved in setting up a computerised assessment system, which may include purchasing expensive software and hardware—for example, a touch sensitive screen—as well as training neuropsychologists and technicians in administration protocols, data storage, and analysis. In contrast, many paper and pencil neuropsychological tests are available freely in the scientific literature and require no computer expertise to administer or score. Furthermore, many computerised cognitive tests do not have sufficient normative and test-retest reliability data, and have not been validated against conventional neuropsychological measures or for use in different disorders and settings. These limitations may all be overcome by further test development and validation. For example, we have recently developed an internet deliverable cognitive test that runs on any platform and can therefore be administered on any PC, with results returned through an existing internet connection.²⁶ This, and cognitive tests developed by other groups,²⁹ are now being validated for use in many settings.

Despite the many advantages of computerised cognitive assessment, the results from such assessments should not be considered in isolation. In fact, computerised testing should be used mainly to inform decisions on fitness to resume participation when the sports physician or trainer is uncertain of the athlete's status after a conventional neurological and physical examination has been conducted. The cardinal signs of concussive episodes should be considered before the athlete's cognitive status—for example, nausea, headache, retrograde and anterograde amnesia. Further, it may be appropriate to conduct a more detailed neuropsychological examination of the athlete after a concussive episode, in order to gain a greater understanding of the domains of cognition persistently affected by the brain injury.

Summary and conclusions

An increasing awareness of the effects of sports related head injury on cognition has led sports physicians to seek fast and accurate assessments of cognitive function, to facilitate management decisions about time of recovery and resumption of participation. Over the past two decades, this has been accomplished through the use of clinical judgment or conventional paper and pencil neuropsychological tests. However, there are limitations associated with using such tests in repeated measures designs. This has led to the development of computerised cognitive test batteries, which are often specifically designed for the serial assessment of cognitive function in the individual. Such computerised batteries offer both scientific and practical advantages over conventional neuropsychological measures. These include high sensitivity and specificity to mild impairments in cognition as occurs in many athletes with sports related head injuries, and the ability to conduct baseline assessments before the season on entire sporting teams in a matter of hours. Recent work suggests that computerised tests of RT allow the detection of very subtle cognitive changes, return to baseline performance, and also the detection of performance variability. The widespread use of many computerised tests and test batteries is limited currently by their high cost and low accessibility; however, some very recently developed test batteries are designed to be internet deliverable and inexpensive. Another limitation of most computerised test batteries is the lack of normative data; however, this is rapidly being overcome as these test batteries become more commonly used.

It is likely that, in the near future, cognitive testing programs will be more widely implemented using one or other computerised screening test. Such programs would perform baseline testing before the season to establish optimum non-impaired performance. The tool used would preferably be self instructing and brief (10–15 minutes). It should be readily available for athletes to practice before supervised testing by sporting associations or teams. Results would then be stored safely and available for comparison with repeat tests after head injury at appropriate medically based intervals. For example, a mild concussion may be judged during play to be insignificant, which, if backed up by an unchanged computerised test at the time, would allow the player to resume play immediately. A more significant concussion may require testing only when the medical attendant felt the athlete was back to normal. Subtle decrements would indicate persisting impairment, and retesting could continue until return to baseline performance had occurred. If no such return to baseline was achieved, then further neuropsychological and medical assessments would be indicated. Such testing programs are predicted to be available inexpensively via the internet for all levels of sports men and women including amateurs.

In conclusion, physicians and athletes in sports in which there is a risk of concussive head injury may benefit greatly from the use of computerised tests of cognitive function. For the sports physician, this benefit may come in the form of more accurate and informed decision making with regard to the athlete's resumption of participation, and a reduction in time to conduct and interpret baseline assessments of athletes and those made after concussion. For the athlete, dual benefits arise, with a reduced risk of long term deficits in cognitive function caused by early re-entry into the sporting arena, and return to baseline test performance predictive of continued optimal on field accomplishment.