Theoretical and operational definitions of injury

To understand the size of an injury issue, the injury being counted needs to be well defined. Theoretical definitions of injury specify what injuries ‘are’4; operational definitions of injury specify which injuries become cases, that is, how injuries can, or will, be counted in a given context.5–7

Although controversies remain about exactly what does and does not fall within a good theoretical definition of an injury,4 most definitions include some reference to an abnormal transfer of energy, and the resulting outcome(s). Theoretical definitions of injury are helpful in providing a conceptual framework by which to consider injuries, however, they leave open the question of operationalising the definition for the purposes of injury surveillance, and subsequent risk management and injury prevention initiatives. It is for this reason that various sports medicine groups have produced ‘injury definitions’ guidelines specifying which events meet the criteria for inclusion in injury counts.6 ,8

The Zürich definition of concussion

The Zürich Consensus Statement on Concussion (hereafter referred to as the Zürich Guidelines) provides a relatively clear theoretical definition of concussion.9 The concept of a transfer of injury resulting in some physical change is reflected in the Zürich Guidelines. To paraphrase, the Zürich Guidelines specify that concussion is a brain injury caused by traumatic forces that alters the way the brain functions and presents with a range of signs and/or symptoms that may or may not include loss of consciousness.

To assist with concussion diagnosis, readers of the guidelines are directed to the Sport Concussion Assessment Tool 3 (SCAT3; or the child version thereof), for ‘the detailed clinical assessment of concussion’. In the SCAT3 and the body of the Zürich Guidelines there are statements to the effect that the final determination of a concussion diagnosis is a medical decision based on clinical judgement.

Limitations of the Zürich definition of concussion

In their recent critique of the Zürich Guidelines, Craton and Leslie10 identify a number of shortcomings with the document, most of which stem from the definition of concussion that has been adopted. Craton and Leslie focus on the clinical implications that result from the limitations to the existing definitions inherent in the Zürich Guidelines. The limitations include:

• Lack of diagnostic specificity;

• Management strategies that are not evidence based;

• Rehabilitation goals that are not attainable.

The Zürich Guidelines and the SCAT3 state that the signs and symptoms of concussion are non-specific—in other words, the signs and symptoms that characterise concussion are not unique to concussion, and can be present concurrently with other conditions or, indeed, with no morbidity. For example, a single sign or symptom from the SCAT3 (or some combination of signs and/or symptoms) following a blow to the head or body may indicate a concussion, some other condition (eg, a subdural haematoma, cervical, ocular, labyrinth or intracranial injury), some combination of conditions, or no injury (eg, a player may have had a pre-existing exertional headache, which he/she reported as a symptom when questioned about current symptoms following a blow). Further, both Craton and Leslie10 and Guskiewicz et al11 have pointed out that baseline scores on the symptom evaluation included in the SCAT3 are typically non-zero.

Beyond the clinical implications noted by Craton and Leslie,10 there are important issues for managing risks on a population basis that arise from the current definitions in the Zürich Guidelines.

As the Zürich concussion definition is theoretical, rather than operational, little direction is provided about which injuries should be counted as concussions for the purposes of injury surveillance. Although the definition, and the SCAT3 instrument, are probably sensitive to capturing events that are concussions,10 many events that are not concussions fall within the scope of the definition (and will be captured by the SCAT3) as they are currently framed. If a literal interpretation of the SCAT3 is taken, then concussion should be ‘suspected’ whenever one or more of the signs or symptoms associated with concussion is/are present subsequent to a blow to the head or body. As noted by Craton and Leslie,10 a clinical diagnosis of concussion in particular sports can have implications on entry into rehabilitation processes/return to play protocols. If concussions are being managed conservatively, it may be that some injuries that are not concussions, but that have resulted in a player having a sign or symptom on the SCAT3, are being treated as such. Should these be counted in injury surveillance systems as concussions? If they are, they will provide higher rates of concussion than studies reported to date, which have typically set the criteria for a concussion at a higher threshold. Counting any ‘suspected’ concussion as a concussion for the purposes of injury surveillance, as was carried out recently in a paper by King et al,12 led to an approximately 10 times greater rate of ‘concussion’ than had previously been reported in rugby union football. It is important for people to understand that the rate of concussion in rugby has not recently undergone a 10-fold increase. What has happened is that different things are being counted in different studies.

Time course of concussion—when should the SCAT3 be performed?

It is also understood that the signs and symptoms of concussion evolve—and resolve—over time. The signs and symptoms apparent immediately following an injury may be different from those the injured player has an hour, or 24 h, later. Yet there is no guidance in the SCAT3 as to when the test should be conducted. Standardising, as far as possible, the time at which the SCAT3 is performed following injury would help provide consistency for injury surveillance. If the SCAT3, or elements of it, are serially applied to a player following an injury, then agreement needs to be reached about which set of scores should be used for the purposes of data collection for injury surveillance (presumably the first, but again, this has not been specified).

Implications for injury surveillance of ‘clinical judgement’ as a gold standard for concussion diagnosis

On the basis of the Zürich Guidelines, the ‘gold standard’ of concussion diagnosis is a clinical decision that a sports participant has been concussed. Although there are other areas of medicine besides concussion in which a clinical judgement is the de facto ‘gold standard’ for a diagnosis,13 for the purposes of injury surveillance it is problematic that the Zürich Guidelines have endorsed clinical decisions as being the basis for concussion diagnosis without having provided any information about the typical validity or interclinician and intraclinician reliability of concussion diagnosis. If concussion diagnosis by clinicians is regarded as a test (is the player concussed or not concussed?), what is the sensitivity and specificity of the test? Such information does not seem to have been published to-date in the sports medicine literature. The current state of affairs means that researchers attempting to validate instruments (such as the SCAT3 or King-Devick test) to assist with concussion diagnosis, management and injury surveillance have nothing against which to validate them. In our view, the Zürich Guidelines are deficient in this respect.

Coggon et al13 have pointed out that using diagnoses as a means of classifying people to prevent or manage illness rather than as a label designating a specific disease process may be a useful approach to take. They state ‘where there is no established underlying pathology or no credible gold standard for the presumed underlying pathology, case definitions should be evaluated according to their practical utility in the elucidation of preventable causes and the optimisation of clinical care’.

Concussion signs and symptoms, and clinical diagnoses

An important element lacking in the operationalising of a concussion definition based on the SCAT3 is the extent to which any given sign or symptom (or combination thereof) is typically associated with a clinical diagnosis of concussion being made. Whether, and to what extent, a given combination of signs or symptoms is valid in the absence of baseline SCAT3 tests is yet to be established. For clinical management, two different sets of validity studies may be required, one for change scores from baseline, and one for scores for players who have no baseline available. For the purposes of injury surveillance, the scores on a postinjury SCAT3 would suffice if enough data on the relationships between constellations of signs and symptoms on the SCAT3 and concussion diagnoses by clinicians were available.

As opposed to a binary decision of ‘yes this person is concussed’ versus ‘no this person is not concussed’ it may, for the purposes of injury surveillance, be more useful to consider concussions as probabilistic events based on SCAT3 scores (or other instruments, if they have been properly validated), and to count any events that are over a given (consensus derived) probability of being a concussion as concussions, regardless of whether the actual injury was a concussion alone, a concussion and some other injury, or neither. Both the sensitivity and specificity of the SCAT3 (or, for that matter, any other postinjury evaluation tool for concussion) are important considerations for clinical management based on a diagnosis of concussion. For injury surveillance, a case can be made that high sensitivity is more important than high specificity as long as a consistent approach is taken—although higher specificity is obviously more desirable than lower specificity. Such an approach would lead to almost all events that actually are concussions being counted in injury surveillance systems, alongside some events that are not. It is likely that most of the injuries resulting in signs or symptoms on the SCAT3 that are not concussions are some other type of head or neck injury that we would wish to prevent, and if they are caused by the same or similar mechanisms as concussion then injury prevention efforts targeted at concussions would be expected to affect the other injuries as well. Ongoing monitoring would allow evaluation of whether rates of concussion and the other injuries picked up by the signs and symptoms assayed by the SCAT3 are increasing or decreasing over time, and facilitate comparisons of rates between activities.

Towards an operational definition of concussion

At present, there is a lack of published information about the behaviour of the SCAT3 as a scientific instrument. While information is available about the various components of the instrument,11 the SCAT3 as a whole is yet to be validated. If the sports medicine community at large is going to adopt and endorse the SCAT3, then establishing the scientific properties of the instrument are crucially important for progressing both clinical care of potentially concussed sports participants, and establishing the size of the concussion problem via injury surveillance. A potential way of improving the operational definition of concussion via the use of the SCAT3 as a diagnostic tool would be to:

1. Standardise the time at which the SCAT3 is applied postinjury.

2. Have an expert group of clinicians ‘diagnose’ whether a concussion has occurred given various sets of signs and symptoms on the SCAT3 for change scores from baseline and also for absolute scores—such information could be obtained retrospectively from sports where the SCAT3 is already being used, alongside the clinical diagnosis made in each case.

3. Conduct a principal component analysis to derive unrelated components of concussion signs and symptoms across the various elements of the SCAT3.

4. Use the components thus derived as predictor variables in a discriminant function or logistic regression (the approaches are essentially equivalent) to produce a likelihood of a concussion diagnosis being associated with a given set of signs or symptoms from the SCAT3, with a clinical diagnosis of concussion (as derived in point 2 above) being the dependent variable. If data are available regarding other diagnoses made concurrently with, or alternatively to, a diagnosis of concussion, fuzzy discriminant analysis may be a useful approach.

5. Use an expert group to produce a consensus about what probability of a concussion is required for counting a particular injury as a concussion for the purposes of injury surveillance.

We can expect to see considerable variability in reported rates of concussion within sports until a standard operational definition is developed and implemented. The current situation with respect to the lack of a generally accepted operational definition of concussion is unsatisfactory for a number of groups. Sports participants (or in the case of children, their parents or guardians), administrators, risk managers, the media and the public are struggling to understand the real size of the concussion issue in sports, in part because they receive conflicting reports about the incidence and severity of the injury from different studies.

As well as helping to provide clarity about the actual size of the concussion issue by sport, a standard operational definition will facilitate better data quality for the purposes of studies intended to follow participants over extended periods. Such prospective follow-up studies can be used to examine the relationships between sports concussion and long-term health outcomes. We would like the next iteration of the Zürich Guidelines to seek to provide clinicians and researchers with an operational concussion definition that allows us to undertake comparative injury surveillance on concussion within and across sports.