Injury details can be obtained from a variety of sources, including self-reports, clinic records, registers, and so on. However, the accuracy of such information can vary across sources because of differences in data collection methodologies and inherent biases. When using data from these sources—particularly when drawing conclusions about the incidence of injury—it is important to consider the validity of the information.¹

Although hospital records only represent a small proportion of all cases, they are a useful source of sports injury data.² They can identify severe and/or acute injuries, but injury data from other health practitioners (for example, general practitioners, sports medicine clinics, physiotherapists, and so on) are needed if comprehensive injury statistics are to be collected for the general community.³ Injuries receiving treatment from health practitioners represent injuries of a moderate to severe nature and only a small proportion of minor injuries will present to them.

The only source that can provide information on all injuries, regardless of severity, is the injured person themself. Many studies rely on self-reported data gathered either by interview or self-administered questionnaires. Self-reported data are often sought when more objective information is not available, or medical records are difficult to access. However, the validity of self-report sports injury data has largely not been established.

This study establishes the validity of a four week self-reported sports injury recall for determining the severity and nature of injury, and the body part and level of treatment sought. A sample of self-reported data collected as part of a larger study, the Western Australian Sports Injury Study (WASIS),^4,5 was validated against data collected from external sources, including hospital records.

METHODS

The WASIS cohort study was conducted during 1997–98, over the five month winter sports seasons (May to September), and involved participants from a random sample of community level Australian football (n = 547), netball (n = 379), field hockey (n = 393), and basketball (n = 193) sporting clubs from the Perth metropolitan area.^4,5 The mean age of the participants (n = 1512) was 23 years (range 15–56 years), with 54% (n = 814) of them being male. Participants were followed up via telephone interview every four weeks to collect incidence and severity details for all injuries sustained during the previous four weeks. Self-reported data included the type of injury, body part involved, and treatment received.

Self-reported injury data were classified into three levels of injury severity:

• severe—required hospitalisation;

• moderate—received other healthcare attention;

• minor—self-treated, or treated by a sports trainer, coach, parent, or similar.

To validate the self-reported injury data, a 10% random sample of injured cases from the 1998 season was selected. A random sample of participants was selected each month based on self-reported injury severity and the proportion of injuries sustained in each sport. Interviewers obtained the following information from injured participants: name, contact number, name of person/place who provided treatment/advice, and date of treatment. Formal consent to access their injury details from the person/place of treatment/advice was also obtained.

The WASIS study also involved a nested case control study of knee injury and those participants who sustained an injury completed an additional questionnaire providing details specific to the knee. Fourteen participants who sustained a knee injury during the 1997 season, and who had specified the treating health professional/hospital, were also included in the study.

A number of procedures were adopted to validate the self-reported injuries against an external source. Data for the moderate to severe injuries were obtained from the participant’s medical/patient record from a hospital or health professional. Hospital data collection required approval from the specific Hospital ethics committees and when consent forms were received for hospital treated injured participants, the Medical Records Department was contacted.

Data for moderate severity injuries were obtained from health professionals by mail. The health professional was asked to supply their name, practice address, number of consultations for treatment of this injury, injury description, and treatment given. As there was no specific external source for the validation of minor injuries, these participants nominated someone (for example, a coach, sports trainer, parent) who was contacted by phone to verify their injury. Information collected included a description of the injury, any advice/treatment given, and relationship to the participant.

The WASIS injury severity classification was applied to the external source data to enable comparisons with the self-report data. This classification was then compared against a “gold standard”, the Injury Severity Score (ISS).⁶ The ISS was calculated by taking the sum of the squares of the highest Abbreviated Injury Scale score and the scoring was undertaken by one of the authors (GV) and then independently verified. The ISS was used because of its design as “a method of numerically describing the overall severity of injury” and was categorised as: 1–3 = minor, 4–8 = moderate, and ⩾9 = severe.⁷ Sensitivity, specificity, and positive predictive values were calculated to compare the WASIS injury severity classification and the ISS categories. All external records were reviewed by two of the authors (GV and MS) and classified according to the ISS classification. The external record, predominantly the clinician’s treatment record, had detail to accurately allocate an abbreviated injury scale, and therefore an ISS.

The kappa statistic⁸ was used to evaluate the agreement between the self-reported data and the external source taking into account agreement due to chance. Excel software was used to calculate kappa so that missing values could be taken into account.⁸ Published categories to assess the level of agreement were used and these ranged from no agreement, which is any value less than 0, to complete agreement, κ = 1.⁹ The null standard error (SE₀) was used to test for a significant difference from chance agreement where κ is significant at 0.05 level for a Z>1.96 or 0.01 level for a Z>2.58.⁸ The standard error (SE) was used to determine the confidence intervals.⁸ Kappa can become unstable when the distribution of the data is skewed, particularly when dealing with rare events, and the results can provide severe underestimates of agreement.⁸

RESULTS

Of the 140 selected participants, 46 were not contactable after three attempts (n = 23, call not answered; n = 20, not at home; n = 3, had left address). Of the 94 participants successfully contacted, 76 people (81%) agreed to have their data validated but 13 did not return a consent form. An additional 14 cases were included from the nested case control study. This gave a final sample of 77 cases for validation.

After contacting the various external sources, 65 participants comprised the study sample because some data forms were not returned by the external source (n = 7) or the external source could not be contacted (n = 5). The final validation sample consisted of 66 data points because one health professional gave details for two separate injury events.

Most cases sought treatment/advice from a physiotherapist or general practitioner (table 1). In one instance, the participant incorrectly reported their practitioner’s profession. This was because they had attended a joint physiotherapy/podiatry practice. The number of participants attending a hospital for treatment was the same in both data sources. The level of agreement between the two data sources was good (κ = 0.76, p<0.001).

From the self-report data, 51% of participants reported having a sprain/strain, 18% bruising, 10% fractures, and 3% both cuts and dislocations. Data from the external sources showed that 57% of participants had sustained a sprain/strain; 15% bruises, 4% both fractures and cuts. In addition, health professionals reported that 7% sustained an overuse injury, which may have been initially reported as a sprain/strain. An acceptable level of agreement⁸ was achieved between the two sources (κ = 0.42, p<0.001).

Table 2 shows that over a third of the injuries were to the knee in both data sources. Other commonly injured body parts were: ankle (9% in both sources), thigh (7% WASIS, 9% external), lower leg (7% in both sources), and foot (1% WASIS, 3% external). There was good agreement between the two sources with respect to body part (κ = 0.78, p<0.001).

Comparisons of the WASIS and ISS injury severity classifications are given in table 3. Although the sensitivity was 0.08, the minor injuries that were classified were all correctly identified, with a positive predictive value of 1. Therefore, when a participant reported a minor injury, we could be certain it really was a minor injury.

Participants correctly identified as having sustained moderate injuries based on the ISS (sensitivity = 0.87). Unfortunately, a high proportion of non-moderate cases (according to the ISS), were not identified as such by the participants (specificity = 0.19). Finally, the participants identified eight cases as being severe, whereas the ISS only identified one case (sensitivity = 1, specificity = 0.89).

DISCUSSION

The ability of sports injury studies to provide good incidence estimates depends on accurate injury data. One of the most commonly used methods of collecting injury data is through self-report, but the validity of such information is largely unknown. Often, self-report data are collected over a period of time and consequently is associated with recall bias.

In the WASIS, injury data were self-reported prospectively during monthly surveys, requiring a four week recall period. This period of recall is unlikely to be as inaccurate as other studies that have used a 12 month recall but it is still essential that the information accurately reflects the sustained injury as it forms the basis of population based incidence rates.

The self-reported injury details were validated against hospital and other health professional records, as well as with third parties, for minor injuries. Overall, we have demonstrated moderate to good levels of agreement between self-report and more objective data such as medical or health professional records. In particular, the study has indicated that the self-report of specific sports injury details such as the body part injured and the level of injury treatment sought, is valid. Almost perfect agreement (rather than the reported κ = 0.78) would have been expected between self-report and the medical/health professional records with respect to the body part injured. This less than perfect agreement can be attributed, in part, to participants using less specific terminology than the professionals when reporting body part—for example, reporting the ankle when it was the foot.

Of particular interest was the categorisation of injury severity. This is important because prevention strategies and resourcing need to be appropriately targeted at the most severe injuries. Our validation of the injury severity classification found the level of agreement to be low, albeit adequate for the treatment-person classification.

In conclusion, our study demonstrates that a four week self-reported recall of sports injury is a valid means of obtaining population level information about the nature of the injury, the body part injured, and the treatment sought. However, the reporting of injury severity is not valid. This latter finding is due, in part, to the fact that the ISS is a scoring system used extensively to reflect “a threat to life” or severe injury, and is therefore not an appropriate “gold standard” upon which to validate self-reported sports injuries; injuries that are predominantly of minor to moderate severity.