The classification of diagnoses is an important component of epidemiological research for analysis of data and for comparison between studies. A number of sports injury classification systems have been proposed, including the International Classification of Diseases Version 10 (ICD-10),1 Orchard Sports Injury Classification System (OSICS),2 Sport Medicine Diagnostic Coding System (SMDCS),3 and National Athletic Injury/Illness Reporting System (NAIRS).4

OSICS-8

The OSICS codes were developed in 1992 specifically for sports medicine diagnoses5 and have undergone a number of revisions. OSICS has been used in various studies of different sports6^–14 and is currently used by the Rugby Football Union (RFU) and English Cricket Board (ECB) for ongoing surveillance of elite professionals in the United Kingdom. It has grown in popularity as sport-specific versions have become available and it is free to use. Using a three-character classification system, Version 8 is concise at just seven pages long and contains sports-specific modifications for cricket and Australian Rules football. The level of inter-rater reliability of OSICS-8 was found to be below acceptable limits at 57.2%, and it has been criticised for a poor hierarchical system, lack of “other” or “not otherwise specified” codes, no redirection guides and poor covering of paediatric conditions.15

OSICS-10

Version 10 of the Orchard Sports Injury Classification System was recently developed, aiming to:5

1. Make it easier and better able to cater for all sports injury diagnoses encountered in a sports medicine setting

2. Provide greater depth of coding for the benefit of those looking for greater diagnostic accuracy

3. Easily collapse down into parent classifications

4. Improve inter-user agreement

This four-letter classification system is organised in four tiers for progressive diagnostic specificity. Compared with previous versions, it has been reorganised to introduce standard pathology sections within each anatomical region, location-not-specified codes, not-otherwise-mentioned codes, and inclusion and exclusion criteria to reduce ambiguity. Authors of OSICS-10 suggest that advantages of the new system include the fact that diagnoses can be specifically classified but are also expandable to their broader parent classifications (anatomical region/general pathology) for more general analysis, and easy mapping back to OSICS-8.16

It is suggested that good classification systems should demonstrate a range of properties including completeness, clarity, mapping, hierarchies and definitions.17 No evaluation of OSICS-10 against any of these concepts has taken place to date. This study assessed OSICS-8 and OSICS-10 coding with injury diagnoses from football (soccer), cricket and rugby union. The study aimed to address whether the recent developments have reduced the problems associated with OSICS-8 and increased the completeness of the system’s coding injuries from three different sports, and to quantify the inter-rater reliability of OSICS-10.

METHODS

A 2 year prospective cohort study of three professional sports teams was undertaken. The first team squads of an English League 1 football club (n = 29) and premiership rugby union club (n = 45), and the first and second XI of a first-class county cricket club (n = 25), were included in the study. Players and team physiotherapists consented to participate and the study was approved by the University of Nottingham Ethics Committee.

Injuries sustained within the observation period, subject to the definition of injury described below, were recorded on a purpose-designed Injury Reporting Form by the team physiotherapist following diagnosis. The form recorded information including diagnosis, mechanism, circumstances and consequences of injury. The physiotherapists were instructed to complete the diagnosis section in the same way as they would usually complete their case notes.

The 335 recorded injuries (football n = 83, rugby n = 219, cricket n = 33), provided a range of pathologies to various anatomical regions against which the classification systems were evaluated. Although lower limb injuries dominated the injury surveillance findings, all body regions were affected (4.5% to head and face; 3.0% to neck; 14.9% to trunk, thoracic and lumbar spine; 10.7% to upper limb; 66.9% to lower limb).

Definition of injury

The issue of injury definition and the effects of restricted versus broad definitions have been discussed by a number of authors.18^–22 Within the present study, injury was defined as: “Any pain sustained during a match, practice or training session, which caused the player to seek medical advice, which prevents the player from playing from the time of injury or from the end of the match or training session in which the injury occurred, resulting in missing the next training session or match”.

Coding with OSICS-8 and OSICS-10

Coding of the written diagnoses provided by the diagnosing physiotherapists was conducted by the lead author (secondary coding), who is experienced in classification using OSICS. Coding was repeated for a 10% sample of the diagnoses randomly selected from the database to assess intra-tester reliability. Level of agreement between initial and repeat coding was established at 98% for OSICS-8 and 93% for OSICS-10.

In the case of OSICS-8, the most appropriate available code was utilised. A noncodeable diagnosis was defined as “where no code encompasses all details of the diagnosis due to inadequate description of location and/or anatomy and/or pathology”. Noncodeable injuries were further classified as either best-fit codes, where a code “partially fulfils the details of the diagnosis but is not ideal”, or absolutely noncodeable, where there was “no code resembling the diagnosis”. For OSICS-10, the highest tier of coding was used where possible. Where no appropriate OSICS-10 tier 4 codes were available, the relevant tier 3 code was used and so on. In the event of a surveillance injury having multiple diagnoses, each was coded separately. For example, a knee injury diagnosed as anterior cruciate ligament (ACL) rupture, medial collateral ligament (MCL) rupture and medial meniscus lesion would contain three separate codes.

Inter-rater reliability testing of OSICS-10

Eight coders, comprising the lead author, orthopaedic surgeons, physiotherapists and general practitioners with a specialist interest in sports medicine, classified a list of 20 diagnoses with OSICS-10. These diagnoses were selected independently by WJR from the database to represent a range of injuries covering various pathologies and anatomical regions. The selection contained five limb fractures, five joint injuries, five injuries to the axial skeleton and five injuries to the soft tissues of the limbs. Participants were provided with standardised information about the coding system and a copy of the codes, and were required to classify the diagnoses using the highest possible tier of diagnostic specificity where possible. No time limit was set on the task.

Analysis

Analysis of OSICS-8 and OSICS-10 comprised:

1. Frequency of code-diagnosis agreement for OSICS-8 and OSICS-10

2. Analysis of nature, location and severity of OSICS-8 noncodeable, best-fit and absolutely noncodeable diagnoses

3. Analysis of coding tier used (1–4) for OSICS-10. Nature, severity and reason for coding at OSICS-10 tier 2 were examined, with tier 2 codes categorised as either:

4. • Lack of detail in the diagnosis (e.g. ankle sprain)

   • Lack of detail in the code (e.g. tensor fascia lata strain coded as Hip and Groin Muscle Strain/Tear)

   • Where the specificity of diagnosis and code were appropriate but no more specific code was available (e.g. Whiplash)

5. Inter-rater reliability assessment comprised overall agreement (OSICS-10 tier 4) and agreement at tiers 1–3, quantified using Fleiss’ Kappa (k)23

RESULTS

OSICS-8

A total of 348 codes were generated with OSICS-8 from the 335 injuries that occurred. Forty-four instances of noncodeable diagnoses occurred using OSICS-8, representing 13% of all diagnoses coded. Four of the noncodeable diagnoses occurred more than once. Within each sport, the greatest percentage of noncodeable diagnoses was found in cricket (23.5%) compared with football (17.4%) and rugby (9.2%). Further examination of noncodeable diagnoses revealed that 75% (representing 9.5% of all codes) could be assigned a best-fit code and the remaining 25% (representing 3.2% of all codes) were absolutely noncodeable.

Almost half (45.5%, n = 20) of all noncodeable diagnoses were contusions (see table 1). Contusions represented 10.4% of all reported injuries. 65.9% of all noncodeable diagnoses affected the lower limb. Noncodeable injuries resulted in missed time ranging from 3 to 143 days (mean of 22.2 days). Noncodeable contusions could most commonly be assigned a best-fit code.

OSICS-10

A total of 352 codes were generated with OSICS-10 from the 335 injuries that occurred. The marginal increase compared with OSICS-8 was due to the more specific nature of some available codes.

There were no diagnoses which were noncodeable using OSICS-10. The majority of codes (80.1%) used were at OSICS-10 tiers 3 and 4 (Tier 1: 0%, Tier 2: 19.9%, Tier 3: 47.4%, Tier 4: 32.7%).

OSICS-10 tier 2 codes resulted in a mean of 19.3 days of missed training/matches (range 2 to 221 days). Of these, half of the codes selected were due to lack of detail in the available codes (see table 2).

Inter-rater reliability

The magnitude of agreement between raters decreased with increasing specificity of diagnostic codes (see table 3). Overall, a moderate level of inter-rater agreement was found (k = 0.56).

DISCUSSION

Injury classification systems should encompass the full range of sports medicine diagnoses and should be able to be applied reliably. OSICS-8 has been found to have a lower than expected level of reliability 15; however, the extent to which codes fit the diagnoses has not been fully assessed. This is the first study to evaluate OSICS-10 with a range of injury diagnoses collected from three different sports.

The findings of the present study suggest that OSICS-10 has managed to “cater for all sports injury diagnoses encountered in a sports medicine setting”,5 as a code was assigned to all of the diagnoses generated by a 2 year surveillance study. This has addressed a major limitation of OSICS-8, in which a proportion of these diagnoses were noncodeable. OSICS-10 was able to cater for all diagnoses because of the hierarchical organisation of progressive diagnostic specificity. That the majority of diagnoses were coded at OSICS-10 tier 3 or 4 indicates that OSICS-10 caters well for most specific diagnoses and has addressed the aim of “providing greater depth of coding for the benefit of those looking for greater diagnostic accuracy”.5

As would be expected, the magnitude of agreement between coders was greatest at the lowest level of diagnostic specificity (anatomical region, e.g. shoulder). An overall agreement level of moderate (k = 0.56) indicates that OSICS-10 can be applied fairly reliably when used by a number of different raters, which is important for those using surveillance databases. Differences in selection of codes may be due to choices by raters based on anatomy, location or pathology, or may be due to differences in interpretation of written diagnoses. The degree of training required by individuals to improve the level of agreement is as yet unknown and it may be that further standardisation or instruction is required to improve this value.

Hierarchy, the logical linking of general and specific terms,17 is a property of good classification systems. An advantage of the hierarchical system introduced to OSICS-10 is that it enables choice in level of diagnostic detail for analysis, that is, to consider very specific diagnoses or to gain an overview by using lower OSICS-10 tier codes. This is an important feature of the system, and addresses the aim to “easily collapse down into parent classifications”,5 which has useful implications for clinicians and researchers alike. Rae and Orchard (2007) suggest one use of this to be the creation of a database from which a case series of a particular injury diagnosis can be extracted for further study.16 A number of OSICS-10 tier 2 codes were utilised in the present study, for reasons including where the diagnosis lacked detail, where the code lacked detail or where the diagnostic and coding detail were appropriate but no more specific code was available. Coding at OSICS-10 tier 2 would not prohibit the creation of a useful injury database, as it classifies at a general level. However, if the intention is to follow up a case series with a specific diagnosis that is not catered for, the classification system would hinder the process and reveal diagnoses from a more general grouping only. This was shown in this study to be problematic for strains, joint effusions and contusions. Half of the OSICS-10 tier 2 codes used were selected due to lack of detail in the available codes; the need for use of general codes when more diagnostic detail is provided suggests that the property of completeness, where codes cover the entire clinical spectrum in sufficient depth and level of detail,17 may not have been satisfied by the revisions made. However, one-third of classifications at OSICS-10 tier 2 occurred due to a lack of detail in the diagnosis provided. To address this, clinicians should be encouraged to provide as much detail as possible in their diagnoses.

For OSICS-8, no specific under-represented pathologies have previously been identified, although paediatric conditions are acknowledged as being under-represented or omitted.15 A major flaw in OSICS-8 revealed in the present study is the lack of contusion diagnoses that could be successfully coded. OSICS-8 offers a number of haematoma codes, but few contusion/bruising codes. These are clinically distinct injuries, contusion defined as bleeding in soft tissue resulting from a direct blow, and haematoma defined as swelling caused by the accumulation of clotted blood in the tissues. When assigned a best-fit code, most contusion diagnoses could be catered for under haematoma codes; although these codes are not ideal, they do enable classification to take place. However, problems arising from this practice include implications about the accuracy of a database created using these codes, and the reliability of best-fit coding (particularly when secondary coding). Despite the restrictions on the injury definition, 10.4% of all injuries reported during the 2 year surveillance were contusions, suggesting that they should be considered as injuries that may be severe enough to warrant inclusion in an injury coding system.

Further issues associated with OSICS-8 include the fact that lower limb injuries appear to be poorly catered for, as 65.9% of the noncodeable diagnoses affected the lower limb. This is a similar percentage to the amount of lower limb injuries reported in total. A larger-scale study, with equal distribution of injuries to the gross anatomical regions, would be required to assess whether the prevalence of noncodeable lower limb diagnoses is a reflection of an inadequate coding system or the large numbers of lower limb injuries that featured in the database against which the system was assessed.

The limitations in OSICS-8 revealed by the present study are primarily due to its concise nature; it lacks the extensive choice of codes presented in OSICS-10. Being more concise may make it easier to use; however, this was not assessed within the study. It is still a preferable system to use over the more expansive ICD-10 as it is sport-specific.15

LIMITATIONS

It should be borne in mind that secondary coding took place within this study. The effects of secondary coding are not known; however, it would be reasonable to suggest that it is not as accurate as primary coding, due to greater knowledge of the clinical presentation of injury with the latter. Secondary coding may particularly adversely affect reliability when assigning a best-fit code, and its accuracy relies heavily upon sufficient diagnostic detail. However, in the present study, a good level of intra-tester reliability was established (>90%). This removed potential inter-rater agreement issues between the diagnosing physiotherapists. Further research could examine the effects of primary and secondary coding, which may influence methods by which large-scale injury surveillance is administered in future.

The relatively small sample size and the cases used may limit the study findings as they do not represent all possible diagnoses from all sports. Although featuring a wide range of pathologies to various anatomical regions, the dataset used was dominated by traumatic soft tissue injuries to the lower limb. How extensively the classification system was evaluated in terms of depth and breadth is therefore unknown.

The selection of diagnoses for inter-rater reliability testing took place to ensure that a range of pathologies to all of the gross anatomical regions was represented. This selection may have introduced biases relating to the type of injury or level of diagnostic detail reported, which might not have been present in a randomly selected sample.

As all diagnoses used within the study were provided by physiotherapists, the extent to which the findings apply to other sports medicine professionals is unknown. There may be differences in use of terminology, particularly with surgical terms, and access to diagnostic imaging equipment that may inform or enhance diagnostic detail.

CONCLUSIONS

The cases of noncodeable diagnoses with OSICS-8 and low-specificity tier codes with OSICS-10 indicate that, in addition to a diagnostic code, injury surveillance systems should capture a written diagnosis that contains as much detail as possible. In the United Kingdom, the English Cricket Board conducts surveillance of professional players, requiring a coded diagnosis (OSICS-8) but no written diagnosis. Individuals may deal with noncodeable diagnoses differently, adding to the questionable inter-rater reliability that this system already has. Requiring a detailed written diagnosis will provide clarification and minimise inappropriate code selection as an extraneous variable.

By naming the new system OSICS-10, the authors have reserved the ability to develop OSICS-9 via minor revisions to OSICS-8, retaining the three-digit code.16 Such a development may be valuable to researchers or practitioners requiring a concise system, for example, when conducting small-scale non-computerised research. Conversely, it could be argued that, if a version has been shown to be inferior to the latest modification, there is limited value in creating an interim version. Based on the present study, the following recommendations would be made to improve OSICS-8:

• Inclusion of “Contusion/bruising” codes, either as separate codes or as an addition to the existing “Haematoma” code

• Organisation into logical repeating sections (similar to OSICS-10) to ease navigation of the system

• Standardisation of degree of diagnostic detail to reduce the incidence of decision-making based on relative importance of anatomy, location or pathology

• Inclusion of “Other” codes in each anatomical section, to enable all possible diagnoses to have a code assigned

OSICS-10 has dealt successfully with a number of the problems associated with OSICS-8 and is a preferable system to use when classifying sports medicine diagnoses. However, future research, using larger numbers of injuries from more sports, should assess whether revision should occur to deal with the specific codes relating to the diagnoses of contusion, strain and effusion.