You are here

  • Home
  • Archive
  • Volume 48, Issue 9
  • Online registration of monthly sports participation after anterior cruciate ligament injury: a reliability and validity study

Article Text

Article menu

Download PDFPDF

Original article
Online registration of monthly sports participation after anterior cruciate ligament injury: a reliability and validity study
  1. Hege Grindem1,
  2. Ingrid Eitzen2,
  3. Lynn Snyder-Mackler3,
  4. May Arna Risberg1
  1. 1Norwegian Research Center for Active Rehabilitation (NAR), Department of Sports Medicine, Norwegian School of Sport Sciences, Oslo, Norway
  2. 2Norwegian Research Center for Active Rehabilitation (NAR), Department of Orthopaedics, Oslo University Hospital, Oslo, Norway
  3. 3Department of Physical Therapy, College of Health Sciences, University of Delaware, Newark, Delaware, USA
  1. Correspondence to Hege Grindem, NIMI/NAR, Pb. 3843 Ullevål Stadion, 0855 Oslo, Norway, hege.grindem{at}nih.no

Abstract

Background The current methods measuring sports activity after anterior cruciate ligament (ACL) injury are commonly restricted to the most knee-demanding sports, and do not consider participation in multiple sports. We therefore developed an online activity survey to prospectively record the monthly participation in all major sports relevant to our patient-group.

Objective To assess the reliability, content validity and concurrent validity of the survey and to evaluate if it provided more complete data on sports participation than a routine activity questionnaire.

Methods 145 consecutively included ACL-injured patients were eligible for the reliability study. The retest of the online activity survey was performed 2 days after the test response had been recorded. A subsample of 88 ACL-reconstructed patients was included in the validity study. The ACL-reconstructed patients completed the online activity survey from the first to the 12th postoperative month, and a routine activity questionnaire 6 and 12 months postoperatively.

Results The online activity survey was highly reliable (κ ranging from 0.81 to 1). It contained all the common sports reported on the routine activity questionnaire. There was a substantial agreement between the two methods on return to preinjury main sport (κ=0.71 and 0.74 at 6 and 12 months postoperatively). The online activity survey revealed that a significantly higher number of patients reported to participate in running, cycling and strength training, and patients reported to participate in a greater number of sports.

Conclusions The online activity survey is a highly reliable way of recording detailed changes in sports participation after ACL injury. The findings of this study support the content and concurrent validity of the survey, and suggest that the online activity survey can provide more complete data on sports participation than a routine activity questionnaire.

  • ACL
  • Athletics
  • Sporting injuries

https://doi.org/10.1136/bjsports-2012-092075

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Introduction

    After an anterior cruciate ligament (ACL) injury, a major concern for athletically active patients is when, if at all, they will be able to return to sports.1 Return to sports has therefore become a commonly used measure of treatment success.2 A multitude of different methods of recording activity level after ACL injury has been reported in the literature, including project-specific questionnaires, patient interviews and activity rating scales.3–5 The reported outcome is frequently based on whether or not the patient resumes one specific sport,6 or on a score reflecting only the most demanding sport patients participate in postinjury (eg, the Tegner activity scale).7

    The current methods only provide a limited view of the full complexity of sports participation. Although return to sports is considered indicative of a successful outcome, it does not guarantee normal knee function8 or sustained sports participation.9 Further, it may expose the athlete to considerable risk of additional injuries or reinjuries.1 ,10 In studies reporting reinjuries, sports participation between index and secondary injury can provide valuable data on patient-time at risk,11 although this is rarely reported in the literature. Additionally, the ACL-injured patients often participate in multiple sports both prior to and following ACL injury. While it has been shown that many patients do not return to their preinjury activity level,2 less attention has been paid to the extent to which the patients return to, or take up, alternative sports activities. Hence, the assessments of sports participation after ACL injury should include not only if the patients return to sport or not, but also how the full sports activity profile changes. Finally, as sports participation is highly dynamic, optimal quantification requires repeated reports during the observation time. Frequent reporting also limits the risk of inaccurate recall by reducing the retrospective period.

    In order to address these concerns, we developed an online activity survey to prospectively record the monthly participation in all major sports activities relevant to ACL-injured patients treated at our clinic. The aims of this study were (1) to assess the test-retest reliability of the online activity survey in a sample of non-operatively and operatively treated ACL-injured patients; (2) to evaluate if the content of the online activity survey provided a valid representation of sports participation at 6 and 12 months after ACL reconstruction; (3) to assess the concurrent validity of the questionnaire with respect to return to preinjury main sport at 6 and 12 months after ACL reconstruction; and (4) to evaluate if the online activity survey provided more complete data on sports participation compared with a routine activity questionnaire.

    Methods

    Subjects

    All the patients were enrolled in a prospective cohort study conducted at the Norwegian Sports Medicine Clinic (Nimi) between 2007 and 2012.12 In order to be included in the study, the patients had to have a sustained unilateral ACL rupture within the previous 3 months. Diagnosis was confirmed with MRI and a side-to-side KT-1000 difference of ≥3 mm.13 Other inclusion criteria included age 13–60 years and preinjury participation in level I or level II sports (table 1)14 ≥ twice a week. Patients were excluded if they had bilateral injuries, previous injuries to either knee or if the MRI showed other grade III ligamentous injury, fracture or full-thickness articular cartilage damage. Patients with meniscal injuries were excluded only if they had symptoms during plyometric activities that were not resolved within 3 months from injury. Finally, patients who were not able to understand written and spoken Norwegian were also excluded.

    View this table:
    Table 1

    Sports recorded in the monthly online activity survey classified according to activity level14

    All patients signed an informed consent prior to inclusion. The study was carried out in accordance with the directives given in the Declaration of Helsinki, and was approved by the Regional Ethical Committee for South-Eastern Norway.

    Data collection

    Online activity survey

    An online registration was carried out with an online survey tool (Questback V. 9.6, Questback AS, Oslo, Norway). Every month, each patient received a standardised e-mail that contained a unique link to the online activity survey. A reminder was automatically sent to those patients who had not responded after 1 week.

    The online activity survey consisted of the question: “Which of the following sports have you participated in during the last 4 weeks?”, followed by the sports listed in table 1. The patients were then asked: “How many times per week have you, on average, participated in these sports?” The listed sports were intended to cover all major sports the ACL-injured patients in our geographical region participated in. The selection of sports was based on previously published studies on ACL-injured patients, discussions with physiotherapists with extensive experience in ACL-rehabilitation and the results from a previous cohort study on ACL-injured patients performed at our institution.15 In this latter study, the patients completed a questionnaire in which they were asked to list all sports they participated in.

    Routine activity questionnaire

    All the patients attended the follow-up visits at our clinic 6 and 12 months postoperatively. Following an established test battery,12 they completed a routine activity questionnaire without supervision in the clinic waiting room. On the routine activity questionnaire, the patients listed the types of sports or exercise they currently participated in, and how many times per week they participated in sports or exercise.

    Reliability (aim 1)

    In February 2012, all patients (n=145) were invited to participate in the test–retest of the online activity survey (figure 1). Based on our power calculations,16 it was estimated that 50 subjects would be needed to detect a κ of ≥0.4 at 0.80 power. The retest was sent 2 days after the test response was recorded. To avoid measuring true changes in sports participation, we decided a priori to exclude patients who responded to the retest ≥5 days after their test response.

    Figure 1
    Figure 1

    Flowchart of patient participation in the reliability and validity assessment of the online activity survey.

    Validity (aims 2–4)

    All patients who had undergone ACL reconstruction between 2007 and 2010 (n=88, figure 1) were included in the comparisons between methods. Non-operatively treated patients were excluded from aims 2 to 4 because a merged sample of non-operatively and operatively treated patients would not allow for clear clinical interpretation of the data. Patients who had undergone ACL-reconstruction after 2010 were excluded due to incomplete follow-up data.

    To evaluate if the content of the online activity survey included all common sports, we used the responses from the routine activity questionnaire at 6 and 12 months postoperatively. “Common sports” was operationally defined as sports that ≥10% of patients participated in. We examined if the patients reported sports that were not part of the online activity survey, and how many patients participated in those sports. We expected that the routine activity questionnaire would not disclose any sports, in which ≥10% of the patients reported to participate, other than those included in the online activity survey.

    To evaluate the concurrent validity of the online activity survey, we examined the agreement between return to preinjury main sports recorded with the online activity survey and the routine activity questionnaire at 6 and 12 months postoperatively. We expected substantial agreement (κ>0.6)17 between the online activity survey and the routine activity questionnaire.

    To evaluate if the online activity survey provided more complete data on sports participation than our routine method, we compared the number of sports patients participated in 12 months postoperatively, the number of patients who participated in each sport 12 months postoperatively, and the number of patients who participated in level I, II and III sports at 6 and 12 months postoperatively. The online activity survey was designed with a list of specific sports in order to avoid bias that may come from patients having to decide which sports are relevant to report. Therefore, we expected that patients would report participation in a higher number of sports, and that more patients would report to participate in some sports, compared with the routine activity questionnaire.

    Data management and statistics

    Participation in level I, level II and level III activities was defined as the reported participation in at least one sport of the respective level (table 1). The patients were classified as having returned to preinjury main sport if they reported participation in their preinjury main sport, regardless of their level of participation.

    Test–retest reliability was quantified using κ with 95% CIs. Linearly weighted κ was used for the frequency of sports participation. Following recent guidelines for reporting reliability and agreement studies,18 the proportion of absolute agreement was reported. Agreement between methods in assessing return to preinjury main sports was quantified using κ. Paired t test and McNemar's test were used for between-methods comparisons of the number of sports patients participated in, the number of patients who participated in specific sports and the number of patients who participated in sports of different levels. All statistical analyses were conducted with SPSS V. 17 (SPSS Inc, Chicago, Illinois, USA).

    Results

    Reliability

    Of the 145 eligible patients, 101 (69%) responded to the test questionnaire, and 90 (62%) also responded to the retest questionnaire 2–4 days later (figure 1). Sixty-seven patients (74.4%) had undergone reconstructive surgery and 23 patients (25.6%) were non-operatively treated (table 2). The most frequent preinjury main sports were football (28.9%) and handball (25.6%). Both the test and the retest responses showed that most patients participated in sports 2–3 times/week, with the most frequent sports activities being strength training, running, cross-country skiing and cycling (table 3). Data collection was carried out in February, which is reflected by the high number of patients participating in winter sports. κ ranged from 0.81 to 1, and the proportion of agreement ranged from 0.91 to 1.

    View this table:
    Table 2

    Characteristics of the reliability sample (n=90)

    View this table:
    Table 3

    Test-retest reliability of the online activity questionnaire (n=90)

    Validity

    Of 88 ACL-reconstructed patients, 74 (84.1%) completed the monthly online activity survey and attended follow-ups 6 and 12 months postoperatively (figure 1). This group included 39 (52.7%) women and 35 (47.3%) men with a mean age of 24.5 (6.9) years. The most frequent preinjury main sports were football (39.2%) and handball (28.4%). The sixth online activity survey was completed 6.3 (SD: 0.4) months postoperatively, and the 6 month routine questionnaire was completed 6.1 (0.3) months postoperatively. The 12th online activity survey and the 12-month routine questionnaire were completed 12.4 (0.5) and 12.1 (0.5) months postoperatively, respectively.

    Content validity of the online activity survey

    Eleven sports activities that were not included in the online activity survey were reported on the routine activity questionnaire (table 4), either at 6 or at 12 months postoperatively. The routine activity questionnaire did not disclose any additional sports in which ≥10% of the patients participated.

    View this table:
    Table 4

    Activities reported on the routine activity questionnaire that were not part of the online activity survey (n=74)

    Concurrent validity of the online activity survey

    Six months postoperatively, the return to preinjury main sports rates were 21.9% based on the online activity survey, and 23.4% based on the routine activity questionnaire (figure 2). At 12 months postoperatively, the return to preinjury main sport rates were 59.7% and 55.6% for the online activity survey and the routine activity questionnaire, respectively. The κ between the two methods was 0.71 (95% CI 0.51 to 0.91) and 0.74 (0.59 to 0.90) for 6 months and 12 months postoperatively, respectively.

    Figure 2
    Figure 2

    Return to preinjury main sport from 1 month to 1 year postoperatively, as measured with the online activity survey and the routine activity questionnaire (n=74).

    Completeness of the data on sports participation

    Twelve months postoperatively, the online activity survey showed participation in a significantly greater number of sports activities (3.7 (SD 2.3)) compared with the routine activity questionnaire (2.3 (1.2), p<0.001). The analysis of specific sports at 12 months postoperatively (figure 3) revealed that a significantly higher number of patients participated in running (online activity survey: 63.5%, routine activity questionnaire: 36.5%, p<0.001), cycling (online activity survey: 67.6%, routine activity questionnaire: 39.2%, p<0.001) and strength training (online activity survey: 78.4%, routine activity questionnaire: 66.2%, p=0.049).

    Figure 3
    Figure 3

    Percentage of patients participating in specific sports activities 12 months postoperatively, as measured with the online activity survey and the routine activity questionnaire (n=74). *p<0.05.

    Six months postoperatively, the online activity survey showed that a significantly higher number of patients participated in level II sports (online activity survey: 18.8%, routine activity questionnaire: 6.8%, p=0.040, figure 4). Twelve months postoperatively, the online activity survey showed that a significantly higher number of patients participated in level II (online activity survey: 39.2%, routine activity questionnaire: 27%, p=0.035) and level III sports (online activity survey: 93.2%, routine activity questionnaire: 81.1%, p=0.049). There were no other significant differences between the methods (all p>0.227).

    Figure 4
    Figure 4

    Percentage of patients participating in level I, II and III sports from 1 month to 1 year postoperatively, as measured with the online activity survey and the routine activity questionnaire (n=74).

    Discussion

    The results of this study showed that the online activity survey was highly reliable. It also included all common sport activities in this patient-group, supporting its content validity. Further, there was a substantial agreement between the methods on return to preinjury main sport, supporting the concurrent validity. Finally, the participation in a greater number of sports and a higher number of patients participating in some sports indicated that the online activity survey offers more complete data on sports participation compared with the routine activity questionnaire.

    Sports participation after an ACL injury can be recorded for at least three purposes. Most commonly, the authors report proportions of patients who have returned to sport as a measure of treatment outcome.7 ,15 ,19 ,20 However, as returning to sport entails a high risk of reinjury, several authors have questioned if return to sports should be the main aim of the treatment.1 ,21 ,22 Rerupture and subsequent injuries have thus become increasingly important in evaluating treatment outcome. Although sports participation places the patient at a higher risk of injury,1 ,10 few studies on reinjuries after ACL injuries account for sports exposure,23 ,24 a key factor in sports injury epidemiological research.11 The second purpose of recording sports participation should therefore be to adjust for sports exposure in analyses of reinjuries. Lastly, it is unknown how an ACL injury affects general physical activity levels in the long term. While it is well documented that many patients do not return to sports,2 ,25 less attention has been paid to whether patients become inactive after quitting their preinjury sports or if they take up other forms of physical activity. The online activity survey utilised in this study holds the potential of providing data that may elucidate all three areas.

    All items in the online activity survey showed almost perfect agreement, defined as κ>0.80.17 Furthermore, all items except participation in cycling and dancing/aerobics showed a κ>0.87. The period between test and retest registrations was limited to 2–4 days because participation in some of the recorded sports is highly dynamic. Participation in organised sports, such as football and handball, may be more consistent than participation in unorganised activities. We did not find an overall pattern of differences between organised and unorganised sports. Thus, we believe the results were not largely affected by the changes in true participation rates. A disadvantage of having a short period between the test–retest registrations is the chance that the retest response is influenced by recollection of the test response. Based on the feedback from patients, the online activity survey is very easy to complete and takes no more than 1 min. While the possibility that the results were influenced by patient recall cannot be excluded, the potential for bias is likely smaller with this method compared with methods that require more time and deliberation to complete.

    Regarding the content validity of the online activity survey, the routine activity questionnaire did not disclose any sports activities, in which more than 10% of patients participated, that was not included in the online activity survey. This supports the content validity of the online activity survey, in that it includes all common sports activities in this patient population (table 4). There was also substantial agreement between the two methods on return to preinjury main sport at 6 and 12 months post-ACL reconstruction. This indicates that the online activity survey can be used to determine return to preinjury main sport. The online activity survey also offers a more detailed knowledge on the timing of return to sports (figure 2). How long they will be out of sports is a main concern for the majority of ACL-injured patients,1 and a detailed knowledge on this topic enables clinicians to provide a more accurate timeline for the resumption of sports participation.

    The online activity survey showed participation in a significantly greater number of sports, and also that a higher number of patients reported to participate in low-level sports compared with the routine activity questionnaire. In the online activity survey, patients ticked a box if they had participated in any of the listed activities. In contrast, the routine activity questionnaire contained an open-ended question where patients listed the sports they were participating in. Responses to open-ended questions rely on the assumptions about what constitutes an informative answer, which may lead to under-reporting of sports patients do not come to think of or deem less important.26 Our results are likely explained by patients under-reporting sports that were perceived as less important when filling out the routine activity questionnaire. Thus, the online activity survey seems to offer more complete data on sports participation.

    We acknowledge that there are limitations to this study. First, the absence of established, comparable methods inevitably hampers the ability of demonstrating validity of the online activity survey. In this study, we compared the online activity survey with a routine activity questionnaire where patients listed the sports or exercises they participated in. While the psychometric properties of the routine activity questionnaire are unknown, we are not aware of any instruments with established validity and reliability that measure participation in several specific sports. Activity scales such as the Tegner activity scale,5 Cincinnati3 and MARS4 provide scores based on knee-demanding sports. In contrast, the online activity survey was not designed as a scale, but as an easy-to-use tool that would provide more detailed data on sports participation. Second, the sports recorded were selected to represent all major sports ACL-injured patients at our institution participated in, and do not provide an accurate representation of sports participation in other cultural or geographical settings. Thus, if used in a different setting, the survey should be modified based on what sports the patient-group participates in. To avoid under-reporting, we recommend that the survey includes a list of specific sports. However, adding a free-text choice where patients can report sports that are not included in the survey should be considered to ensure that no major sports are missed. Third, the monthly online activity registration might have led to reporting bias at the 6 and 12 month follow-ups, as patients may have become accustomed to reporting sports listed in the online activity survey. A fourth limitation is the fact that the online activity survey was not intended to measure treatment success, but rather to provide a broader perspective on changes in sports participation after an ACL injury. If return to sport is to be used as a criterion for successful outcome, the survey should be modified to include questions regarding sport performance and reasons for not returning to sport. Finally, sports frequency was not registered for each individual sport. Recording the number of hours the patients spend in different sports could add important data in future studies. However, if further development of this method is performed, the benefit of increasing the level of detail in the registration should be carefully weighed against the risk of adversely affecting response rates.

    Conclusion

    This study showed that the online activity survey was highly reliable and provided a valid representation of sports participation in a sample of ACL-injured patients. There was a substantial agreement between the online activity survey and a routine activity questionnaire on determining return to preinjury main sport; however, the online activity survey provided more complete data on sports participation. While the existing literature is predominantly focused on activity scales and return to preinjury sport, this method provides a broader perspective on changes in sports participation after ACL injury and surgery.

    What are the new findings?

    • The online activity survey utilised in this study can provide more detailed longitudinal data on sports participation after anterior cruciate ligament (ACL) injury than commonly used measures like return to sports and activity rating scales.

    • The use of an online activity survey is a highly reliable way of collecting data on sports participation after ACL injury.

    • In addition to providing a broader perspective on sports participation, the online activity survey can be used to prospectively monitor return to preinjury main sport.

    How might it impact on clinical practice in the near future?

    • This study may lead to a change in methods that are used to monitor sports participation following ACL injuries.

    • The online activity survey can provide clinicians with more detailed information about the changes in patients’ sports participation after injury.

    • By incorporating more detailed data on sports participation after injury, future studies might provide more accurate prognoses for future sports participation, and more accurate estimates of the risk of reinjuries.

    Acknowledgments

    We would like to especially acknowledge physiotherapist Håvard Moksnes for his valuable contributions to the development of the online activity survey. We also thank physiotherapists Annika Storevold, Ida Svege, Espen Selboskar, Karin Rydevik and Marte Lund for their assistance in data collection for this study. We acknowledge the Norwegian Sports Medicine Clinic, Nimi (http://www.nimi.no), for supporting the Norwegian Research Center for Active Rehabilitation (NAR; http://www.active-rehab.no) with rehabilitation facilities and research staff.

    References

    1. ↵
      1. Myklebust G,
      2. Bahr R
      . Return to play guidelines after anterior cruciate ligament surgery. Br J Sports Med 2005;39:127–31.
      OpenUrlFREE Full Text
    2. ↵
      1. Ardern CL,
      2. Webster KE,
      3. Taylor NF,
      4. et al
      . Return to sport following anterior cruciate ligament reconstruction surgery: a systematic review and meta-analysis of the state of play. Br J Sports Med 2011;45:596–606.
      OpenUrlAbstract/FREE Full Text
    3. ↵
      1. Barber-Westin SD,
      2. Noyes FR,
      3. McCloskey JW
      . Rigorous statistical reliability, validity, and responsiveness testing of the Cincinnati knee rating system in 350 subjects with uninjured, injured, or anterior cruciate ligament-reconstructed knees. Am J Sports Med 1999;27:402–16.
      OpenUrlAbstract/FREE Full Text
    4. ↵
      1. Marx RG,
      2. Stump TJ,
      3. Jones EC,
      4. et al
      . Development and evaluation of an activity rating scale for disorders of the knee. Am J Sports Med 2001;29:213–18.
      OpenUrlAbstract/FREE Full Text
    5. ↵
      1. Tegner Y,
      2. Lysholm J
      . Rating systems in the evaluation of knee ligament injuries. Clin Orthop Relat Res 1985;198;43–9.
      OpenUrlPubMed
    6. ↵
      1. Grindem H,
      2. Eitzen I,
      3. Moksnes H,
      4. et al
      . A pair-matched comparison of return to pivoting sports at 1 year in anterior cruciate ligament-injured patients after a nonoperative versus an operative treatment course. Am J Sports Med 2012;40:2509–16.
      OpenUrlAbstract/FREE Full Text
    7. ↵
      1. Frobell RB,
      2. Roos EM,
      3. Roos HP,
      4. et al
      . A randomized trial of treatment for acute anterior cruciate ligament tears. N Engl J Med 2010;363:331–42.
      OpenUrlCrossRefPubMedWeb of Science
    8. ↵
      1. Frobell RB,
      2. Svensson E,
      3. Gothrick M,
      4. et al
      . Self-reported activity level and knee function in amateur football players: the influence of age, gender, history of knee injury and level of competition. Knee Surg Sports Traumatol Arthrosc 2008;16:713–19.
      OpenUrlCrossRefPubMedWeb of Science
    9. ↵
      1. Roos H,
      2. Ornell M,
      3. Gardsell P,
      4. et al
      . Soccer after anterior cruciate ligament injury—an incompatible combination? A national survey of incidence and risk factors and a 7-year follow-up of 310 players. Acta Orthop Scand 1995;66:107–12.
      OpenUrlCrossRefPubMedWeb of Science
    10. ↵
      1. Salmon L,
      2. Russell V,
      3. Musgrove T,
      4. et al
      . Incidence and risk factors for graft rupture and contralateral rupture after anterior cruciate ligament reconstruction. Arthroscopy 2005;21:948–57.
      OpenUrlCrossRefPubMedWeb of Science
    11. ↵
      1. de Loes M
      . Exposure data. Why are they needed? Sports Med 1997;24:172–5.
      OpenUrlCrossRefPubMedWeb of Science
    12. ↵
      1. Eitzen I,
      2. Moksnes H,
      3. Snyder-Mackler L,
      4. et al
      . A progressive 5-week exercise therapy program leads to significant improvement in knee function early after anterior cruciate ligament injury. J Orthop Sports Phys Ther 2010;40:705–21.
      OpenUrlCrossRefPubMedWeb of Science
    13. ↵
      1. Daniel DM,
      2. Stone ML,
      3. Sachs R,
      4. et al
      . Instrumented measurement of anterior knee laxity in patients with acute anterior cruciate ligament disruption. Am J Sports Med 1985;13:401–7.
      OpenUrlAbstract/FREE Full Text
    14. ↵
      1. Hefti F,
      2. Muller W,
      3. Jakob RP,
      4. et al
      . Evaluation of knee ligament injuries with the IKDC form. Knee Surg Sports Traumatol Arthrosc 1993;1:226–34.
      OpenUrlCrossRefPubMed
    15. ↵
      1. Moksnes H,
      2. Risberg MA
      . Performance-based functional evaluation of non-operative and operative treatment after anterior cruciate ligament injury. Scand J Med Sci Sports 2009;19:345–55.
      OpenUrlCrossRefPubMedWeb of Science
    16. ↵
      1. Donner A,
      2. Eliasziw M
      . A goodness-of-fit approach to inference procedures for the kappa statistic: confidence interval construction, significance-testing and sample size estimation. Stat Med 1992;11:1511–19.
      OpenUrlCrossRefPubMedWeb of Science
    17. ↵
      1. Landis JR,
      2. Koch GG
      . The measurement of observer agreement for categorical data. Biometrics 1977;33:159–74.
      OpenUrlCrossRefPubMedWeb of Science
    18. ↵
      1. Kottner J,
      2. Audige L,
      3. Brorson S,
      4. et al
      . Guidelines for Reporting Reliability and Agreement Studies (GRRAS) were proposed. J Clin Epidemiol 2011;64:96–106.
      OpenUrlCrossRefPubMed
    19. ↵
      1. Fithian DC,
      2. Paxton EW,
      3. Stone ML,
      4. et al
      . Prospective trial of a treatment algorithm for the management of the anterior cruciate ligament-injured knee. Am J Sports Med 2005;33:335–46.
      OpenUrlAbstract/FREE Full Text
    20. ↵
      1. Holm I,
      2. Oiestad BE,
      3. Risberg MA,
      4. et al
      . No difference in knee function or prevalence of osteoarthritis after reconstruction of the anterior cruciate ligament with 4-strand hamstring autograft versus patellar tendon-bone autograft: a randomized study with 10-year follow-up. Am J Sports Med 2010;38:448–54.
      OpenUrlAbstract/FREE Full Text
    21. ↵
      1. Gillquist J
      . Repair and reconstruction of the ACL: is it good enough? Arthroscopy 1993;9:68–71.
      OpenUrlCrossRefPubMedWeb of Science
    22. ↵
      1. Roos H,
      2. Karlsson J
      . Anterior cruciate ligament instability and reconstruction. Review of current trends in treatment. Scand J Med Sci Sports 1998;8:426–31.
      OpenUrlPubMedWeb of Science
    23. ↵
      1. Paterno MV,
      2. Rauh MJ,
      3. Schmitt LC,
      4. et al
      . Incidence of contralateral and ipsilateral anterior cruciate ligament (ACL) injury after primary ACL reconstruction and return to sport. Clin J Sport Med 2012;22:116–21.
      OpenUrlCrossRefPubMed
    24. ↵
      1. Walden M,
      2. Hagglund M,
      3. Ekstrand J
      . High risk of new knee injury in elite footballers with previous anterior cruciate ligament injury. Br J Sports Med 2006;40:158–62.
      OpenUrlAbstract/FREE Full Text
    25. ↵
      1. Muaidi QI,
      2. Nicholson LL,
      3. Refshauge KM,
      4. et al
      . Prognosis of conservatively managed anterior cruciate ligament injury: a systematic review. Sports Med 2007;37:703–16.
      OpenUrlCrossRefPubMedWeb of Science
    26. ↵
      1. Schwarz N,
      2. Oyserman D
      . Asking questions about behavior: cognition, communication, and questionnaire construction. Am J Eval 2001;22:127–60.
      OpenUrlWeb of Science

    Footnotes

    • Contributors All authors contributed to the conception and design, interpretation of the data, critical revision and final approval of the article. IE and HG were responsible for data acquisition, HG was responsible for data analysis.

    • Funding This study was funded by the National Institutes of Health (NIH R01 HD37985).

    • Competing interests None.

    • Ethics approval Regional Ethical Committee for South-Eastern Norway.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    Linked Articles