Article Text

other Versions

Download PDFPDF

Can even experienced orthopaedic surgeons predict who will benefit from surgery when patients present with degenerative meniscal tears? A survey of 194 orthopaedic surgeons who made 3880 predictions
  1. Victor van de Graaf1,2,
  2. Coen H Bloembergen MD1,3,
  3. Nienke W Willigenburg PhD1,
  4. Julia C A Noorduyn MSc1,
  5. Daniel Saris2,4,
  6. Ian Harris5,6,
  7. Rudolf Poolman1
  8. for the ESCAPE Research Group
  1. 1Orthopaedic Surgery, Joint Research, OLVG, Amsterdam, The Netherlands
  2. 2Orthopaedic Surgery, University Medical Centre, Utrecht, The Netherlands
  3. 3Department of Orthopaedics, CORAL - Center for Orthopaedic Research Alkmaar, Noordwest Ziekenhuisgroep, Alkmaar, The Netherlands
  4. 4Orthopaedic Surgery, Mayo Clinic, Rochester, Minnesota, USA
  5. 5Injury and Rehabilitation Research Department, Whitlam Orthopaedic Research Centre, Ingham Institute for Applied Medical Research, Liverpool, New South Wales, Australia
  6. 6Orthopaedic Department, South Western Sydney Local Health District, Liverpool Hospital, Liverpool, New South Wales, Australia
  1. Correspondence to Dr Victor van de Graaf, Orthopaedic Surgery, Joint Research, OLVG, Oosterpark 9, 1091 AC, Amsterdam, The Netherlands; v.a.vandegraaf{at}olvg.nl

Abstract

Objectives To examine the ability of surgeons to predict the outcome of treatment for meniscal tears by arthroscopic partial meniscectomy (APM) and exercise therapy in middle-aged patients.

Design and setting Electronic survey. Orthopaedic surgeon survey participants were presented 20 patient profiles. These profiles were derived from a randomised clinical trial comparing APM with exercise therapy in middle-aged patients with symptomatic non-obstructive meniscal tears. From each treatment group (APM and exercise therapy), we selected five patients with the best (responders) and five patients with the worst (non-responders) knee function after treatment. 1111 orthopaedic surgeons and residents in the Netherlands and Australia were invited to participate in the survey.

Interventions For each of the 20 patient profiles, surgeons (unaware of treatment allocation) had to choose between APM and exercise therapy as preferred treatment and subsequently had to estimate the expected change in knee function for both treatments on a 5-point Likert Scale. Finally, surgeons were asked which patient characteristics affected their treatment choice.

Main outcomes The primary outcome was the surgeons’ percentage correct predictions. We also compared this percentage between experienced knee surgeons and other orthopaedic surgeons, and between treatment responders and non-responders.

Results We received 194 (17%) complete responses for all 20 patient profiles, resulting in 3880 predictions. Overall, 50.0% (95% CI 39.6% to 60.4%) of the predictions were correct, which equals the proportion expected by chance. Experienced knee surgeons were not better in predicting outcome than other orthopaedic surgeons (50.4% vs 49.5%, respectively; p=0.29). The percentage correct predictions was lower for patient profiles of non-responders (34%; 95% CI 21.3% to 46.6%) compared with responders (66.0%; 95% CI 57.0% to 75.0%; p=0.01).

In general, bucket handle tears, knee locking and failed non-operative treatment directed the surgeons’ choice towards APM, while higher level of osteoarthritis, degenerative aetiology and the absence of locking complaints directed the surgeons’ choice towards exercise therapy.

Conclusions Surgeons’ criteria for deciding that surgery was indicated did not pass statistical examination. This was true regardless of a surgeon’s experience. These results suggest that non-surgical management is appropriate as first-line therapy in middle-aged patients with symptomatic non-obstructive meniscal tears.

Clinical trial registration ClinicalTrials.gov Identifier: NCT03462134.

  • meniscus
  • knee
  • arthroscopic surgery
  • meniscal pathology
  • physiotherapy

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Key messages

What is already known on this topic?

  • Arthroscopic partial meniscectomy (APM) is one of the most common orthopaedic procedures. Although several randomised controlled trials demonstrated no superiority of surgery over non-operative alternatives or sham surgery, the decline in the number of meniscectomies is less than expected. One reason for this is a surgeon’s belief that he or she is able to identify which patients will benefit from surgery.

What this study adds

  • This is the first study that examined the ability of orthopaedic surgeons to predict the treatment response to APM and exercise therapy in patients with non-obstructive meniscal tears. We found that surgeons’ predictions of outcome from APM and exercise therapy in these patients were similar to prediction expected by chance alone, regardless of clinical experience. Surgeons are limited in their ability to predict outcomes in such patients with non-obstructive meniscal tears. We recommend surgeons carefully consider the objective evidence from the literature when considering treatment options.

Introduction

The indication for arthroscopic partial meniscectomy (APM) is one of the most commonly made decisions in orthopaedic practice,1 and 75% of APMs are performed in patients older than 40 years of age.2 However, meniscal tears are common incidental findings in the general population. Incidental meniscal tears are found on MRI in 60% of asymptomatic adults older than 50 years with radiographic evidence of osteoarthritis.3 Therefore, meniscal tears can be seen as part of a degenerative process of the knee.

Although several randomised controlled trials failed to demonstrate a clinically important benefit of APM over non-operative alternatives4–10 or sham surgery11 in middle-aged and older patients with symptomatic meniscal tears, these results have not led to a consistent decline in the number of APMs performed in daily practice.2 12 Common arguments for performing APM include being a difficult habit to break, being influenced by personal experiences (observational evidence), criticism of the experimental evidence (eg, low external validity) and a surgeon’s belief in being capable to identify which patient may still benefit more from surgery.13–19 Therefore, it is suggested to be up to the judgement of the treating surgeon to decide what is best for the individual patient.18

In this survey, we examined the ability of orthopaedic surgeons to predict the outcome in patients treated for meniscal tears. We also determined differences between surgeons with and without expertise in managing patients with knee pain, and how predictions differed between responders and non-responders to treatment. Finally, we evaluated which patient characteristics directed orthopaedic surgeons towards APM or non-surgical treatment.

Methods

Participants and setting

Between December 2017 and March 2018, an online survey was conducted among orthopaedic surgeons and orthopaedic surgery residents. The survey was sent to 1111 orthopaedic surgeons and residents active in the Netherlands (950 orthopaedic surgeons and residents) and Australia (161 orthopaedic surgeons). The Dutch participants were invited by the Dutch Orthopaedic Association (Nederlandse Orthopaedische Vereniging) and the Australian participants were invited by one of the authors (IH). The survey was constructed and distributed using Castor Electronic Data Capture 2018, Ciwit BV, Amsterdam, The Netherlands. In December 2017, the first invitation was sent, and in January and February 2018 a maximum of two reminders were sent to all participants. Only 100% completed surveys were used for data analysis. We registered the study at clinicaltrials.gov (NCT03462134). Ethics approval was not required.

Patient profiles

Each participating surgeon was presented 20 patient profiles. These profiles represented participants from the ESCAPE Trial,20 a multicentre randomised controlled trial that compared APM with exercise therapy under supervision of a physical therapist in middle-aged patients (45–70 years) with a non-obstructive meniscal tear.9 A case description of each patient was presented in the survey, as shown in figure 1. The profiles consisted of demographics, a description of symptoms, baseline knee function, baseline pain score, the results of physical examination, type of meniscal tear on MRI (on the Modified International Cartilage Repair Society classification)21 and osteoarthritis level (Kellgren-Lawrence classification).22

Figure 1

Example of a patient profile in the survey. Each patient profile consisted of demographics, a description of symptoms, baseline knee function on the IKDC, baseline pain score on the VAS, the results of physical examination, type of meniscal tear on MRI and osteoarthritis level. The information in the figure above corresponds to patient profile 1. A clarification of the terms and clinical tests is presented in online supplement appendix 2, ‘Patient profiles 1–20 – explanation of terms and abbreviations used’. For each profile, surgeons were asked to choose between meniscectomy and exercise therapy (in the survey referred to as physical therapy) as the preferred treatment. Subsequently, the surgeons had to estimate the expected effect on the patients’ knee function after 2 years, twice: first for their preferred treatment and second for the other (non-preferred) treatment on a 5-point Likert Scale. BMI, body mass index; IKDC, International Knee Documentation Committee; MRI, magnetic resonance imaging; OA, osteoarthritis; VAS, Visual Analogue Scale; y, years.

Knee function was quantified as a score on the International Knee Documentation Committee (IKDC) Subjective Knee Form, which is a patient reported knee function with a score ranging from 0 to 100 points, with a score of 0 corresponding to maximum knee symptoms and limitations in daily or sporting activities and a score of 100 reflecting no knee symptoms or limitations in daily or sporting activities.23 Knee pain was quantified using a Visual Analogue Scale, ranging from 0 mm to 100 mm, with a score of 0 mm corresponding to no pain and a score of 100 mm reflecting the worst possible pain.24 Pain scores during rest and during weight-bearing were both presented.

These selected patient profiles represented the top five and bottom five participants from the ESCAPE Trial with complete baseline data from each treatment group, discarding patients who where allocated to exercise therapy but received delayed APM. The top five were the patients with the most improvement on the IKDC Score after 24 months follow-up, therefore categorised as ‘responders’, with a mean improvement in IKDC Score of 64.6 (SD 4.6) points in the patients allocated to APM and 54.0 (SD 5.9) points in the patients allocated to exercise therapy. The bottom five per treatment group were the patients who deteriorated or had the least improvement on the IKDC Score after 24 months follow-up, therefore categorised as ‘non-responders’, with a mean deterioration in IKDC Score of −11.5 (SD 6.0) points in the patients allocated to APM and −13.1 (SD 6.6) points in the patients allocated to exercise therapy All 20 patient profiles are presented in online supplement 1.

Survey

The participating surgeons, who were unaware of the treatment received, were asked to choose between APM and exercise therapy as the preferred treatment per profile. Subsequently, the surgeons had to estimate the expected effect on the patients’ knee function after 2 years, twice: first for their preferred treatment and second for the other (non-preferred) treatment. The treatment effect on knee function was scored on a 5-point Likert Scale (strong deterioration, mild deterioration, no relevant change, mild improvement, strong improvement, see figure 1).

Furthermore, the surgeons were asked for their years of experience, field of expertise and opinion regarding the quality of the literature. In addition, they were presented a list of patients’ characteristics, and were asked whether these typically affect their choice towards APM or exercise therapy, or does not affect their choice of preferred treatment. The complete survey content is attached in online supplement 2.

Outcome measures

The primary outcome was the percentage of correct predictions of treatment outcome. We assessed differences between surgeons with and without knee expertise, and how predictions differed between responders and non-responders to treatment. Secondary outcomes included the ratio of treatment choice between APM and exercise therapy, the surgeons’ opinion towards the literature, and an overview of patient characteristics that direct the surgeons’ choice towards APM or non-surgical treatment.

Data analysis

The predictions on the 5-point Likert Scale were dichotomised to discriminate between identified non-responders (Likert Scores 1, 2 and 3) and identified responders (Likert Scores 4 and 5) to treatment. The overall percentage of correct predictions (correct identification as either responder or non-responder) was first calculated over profiles per surgeon and then averaged over surgeons, with the 95% CI representing the reliability of the average estimate over all surgeons.

Surgeons were then divided in two groups based on expertise. The criterion for the group ‘experienced knee surgeons’ was a minimum of 5 years of experience in knee surgery. We used the χ2 test to compare the percentage of correct predictions between the surgeon groups (experienced knee surgeons vs other surgeons) and to compare the percentages of correct predictions in the responders and non-responders to treatment.

All other outcomes were analysed descriptively. Level of significance was set at 0.05. All analyses were performed using SPSS V.22 (IBM Corporation, Armonk, New York, USA).

Patient and public involvement

No patients or the public were involved in designing the study, nor were they involved in developing plans for recruitment, design or implementation of the study. No patients were asked to advise on interpretation or writing up of results.

Results

Participant demographics

Of the 1111 invitations sent, we received 194 (17%) complete responses, 139 from The Netherlands and 55 from Australia. Of the participants, 163 (84%) were orthopaedic surgeons while 31 (16%) were residents in orthopaedic surgery. A total of 101 (52%) participants were experienced knee surgeons and 93 (48%) participants were residents, less experienced or had no knee expertise. An overview of the surgeons’ characteristics is provided in online supplement 3.

Figure 2 presents the results of the predicted outcome per treatment group. Overall, 50.0% (95% CI 39.6% to 60.4%) of all predictions were correct. This percentage was similar between experienced knee surgeons and the other surgeons, 50.4% (95% CI 48.6% to 52.2%) vs 49.5% (95% CI 48.0% to 51.1%), respectively (p=0.58).

Figure 2

Predicted outcome per treatment group. The figure above demonstrates the distribution between the correct and incorrect predictions for each of the categorised profiles (responders and non-responders to meniscectomy and exercise therapy). A correct prediction in the ‘responders meniscectomy’ and ‘responders exercise therapy’ corresponds to the options ‘some improvement’ and ‘strong improvement’, and a correct prediction in the ‘non-responders meniscectomy’ and ‘non-responders exercise therapy’ corresponds to the options ‘strong deterioration’, ‘mild deterioration’ and ‘no relevant difference’ from the 5-point Likert Scale.

The percentage of correct predictions was 66.0% (95% CI 57.0% to 75.0%) in the group of treatment responders vs 34.0% (95% CI 21.3% to 46.6%) in the group of treatment non-responders (p<0.001).

Table 1 presents an overview of the survey results for each patient profile. Overall, 21.6% surgeons chose APM and 78.4% of surgeons chose exercise therapy as the preferred treatment. There was no difference in treatment preference between the level of experience, with 23.7% of experienced knee surgeons choosing APM as the preferred treatment compared with 19.5% of the other surgeons.

Table 1

Results for responders/non-responders per treatment group

Fifty-one per cent of the surgeons reported evidence-based medicine to be more important than personal experience in their clinical decision making, and 77% considered themselves to be completely up to date with the literature for treatment of meniscal tears. The available evidence was convincing to 74% of the participants, and 76% felt confident in choosing between APM and exercise therapy. Seventy-seven per cent indicated that exercise therapy is a good option as initial treatment for non-obstructive meniscal tears, and 89% disagreed with APM being a good option as initial treatment. A complete overview of the results, as well as the distribution per expertise group, is presented in online supplement 4.

Patient characteristics that direct surgeons towards APM include bucket handle tears (94% of surgeons), knee locking (82%), failed non-operative treatment (82%), traumatic aetiology (76%) and age <45 years (74%), while characteristics that direct surgeons towards exercise therapy include moderate to severe osteoarthritis (96%), degenerative aetiology (92%), no obstructive complaints (88%), age >45 years (87%) and obesity (79%). Education level, gender and location of tear do not affect treatment choice. An overview of the results of all characteristics that were presented in the survey is shown in online supplement 5.

Discussion

The survey results indicate that orthopaedic surgeons are unable to identify whether a patient with a non-obstructive meniscal tear will benefit from APM or exercise therapy. The percentage of correct predictions was similar to prediction expected by chance alone, regardless of clinical expertise.

Comparison with literature

The present survey is the first study that determines whether orthopaedic surgeons are able to predict treatment outcome in patients with meniscal tears based on patient profiles. In a recent systematic review evaluating clinicians’ general expectations of any treatment, test or screening test, the authors reported that clinicians often have inaccurate expectations of treatment response.25 With an underestimation of the harms and an overestimation of the benefits, the authors concluded that these inaccurate predictions are likely to result in suboptimal clinical management choices.25 These results are comparable to the findings in this study, in which the surgeons’ ability to predict the outcome was poorer in a group of treatment non-responders, that is, an overestimation of treatment response in this group.

We have found only one study that determined whether orthopaedic surgeons are able to predict a treatment response.26 In patients with sarcomas, orthopaedic oncologists were also incapable of accurately predicting the outcome of limb salvage surgery.26

Furthermore, it has previously been shown in a population with knee disorders that surgeons tend to be (over)optimistic with respect to treatment outcome.27 28 This was supported by our findings, as two-thirds of the non-responders were expected to respond well.

Implications

In the present study, 89% of orthopaedic surgeon participants disagreed with APM being a good option for the initial treatment. However, in 22% of the cases APM was chosen as the preferred treatment. Interestingly, the percentage of respondents who recommended meniscectomy was highest (25.2%) for the descriptions of patients who did not benefit from surgery (non-responders to surgery).

This discrepancy—the greater propensity to recommend surgery for those patient descriptions that were associated with non-responders to treatment—and the poor ability of orthopaedic surgeons to predict who will respond well after surgery, suggests that surgeons should rely more on objective evidence from the literature when choosing treatment modalities in middle-aged patients with non-obstructive meniscal tears.

The participating surgeons were mainly focused on knee-specific characteristics that influenced treatment outcome. Among the most chosen variables that directed surgeons towards meniscectomy were obstructive complaints and traumatic aetiology. However, these convictions are not supported by the most recent literature. Obstructive complaints are associated with poor treatment response in general and meniscectomy in these patients has no added benefit over sham surgery.29 30 There is no difference in improvement from meniscectomy between patients with a traumatic or a degenerative aetiology.31 These misconceptions contribute to the large numbers of meniscectomies still performed.

Instead, considering the whole person in clinical decision making—by including characteristics such as education level, gender and activity level—may improve the surgeon’s predictive ability. Psychological, mental health and socioeconomic variables are known to influence a persons’ health status and mobility in patients with other knee injuries.32 33 Future research should focus on the effects of these variables and on finding other variables that influence treatment outcome in patients with meniscal tears.

Of the 22% of cases in which surgeons recommended meniscectomy as the preferred treatment, the mean expected change from meniscectomy was 4.3 points (on the 5-point Likert Scale), whereas the mean expected change from the non-preferred exercise therapy in these cases was 3.2 points (mean difference 1.1 point). This information provides insight into the criteria used by surgeons for deciding that surgery is indicated. The participating surgeons in this survey considered it sufficient to recommend meniscectomy if they expected no relevant change from exercise therapy (3.2 points on the 5-point Likert Scale) and a mild improvement from meniscectomy as compared with exercise therapy (1.1 points on the 5-point Likert Scale).

In a European survey, prior to the publication of the randomised controlled trials,4–11 75% of surgeons recommended APM as the first treatment in patients with knee osteoarthritis and meniscal tears.34 In the present study, 22% of orthopaedic surgeons chose APM as the preferred treatment in patients over 45 years old with a non-obstructive meniscal tear. These numbers demonstrate the willingness to change clinical practice from an initial surgical approach towards a conservative approach. However, the most recent data do not show a similar decrease in the number of APMs performed.2 12 To further reduce the number of APMs, more effort is needed such as the implementation of administrative measures or (local) policy changes, which were earlier proven to be effective in reducing the number of knee arthroscopies in Norway and Australia.35 36

With 36% and 33% failures of initial exercise therapy (those who underwent meniscectomy during follow-up) reported in the literature after 1 year and 2 years, respectively,5 9 especially this group of patients should be identified in the outpatient clinic. However, in the present study, the surgeons’ ability to predict the outcome was poorest in the non-responders to treatment. According to the literature, there is currently insufficient evidence to allow prediction at an individual level in patients with meniscal tears. Future studies, such as prediction models and individual patient data meta-analyses, could help improving identification of treatment (non-)responders.

Strengths and limitations

To our knowledge, this is the first survey of orthopaedic surgeons to glean insight into their perception and expectations of treating patients with meniscal tears. By selecting the five patients with the best and worse outcomes per treatment group, we aimed for the most unarguable distinction between treatment responders and non-responders for the surgeons.

This study has several limitations. First, a digital survey was the only feasible way to obtain predictions of many surgeons for each of the 20 patient profiles. We made an effort to provide the most relevant information for clinical decision making, but acknowledge that this is still different from a real patient. Predictions in a real clinical setting might be more accurate. However, the majority of patients in a real clinical setting will have a more moderate treatment outcome (compared with the top five responders and bottom five non-responders included in this survey), which is likely harder to predict. Second, although we retrieved 194 responses, the response rate was low (17%)—this raises the potential for selection bias. Most surgeons indicated that they were all up to date with the literature and they were convinced by the evidence (online supplement 4). A higher response rate therefore might have led to a higher percentage of preferred APM and a worse prediction capability. Third, the poor prediction ability could also be due to unknown variables that determine the outcome after treatment of meniscal tears. The duration of symptoms, radiographic level of knee osteoarthritis and the amount of resected meniscus are known variables to be associated with the outcome following meniscectomy.37 Only the level of osteoarthritis was provided to the surgeons. Knowledge of the other variables might have increased the accuracy of their prediction.

Conclusion

Surgeons’ criteria used for deciding that surgery is indicated in a sample of patients with degenerative meniscal tears resulted in a prediction as accurate as a coin toss. This was true regardless of a surgeon’s experience. This suggests that non-surgical management is appropriate as the first-line therapy in these patients. We respectfully recommend that orthopaedic surgeons should rely more on the objective evidence from the literature when choosing treatment options.

References

  1. 1.
  2. 2.
  3. 3.
  4. 4.
  5. 5.
  6. 6.
  7. 7.
  8. 8.
  9. 9.
  10. 10.
  11. 11.
  12. 12.
  13. 13.
  14. 14.
  15. 15.
  16. 16.
  17. 17.
  18. 18.
  19. 19.
  20. 20.
  21. 21.
  22. 22.
  23. 23.
  24. 24.
  25. 25.
  26. 26.
  27. 27.
  28. 28.
  29. 29.
  30. 30.
  31. 31.
  32. 32.
  33. 33.
  34. 34.
  35. 35.
  36. 36.
  37. 37.

Footnotes

  • Collaborators Vanessa AB Scholtes, Eduard LAR Mutsaerts, Julius Wolkenfelt, Matthijs R Krijnen, Derek FP van Deurzen, Dirk Jan F Moojen, Ise K Butter, Arthur de Gast, Thom Snijders, Jelle J Halma, Nienke Wolterbeek, Camille Neeter, Gino MMJ Kerkhoffs, Rolf W Peters, Igor CJB van den Brand, Suzanne de Vos-Jakobs, Andy B Spoor, Taco Gosens, Wahid Rezaie, Dirk Jan Hofstee, Bart J Burger, Daniël Haverkamp, Anton MJS Vervest, Thijs A van Rheenen, Anne E Wijsbek, Ewoud RA van Arkel, Bregje JW Thomassen, Sheila Sprague, Ben W Mol, Maurits W van Tulder.

  • Contributors VG and CB equally contributed to this work. VG, CB and RP were co-chief investigators. VG, CB, NW, DS, IH and RP were involved in the design of the study and its implementation. VG, CB, NW, IH and RP designed the prespecified statistical analysis plan. The members of the ESCAPE Research Group were responsible for study progress of the ESCAPE Trial and data collection at site. VG, CB and NW did the statistical analyses. VG and CB wrote the first draft of the manuscript; NW, JN, DS, IH and RP made revisions. All authors and collaborators read and approved the final manuscript.

  • Funding This study was funded by the Netherlands Organisation for Health Research and Development (in Dutch: ZonMw; grant number 837002009), Zilverenkruis Health Insurance (grant number Z436) and the foundation of medical research of the OLVG, Amsterdam (grant number 15u.025). The funders had no role in design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication. All authors had full access to all of the data in the study and can take responsibility for the integrity of the data and the accuracy of the data analysis.

  • Competing interests VG, NW, JN, and RP received financial support from The Netherlands Organisation for Health Research and Development (in Dutch: ZonMw) for the submitted work; the Achmea Healthcare Foundation (in Dutch Stichting Achmea Gezonheidszorg fonds), and the foundation of medical research at the OLVG, Amsterdam, the Netherlands; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work.

  • Patient consent for publication Not required.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data availability statement Data are available upon request.