We thank Freke et al. (1) for their systematic review about physical impairments in patients with symptomatic femoroacetabular impingement, nonetheless we have some remarks about methods and results of the article, in particular for range of motion (ROM) outcome.
A meta-analysis of ROM was performed without reporting an overall estimate. Taking into account the amount of studies included and their information, a meta-analysis should have been accomplished. Nonetheless, authors concluded that individuals with symptomatic FAI demonstrated no difference in hip ROM in any direction of movement. This conclusion was unexpected taking into account the findings reported in the primary studies included), and in the previous systematic review published in 2015 (2), that showed instead a reduced ROM.
This discrepancy in literature is already discussed by the Warwick agreement (3), where authors stated that “the evidence on hip range of motion (ROM) in FAI syndrome is surprisingly contradictory” due to contrasting published systematic reviews (1) (2).
Therefore, we checked the accuracy of results reported, analyzing the data reported for every movement assessed in primary studies comparing those reported in this systematic review. We noted some issues in the represented forest plots.
Firstly, some included studies (4), (5), (6), (7) were reported twice in the meta-analysis for different times points or reporting double data of the same patients obtained by two instruments to measure ROM. The number of observations in the analysis should match the number of individuals (unit) that are randomized or allocated: reporting twice data from the same patients is not appropriate, resulting in an error of unit of analysis inflating the sample size. Secondarily, two studies were included in the meta-analyses selecting an inappropriate control group. In particular, the control group in one study was represented by post-surgery impingement population instead of the healthy control, even if the displaced forest plot indicated the comparisons “pre versus control” (8). Analogously, an inappropriate population was selected in another study (6) for the femoroacetabular group where authors chose arbitrarily data from the subgroup of patients with cam impingement, while they should have chosen data reported from combined cam or pincer impingement, since the aim of the review was to measure physical impairment in patients with any kind of FAI.
We are confident with data published in this review of Freke et al (1): checking data from all primary studies we confirmed that data were correctly reported by authors in tables 1 and 2, however inconsistency between data extraction and analysis is present. It is unclear the estimate and related confidence intervals reported in each forest plot. Reasons could be related to authors obtain missing outcome data from contacting primary authors, nevertheless, this information should be transparently reported in the systematic review. For every direction of movement analyzed, the confidence intervals appear to be too large and, in some cases, the SMD calculated seemed to be wrong, making non-significant some of the significant differences published in the primary studies.
Considering the errors in selective reporting and aggregation of data we claim the conclusion of the review about ROM is not correct. Even if the authors did not show the diamond of the overall results in forest plots, they concluded that there is no statistically significant deficit in ROM in any plane of movement, and visually impressed stakeholders by the forest plots that sustain this conclusion. Anyway, we correctly re-run the analyses and we found that there is a statistically significant ROM deficit in flexion, abduction, external rotation and internal rotation in patients with FAI versus controls.
Errors in published systematic reviews are possible (9), but they can limit validity of conclusions of systematic reviews and resulting statements, agreements or clinical guidelines. We understand that peer-reviewers cannot check every calculation or data analysis but the importance of meta-analysis in the hierarchy of evidence need high level of attention and hopefully technical support to the reviewers to avoid relevant mistakes. At the same time, to improve transparency and help the review process, Journals should require authors to submit the raw data and to share the dataset used for the analyses, when they are not reported in the forest plot.
References
1. Freke MD, Kemp J, Svege I, Risberg MA, Semciw A, Crossley KM. Physical impairments in symptomatic femoroacetabular impingement: A systematic review of the evidence. Br J Sports Med. 2016;50(19):1180.
2. Diamond LE, Dobson FL, Bennell KL, Wrigley T V., Hodges PW, Hinman RS. Physical impairments and activity limitations in people with femoroacetabular impingement: A systematic review. Br J Sports Med. 2015;49(4):230–42.
3. Griffin DR, Dickenson EJ, O’Donnell J, Agricola R, Awan T, Beck M, et al. The Warwick Agreement on femoroacetabular impingement syndrome (FAI syndrome): An international consensus statement. Br J Sports Med. 2016;50(19):1169–76.
4. Nussbaumer S, Leunig M, Glatthorn JF, Stauffacher S, Gerber H, Maffiuletti NA. Validity and test-retest reliability of manual goniometers for measuring passive hip range of motion in femoroacetabular impingement patients. BMC Musculoskelet Disord. 2010;11(194).
5. Bedi A, Dolan M, Hetsroni I, Magennis E, Lipman J, Buly R, et al. Surgical Treatment of Femoroacetabular Impingement Improves Hip Kinematics. Am J Sports Med. 2011;39(1 Suppl):43–9.
6. Harris-hayes M, Mueller MJ, Sahrmann SA, Bloom NJ, Steger-may K, Clohisy MAJC, et al. Persons With Chronic Hip Joint Pain Exhibit Reduced Hip Muscle Strength. J Orthop Sport Phys Ther. 2014;44(11):890–8.
7. Audenaert EA, Peeters I, Vigneron L, Baelde N, Pattyn C. Hip Morphological Characteristics and Range of Internal Rotation in Femoroacetabular Impingement. Am J Sports Med. 2012;40(6):1329–36.
8. Kubiak-Langer M, Tannast M, Murphy SB, Siebenrock K a, Langlotz F. Range of motion in anterior femoroacetabular impingement. Clin Orthop Relat Res [Internet]. 2007 May [cited 2012 Mar 9];458(458):117–24. Available from: http://www.ncbi.nlm.nih.gov/pubmed/17206102
9. Yip R, Islami F, Zhao S, Tao M, Yankelevitz DF, Boffetta P. Errors in systematic reviews : an example of computed tomography screening for lung cancer. Eur J Cancer Prev. 2014;23(i):43–8.
We thank Freke et al. (1) for their systematic review about physical impairments in patients with symptomatic femoroacetabular impingement, nonetheless we have some remarks about methods and results of the article, in particular for range of motion (ROM) outcome.
A meta-analysis of ROM was performed without reporting an overall estimate. Taking into account the amount of studies included and their information, a meta-analysis should have been accomplished. Nonetheless, authors concluded that individuals with symptomatic FAI demonstrated no difference in hip ROM in any direction of movement. This conclusion was unexpected taking into account the findings reported in the primary studies included), and in the previous systematic review published in 2015 (2), that showed instead a reduced ROM.
This discrepancy in literature is already discussed by the Warwick agreement (3), where authors stated that “the evidence on hip range of motion (ROM) in FAI syndrome is surprisingly contradictory” due to contrasting published systematic reviews (1) (2).
Therefore, we checked the accuracy of results reported, analyzing the data reported for every movement assessed in primary studies comparing those reported in this systematic review. We noted some issues in the represented forest plots.
Firstly, some included studies (4), (5), (6), (7) were reported twice in the meta-analysis for different times points or reporting double data of the same patients obtained by two instruments to measure ROM. The number of observations in the analysis should match the number of individuals (unit) that are randomized or allocated: reporting twice data from the same patients is not appropriate, resulting in an error of unit of analysis inflating the sample size. Secondarily, two studies were included in the meta-analyses selecting an inappropriate control group. In particular, the control group in one study was represented by post-surgery impingement population instead of the healthy control, even if the displaced forest plot indicated the comparisons “pre versus control” (8). Analogously, an inappropriate population was selected in another study (6) for the femoroacetabular group where authors chose arbitrarily data from the subgroup of patients with cam impingement, while they should have chosen data reported from combined cam or pincer impingement, since the aim of the review was to measure physical impairment in patients with any kind of FAI.
We are confident with data published in this review of Freke et al (1): checking data from all primary studies we confirmed that data were correctly reported by authors in tables 1 and 2, however inconsistency between data extraction and analysis is present. It is unclear the estimate and related confidence intervals reported in each forest plot. Reasons could be related to authors obtain missing outcome data from contacting primary authors, nevertheless, this information should be transparently reported in the systematic review. For every direction of movement analyzed, the confidence intervals appear to be too large and, in some cases, the SMD calculated seemed to be wrong, making non-significant some of the significant differences published in the primary studies.
Considering the errors in selective reporting and aggregation of data we claim the conclusion of the review about ROM is not correct. Even if the authors did not show the diamond of the overall results in forest plots, they concluded that there is no statistically significant deficit in ROM in any plane of movement, and visually impressed stakeholders by the forest plots that sustain this conclusion. Anyway, we correctly re-run the analyses and we found that there is a statistically significant ROM deficit in flexion, abduction, external rotation and internal rotation in patients with FAI versus controls.
Errors in published systematic reviews are possible (9), but they can limit validity of conclusions of systematic reviews and resulting statements, agreements or clinical guidelines. We understand that peer-reviewers cannot check every calculation or data analysis but the importance of meta-analysis in the hierarchy of evidence need high level of attention and hopefully technical support to the reviewers to avoid relevant mistakes. At the same time, to improve transparency and help the review process, Journals should require authors to submit the raw data and to share the dataset used for the analyses, when they are not reported in the forest plot.
References
1. Freke MD, Kemp J, Svege I, Risberg MA, Semciw A, Crossley KM. Physical impairments in symptomatic femoroacetabular impingement: A systematic review of the evidence. Br J Sports Med. 2016;50(19):1180.
2. Diamond LE, Dobson FL, Bennell KL, Wrigley T V., Hodges PW, Hinman RS. Physical impairments and activity limitations in people with femoroacetabular impingement: A systematic review. Br J Sports Med. 2015;49(4):230–42.
3. Griffin DR, Dickenson EJ, O’Donnell J, Agricola R, Awan T, Beck M, et al. The Warwick Agreement on femoroacetabular impingement syndrome (FAI syndrome): An international consensus statement. Br J Sports Med. 2016;50(19):1169–76.
4. Nussbaumer S, Leunig M, Glatthorn JF, Stauffacher S, Gerber H, Maffiuletti NA. Validity and test-retest reliability of manual goniometers for measuring passive hip range of motion in femoroacetabular impingement patients. BMC Musculoskelet Disord. 2010;11(194).
5. Bedi A, Dolan M, Hetsroni I, Magennis E, Lipman J, Buly R, et al. Surgical Treatment of Femoroacetabular Impingement Improves Hip Kinematics. Am J Sports Med. 2011;39(1 Suppl):43–9.
6. Harris-hayes M, Mueller MJ, Sahrmann SA, Bloom NJ, Steger-may K, Clohisy MAJC, et al. Persons With Chronic Hip Joint Pain Exhibit Reduced Hip Muscle Strength. J Orthop Sport Phys Ther. 2014;44(11):890–8.
7. Audenaert EA, Peeters I, Vigneron L, Baelde N, Pattyn C. Hip Morphological Characteristics and Range of Internal Rotation in Femoroacetabular Impingement. Am J Sports Med. 2012;40(6):1329–36.
8. Kubiak-Langer M, Tannast M, Murphy SB, Siebenrock K a, Langlotz F. Range of motion in anterior femoroacetabular impingement. Clin Orthop Relat Res [Internet]. 2007 May [cited 2012 Mar 9];458(458):117–24. Available from: http://www.ncbi.nlm.nih.gov/pubmed/17206102
9. Yip R, Islami F, Zhao S, Tao M, Yankelevitz DF, Boffetta P. Errors in systematic reviews : an example of computed tomography screening for lung cancer. Eur J Cancer Prev. 2014;23(i):43–8.