The BJSM recently rejected our request of retraction or errata corrige of the editorials by Blanch and Gabbett(1) and Gabbett (2) presenting the relation between the Acute:Chronic Workload Ratio (ACWR) and likelihood of injuries. The preprint and a list of some of the errors presented in that figure can be found here: https://osf.io/preprints/sportrxiv/gs8yu/. In challenging our request, it was underlined several times by the Editor in Chief of BJSM that the “model” was presented as illustrative only, and this seems to make errors acceptable like if the editorials are a “safe zone” where for illustrative purposes it is possible to bend and even break scientific rules and methods, presenting models using unpublished and uncontrollable data.
However, the reason of this communication is to warn the members of the consensus (and readers) that the ACWR model published in the IOC consensus(3) as a validated model has in fact not been validated at all: [page 1034] “The model has currently been validated through data from three different sports (Australian football, cricket and rugby league)(187)”. The reference 187 is one of the two editorials(1) for which we asked the retraction. So on one side the Editor in Chief insists that it is just an illustrative (flawed) model, but on the other side the same Editor in Chief, co-author (with one of the proponents of the model) of the IOC consensus wrote and published that it...
The BJSM recently rejected our request of retraction or errata corrige of the editorials by Blanch and Gabbett(1) and Gabbett (2) presenting the relation between the Acute:Chronic Workload Ratio (ACWR) and likelihood of injuries. The preprint and a list of some of the errors presented in that figure can be found here: https://osf.io/preprints/sportrxiv/gs8yu/. In challenging our request, it was underlined several times by the Editor in Chief of BJSM that the “model” was presented as illustrative only, and this seems to make errors acceptable like if the editorials are a “safe zone” where for illustrative purposes it is possible to bend and even break scientific rules and methods, presenting models using unpublished and uncontrollable data.
However, the reason of this communication is to warn the members of the consensus (and readers) that the ACWR model published in the IOC consensus(3) as a validated model has in fact not been validated at all: [page 1034] “The model has currently been validated through data from three different sports (Australian football, cricket and rugby league)(187)”. The reference 187 is one of the two editorials(1) for which we asked the retraction. So on one side the Editor in Chief insists that it is just an illustrative (flawed) model, but on the other side the same Editor in Chief, co-author (with one of the proponents of the model) of the IOC consensus wrote and published that it was a validated model. There is something of great concern here. The publication of this figure in your consensus has given that model the credit it does not deserve. Other than not being validated, the errors in developing that figure are evident and I am sure the members can easily realize this by reading the original editorials proposing the model in the first place or by reading the problems in our request of retraction or errata. I believe that the consensus members gave too much credit to some participants involved in the consensus that proposed the model as validated. I further underline that the ACWR model you have presented also shows that if you taper before a competition or you complete a recovery week (i.e. you are outside the “sweet spot”), you are at higher risk of injuries. This is, unfortunately, an exemplificative “practical” translation of one part of the model you have contributed (unknowingly) to be popularized through including it in a consensus statement. The problem is that the metric (ACWR) is also deceptive and the members probably did not realize the meaning and hence interpretation of the ratio and figure. Furthermore, the ACWR-injury relation it is not established and in the literature various and contrasting relations can be found. Clearly more studies, hopefully with lower risk of bias, are needed. Therefore, since I value your contribution to better science and evidence-based recommendations, it would be appreciated to know whether at least the IOC panel will acknowledge and advise the reader the fact that the model has not been validated and potentially misleading.
References
1. Blanch P, Gabbett TJ. Has the athlete trained enough to return to play safely? The acute:chronic workload ratio permits clinicians to quantify a player's risk of subsequent injury. Br J Sports Med 2016;50(8):471-5. doi: 10.1136/bjsports-2015-095445
2. Gabbett TJ. The training-injury prevention paradox: should athletes be training smarter and harder? Br J Sports Med 2016;50(5):273-80. doi: 10.1136/bjsports-2015-095788
3. Soligard T, Schwellnus M, Alonso JM, et al. How much is too much? (Part 1) International Olympic Committee consensus statement on load in sport and risk of injury. Br J Sports Med 2016;50(17):1030-41. doi: 10.1136/bjsports-2016-096581
Thank you for your thorough response to my initial comment.
I am wondering if you could help me understand the new AE-level as-treated analysis you have done in response to Point 2. This accounts for all non-compliant AEs among all athletes, correct? If I understood you correctly, there were somewhat more than the 711 non-compliant AEs reported in the paper and which you reported in your response to Point 4, correct?
What would be very helpful to see is a.) the number of AEs and b.) the number of SRCs that occurred during those AEs for each of the following groups when considering any non-compliant AE, not just ones from athletes who suffered an SRC while non-compliant or were non-compliant >50% of the time:
Assigned HG/Did Not Wear:
Assigned HG/Did Wear:
Assigned No HG/Did Not Wear:
Assigned No HG/Did Wear:
After careful appraisal and following our own investigations, we are concerned that the article “Is interval training the magic bullet for fat loss? A systematic review and meta-analysis comparing moderate-intensity continuous training with high-intensity interval training (HIIT)” [1] may have some data extraction and analysis errors that warrant further review by the editor and authors, and which more concerningly, may impact the original conclusions of the article.
We were initially concerned about the reported results within the Thomas et al. paper [2], particularly the biological plausibility of a mean between-group fat-loss difference of 13.44 kg over 12 weeks. Given that the authors did not report any study-level data, we decided to investigate the effect size within this paper. However, this study [2] did not report any fat mass data, only % body fat data. Given that the authors of the review [1] reported “When studies provided insufficient data for inclusion in the meta-analysis (five studies), the corresponding authors were contacted via email to determine whether additional data could be provided; however, no corresponding authors responded.”, it is unclear how an unpublished mean difference of -13.44 kg in favour of HIIT/SIT could be presented within the fat mass analysis of this review. Furthermore, when reviewing another of the included studies [3], we found that fat mass data were reported, but not included in the current meta-analysis [1]. Given the m...
After careful appraisal and following our own investigations, we are concerned that the article “Is interval training the magic bullet for fat loss? A systematic review and meta-analysis comparing moderate-intensity continuous training with high-intensity interval training (HIIT)” [1] may have some data extraction and analysis errors that warrant further review by the editor and authors, and which more concerningly, may impact the original conclusions of the article.
We were initially concerned about the reported results within the Thomas et al. paper [2], particularly the biological plausibility of a mean between-group fat-loss difference of 13.44 kg over 12 weeks. Given that the authors did not report any study-level data, we decided to investigate the effect size within this paper. However, this study [2] did not report any fat mass data, only % body fat data. Given that the authors of the review [1] reported “When studies provided insufficient data for inclusion in the meta-analysis (five studies), the corresponding authors were contacted via email to determine whether additional data could be provided; however, no corresponding authors responded.”, it is unclear how an unpublished mean difference of -13.44 kg in favour of HIIT/SIT could be presented within the fat mass analysis of this review. Furthermore, when reviewing another of the included studies [3], we found that fat mass data were reported, but not included in the current meta-analysis [1]. Given the marginal level of significance for the fat mass outcome in the review (MD, CI; -2.28, [-4.00, -0.56]), it is possible that if these data had been excluded/included the primary conclusion of this article (that HIIT training resulted in superior fat mass loss compared to moderate intensity aerobic training), may be altered.
Additionally, upon review of the first two data points in the % body fat plot, we also have concerns regarding the calculated effect sizes. We are unable to confirm the accuracy of the calculations, given the lack of study-level data reported and the absence of effect size (ES) calculation methods in either the review article methods or the registered protocol on PROSPERO (CRD42018089427). Whilst the Thomas et al. [2] paper reported % body fat data graphically, and we acknowledge there may be some minor differences in data points depending on which graph analysis tool was used, the Trapp et al. [3] paper reported all pre- and post- mean and standard error data, so our ES and CI calculations should be identical to those in the review if the same calculation method was used. However, our calculated ES using the pooled baseline SD for ES calculation (-3.30 [-13.36, 6.76]) varied significantly from that reported by the Viana et al (-9.59 [-16.97, -2.20]). Presuming that the authors are presenting mean difference data (as suggested in their forest plot figures), there should be no disputing this value as -3.30, based solely on study-level data. Our analyses only examined data from these two studies, so we can neither verify nor refute the results presented for the remaining 34 included studies.
Finally, we question the inclusion of one of the studies in the review. Viana et al. have included the study by Boer and Moss [4] that used 70-80% VO2peak for their moderate intensity exercise. This is outside the 40-60% VO2max ‘criteria’ for moderate intensity provided by the authors.
In conclusion, we have grave concerns about the accuracy of the data extraction and analysis, as well as study selection in the Viana et al review [1]. We would ask the editors therefore to request clarification and transparency in the methods used to derive the statistical comparisons presented in the published review. At minimum, a published correction of the article is needed, given the likelihood that this review will be highly cited by others who do not have the time or the statistical knowledge to question the findings. However, if the conclusions of the article are in fact driven by the errors in data extraction and analysis noted above, then full retraction may be indicated.
[1] Viana RB, Naves JPA, Coswig VS, et al Is interval training the magic bullet for fat loss? A systematic review and meta-analysis comparing moderate-intensity continuous training with high-intensity interval training (HIIT) Br J Sports Med. Published Online First: 14 February 2019.
[2] Thomas TR, Adeniran SB, Etheridge GL. Effects of different running programs on VO2max, percent fat, and plasma lipids. Can J Appl Sport Sci 1984;9:55–62.
[3] Trapp EG, Chisholm DJ, Freund J, et al. The effects of high-intensity intermittent exercise training on fat loss and fasting insulin levels of young women. Int J Obes 2008;32:684–91.
[4] Boer PH , Moss SJ . Effect of continuous aerobic vs. interval training on selected anthropometrical, physiological and functional parameters of adults with Down syndrome. J Intellect Disabil Res 2016;60:322–34
We are grateful for Dr. Binney’s interest in our study and his consideration of a portion of the results presented in the manuscript.
Listed below are our responses to each of the concerns raised in the letter.
1. In the as-treated analysis you have a very strange result. Your multivariate risk ratio (which is actually a rate ratio) is 0.63 for everyone overall, 0.64 for females, and 0.93 for males. The result for everyone should be between the results for males and females. Can you please clarify how you got these results, including the exact model(s) you used and how you calculated the rate ratios? Did you use a group*sex interaction term to get the sex-specific results?
Response: We thank you for noticing the mathematical inconsistency in Table 4 rate ratio results for the as-treated analyses. You are correct that if these results were from one model, the overall rate ratio estimate would need to be in-between the male/female estimates. We should note that these were actually 3 separate mixed-effects models: (1) the overall model adjusting for all variables including sex, (2) female sub-group model adjusting for all variables –excluding sex, and (3) male sub-group model adjusting for all variables –excluding sex. We apologize that the footnote in the table is unclear in this regard. We did attempt to use interaction models for this analyses, but did not achieve consistent convergence. As such, we opt...
We are grateful for Dr. Binney’s interest in our study and his consideration of a portion of the results presented in the manuscript.
Listed below are our responses to each of the concerns raised in the letter.
1. In the as-treated analysis you have a very strange result. Your multivariate risk ratio (which is actually a rate ratio) is 0.63 for everyone overall, 0.64 for females, and 0.93 for males. The result for everyone should be between the results for males and females. Can you please clarify how you got these results, including the exact model(s) you used and how you calculated the rate ratios? Did you use a group*sex interaction term to get the sex-specific results?
Response: We thank you for noticing the mathematical inconsistency in Table 4 rate ratio results for the as-treated analyses. You are correct that if these results were from one model, the overall rate ratio estimate would need to be in-between the male/female estimates. We should note that these were actually 3 separate mixed-effects models: (1) the overall model adjusting for all variables including sex, (2) female sub-group model adjusting for all variables –excluding sex, and (3) male sub-group model adjusting for all variables –excluding sex. We apologize that the footnote in the table is unclear in this regard. We did attempt to use interaction models for this analyses, but did not achieve consistent convergence. As such, we opted for sub-group analysis. It is curious that the overall rate ratio estimate does not fall within the bounds of the male and female rate ratio estimate; this is something we continue to explore. We regret not publishing the univariable results in conjunction with the multivariable results in Table 4. As noticed in the text below, the univariable results lead to estimates where the overall estimate falls in between that of the subgroup analyses. The cluster adjusted univariable rate ratios [URR (95%CI), p-value] and cox proportional hazard ratios [UHR (95%CI) p-value] comparing the incidence of SRC between the HG and NoHG groups are as follows:
Per protocol analyses; All No HG: n = 1545, SRC’s n = 68, %SRC = 4.4%. All HG: n = 1505, SRCs’ n = 62, %SRCs = 4.1% [RR = 0.96 (0.63–1.46) p = 0.855], HR = 0.99 (95%CI 0.65–1.50) p = 0.951]. Males - no HG: n = 546, SRC’s n = 8, %SRC = 1.5%. Males HG: n = 474, SRC’s n = 14, %SRC’s 3.0%, [RR 1.81 (0.63–5.18) p = 0.271], HR = 2.02 (0.70–5.80) p = 0.286]. Females-NHG: n = 999, SRC’s n = 60, %SRC = 6.0%. Females-HG: n = 1031, SRC’s n = 48, %SRC = 4.7% [RR = 0.87 (0.53-1.42) p = 0.582], [HR 0.83 (0.53-1.32) p = 0.442]
As treated analyses; All No HG; n =1546, SRC’s n = 75, %SRC = 4.9%. All HG n = 1504, SRC’s n = 55, %SRC = 3.7 [RR = 0.66 (0.41-1.08) p = 0.097], [HR = 0.80 (0.51-1.24) p = 0.315]
Males- no HG: n = 548, SRC’s n = 10, %SRC = 1.8%. Males-HG n = 472, SRC’s n = 12, %SRC = 2.5% [RR = 0.97 (0.30-3.20) p = 0.966], [HR = 1.40 (0.49-4.06) p = 0.531]
Females-NHG n = 998, SRC n = 65, %SRC = 6.5%. Females-HG n = 1032, SRC’s n = 43, %SRC = 4.2 [RR = 0.62 (0.37-1.06) p = 0.078], [HR = 0.69 (0.42-1.12) p = 0.134].
Given the fact that the univariable results are plausible, it convinces us that perhaps something is happening when adjusting for other variables or by having a random effect in each separate model that is causing the overall ‘as treated’ HG estimate to be outside the estimates for the male and female subgroup models. We will continue to look at issues of sparsity, multicolinearity, exchangeability, etc. that may be causing this disparity in the multivariable analyses.
2. How you defined the as-treated group is concerning. You state that you only re-classified a subject if they spent >50% of their time in their non-assigned group OR if they were concussed while in their non-assigned group. This approach will bias the results of your as-treated analysis as you are deliberately misclassifying the AEs of people who do not get hurt and the non-concussed AEs of those who do. You need to classify every AE, rather than each athlete, as headgear or no headgear and repeat the as-treated analysis. Otherwise this analysis is highly questionable and should be removed from the paper.
Response: For this study we considered the subjects without SRC to be adherent if they participated in more than 50% of the AE with their assigned group allocation. There are certainly limitations to this decision. One could also argue that our study team should have set the “threshold” of the percentage of exposures that met the assigned group criteria greater than 50% as we did. The 50% threshold appears arbitrary but we felt it was set to give us the most realistic picture of what was occurring at the team level during the study. We recognize that we could have set this threshold at 60%, 70%, 80% or even 90% of the athletic exposures and doing so would alter the distribution of the as-treated group. But given the high rate of compliance, we felt that our results would not sufficiently change by this decision. The distribution of compliance would have to be very different. In fact, we looked into this and determined that the classification of adherent versus non-adherent groups would not change drastically based on our definition of “as-treated”. The sensitivity of classification approach for all participants as follows: ITT HG Group (n = 1505); 0 % adherent n = 0, 1% to 10% adherent n = 0, 11% to 20% adherent n = 0, 21% to 30% adherent n = 0, 41% to 50% adherent n = 0, 51% to 60% adherent n = 0, 61% to 70% n = 3 (0.2%), 71% to 80% n = 2 (0.1%), 81% to 90% n = 33 (2.2%) Note: n = 7 of the 33 subjects sustained a SRC during one of their non-adherent AEs], 91% to 100% n = 1467 (97.5%).
ITT No HG group (n=1545); 0 % adherent n = 5 (0.3%), 1% to 10% adherent n = 0, 11% to 20% adherent n = 0, 21% to 30% adherent n = 0, 41% to 50% adherent n = 1 (0.06%), 51% to 60% adherent n = 0, 61% to 70% n = 0, 71% to 80% n = 0, 81% to 90% n = 0, 91% to 100% n = 1539 (95.4%). Note: The n = 6 non adherent participants in this group were changed to HG group in “as-treated” analyses due to non-adherence (wore their own headgear, not provided by the study team).
In the distribution results above, there are very few athletes relative to the entire sample that were affected by our definition of < 50% adherence. The second component that could change the analyzed group assignment for the as-treated analysis was having a non-adherent headgear status for the athletic exposure in which an SRC occurred. There were only 13 athletes whose group assignment was different between the ITT and as-treated analysis. Six of them changed from No HG to HG group based on adherence (seen in table above), and the other 7 changed from HG to No HG because of not wearing their headgear at time of SRC. We agree that it is a little puzzling that all 7 that changed status from HG to no HG were due to SRC. Could this be due to data collection issues? We explored this possibility and discuss further below.
We also acknowledge that our ‘as-treated’ groupings may not be optimal and the approach of assigning each athlete-exposure, rather than each student, to whether they wore headgear each day or not could be a preferred method to further analyze the as treated group. As suggested, we analyzed the data in this way for the ‘as treated’ analysis. The results of the cluster adjusted Univariable Odds Ratio [UOR (95%CI) p value] and Multivariable Odds Ratios [MVOR (95%CI) p value] comparing the incidence of SRC between the HG and NoHG groups using athletic exposure as individual unit of analysis are as follows: All HG: [UOR = 0.70 (0.45-1.07) p = 0.099], [MVOR = 0.67 (0.42-1.06) p = 0.089]. Males-HG [UOR = 1.09 (0.40-2.94) p = 0.871], [MVOR = 1.06 (0.38-2.97) p = 0.911]. Females-HG [UOR = 0.65 (0.40-1.06) p = 0.083], [MVOR = 0.67 (0.41-1.08) p = 0.101]. The cluster models were adjusted at the school level only due to singularity issues when additionally adjusting at the subject ID level
Note, this analysis method doesn’t apply to an ITT type of analysis because it allows the group assignment to change, while an ITT analysis does not. Also, this analysis method estimates an odds ratio, not a rate ratio since athletic exposures are considered the individual unit. Since there are no repeat concussions in our data, the OR should be similar to the RR. Comparing the results seen here to the published Table 4 as-treated results, does not show an appreciable difference. We have shown through the responses to these concerns that we can analyze the data in multiple different ways and the results have not changed in a manner that should bring the methods selected for publication into major scrutiny.
3. You report extremely high adherence (99.53%). Is this taking into account any non-adherence or only non-adherence from students not adhering for a majority of their AEs or suffering a concussion while non-adherent? It would be very helpful to see total non-adherence since it sounds like your ATs reported that? I would like to emphasize again that this total non-adherence is what should be used in your as-treated analysis. If 0.47% is the total non-adherence for all participants, would you be willing to share some strategies you used to secure such great adherence?
Response: We readily acknowledge the high compliance with the study once the participants were enrolled in the study, and we believe, to the best of our knowledge, this to be an accurate representation of adherence to study protocol. With that said, there are some selection effects that may have contributed to this unusually high compliance rate.
For any RCT, researchers must be concerned with compliance of the subjects to any given treatment or control group. This is even more a concern when the subjects are participants on athletic teams and or adolescents. It is entirely unrealistic to assume that each subject would be compliant for 100% of the athletic exposures. This study is no different. In fact, given our previous work with adolescent athletes, during the initial study planning, the entire study team recognized that we needed to have a plan to monitor and encourage compliance for subjects in the HG group.
To encourage compliance our plan was to stress the importance of the study during all phases of team and potential subject recruitment. Once the study began, we also stressed the importance to coaches and athletic trainers to continually monitor and report how many subjects were compliant to their assigned group.
This high compliance could be attributed to several factors but most likely due to the fact that the coaches who were highly motivated to have their teams participate in the study actually did so (and independently encouraged their athletes to fully comply with study protocol). Even though we contacted 537 teams to participate, only 33% of the female coaches and 23% of the male coaches agreed to take part in the study (see Figure 1 in manuscript).
Anecdotally, coaches who did not want the risk of being randomized to either the HG or NoHG group did not agree to participate at the onset of the study, resulting in a self-selected group of potentially highly motivated schools/coaches to be randomized. Further, motivation of the study participants should be mentioned. For example, once the school agreed to allow their teams to participate, 60% of the players in the NoHG group actually enrolled in the study while only 50% of the players from schools allocated to the HG group enrolled in the study (Figure 1).
At the soccer team recruitment and enrollment meetings, individual core study team members stressed the magnitude of the study. Potential participants (and their parents) were told the importance of being compliant with the study due to the costs and logistics involved in the data collection process. In many cases the coaches stressed these points as well. Many coaches repeated that while participation was voluntary they expected all subjects to be compliant on a daily basis throughout the season if they enrolled in the study. We anticipate but cannot fully prove that as the result of these statements during the subject recruitment meetings, the subjects who were most likely to be compliant enrolled in the study while those not likely to be compliant did not enroll in the study.
In addition, we allowed the subjects in the HG group to choose which brand of headgear to wear during the season. Each of the brands utilized of the study had unique characteristics, were comprised out of various materials and conformed to various head shapes and sizes differently. As a result, the headgear did not fit each subject to the same degree of comfort. The HG group participants were encouraged to try on each different HG brand and to pick the model they felt was the most comfortable as they would be wearing it the entire season, for every practice and competition. By allowing the participants to individually choose the HG model they would wear, we felt we could increase the likelihood the participant to be compliant throughout the season.
During the run up to and throughout the study, the PI contacted each AT collecting data reminding them the importance of compliance and to let the study team know if participants were not compliant. Coaches were contacted personally by the study PI if the school AT reported compliance was a concern for their team participants. In Figure 1, we note that of the n=1599 initially enrolled in the HG arm of the study, only n=1505 began participation (i.e. n=94 (6%) HG participants dropped out of the study after enrollment). This differed in the NHG arm where only n=15 (1%) did not participate with the team after enrollment. Presumably, most of these individuals left because they were cut from the team or quit before data collection began.
Finally, n = 59 participants chose to drop out of the study during the season. These individuals are effectively censored in the analysis (disproportionately in the HG group) and only their data up until the point of withdrawing from the study are included in n=3050. Study oversight rules and regulations prohibited us from contacting any of the players who decided to stop participating in the study. However, at the time of withdrawal, ATs recorded the reason for withdrawing from the study, if known. Most (69%) did not list a reason. We can’t say for sure, but it can be presumed a majority of the HG athletes did not want to wear the HG anymore and thus withdrew from the study. As a result, only data up to the point at which they withdrew from the study are included. Further info on study drop-out by headgear group are as follows:
Head Gear Group: n = 50 (3%) Non soccer related illness (n=1); quit team (n=10), didn’t want to wear (n=3); reason not listed (n=36). No Head Gear Group: n = 9 (0.6%) Quit the team (n=4); reason not listed (n=5)
Thus, although we do have high adherence once subjects started participating in the study, it should be noted that we do have a number of eligible subjects who did not enroll and or exited the study after enrollment. We readily acknowledged this fact and we mentioned this in the limitations section, that we are very aware that selection effect for study participants may be present. Thus selection effect, along with coaches and school ATs stressing compliance during the study, and allowing for subjects to individually choose the headgear brand they would wear are most likely the reasons for the high compliance although we have no way to definitively answer this question.
Further, we only analyzed the data reported to us by the school personnel regarding their participants. In an optimal situation, one could argue that to be most accurate to have a study team member (not school personnel) present at each of the school sites to monitor compliance. While that type of study oversight may be desirable it is entirely unrealistic given the size and scope of this study. We did perform weekly queries and periodically questioned individual school ATs if we felt the data that was reported was incongruent with the data reported from other teams. We are making the assumption that the ATs, who were incentivized for their efforts (note: the soccer programs themselves were also incentivized), accurately recorded HG use and injuries throughout the season. And although we feel relatively confident with this assumption, once we had the final data we felt we had to analyze the data as it was reported to us.
4. Finally, it seems there is an extremely high rate of concussions among the non-adherent AEs. Per Table 4 and the text of your paper, there were at least 7 concussions among the 711 non-adherent AEs for a rate of 9.85 per 1,000 AEs. For the adherent group this leaves 123 concussions in 150,466 AEs for a rate of 0.82 per 1,000 AEs. This suggests the rate of concussions in the non-adherent was at least 12-fold that in the adherent (regardless of whether these involved wearing headgear or not). This is a very strong effect. Do you have any explanation for this vast difference? It is possible that this difference will shrink or disappear if you correctly count all non-adherent AEs as non-adherent.
Response: The estimates of rate of concussion per 1,000 AEs based on non-adherent (1000*7/711 = 9.8) vs adherent (1000*123/150,446 = 0.8) is quite the difference, and worth exploring more. The distribution of concussion by ITT and As-treated assignment are as follows: ITT Assignment: NoHG and the As-treated; NoHG, No SRC = 78,558 while the Yes SRC n = 68. ITT Assignment: NoHG and the As-treated; HG: No SRC = 218 while the Yes SRC n = 0. ITT Assignment: HG and the As treated: NoHG; No SRC = 486 while the Yes SRC n = 7. ITT Assignment HG and the As- treated: HG; No SRC = 71,765 while the Yes SRC = 55.
In addition, here is the breakdown for all 7 subjects that switched from HG to no HG in the as-treated analysis because they suffered a SRC while not wearing HG. Player A; School 07 - 100% adherent from 8/15 – 09/01, then only ~50% adherent from 09/02 – 09/29 (n=13 total AEs; n=7 non-adherent). Suffers SRC on 09/29. 100% adherent after return to play. Player B; School 28 – 100% compliance from 03/20 – 05/01, then 0% compliance from 05/02 – 05/09 (n=5 AEs). Suffers SRC on 05/09. Did not return prior to end of season. Player C; School 13 – 100% adherent from 03/20 – 04/13, then 0% compliance from 04/17 - 04/18 (n=2 AEs). Suffers SRC on 04/18. Did not return prior to end of season. Player D; School 11 - 100% adherent from 03/20 – 03/30, then 0% adherent from 03/31 – 04/04 (n=3 AEs) then 100% adherent after return to play. Player E; School 22 – 100% adherent from 03/23 – 05/01, then 36% adherent from 05/02 – 05/16 (n=11 AEs; 4 non-adherent), then 100% adherent after return to play. Player F; School 29 – 100% adherent from 03/20 – 04/06, then 0% adherent from 04/07 – 04/11 (n=3 AEs), then 100% adherent from 04/12 – 05/15, then 0% adherent from 05/15 – 05/18 (n=4 AEs). Suffers SRC on 05/18 and does not return prior to end of season. Player G; School 18 – 100% adherent from 08/14 – 08/21, then 50% adherent from 08/22 – 09/05 (n=10 AEs; 5 non-adherent. Suffer SRC on 09/05. 100% adherent from return to play to end of season.
All 7 of these athletes were non-adherent for 10-20% of their total AEs. For the 6 athletes that were non-adherent but did not suffer an SRC, all of them come from different schools (01, 03, 05, 10, 13, and 33). Five of them, were non-adherent for the entire season and all had at least 30 AEs. The other, was non-adherent (i.e. wore HG, but was assigned no HG) for 33 of 57 AEs (57.9%).
By looking at the information above, it is noticed that 12 of 13 athletes came from different schools (so this anomaly isn’t athletic trainer specific). All 7 that had SRC had periods of non-adherence prior to sustaining their SRC. The latter could suggest that the anomaly isn’t due to only reporting non-adherence on days when an SRC occurred. It might be that ATs were not as compliant as we expected they were during the recording of compliance during each practice and game; however, given our efforts to promote and ensure compliance, we have no reason to not trust the accuracy of our results.
Thank you again for allowing to respond to these comments.
I'd like to commend you on running a large RCT on such an important topic (assessing the purported effectiveness of concussion-reduction technologies). Unfortunately I have some concerns about some aspects of your data and analysis, particularly the as-treated analysis in Table 4and your reported adherence numbers. I am hoping you can clarify these concerns and re-do parts of your analysis.
1. In the as-treated analysis you have a very strange result. Your multivariate risk ratio (which is actually a rate ratio) is 0.63 for everyone overall, 0.64 for females, and 0.93 for males. The result for everyone should be between the results for males and females. Can you please clarify how you got these results, including the exact model(s) you used and how you calculated the rate ratios? Did you use a group*sex interaction term to get the sex-specific results?
2. How you defined the as-treated group is concerning. You state that you only re-classified a subject if they spent >50% of their time in their non-assigned group OR if they were concussed while in their non-assigned group. This approach will bias the results of your as-treated analysis as you are deliberately misclassifying the AEs of people who do not get hurt and the non-concussed AEs of those who do. You need to classify every AE, rather than each athlete, as headgear or no headgear and repeat the as-treated analysis. Otherwise this analysis is highly questionable and...
I'd like to commend you on running a large RCT on such an important topic (assessing the purported effectiveness of concussion-reduction technologies). Unfortunately I have some concerns about some aspects of your data and analysis, particularly the as-treated analysis in Table 4and your reported adherence numbers. I am hoping you can clarify these concerns and re-do parts of your analysis.
1. In the as-treated analysis you have a very strange result. Your multivariate risk ratio (which is actually a rate ratio) is 0.63 for everyone overall, 0.64 for females, and 0.93 for males. The result for everyone should be between the results for males and females. Can you please clarify how you got these results, including the exact model(s) you used and how you calculated the rate ratios? Did you use a group*sex interaction term to get the sex-specific results?
2. How you defined the as-treated group is concerning. You state that you only re-classified a subject if they spent >50% of their time in their non-assigned group OR if they were concussed while in their non-assigned group. This approach will bias the results of your as-treated analysis as you are deliberately misclassifying the AEs of people who do not get hurt and the non-concussed AEs of those who do. You need to classify every AE, rather than each athlete, as headgear or no headgear and repeat the as-treated analysis. Otherwise this analysis is highly questionable and should be removed from the paper.
3. You report extremely high adherence (99.53%). Is this taking into account any non-adherence or only non-adherence from students not adhering for a majority of their AEs or suffering a concussion while non-adherent? It would be very helpful to see total non-adherence since it sounds like your ATs reported that? I would like to emphasize again that this total non-adherence is what should be used in your as-treated analysis.
If 0.47% is the total non-adherence for all participants, would you be willing to share some strategies you used to secure such great adherence?
4. Finally, it seems there is an extremely high rate of concussions among the non-adherent AEs. Per Table 4 and the text of your paper, there were at least 7 concussions among the 711 non-adherent AEs for a rate of 9.85 per 1,000 AEs. For the adherent group this leaves 123 concussions in 150,466 AEs for a rate of 0.82 per 1,000 AEs. This suggests the rate of concussions in the non-adherent was at least 12-fold that in the adherent (regardless of whether these involved wearing headgear or not). This is a very strong effect. Do you have any explanation for this vast difference? It is possible that this difference will shrink or disappear if you correctly count all non-adherent AEs as non-adherent.
Thank you again for conducting these trial and for your kind attention to these questions!
We would like to thank you for your insightful and interesting comment.
Regarding the first point, we presented the 28.5% to illustrate the relative difference in total absolute fat (kg) change between interventions, so the reader could have information about the relative difference between groups. We would like to highlight that it was only possible to perform this analysis using the within group changes, since the change between group analysis was showed in absolute values.
About the second point, it was not our purpose to analyse lean body mass; however, we agree that this topic is very important for health and athletic performance purposes. This is an unanswered question and we are performing studies to test the effects of interval training on lean body mass to help shedding light in the topic.
In this article the authors discuss their analysis of 21 female and 22 male athletic events. Testing all 43, they find 3 events significant with p<0.05. When testing 43 events, the expectation is that a well-calibrated statistical test will produce 2 false positives with random data, on average, due to the definition of the p-value. The odds of producing 3 false positives are also rather high; for normally distributed simulated data under the null, I found 3 or more false positives approximately 1/3 of the time such an analysis is performed, see here for a simulation notebook: https://github.com/davidasiegel/False-Positive-Rate-for-Multiple-Tests-i....
This is why adjustments for multiple comparisons needs to be performed. It was neglected in their initial study and neglected again in this study. In the 2017 study they state, "These different athletic events were considered as distinct independent analyses and adjustment for multiple comparisons was not required." This doesn't make sense to me; if the analyses are distinct, then all the more reason to correct for multiple comparisons. If a Bonferroni correction were performed, none of the p-values would test significant at the level of the study (p<0.05/43 = 0.001). Therefore I do not see why there is any reason to reject the null hypothesis for any of these results.
In this article the authors discuss their analysis of 21 female and 22 male athletic events. Testing all 43, they find 3 events significant with p<0.05. When testing 43 events, the expectation is that a well-calibrated statistical test will produce 2 false positives with random data, on average, due to the definition of the p-value. The odds of producing 3 false positives are also rather high; for normally distributed simulated data under the null, I found 3 or more false positives approximately 1/3 of the time such an analysis is performed, see here for a simulation notebook: https://github.com/davidasiegel/False-Positive-Rate-for-Multiple-Tests-i....
This is why adjustments for multiple comparisons needs to be performed. It was neglected in their initial study and neglected again in this study. In the 2017 study they state, "These different athletic events were considered as distinct independent analyses and adjustment for multiple comparisons was not required." This doesn't make sense to me; if the analyses are distinct, then all the more reason to correct for multiple comparisons. If a Bonferroni correction were performed, none of the p-values would test significant at the level of the study (p<0.05/43 = 0.001). Therefore I do not see why there is any reason to reject the null hypothesis for any of these results.
Performing a series of tests and then re-testing the subset of events with the most significant results is also poor statistical practice that doesn't add meaning to the study.
In conclusion, taking into account the total number of tests that were performed (43), the results fail to be significant at the level of the study.
For readers who are following the debate about how training load may relate to injury, Dr Johann Windt considers the implication of the correlation that is pointed out here. Thanks to all the authors. k2
The study raises two questions that one of the authors might be able to help with:
First, the authors report both within group and between group changes in body fat in the abstract. But it is unclear why the authors chose the within-group changes (28% fat loss) as the study conclusion than the between-group change.
The within group change showed a fat loss of 0.45 kg (28%) in favor of interval training (IT), while the between-group changes showed a large difference of 2.28 kg of fat loss in favor of IT. Considering the large difference in fat loss, and some studies recommending to avoid within group differences in meta-analysis, it would be helpful if the authors could comment on this.
Second, maintaining lean body mass (LBM) is one of the primary reasons to include exercise as part of a weight loss strategy. So it is not clear why the authors chose not to include lean body mass as one of the outcomes. It would have certainly helped the reader to make a decision regarding the choice of exercise for weight loss.
Finally, congratulations to all the authors for asking a very relevant question!.
We thank Freke et al. (1) for their systematic review about physical impairments in patients with symptomatic femoroacetabular impingement, nonetheless we have some remarks about methods and results of the article, in particular for range of motion (ROM) outcome.
A meta-analysis of ROM was performed without reporting an overall estimate. Taking into account the amount of studies included and their information, a meta-analysis should have been accomplished. Nonetheless, authors concluded that individuals with symptomatic FAI demonstrated no difference in hip ROM in any direction of movement. This conclusion was unexpected taking into account the findings reported in the primary studies included), and in the previous systematic review published in 2015 (2), that showed instead a reduced ROM.
This discrepancy in literature is already discussed by the Warwick agreement (3), where authors stated that “the evidence on hip range of motion (ROM) in FAI syndrome is surprisingly contradictory” due to contrasting published systematic reviews (1) (2).
Therefore, we checked the accuracy of results reported, analyzing the data reported for every movement assessed in primary studies comparing those reported in this systematic review. We noted some issues in the represented forest plots.
Firstly, some included studies (4), (5), (6), (7) were reported twice in the meta-analysis for different times points or reporting double data of the same patients obtained by two...
We thank Freke et al. (1) for their systematic review about physical impairments in patients with symptomatic femoroacetabular impingement, nonetheless we have some remarks about methods and results of the article, in particular for range of motion (ROM) outcome.
A meta-analysis of ROM was performed without reporting an overall estimate. Taking into account the amount of studies included and their information, a meta-analysis should have been accomplished. Nonetheless, authors concluded that individuals with symptomatic FAI demonstrated no difference in hip ROM in any direction of movement. This conclusion was unexpected taking into account the findings reported in the primary studies included), and in the previous systematic review published in 2015 (2), that showed instead a reduced ROM.
This discrepancy in literature is already discussed by the Warwick agreement (3), where authors stated that “the evidence on hip range of motion (ROM) in FAI syndrome is surprisingly contradictory” due to contrasting published systematic reviews (1) (2).
Therefore, we checked the accuracy of results reported, analyzing the data reported for every movement assessed in primary studies comparing those reported in this systematic review. We noted some issues in the represented forest plots.
Firstly, some included studies (4), (5), (6), (7) were reported twice in the meta-analysis for different times points or reporting double data of the same patients obtained by two instruments to measure ROM. The number of observations in the analysis should match the number of individuals (unit) that are randomized or allocated: reporting twice data from the same patients is not appropriate, resulting in an error of unit of analysis inflating the sample size. Secondarily, two studies were included in the meta-analyses selecting an inappropriate control group. In particular, the control group in one study was represented by post-surgery impingement population instead of the healthy control, even if the displaced forest plot indicated the comparisons “pre versus control” (8). Analogously, an inappropriate population was selected in another study (6) for the femoroacetabular group where authors chose arbitrarily data from the subgroup of patients with cam impingement, while they should have chosen data reported from combined cam or pincer impingement, since the aim of the review was to measure physical impairment in patients with any kind of FAI.
We are confident with data published in this review of Freke et al (1): checking data from all primary studies we confirmed that data were correctly reported by authors in tables 1 and 2, however inconsistency between data extraction and analysis is present. It is unclear the estimate and related confidence intervals reported in each forest plot. Reasons could be related to authors obtain missing outcome data from contacting primary authors, nevertheless, this information should be transparently reported in the systematic review. For every direction of movement analyzed, the confidence intervals appear to be too large and, in some cases, the SMD calculated seemed to be wrong, making non-significant some of the significant differences published in the primary studies.
Considering the errors in selective reporting and aggregation of data we claim the conclusion of the review about ROM is not correct. Even if the authors did not show the diamond of the overall results in forest plots, they concluded that there is no statistically significant deficit in ROM in any plane of movement, and visually impressed stakeholders by the forest plots that sustain this conclusion. Anyway, we correctly re-run the analyses and we found that there is a statistically significant ROM deficit in flexion, abduction, external rotation and internal rotation in patients with FAI versus controls.
Errors in published systematic reviews are possible (9), but they can limit validity of conclusions of systematic reviews and resulting statements, agreements or clinical guidelines. We understand that peer-reviewers cannot check every calculation or data analysis but the importance of meta-analysis in the hierarchy of evidence need high level of attention and hopefully technical support to the reviewers to avoid relevant mistakes. At the same time, to improve transparency and help the review process, Journals should require authors to submit the raw data and to share the dataset used for the analyses, when they are not reported in the forest plot.
References
1. Freke MD, Kemp J, Svege I, Risberg MA, Semciw A, Crossley KM. Physical impairments in symptomatic femoroacetabular impingement: A systematic review of the evidence. Br J Sports Med. 2016;50(19):1180.
2. Diamond LE, Dobson FL, Bennell KL, Wrigley T V., Hodges PW, Hinman RS. Physical impairments and activity limitations in people with femoroacetabular impingement: A systematic review. Br J Sports Med. 2015;49(4):230–42.
3. Griffin DR, Dickenson EJ, O’Donnell J, Agricola R, Awan T, Beck M, et al. The Warwick Agreement on femoroacetabular impingement syndrome (FAI syndrome): An international consensus statement. Br J Sports Med. 2016;50(19):1169–76.
4. Nussbaumer S, Leunig M, Glatthorn JF, Stauffacher S, Gerber H, Maffiuletti NA. Validity and test-retest reliability of manual goniometers for measuring passive hip range of motion in femoroacetabular impingement patients. BMC Musculoskelet Disord. 2010;11(194).
5. Bedi A, Dolan M, Hetsroni I, Magennis E, Lipman J, Buly R, et al. Surgical Treatment of Femoroacetabular Impingement Improves Hip Kinematics. Am J Sports Med. 2011;39(1 Suppl):43–9.
6. Harris-hayes M, Mueller MJ, Sahrmann SA, Bloom NJ, Steger-may K, Clohisy MAJC, et al. Persons With Chronic Hip Joint Pain Exhibit Reduced Hip Muscle Strength. J Orthop Sport Phys Ther. 2014;44(11):890–8.
7. Audenaert EA, Peeters I, Vigneron L, Baelde N, Pattyn C. Hip Morphological Characteristics and Range of Internal Rotation in Femoroacetabular Impingement. Am J Sports Med. 2012;40(6):1329–36.
8. Kubiak-Langer M, Tannast M, Murphy SB, Siebenrock K a, Langlotz F. Range of motion in anterior femoroacetabular impingement. Clin Orthop Relat Res [Internet]. 2007 May [cited 2012 Mar 9];458(458):117–24. Available from: http://www.ncbi.nlm.nih.gov/pubmed/17206102
9. Yip R, Islami F, Zhao S, Tao M, Yankelevitz DF, Boffetta P. Errors in systematic reviews : an example of computed tomography screening for lung cancer. Eur J Cancer Prev. 2014;23(i):43–8.
The BJSM recently rejected our request of retraction or errata corrige of the editorials by Blanch and Gabbett(1) and Gabbett (2) presenting the relation between the Acute:Chronic Workload Ratio (ACWR) and likelihood of injuries. The preprint and a list of some of the errors presented in that figure can be found here: https://osf.io/preprints/sportrxiv/gs8yu/. In challenging our request, it was underlined several times by the Editor in Chief of BJSM that the “model” was presented as illustrative only, and this seems to make errors acceptable like if the editorials are a “safe zone” where for illustrative purposes it is possible to bend and even break scientific rules and methods, presenting models using unpublished and uncontrollable data.
However, the reason of this communication is to warn the members of the consensus (and readers) that the ACWR model published in the IOC consensus(3) as a validated model has in fact not been validated at all: [page 1034] “The model has currently been validated through data from three different sports (Australian football, cricket and rugby league)(187)”. The reference 187 is one of the two editorials(1) for which we asked the retraction. So on one side the Editor in Chief insists that it is just an illustrative (flawed) model, but on the other side the same Editor in Chief, co-author (with one of the proponents of the model) of the IOC consensus wrote and published that it...
Show MoreDear Drs. McGuine, Hetzel, and Kliethermes,
Thank you for your thorough response to my initial comment.
I am wondering if you could help me understand the new AE-level as-treated analysis you have done in response to Point 2. This accounts for all non-compliant AEs among all athletes, correct? If I understood you correctly, there were somewhat more than the 711 non-compliant AEs reported in the paper and which you reported in your response to Point 4, correct?
What would be very helpful to see is a.) the number of AEs and b.) the number of SRCs that occurred during those AEs for each of the following groups when considering any non-compliant AE, not just ones from athletes who suffered an SRC while non-compliant or were non-compliant >50% of the time:
Assigned HG/Did Not Wear:
Assigned HG/Did Wear:
Assigned No HG/Did Not Wear:
Assigned No HG/Did Wear:
Thank you again for your thorough response.
After careful appraisal and following our own investigations, we are concerned that the article “Is interval training the magic bullet for fat loss? A systematic review and meta-analysis comparing moderate-intensity continuous training with high-intensity interval training (HIIT)” [1] may have some data extraction and analysis errors that warrant further review by the editor and authors, and which more concerningly, may impact the original conclusions of the article.
We were initially concerned about the reported results within the Thomas et al. paper [2], particularly the biological plausibility of a mean between-group fat-loss difference of 13.44 kg over 12 weeks. Given that the authors did not report any study-level data, we decided to investigate the effect size within this paper. However, this study [2] did not report any fat mass data, only % body fat data. Given that the authors of the review [1] reported “When studies provided insufficient data for inclusion in the meta-analysis (five studies), the corresponding authors were contacted via email to determine whether additional data could be provided; however, no corresponding authors responded.”, it is unclear how an unpublished mean difference of -13.44 kg in favour of HIIT/SIT could be presented within the fat mass analysis of this review. Furthermore, when reviewing another of the included studies [3], we found that fat mass data were reported, but not included in the current meta-analysis [1]. Given the m...
Show MoreTo: The British Journal Sports Medicine
We are grateful for Dr. Binney’s interest in our study and his consideration of a portion of the results presented in the manuscript.
Listed below are our responses to each of the concerns raised in the letter.
1. In the as-treated analysis you have a very strange result. Your multivariate risk ratio (which is actually a rate ratio) is 0.63 for everyone overall, 0.64 for females, and 0.93 for males. The result for everyone should be between the results for males and females. Can you please clarify how you got these results, including the exact model(s) you used and how you calculated the rate ratios? Did you use a group*sex interaction term to get the sex-specific results?
Response: We thank you for noticing the mathematical inconsistency in Table 4 rate ratio results for the as-treated analyses. You are correct that if these results were from one model, the overall rate ratio estimate would need to be in-between the male/female estimates. We should note that these were actually 3 separate mixed-effects models: (1) the overall model adjusting for all variables including sex, (2) female sub-group model adjusting for all variables –excluding sex, and (3) male sub-group model adjusting for all variables –excluding sex. We apologize that the footnote in the table is unclear in this regard. We did attempt to use interaction models for this analyses, but did not achieve consistent convergence. As such, we opt...
Show MoreDear Dr. McGuine et al,
I'd like to commend you on running a large RCT on such an important topic (assessing the purported effectiveness of concussion-reduction technologies). Unfortunately I have some concerns about some aspects of your data and analysis, particularly the as-treated analysis in Table 4and your reported adherence numbers. I am hoping you can clarify these concerns and re-do parts of your analysis.
1. In the as-treated analysis you have a very strange result. Your multivariate risk ratio (which is actually a rate ratio) is 0.63 for everyone overall, 0.64 for females, and 0.93 for males. The result for everyone should be between the results for males and females. Can you please clarify how you got these results, including the exact model(s) you used and how you calculated the rate ratios? Did you use a group*sex interaction term to get the sex-specific results?
2. How you defined the as-treated group is concerning. You state that you only re-classified a subject if they spent >50% of their time in their non-assigned group OR if they were concussed while in their non-assigned group. This approach will bias the results of your as-treated analysis as you are deliberately misclassifying the AEs of people who do not get hurt and the non-concussed AEs of those who do. You need to classify every AE, rather than each athlete, as headgear or no headgear and repeat the as-treated analysis. Otherwise this analysis is highly questionable and...
Show MoreDear Dr. Anoop Balachandran
We would like to thank you for your insightful and interesting comment.
Regarding the first point, we presented the 28.5% to illustrate the relative difference in total absolute fat (kg) change between interventions, so the reader could have information about the relative difference between groups. We would like to highlight that it was only possible to perform this analysis using the within group changes, since the change between group analysis was showed in absolute values.
About the second point, it was not our purpose to analyse lean body mass; however, we agree that this topic is very important for health and athletic performance purposes. This is an unanswered question and we are performing studies to test the effects of interval training on lean body mass to help shedding light in the topic.
Best regards.
In this article the authors discuss their analysis of 21 female and 22 male athletic events. Testing all 43, they find 3 events significant with p<0.05. When testing 43 events, the expectation is that a well-calibrated statistical test will produce 2 false positives with random data, on average, due to the definition of the p-value. The odds of producing 3 false positives are also rather high; for normally distributed simulated data under the null, I found 3 or more false positives approximately 1/3 of the time such an analysis is performed, see here for a simulation notebook: https://github.com/davidasiegel/False-Positive-Rate-for-Multiple-Tests-i....
This is why adjustments for multiple comparisons needs to be performed. It was neglected in their initial study and neglected again in this study. In the 2017 study they state, "These different athletic events were considered as distinct independent analyses and adjustment for multiple comparisons was not required." This doesn't make sense to me; if the analyses are distinct, then all the more reason to correct for multiple comparisons. If a Bonferroni correction were performed, none of the p-values would test significant at the level of the study (p<0.05/43 = 0.001). Therefore I do not see why there is any reason to reject the null hypothesis for any of these results.
Perfor...
Show MoreFor readers who are following the debate about how training load may relate to injury, Dr Johann Windt considers the implication of the correlation that is pointed out here. Thanks to all the authors. k2
https://bjsm.bmj.com/content/early/2018/05/28/bjsports-2017-098925
The study raises two questions that one of the authors might be able to help with:
First, the authors report both within group and between group changes in body fat in the abstract. But it is unclear why the authors chose the within-group changes (28% fat loss) as the study conclusion than the between-group change.
The within group change showed a fat loss of 0.45 kg (28%) in favor of interval training (IT), while the between-group changes showed a large difference of 2.28 kg of fat loss in favor of IT. Considering the large difference in fat loss, and some studies recommending to avoid within group differences in meta-analysis, it would be helpful if the authors could comment on this.
Second, maintaining lean body mass (LBM) is one of the primary reasons to include exercise as part of a weight loss strategy. So it is not clear why the authors chose not to include lean body mass as one of the outcomes. It would have certainly helped the reader to make a decision regarding the choice of exercise for weight loss.
Finally, congratulations to all the authors for asking a very relevant question!.
We thank Freke et al. (1) for their systematic review about physical impairments in patients with symptomatic femoroacetabular impingement, nonetheless we have some remarks about methods and results of the article, in particular for range of motion (ROM) outcome.
Show MoreA meta-analysis of ROM was performed without reporting an overall estimate. Taking into account the amount of studies included and their information, a meta-analysis should have been accomplished. Nonetheless, authors concluded that individuals with symptomatic FAI demonstrated no difference in hip ROM in any direction of movement. This conclusion was unexpected taking into account the findings reported in the primary studies included), and in the previous systematic review published in 2015 (2), that showed instead a reduced ROM.
This discrepancy in literature is already discussed by the Warwick agreement (3), where authors stated that “the evidence on hip range of motion (ROM) in FAI syndrome is surprisingly contradictory” due to contrasting published systematic reviews (1) (2).
Therefore, we checked the accuracy of results reported, analyzing the data reported for every movement assessed in primary studies comparing those reported in this systematic review. We noted some issues in the represented forest plots.
Firstly, some included studies (4), (5), (6), (7) were reported twice in the meta-analysis for different times points or reporting double data of the same patients obtained by two...
Pages