We congratulate O’Keeffe et al. [1] for their research on the comparative efficacy of Cognitive Functional Therapy (CFT) and physiotherapist-delivered group-based exercise and education for individuals with chronic low back pain (CLBP). Their study shows that “CFT can reduce disability, but not pain, at 6 months compared with the group-based exercise and education intervention”. The CFT approach is very promising and has caught the attention and interest of a number of clinicians worldwide in the management of non‐specific disabling CLBP. The study by O’Keeffe et al. [1] has methodological strengths compared to a previous clinical trial by Vibe Fersum et al. [2,3] such as a higher sample size which means it is less vulnerable to type-II error. Nonetheless, some shortcomings threaten substantially the risk of bias and type I error that are worthy of further discussion.
The first is the choice of three physiotherapists for delivering both interventions in this trial. This aspect was considered by O’Keeffe et al. [1] as a strength of the study because it arguably minimized differences in clinicians’ expertise and communication style. Notwithstanding, this fact could also have decreased the treatment effect on the control group. It is important to remember that the trial was performed by the research group that not only developed CFT but also has trained the physiotherapists on such an approach, and thus the enthusiasm and motivation to apply the intervention on the CFT...
We congratulate O’Keeffe et al. [1] for their research on the comparative efficacy of Cognitive Functional Therapy (CFT) and physiotherapist-delivered group-based exercise and education for individuals with chronic low back pain (CLBP). Their study shows that “CFT can reduce disability, but not pain, at 6 months compared with the group-based exercise and education intervention”. The CFT approach is very promising and has caught the attention and interest of a number of clinicians worldwide in the management of non‐specific disabling CLBP. The study by O’Keeffe et al. [1] has methodological strengths compared to a previous clinical trial by Vibe Fersum et al. [2,3] such as a higher sample size which means it is less vulnerable to type-II error. Nonetheless, some shortcomings threaten substantially the risk of bias and type I error that are worthy of further discussion.
The first is the choice of three physiotherapists for delivering both interventions in this trial. This aspect was considered by O’Keeffe et al. [1] as a strength of the study because it arguably minimized differences in clinicians’ expertise and communication style. Notwithstanding, this fact could also have decreased the treatment effect on the control group. It is important to remember that the trial was performed by the research group that not only developed CFT but also has trained the physiotherapists on such an approach, and thus the enthusiasm and motivation to apply the intervention on the CFT group could have been considerably greater. Likewise, if the physiotherapists providing the group-based exercise were not involved in any kind of CFT training and had a strong belief in their intervention, one can argue that the performance of the latter group could have been better. Also, the CFT group received the intervention for an average (SD) of 13.7 (10.9) weeks, while the comparison group received treatment for just 4.4 (2.4) weeks. The same therapist applying both interventions combined with the longer period of the CFT group treatment may have generated performance bias.
The second is the unblinded assessment of the outcomes immediately postintervention. Since CFT therapists had more one-to-one time with each patient and therefore had more opportunities to enhance the therapeutic alliance, this could have influenced patients’ behavior when filling the outcome assessment questionnaires in their presence, arguably not only in the postintervention assessment but also in the blinded follow up periods at 6 and 12-months.
Third, apparently, the authors were not concerned with the multiple primary outcomes (n = 2; pain and disability) and endpoints (n = 4; at 8-14 weeks; 6, 12 and 36 months). The problem of multiple testing is that the overall type-I (false positive) error is much greater than 5%. Assuming a true null hypothesis for all the 6 (independent) effects being tested, the probability that no false positives occur in 6 tests equals (0.95)^6 and hence the probability that at least one false positive occurs is 1–(0.95)^6 = 0.26. This also highlights a discrepancy regarding the primary outcomes between the pre-registered trial protocol on ClinicalTrial.gov (https://clinicaltrials.gov/ct2/show/NCT02145728) and the published protocol by the same authors in 2015 [4]. Notably, O’Keeffe et al. [1] gave up on the postintervention follow up results, which was a primary outcome according to the pre-registered protocol. Therefore, it is mandatory to discuss this aspect and to reduce the number of primary outcomes in future studies, if possible, to 1; in this case, disability at 6 months after randomization would have been recommended.
Finally, 37% of the participants were lost to follow up in the first primary outcome endpoint (postintervention), 28% of loss of follow up on the second primary outcome endpoint (6 months), and 31% on the third (12 months). Although the unmeasured bias was recognized by O’Keeffe et al. [1], it seems that there is a systematic high loss of follow up in CFT clinical trials conducted by the research group of the developers of the method. Similarly to the first trial of Vibe Fersum et al. [2], this trial did not provide high quality evidence about the efficacy of CFT and should be considered as an exploratory study, not confirmatory enough to generate recommendation. Therefore, the next CFT trials should focus on improving the methodological quality. It is recommended that future studies avoid a loss of follow up higher than 15%, blind the assessors, establish up to 2 primary outcomes, and special attention be paid to the quality and duration of the treatment provided to the comparison group.
References
1 O’Keeffe M, O’Sullivan P, Purtill H, et al. Cognitive functional therapy compared with a group-based exercise and education intervention for chronic low back pain: a multicentre randomised controlled trial (RCT). Br J Sports Med 2019;:bjsports-2019-100780. doi:10.1136/bjsports-2019-100780
2 Vibe Fersum K, O’Sullivan P, Skouen JS, et al. Efficacy of classification-based cognitive functional therapy in patients with non-specific chronic low back pain: a randomized controlled trial. Eur J Pain 2013;17:916–28. doi:10.1002/j.1532-2149.2012.00252.x
3 Fersum KV, Smith A, Kvåle A, et al. Cognitive Functional Therapy in patients with Non Specific Chronic Low Back Pain A randomized controlled trial 3-year follow up. Eur J Pain 2019;:ejp.1399. doi:10.1002/ejp.1399
4 O’Keeffe M, Purtill H, Kennedy N, et al. Individualised cognitive functional therapy compared with a combined exercise and pain education class for patients with non-specific chronic low back pain: study protocol for a multicentre randomised controlled trial. BMJ Open 2015;5:e007156. doi:10.1136/bmjopen-2014-007156
Here is my simple response to this absurd proposal. If drugs help those who are not as genetically advantaged to be more competitive with those who are, shall we prohibit the genetically advantaged from taking them? Otherwise, you create the situation where all athletes must take these drugs just to maintain the status quo. Athletes who prefer not to use drugs would suffer the most. Since drug use monitoring will be required anyway for safety, let's prohibit their use as much as possible. Allowing their use only benefits the pharmaceutical companies who sell the drugs. Sports would becomes less about athletic ability and more about who can come up with the best drug formula for competitive success.
Dear editor,
We have read with great interest the article by Wheeler et al1 showing distinct effects of exercise with and without breaks in sitting on cognition. In this study, they also demonstrated that both activity conditions increase serum brain-derived neurotrophic growth factor (BDNF) levels. Although we highly appreciate the efforts of the authors to explore potential mechanisms, we suggest that the followings need to be addressed.
BDNF is an important member of the neurotrophic factors family which enhances neuronal development and plasticity. It is synthesized as the N-glycosylated precursor (brain-derived neurotrophic factor precursor, proBDNF), and secreted into cell matrix processed by Golgi complex. Additionally, BDNF is a novel kind of myokines produced by skeletal muscle after the muscle contraction immediately. Hayashi and coworkers2 observed that both exercise and electrical muscle stimulation could increase the mRNA and protein expression of BDNF in skeletal muscle of rats. In addition, exercise could also enhance gene expression of BDNF and other neuroprotective factors in hippocampus via peroxisome proliferator-activated receptor gamma coactivator-1α-fibronectin type III domain-containing protein 5/irisin (PGC-1α-FNDC5/irisin) pathway.3
BDNF has been reported to play a pivotal role in the improvement of learning and memory function, which might be associated with the phosphorylation of tropomyosin-related kinase B (TrkB) in cognitive-...
Dear editor,
We have read with great interest the article by Wheeler et al1 showing distinct effects of exercise with and without breaks in sitting on cognition. In this study, they also demonstrated that both activity conditions increase serum brain-derived neurotrophic growth factor (BDNF) levels. Although we highly appreciate the efforts of the authors to explore potential mechanisms, we suggest that the followings need to be addressed.
BDNF is an important member of the neurotrophic factors family which enhances neuronal development and plasticity. It is synthesized as the N-glycosylated precursor (brain-derived neurotrophic factor precursor, proBDNF), and secreted into cell matrix processed by Golgi complex. Additionally, BDNF is a novel kind of myokines produced by skeletal muscle after the muscle contraction immediately. Hayashi and coworkers2 observed that both exercise and electrical muscle stimulation could increase the mRNA and protein expression of BDNF in skeletal muscle of rats. In addition, exercise could also enhance gene expression of BDNF and other neuroprotective factors in hippocampus via peroxisome proliferator-activated receptor gamma coactivator-1α-fibronectin type III domain-containing protein 5/irisin (PGC-1α-FNDC5/irisin) pathway.3
BDNF has been reported to play a pivotal role in the improvement of learning and memory function, which might be associated with the phosphorylation of tropomyosin-related kinase B (TrkB) in cognitive-related brain regions. BDNF and its specific receptor TrkB are widely expressed in the animal neural tissues. After binding to TrkB, BDNF could activate mitogen-activated protein kinase (MAPK), phosphatidylinositol 3-kinase (PI3K), and phospholipase C-γ (PLC-γ), thereby eliciting a protective effect on neurons.4 Moreover, BDNF could also enhance long-term potentiation (LTP) via regulating synaptic transmission,4 and the lack or dysfunction of BDNF would be accompanied by severe impairments in learning and memory function.
Very recently, proBDNF, previously considered as a transitional form, could be directly secreted into cell matrix without being cleaved, possessing physiologic functions that are distinct from mature BDNF.5 ProBDNF could specifically bind to p75 neurotrophin receptor (p75NTR), a member of tumor necrosis factor receptor superfamily, and activate c-Jun amino terminal kinase (JNK) pathway to up-regulate p53 expression, finally leading to apoptosis.5 Importantly, proBDNF could also impair the learning and memory function by affecting neurotransmitter release and inhibiting axonal outgrowth.5 However, further studies have indicated that the major pro-apoptotic signal stimulated by proBDNF-p75NTR would be largely suppressed by activation of BDNF-TrkB and the downstream signaling.4 More interestingly, p75NTR could promote the efficacy of BDNF-TrkB signaling, thus improving neuronal survival.4 A study by Luo et al6 suggested that aerobic exercise could increase BDNF/proBDNF ratio, and relatively inhibit the proBDNF-p75NTR pathway, which exerts a protective action against apoptosis.
Taken together, exercise could activate BDNF expression not only in skeletal muscle but also in brain, which might be conducive to enhance BDNF-trkB-mediated neuroprotective pathways and inhibit proBDNF-p75NTR-mediated pro-apoptotic response, finally improving cognitive function. Further detailed studies on this field are greatly needed.
Competing interests
None declared.
Contributions
All the authors conceived the scientific ideas, critically reviewed, approved the final version.
Provenance and peer review
Not commissioned; externally peer reviewed.
Finding sources
This work was supported by grants from the Program of Bureau of Science and Technology Foundation of Changzhou (CJ20179028), Major Science and Technology Project of Changzhou Municipal Commission of Health and Family Planning (ZD201407, ZD201505, ZD201601) and "333 Project" (BRA2016122) of Jiangsu Province.
REFERENCES
1 Wheeler MJ, Green DJ, Ellis KA, et al. Distinct effects of acute exercise and breaks in sitting on working memory and executive function in older adults: a three-arm, randomised cross-over trial to evaluate the effects of exercise with and without breaks in sitting on cognition. Br J Sports Med 2019.
2 Hayashi N, Himi N, Nakamura-Maruyama E, et al. Improvement of motor function induced by skeletal muscle contraction in spinal cord-injured rats. Spine J 2019;19:1094-105.
3 Wrann CD. FNDC5/irisin - their role in the nervous system and as a mediator for beneficial effects of exercise on the brain. Brain Plast 2015;1:55-61.
4 Kowianski P, Lietzau G, Czuba E, et al. BDNF: A Key Factor with Multipotent Impact on Brain Signaling and Synaptic Plasticity. Cell Mol Neurobiol 2018;38:579-93.
5 Chen J, Zhang T, Jiao S, et al. proBDNF Accelerates Brain Amyloid-beta Deposition and Learning and Memory Impairment in APPswePS1dE9 Transgenic Mice. J Alzheimers Dis 2017;59:941-9.
6 Luo L, Li C, Du X, et al. Effect of aerobic exercise on BDNF/proBDNF expression in the ischemic hippocampus and depression recovery of rats after stroke. Behav Brain Res 2019;362:323-31.
The BJSM recently rejected our request of retraction or errata corrige of the editorials by Blanch and Gabbett(1) and Gabbett (2) presenting the relation between the Acute:Chronic Workload Ratio (ACWR) and likelihood of injuries. The preprint and a list of some of the errors presented in that figure can be found here: https://osf.io/preprints/sportrxiv/gs8yu/. In challenging our request, it was underlined several times by the Editor in Chief of BJSM that the “model” was presented as illustrative only, and this seems to make errors acceptable like if the editorials are a “safe zone” where for illustrative purposes it is possible to bend and even break scientific rules and methods, presenting models using unpublished and uncontrollable data.
However, the reason of this communication is to warn the members of the consensus (and readers) that the ACWR model published in the IOC consensus(3) as a validated model has in fact not been validated at all: [page 1034] “The model has currently been validated through data from three different sports (Australian football, cricket and rugby league)(187)”. The reference 187 is one of the two editorials(1) for which we asked the retraction. So on one side the Editor in Chief insists that it is just an illustrative (flawed) model, but on the other side the same Editor in Chief, co-author (with one of the proponents of the model) of the IOC consensus wrote and published that it...
The BJSM recently rejected our request of retraction or errata corrige of the editorials by Blanch and Gabbett(1) and Gabbett (2) presenting the relation between the Acute:Chronic Workload Ratio (ACWR) and likelihood of injuries. The preprint and a list of some of the errors presented in that figure can be found here: https://osf.io/preprints/sportrxiv/gs8yu/. In challenging our request, it was underlined several times by the Editor in Chief of BJSM that the “model” was presented as illustrative only, and this seems to make errors acceptable like if the editorials are a “safe zone” where for illustrative purposes it is possible to bend and even break scientific rules and methods, presenting models using unpublished and uncontrollable data.
However, the reason of this communication is to warn the members of the consensus (and readers) that the ACWR model published in the IOC consensus(3) as a validated model has in fact not been validated at all: [page 1034] “The model has currently been validated through data from three different sports (Australian football, cricket and rugby league)(187)”. The reference 187 is one of the two editorials(1) for which we asked the retraction. So on one side the Editor in Chief insists that it is just an illustrative (flawed) model, but on the other side the same Editor in Chief, co-author (with one of the proponents of the model) of the IOC consensus wrote and published that it was a validated model. There is something of great concern here. The publication of this figure in your consensus has given that model the credit it does not deserve. Other than not being validated, the errors in developing that figure are evident and I am sure the members can easily realize this by reading the original editorials proposing the model in the first place or by reading the problems in our request of retraction or errata. I believe that the consensus members gave too much credit to some participants involved in the consensus that proposed the model as validated. I further underline that the ACWR model you have presented also shows that if you taper before a competition or you complete a recovery week (i.e. you are outside the “sweet spot”), you are at higher risk of injuries. This is, unfortunately, an exemplificative “practical” translation of one part of the model you have contributed (unknowingly) to be popularized through including it in a consensus statement. The problem is that the metric (ACWR) is also deceptive and the members probably did not realize the meaning and hence interpretation of the ratio and figure. Furthermore, the ACWR-injury relation it is not established and in the literature various and contrasting relations can be found. Clearly more studies, hopefully with lower risk of bias, are needed. Therefore, since I value your contribution to better science and evidence-based recommendations, it would be appreciated to know whether at least the IOC panel will acknowledge and advise the reader the fact that the model has not been validated and potentially misleading.
References
1. Blanch P, Gabbett TJ. Has the athlete trained enough to return to play safely? The acute:chronic workload ratio permits clinicians to quantify a player's risk of subsequent injury. Br J Sports Med 2016;50(8):471-5. doi: 10.1136/bjsports-2015-095445
2. Gabbett TJ. The training-injury prevention paradox: should athletes be training smarter and harder? Br J Sports Med 2016;50(5):273-80. doi: 10.1136/bjsports-2015-095788
3. Soligard T, Schwellnus M, Alonso JM, et al. How much is too much? (Part 1) International Olympic Committee consensus statement on load in sport and risk of injury. Br J Sports Med 2016;50(17):1030-41. doi: 10.1136/bjsports-2016-096581
Thank you for your thorough response to my initial comment.
I am wondering if you could help me understand the new AE-level as-treated analysis you have done in response to Point 2. This accounts for all non-compliant AEs among all athletes, correct? If I understood you correctly, there were somewhat more than the 711 non-compliant AEs reported in the paper and which you reported in your response to Point 4, correct?
What would be very helpful to see is a.) the number of AEs and b.) the number of SRCs that occurred during those AEs for each of the following groups when considering any non-compliant AE, not just ones from athletes who suffered an SRC while non-compliant or were non-compliant >50% of the time:
Assigned HG/Did Not Wear:
Assigned HG/Did Wear:
Assigned No HG/Did Not Wear:
Assigned No HG/Did Wear:
After careful appraisal and following our own investigations, we are concerned that the article “Is interval training the magic bullet for fat loss? A systematic review and meta-analysis comparing moderate-intensity continuous training with high-intensity interval training (HIIT)” [1] may have some data extraction and analysis errors that warrant further review by the editor and authors, and which more concerningly, may impact the original conclusions of the article.
We were initially concerned about the reported results within the Thomas et al. paper [2], particularly the biological plausibility of a mean between-group fat-loss difference of 13.44 kg over 12 weeks. Given that the authors did not report any study-level data, we decided to investigate the effect size within this paper. However, this study [2] did not report any fat mass data, only % body fat data. Given that the authors of the review [1] reported “When studies provided insufficient data for inclusion in the meta-analysis (five studies), the corresponding authors were contacted via email to determine whether additional data could be provided; however, no corresponding authors responded.”, it is unclear how an unpublished mean difference of -13.44 kg in favour of HIIT/SIT could be presented within the fat mass analysis of this review. Furthermore, when reviewing another of the included studies [3], we found that fat mass data were reported, but not included in the current meta-analysis [1]. Given the m...
After careful appraisal and following our own investigations, we are concerned that the article “Is interval training the magic bullet for fat loss? A systematic review and meta-analysis comparing moderate-intensity continuous training with high-intensity interval training (HIIT)” [1] may have some data extraction and analysis errors that warrant further review by the editor and authors, and which more concerningly, may impact the original conclusions of the article.
We were initially concerned about the reported results within the Thomas et al. paper [2], particularly the biological plausibility of a mean between-group fat-loss difference of 13.44 kg over 12 weeks. Given that the authors did not report any study-level data, we decided to investigate the effect size within this paper. However, this study [2] did not report any fat mass data, only % body fat data. Given that the authors of the review [1] reported “When studies provided insufficient data for inclusion in the meta-analysis (five studies), the corresponding authors were contacted via email to determine whether additional data could be provided; however, no corresponding authors responded.”, it is unclear how an unpublished mean difference of -13.44 kg in favour of HIIT/SIT could be presented within the fat mass analysis of this review. Furthermore, when reviewing another of the included studies [3], we found that fat mass data were reported, but not included in the current meta-analysis [1]. Given the marginal level of significance for the fat mass outcome in the review (MD, CI; -2.28, [-4.00, -0.56]), it is possible that if these data had been excluded/included the primary conclusion of this article (that HIIT training resulted in superior fat mass loss compared to moderate intensity aerobic training), may be altered.
Additionally, upon review of the first two data points in the % body fat plot, we also have concerns regarding the calculated effect sizes. We are unable to confirm the accuracy of the calculations, given the lack of study-level data reported and the absence of effect size (ES) calculation methods in either the review article methods or the registered protocol on PROSPERO (CRD42018089427). Whilst the Thomas et al. [2] paper reported % body fat data graphically, and we acknowledge there may be some minor differences in data points depending on which graph analysis tool was used, the Trapp et al. [3] paper reported all pre- and post- mean and standard error data, so our ES and CI calculations should be identical to those in the review if the same calculation method was used. However, our calculated ES using the pooled baseline SD for ES calculation (-3.30 [-13.36, 6.76]) varied significantly from that reported by the Viana et al (-9.59 [-16.97, -2.20]). Presuming that the authors are presenting mean difference data (as suggested in their forest plot figures), there should be no disputing this value as -3.30, based solely on study-level data. Our analyses only examined data from these two studies, so we can neither verify nor refute the results presented for the remaining 34 included studies.
Finally, we question the inclusion of one of the studies in the review. Viana et al. have included the study by Boer and Moss [4] that used 70-80% VO2peak for their moderate intensity exercise. This is outside the 40-60% VO2max ‘criteria’ for moderate intensity provided by the authors.
In conclusion, we have grave concerns about the accuracy of the data extraction and analysis, as well as study selection in the Viana et al review [1]. We would ask the editors therefore to request clarification and transparency in the methods used to derive the statistical comparisons presented in the published review. At minimum, a published correction of the article is needed, given the likelihood that this review will be highly cited by others who do not have the time or the statistical knowledge to question the findings. However, if the conclusions of the article are in fact driven by the errors in data extraction and analysis noted above, then full retraction may be indicated.
[1] Viana RB, Naves JPA, Coswig VS, et al Is interval training the magic bullet for fat loss? A systematic review and meta-analysis comparing moderate-intensity continuous training with high-intensity interval training (HIIT) Br J Sports Med. Published Online First: 14 February 2019.
[2] Thomas TR, Adeniran SB, Etheridge GL. Effects of different running programs on VO2max, percent fat, and plasma lipids. Can J Appl Sport Sci 1984;9:55–62.
[3] Trapp EG, Chisholm DJ, Freund J, et al. The effects of high-intensity intermittent exercise training on fat loss and fasting insulin levels of young women. Int J Obes 2008;32:684–91.
[4] Boer PH , Moss SJ . Effect of continuous aerobic vs. interval training on selected anthropometrical, physiological and functional parameters of adults with Down syndrome. J Intellect Disabil Res 2016;60:322–34
We are grateful for Dr. Binney’s interest in our study and his consideration of a portion of the results presented in the manuscript.
Listed below are our responses to each of the concerns raised in the letter.
1. In the as-treated analysis you have a very strange result. Your multivariate risk ratio (which is actually a rate ratio) is 0.63 for everyone overall, 0.64 for females, and 0.93 for males. The result for everyone should be between the results for males and females. Can you please clarify how you got these results, including the exact model(s) you used and how you calculated the rate ratios? Did you use a group*sex interaction term to get the sex-specific results?
Response: We thank you for noticing the mathematical inconsistency in Table 4 rate ratio results for the as-treated analyses. You are correct that if these results were from one model, the overall rate ratio estimate would need to be in-between the male/female estimates. We should note that these were actually 3 separate mixed-effects models: (1) the overall model adjusting for all variables including sex, (2) female sub-group model adjusting for all variables –excluding sex, and (3) male sub-group model adjusting for all variables –excluding sex. We apologize that the footnote in the table is unclear in this regard. We did attempt to use interaction models for this analyses, but did not achieve consistent convergence. As such, we opt...
We are grateful for Dr. Binney’s interest in our study and his consideration of a portion of the results presented in the manuscript.
Listed below are our responses to each of the concerns raised in the letter.
1. In the as-treated analysis you have a very strange result. Your multivariate risk ratio (which is actually a rate ratio) is 0.63 for everyone overall, 0.64 for females, and 0.93 for males. The result for everyone should be between the results for males and females. Can you please clarify how you got these results, including the exact model(s) you used and how you calculated the rate ratios? Did you use a group*sex interaction term to get the sex-specific results?
Response: We thank you for noticing the mathematical inconsistency in Table 4 rate ratio results for the as-treated analyses. You are correct that if these results were from one model, the overall rate ratio estimate would need to be in-between the male/female estimates. We should note that these were actually 3 separate mixed-effects models: (1) the overall model adjusting for all variables including sex, (2) female sub-group model adjusting for all variables –excluding sex, and (3) male sub-group model adjusting for all variables –excluding sex. We apologize that the footnote in the table is unclear in this regard. We did attempt to use interaction models for this analyses, but did not achieve consistent convergence. As such, we opted for sub-group analysis. It is curious that the overall rate ratio estimate does not fall within the bounds of the male and female rate ratio estimate; this is something we continue to explore. We regret not publishing the univariable results in conjunction with the multivariable results in Table 4. As noticed in the text below, the univariable results lead to estimates where the overall estimate falls in between that of the subgroup analyses. The cluster adjusted univariable rate ratios [URR (95%CI), p-value] and cox proportional hazard ratios [UHR (95%CI) p-value] comparing the incidence of SRC between the HG and NoHG groups are as follows:
Per protocol analyses; All No HG: n = 1545, SRC’s n = 68, %SRC = 4.4%. All HG: n = 1505, SRCs’ n = 62, %SRCs = 4.1% [RR = 0.96 (0.63–1.46) p = 0.855], HR = 0.99 (95%CI 0.65–1.50) p = 0.951]. Males - no HG: n = 546, SRC’s n = 8, %SRC = 1.5%. Males HG: n = 474, SRC’s n = 14, %SRC’s 3.0%, [RR 1.81 (0.63–5.18) p = 0.271], HR = 2.02 (0.70–5.80) p = 0.286]. Females-NHG: n = 999, SRC’s n = 60, %SRC = 6.0%. Females-HG: n = 1031, SRC’s n = 48, %SRC = 4.7% [RR = 0.87 (0.53-1.42) p = 0.582], [HR 0.83 (0.53-1.32) p = 0.442]
As treated analyses; All No HG; n =1546, SRC’s n = 75, %SRC = 4.9%. All HG n = 1504, SRC’s n = 55, %SRC = 3.7 [RR = 0.66 (0.41-1.08) p = 0.097], [HR = 0.80 (0.51-1.24) p = 0.315]
Males- no HG: n = 548, SRC’s n = 10, %SRC = 1.8%. Males-HG n = 472, SRC’s n = 12, %SRC = 2.5% [RR = 0.97 (0.30-3.20) p = 0.966], [HR = 1.40 (0.49-4.06) p = 0.531]
Females-NHG n = 998, SRC n = 65, %SRC = 6.5%. Females-HG n = 1032, SRC’s n = 43, %SRC = 4.2 [RR = 0.62 (0.37-1.06) p = 0.078], [HR = 0.69 (0.42-1.12) p = 0.134].
Given the fact that the univariable results are plausible, it convinces us that perhaps something is happening when adjusting for other variables or by having a random effect in each separate model that is causing the overall ‘as treated’ HG estimate to be outside the estimates for the male and female subgroup models. We will continue to look at issues of sparsity, multicolinearity, exchangeability, etc. that may be causing this disparity in the multivariable analyses.
2. How you defined the as-treated group is concerning. You state that you only re-classified a subject if they spent >50% of their time in their non-assigned group OR if they were concussed while in their non-assigned group. This approach will bias the results of your as-treated analysis as you are deliberately misclassifying the AEs of people who do not get hurt and the non-concussed AEs of those who do. You need to classify every AE, rather than each athlete, as headgear or no headgear and repeat the as-treated analysis. Otherwise this analysis is highly questionable and should be removed from the paper.
Response: For this study we considered the subjects without SRC to be adherent if they participated in more than 50% of the AE with their assigned group allocation. There are certainly limitations to this decision. One could also argue that our study team should have set the “threshold” of the percentage of exposures that met the assigned group criteria greater than 50% as we did. The 50% threshold appears arbitrary but we felt it was set to give us the most realistic picture of what was occurring at the team level during the study. We recognize that we could have set this threshold at 60%, 70%, 80% or even 90% of the athletic exposures and doing so would alter the distribution of the as-treated group. But given the high rate of compliance, we felt that our results would not sufficiently change by this decision. The distribution of compliance would have to be very different. In fact, we looked into this and determined that the classification of adherent versus non-adherent groups would not change drastically based on our definition of “as-treated”. The sensitivity of classification approach for all participants as follows: ITT HG Group (n = 1505); 0 % adherent n = 0, 1% to 10% adherent n = 0, 11% to 20% adherent n = 0, 21% to 30% adherent n = 0, 41% to 50% adherent n = 0, 51% to 60% adherent n = 0, 61% to 70% n = 3 (0.2%), 71% to 80% n = 2 (0.1%), 81% to 90% n = 33 (2.2%) Note: n = 7 of the 33 subjects sustained a SRC during one of their non-adherent AEs], 91% to 100% n = 1467 (97.5%).
ITT No HG group (n=1545); 0 % adherent n = 5 (0.3%), 1% to 10% adherent n = 0, 11% to 20% adherent n = 0, 21% to 30% adherent n = 0, 41% to 50% adherent n = 1 (0.06%), 51% to 60% adherent n = 0, 61% to 70% n = 0, 71% to 80% n = 0, 81% to 90% n = 0, 91% to 100% n = 1539 (95.4%). Note: The n = 6 non adherent participants in this group were changed to HG group in “as-treated” analyses due to non-adherence (wore their own headgear, not provided by the study team).
In the distribution results above, there are very few athletes relative to the entire sample that were affected by our definition of < 50% adherence. The second component that could change the analyzed group assignment for the as-treated analysis was having a non-adherent headgear status for the athletic exposure in which an SRC occurred. There were only 13 athletes whose group assignment was different between the ITT and as-treated analysis. Six of them changed from No HG to HG group based on adherence (seen in table above), and the other 7 changed from HG to No HG because of not wearing their headgear at time of SRC. We agree that it is a little puzzling that all 7 that changed status from HG to no HG were due to SRC. Could this be due to data collection issues? We explored this possibility and discuss further below.
We also acknowledge that our ‘as-treated’ groupings may not be optimal and the approach of assigning each athlete-exposure, rather than each student, to whether they wore headgear each day or not could be a preferred method to further analyze the as treated group. As suggested, we analyzed the data in this way for the ‘as treated’ analysis. The results of the cluster adjusted Univariable Odds Ratio [UOR (95%CI) p value] and Multivariable Odds Ratios [MVOR (95%CI) p value] comparing the incidence of SRC between the HG and NoHG groups using athletic exposure as individual unit of analysis are as follows: All HG: [UOR = 0.70 (0.45-1.07) p = 0.099], [MVOR = 0.67 (0.42-1.06) p = 0.089]. Males-HG [UOR = 1.09 (0.40-2.94) p = 0.871], [MVOR = 1.06 (0.38-2.97) p = 0.911]. Females-HG [UOR = 0.65 (0.40-1.06) p = 0.083], [MVOR = 0.67 (0.41-1.08) p = 0.101]. The cluster models were adjusted at the school level only due to singularity issues when additionally adjusting at the subject ID level
Note, this analysis method doesn’t apply to an ITT type of analysis because it allows the group assignment to change, while an ITT analysis does not. Also, this analysis method estimates an odds ratio, not a rate ratio since athletic exposures are considered the individual unit. Since there are no repeat concussions in our data, the OR should be similar to the RR. Comparing the results seen here to the published Table 4 as-treated results, does not show an appreciable difference. We have shown through the responses to these concerns that we can analyze the data in multiple different ways and the results have not changed in a manner that should bring the methods selected for publication into major scrutiny.
3. You report extremely high adherence (99.53%). Is this taking into account any non-adherence or only non-adherence from students not adhering for a majority of their AEs or suffering a concussion while non-adherent? It would be very helpful to see total non-adherence since it sounds like your ATs reported that? I would like to emphasize again that this total non-adherence is what should be used in your as-treated analysis. If 0.47% is the total non-adherence for all participants, would you be willing to share some strategies you used to secure such great adherence?
Response: We readily acknowledge the high compliance with the study once the participants were enrolled in the study, and we believe, to the best of our knowledge, this to be an accurate representation of adherence to study protocol. With that said, there are some selection effects that may have contributed to this unusually high compliance rate.
For any RCT, researchers must be concerned with compliance of the subjects to any given treatment or control group. This is even more a concern when the subjects are participants on athletic teams and or adolescents. It is entirely unrealistic to assume that each subject would be compliant for 100% of the athletic exposures. This study is no different. In fact, given our previous work with adolescent athletes, during the initial study planning, the entire study team recognized that we needed to have a plan to monitor and encourage compliance for subjects in the HG group.
To encourage compliance our plan was to stress the importance of the study during all phases of team and potential subject recruitment. Once the study began, we also stressed the importance to coaches and athletic trainers to continually monitor and report how many subjects were compliant to their assigned group.
This high compliance could be attributed to several factors but most likely due to the fact that the coaches who were highly motivated to have their teams participate in the study actually did so (and independently encouraged their athletes to fully comply with study protocol). Even though we contacted 537 teams to participate, only 33% of the female coaches and 23% of the male coaches agreed to take part in the study (see Figure 1 in manuscript).
Anecdotally, coaches who did not want the risk of being randomized to either the HG or NoHG group did not agree to participate at the onset of the study, resulting in a self-selected group of potentially highly motivated schools/coaches to be randomized. Further, motivation of the study participants should be mentioned. For example, once the school agreed to allow their teams to participate, 60% of the players in the NoHG group actually enrolled in the study while only 50% of the players from schools allocated to the HG group enrolled in the study (Figure 1).
At the soccer team recruitment and enrollment meetings, individual core study team members stressed the magnitude of the study. Potential participants (and their parents) were told the importance of being compliant with the study due to the costs and logistics involved in the data collection process. In many cases the coaches stressed these points as well. Many coaches repeated that while participation was voluntary they expected all subjects to be compliant on a daily basis throughout the season if they enrolled in the study. We anticipate but cannot fully prove that as the result of these statements during the subject recruitment meetings, the subjects who were most likely to be compliant enrolled in the study while those not likely to be compliant did not enroll in the study.
In addition, we allowed the subjects in the HG group to choose which brand of headgear to wear during the season. Each of the brands utilized of the study had unique characteristics, were comprised out of various materials and conformed to various head shapes and sizes differently. As a result, the headgear did not fit each subject to the same degree of comfort. The HG group participants were encouraged to try on each different HG brand and to pick the model they felt was the most comfortable as they would be wearing it the entire season, for every practice and competition. By allowing the participants to individually choose the HG model they would wear, we felt we could increase the likelihood the participant to be compliant throughout the season.
During the run up to and throughout the study, the PI contacted each AT collecting data reminding them the importance of compliance and to let the study team know if participants were not compliant. Coaches were contacted personally by the study PI if the school AT reported compliance was a concern for their team participants. In Figure 1, we note that of the n=1599 initially enrolled in the HG arm of the study, only n=1505 began participation (i.e. n=94 (6%) HG participants dropped out of the study after enrollment). This differed in the NHG arm where only n=15 (1%) did not participate with the team after enrollment. Presumably, most of these individuals left because they were cut from the team or quit before data collection began.
Finally, n = 59 participants chose to drop out of the study during the season. These individuals are effectively censored in the analysis (disproportionately in the HG group) and only their data up until the point of withdrawing from the study are included in n=3050. Study oversight rules and regulations prohibited us from contacting any of the players who decided to stop participating in the study. However, at the time of withdrawal, ATs recorded the reason for withdrawing from the study, if known. Most (69%) did not list a reason. We can’t say for sure, but it can be presumed a majority of the HG athletes did not want to wear the HG anymore and thus withdrew from the study. As a result, only data up to the point at which they withdrew from the study are included. Further info on study drop-out by headgear group are as follows:
Head Gear Group: n = 50 (3%) Non soccer related illness (n=1); quit team (n=10), didn’t want to wear (n=3); reason not listed (n=36). No Head Gear Group: n = 9 (0.6%) Quit the team (n=4); reason not listed (n=5)
Thus, although we do have high adherence once subjects started participating in the study, it should be noted that we do have a number of eligible subjects who did not enroll and or exited the study after enrollment. We readily acknowledged this fact and we mentioned this in the limitations section, that we are very aware that selection effect for study participants may be present. Thus selection effect, along with coaches and school ATs stressing compliance during the study, and allowing for subjects to individually choose the headgear brand they would wear are most likely the reasons for the high compliance although we have no way to definitively answer this question.
Further, we only analyzed the data reported to us by the school personnel regarding their participants. In an optimal situation, one could argue that to be most accurate to have a study team member (not school personnel) present at each of the school sites to monitor compliance. While that type of study oversight may be desirable it is entirely unrealistic given the size and scope of this study. We did perform weekly queries and periodically questioned individual school ATs if we felt the data that was reported was incongruent with the data reported from other teams. We are making the assumption that the ATs, who were incentivized for their efforts (note: the soccer programs themselves were also incentivized), accurately recorded HG use and injuries throughout the season. And although we feel relatively confident with this assumption, once we had the final data we felt we had to analyze the data as it was reported to us.
4. Finally, it seems there is an extremely high rate of concussions among the non-adherent AEs. Per Table 4 and the text of your paper, there were at least 7 concussions among the 711 non-adherent AEs for a rate of 9.85 per 1,000 AEs. For the adherent group this leaves 123 concussions in 150,466 AEs for a rate of 0.82 per 1,000 AEs. This suggests the rate of concussions in the non-adherent was at least 12-fold that in the adherent (regardless of whether these involved wearing headgear or not). This is a very strong effect. Do you have any explanation for this vast difference? It is possible that this difference will shrink or disappear if you correctly count all non-adherent AEs as non-adherent.
Response: The estimates of rate of concussion per 1,000 AEs based on non-adherent (1000*7/711 = 9.8) vs adherent (1000*123/150,446 = 0.8) is quite the difference, and worth exploring more. The distribution of concussion by ITT and As-treated assignment are as follows: ITT Assignment: NoHG and the As-treated; NoHG, No SRC = 78,558 while the Yes SRC n = 68. ITT Assignment: NoHG and the As-treated; HG: No SRC = 218 while the Yes SRC n = 0. ITT Assignment: HG and the As treated: NoHG; No SRC = 486 while the Yes SRC n = 7. ITT Assignment HG and the As- treated: HG; No SRC = 71,765 while the Yes SRC = 55.
In addition, here is the breakdown for all 7 subjects that switched from HG to no HG in the as-treated analysis because they suffered a SRC while not wearing HG. Player A; School 07 - 100% adherent from 8/15 – 09/01, then only ~50% adherent from 09/02 – 09/29 (n=13 total AEs; n=7 non-adherent). Suffers SRC on 09/29. 100% adherent after return to play. Player B; School 28 – 100% compliance from 03/20 – 05/01, then 0% compliance from 05/02 – 05/09 (n=5 AEs). Suffers SRC on 05/09. Did not return prior to end of season. Player C; School 13 – 100% adherent from 03/20 – 04/13, then 0% compliance from 04/17 - 04/18 (n=2 AEs). Suffers SRC on 04/18. Did not return prior to end of season. Player D; School 11 - 100% adherent from 03/20 – 03/30, then 0% adherent from 03/31 – 04/04 (n=3 AEs) then 100% adherent after return to play. Player E; School 22 – 100% adherent from 03/23 – 05/01, then 36% adherent from 05/02 – 05/16 (n=11 AEs; 4 non-adherent), then 100% adherent after return to play. Player F; School 29 – 100% adherent from 03/20 – 04/06, then 0% adherent from 04/07 – 04/11 (n=3 AEs), then 100% adherent from 04/12 – 05/15, then 0% adherent from 05/15 – 05/18 (n=4 AEs). Suffers SRC on 05/18 and does not return prior to end of season. Player G; School 18 – 100% adherent from 08/14 – 08/21, then 50% adherent from 08/22 – 09/05 (n=10 AEs; 5 non-adherent. Suffer SRC on 09/05. 100% adherent from return to play to end of season.
All 7 of these athletes were non-adherent for 10-20% of their total AEs. For the 6 athletes that were non-adherent but did not suffer an SRC, all of them come from different schools (01, 03, 05, 10, 13, and 33). Five of them, were non-adherent for the entire season and all had at least 30 AEs. The other, was non-adherent (i.e. wore HG, but was assigned no HG) for 33 of 57 AEs (57.9%).
By looking at the information above, it is noticed that 12 of 13 athletes came from different schools (so this anomaly isn’t athletic trainer specific). All 7 that had SRC had periods of non-adherence prior to sustaining their SRC. The latter could suggest that the anomaly isn’t due to only reporting non-adherence on days when an SRC occurred. It might be that ATs were not as compliant as we expected they were during the recording of compliance during each practice and game; however, given our efforts to promote and ensure compliance, we have no reason to not trust the accuracy of our results.
Thank you again for allowing to respond to these comments.
I'd like to commend you on running a large RCT on such an important topic (assessing the purported effectiveness of concussion-reduction technologies). Unfortunately I have some concerns about some aspects of your data and analysis, particularly the as-treated analysis in Table 4and your reported adherence numbers. I am hoping you can clarify these concerns and re-do parts of your analysis.
1. In the as-treated analysis you have a very strange result. Your multivariate risk ratio (which is actually a rate ratio) is 0.63 for everyone overall, 0.64 for females, and 0.93 for males. The result for everyone should be between the results for males and females. Can you please clarify how you got these results, including the exact model(s) you used and how you calculated the rate ratios? Did you use a group*sex interaction term to get the sex-specific results?
2. How you defined the as-treated group is concerning. You state that you only re-classified a subject if they spent >50% of their time in their non-assigned group OR if they were concussed while in their non-assigned group. This approach will bias the results of your as-treated analysis as you are deliberately misclassifying the AEs of people who do not get hurt and the non-concussed AEs of those who do. You need to classify every AE, rather than each athlete, as headgear or no headgear and repeat the as-treated analysis. Otherwise this analysis is highly questionable and...
I'd like to commend you on running a large RCT on such an important topic (assessing the purported effectiveness of concussion-reduction technologies). Unfortunately I have some concerns about some aspects of your data and analysis, particularly the as-treated analysis in Table 4and your reported adherence numbers. I am hoping you can clarify these concerns and re-do parts of your analysis.
1. In the as-treated analysis you have a very strange result. Your multivariate risk ratio (which is actually a rate ratio) is 0.63 for everyone overall, 0.64 for females, and 0.93 for males. The result for everyone should be between the results for males and females. Can you please clarify how you got these results, including the exact model(s) you used and how you calculated the rate ratios? Did you use a group*sex interaction term to get the sex-specific results?
2. How you defined the as-treated group is concerning. You state that you only re-classified a subject if they spent >50% of their time in their non-assigned group OR if they were concussed while in their non-assigned group. This approach will bias the results of your as-treated analysis as you are deliberately misclassifying the AEs of people who do not get hurt and the non-concussed AEs of those who do. You need to classify every AE, rather than each athlete, as headgear or no headgear and repeat the as-treated analysis. Otherwise this analysis is highly questionable and should be removed from the paper.
3. You report extremely high adherence (99.53%). Is this taking into account any non-adherence or only non-adherence from students not adhering for a majority of their AEs or suffering a concussion while non-adherent? It would be very helpful to see total non-adherence since it sounds like your ATs reported that? I would like to emphasize again that this total non-adherence is what should be used in your as-treated analysis.
If 0.47% is the total non-adherence for all participants, would you be willing to share some strategies you used to secure such great adherence?
4. Finally, it seems there is an extremely high rate of concussions among the non-adherent AEs. Per Table 4 and the text of your paper, there were at least 7 concussions among the 711 non-adherent AEs for a rate of 9.85 per 1,000 AEs. For the adherent group this leaves 123 concussions in 150,466 AEs for a rate of 0.82 per 1,000 AEs. This suggests the rate of concussions in the non-adherent was at least 12-fold that in the adherent (regardless of whether these involved wearing headgear or not). This is a very strong effect. Do you have any explanation for this vast difference? It is possible that this difference will shrink or disappear if you correctly count all non-adherent AEs as non-adherent.
Thank you again for conducting these trial and for your kind attention to these questions!
We would like to thank you for your insightful and interesting comment.
Regarding the first point, we presented the 28.5% to illustrate the relative difference in total absolute fat (kg) change between interventions, so the reader could have information about the relative difference between groups. We would like to highlight that it was only possible to perform this analysis using the within group changes, since the change between group analysis was showed in absolute values.
About the second point, it was not our purpose to analyse lean body mass; however, we agree that this topic is very important for health and athletic performance purposes. This is an unanswered question and we are performing studies to test the effects of interval training on lean body mass to help shedding light in the topic.
In this article the authors discuss their analysis of 21 female and 22 male athletic events. Testing all 43, they find 3 events significant with p<0.05. When testing 43 events, the expectation is that a well-calibrated statistical test will produce 2 false positives with random data, on average, due to the definition of the p-value. The odds of producing 3 false positives are also rather high; for normally distributed simulated data under the null, I found 3 or more false positives approximately 1/3 of the time such an analysis is performed, see here for a simulation notebook: https://github.com/davidasiegel/False-Positive-Rate-for-Multiple-Tests-i....
This is why adjustments for multiple comparisons needs to be performed. It was neglected in their initial study and neglected again in this study. In the 2017 study they state, "These different athletic events were considered as distinct independent analyses and adjustment for multiple comparisons was not required." This doesn't make sense to me; if the analyses are distinct, then all the more reason to correct for multiple comparisons. If a Bonferroni correction were performed, none of the p-values would test significant at the level of the study (p<0.05/43 = 0.001). Therefore I do not see why there is any reason to reject the null hypothesis for any of these results.
In this article the authors discuss their analysis of 21 female and 22 male athletic events. Testing all 43, they find 3 events significant with p<0.05. When testing 43 events, the expectation is that a well-calibrated statistical test will produce 2 false positives with random data, on average, due to the definition of the p-value. The odds of producing 3 false positives are also rather high; for normally distributed simulated data under the null, I found 3 or more false positives approximately 1/3 of the time such an analysis is performed, see here for a simulation notebook: https://github.com/davidasiegel/False-Positive-Rate-for-Multiple-Tests-i....
This is why adjustments for multiple comparisons needs to be performed. It was neglected in their initial study and neglected again in this study. In the 2017 study they state, "These different athletic events were considered as distinct independent analyses and adjustment for multiple comparisons was not required." This doesn't make sense to me; if the analyses are distinct, then all the more reason to correct for multiple comparisons. If a Bonferroni correction were performed, none of the p-values would test significant at the level of the study (p<0.05/43 = 0.001). Therefore I do not see why there is any reason to reject the null hypothesis for any of these results.
Performing a series of tests and then re-testing the subset of events with the most significant results is also poor statistical practice that doesn't add meaning to the study.
In conclusion, taking into account the total number of tests that were performed (43), the results fail to be significant at the level of the study.
We congratulate O’Keeffe et al. [1] for their research on the comparative efficacy of Cognitive Functional Therapy (CFT) and physiotherapist-delivered group-based exercise and education for individuals with chronic low back pain (CLBP). Their study shows that “CFT can reduce disability, but not pain, at 6 months compared with the group-based exercise and education intervention”. The CFT approach is very promising and has caught the attention and interest of a number of clinicians worldwide in the management of non‐specific disabling CLBP. The study by O’Keeffe et al. [1] has methodological strengths compared to a previous clinical trial by Vibe Fersum et al. [2,3] such as a higher sample size which means it is less vulnerable to type-II error. Nonetheless, some shortcomings threaten substantially the risk of bias and type I error that are worthy of further discussion.
The first is the choice of three physiotherapists for delivering both interventions in this trial. This aspect was considered by O’Keeffe et al. [1] as a strength of the study because it arguably minimized differences in clinicians’ expertise and communication style. Notwithstanding, this fact could also have decreased the treatment effect on the control group. It is important to remember that the trial was performed by the research group that not only developed CFT but also has trained the physiotherapists on such an approach, and thus the enthusiasm and motivation to apply the intervention on the CFT...
Show MoreHere is my simple response to this absurd proposal. If drugs help those who are not as genetically advantaged to be more competitive with those who are, shall we prohibit the genetically advantaged from taking them? Otherwise, you create the situation where all athletes must take these drugs just to maintain the status quo. Athletes who prefer not to use drugs would suffer the most. Since drug use monitoring will be required anyway for safety, let's prohibit their use as much as possible. Allowing their use only benefits the pharmaceutical companies who sell the drugs. Sports would becomes less about athletic ability and more about who can come up with the best drug formula for competitive success.
Dear editor,
Show MoreWe have read with great interest the article by Wheeler et al1 showing distinct effects of exercise with and without breaks in sitting on cognition. In this study, they also demonstrated that both activity conditions increase serum brain-derived neurotrophic growth factor (BDNF) levels. Although we highly appreciate the efforts of the authors to explore potential mechanisms, we suggest that the followings need to be addressed.
BDNF is an important member of the neurotrophic factors family which enhances neuronal development and plasticity. It is synthesized as the N-glycosylated precursor (brain-derived neurotrophic factor precursor, proBDNF), and secreted into cell matrix processed by Golgi complex. Additionally, BDNF is a novel kind of myokines produced by skeletal muscle after the muscle contraction immediately. Hayashi and coworkers2 observed that both exercise and electrical muscle stimulation could increase the mRNA and protein expression of BDNF in skeletal muscle of rats. In addition, exercise could also enhance gene expression of BDNF and other neuroprotective factors in hippocampus via peroxisome proliferator-activated receptor gamma coactivator-1α-fibronectin type III domain-containing protein 5/irisin (PGC-1α-FNDC5/irisin) pathway.3
BDNF has been reported to play a pivotal role in the improvement of learning and memory function, which might be associated with the phosphorylation of tropomyosin-related kinase B (TrkB) in cognitive-...
The BJSM recently rejected our request of retraction or errata corrige of the editorials by Blanch and Gabbett(1) and Gabbett (2) presenting the relation between the Acute:Chronic Workload Ratio (ACWR) and likelihood of injuries. The preprint and a list of some of the errors presented in that figure can be found here: https://osf.io/preprints/sportrxiv/gs8yu/. In challenging our request, it was underlined several times by the Editor in Chief of BJSM that the “model” was presented as illustrative only, and this seems to make errors acceptable like if the editorials are a “safe zone” where for illustrative purposes it is possible to bend and even break scientific rules and methods, presenting models using unpublished and uncontrollable data.
However, the reason of this communication is to warn the members of the consensus (and readers) that the ACWR model published in the IOC consensus(3) as a validated model has in fact not been validated at all: [page 1034] “The model has currently been validated through data from three different sports (Australian football, cricket and rugby league)(187)”. The reference 187 is one of the two editorials(1) for which we asked the retraction. So on one side the Editor in Chief insists that it is just an illustrative (flawed) model, but on the other side the same Editor in Chief, co-author (with one of the proponents of the model) of the IOC consensus wrote and published that it...
Show MoreDear Drs. McGuine, Hetzel, and Kliethermes,
Thank you for your thorough response to my initial comment.
I am wondering if you could help me understand the new AE-level as-treated analysis you have done in response to Point 2. This accounts for all non-compliant AEs among all athletes, correct? If I understood you correctly, there were somewhat more than the 711 non-compliant AEs reported in the paper and which you reported in your response to Point 4, correct?
What would be very helpful to see is a.) the number of AEs and b.) the number of SRCs that occurred during those AEs for each of the following groups when considering any non-compliant AE, not just ones from athletes who suffered an SRC while non-compliant or were non-compliant >50% of the time:
Assigned HG/Did Not Wear:
Assigned HG/Did Wear:
Assigned No HG/Did Not Wear:
Assigned No HG/Did Wear:
Thank you again for your thorough response.
After careful appraisal and following our own investigations, we are concerned that the article “Is interval training the magic bullet for fat loss? A systematic review and meta-analysis comparing moderate-intensity continuous training with high-intensity interval training (HIIT)” [1] may have some data extraction and analysis errors that warrant further review by the editor and authors, and which more concerningly, may impact the original conclusions of the article.
We were initially concerned about the reported results within the Thomas et al. paper [2], particularly the biological plausibility of a mean between-group fat-loss difference of 13.44 kg over 12 weeks. Given that the authors did not report any study-level data, we decided to investigate the effect size within this paper. However, this study [2] did not report any fat mass data, only % body fat data. Given that the authors of the review [1] reported “When studies provided insufficient data for inclusion in the meta-analysis (five studies), the corresponding authors were contacted via email to determine whether additional data could be provided; however, no corresponding authors responded.”, it is unclear how an unpublished mean difference of -13.44 kg in favour of HIIT/SIT could be presented within the fat mass analysis of this review. Furthermore, when reviewing another of the included studies [3], we found that fat mass data were reported, but not included in the current meta-analysis [1]. Given the m...
Show MoreTo: The British Journal Sports Medicine
We are grateful for Dr. Binney’s interest in our study and his consideration of a portion of the results presented in the manuscript.
Listed below are our responses to each of the concerns raised in the letter.
1. In the as-treated analysis you have a very strange result. Your multivariate risk ratio (which is actually a rate ratio) is 0.63 for everyone overall, 0.64 for females, and 0.93 for males. The result for everyone should be between the results for males and females. Can you please clarify how you got these results, including the exact model(s) you used and how you calculated the rate ratios? Did you use a group*sex interaction term to get the sex-specific results?
Response: We thank you for noticing the mathematical inconsistency in Table 4 rate ratio results for the as-treated analyses. You are correct that if these results were from one model, the overall rate ratio estimate would need to be in-between the male/female estimates. We should note that these were actually 3 separate mixed-effects models: (1) the overall model adjusting for all variables including sex, (2) female sub-group model adjusting for all variables –excluding sex, and (3) male sub-group model adjusting for all variables –excluding sex. We apologize that the footnote in the table is unclear in this regard. We did attempt to use interaction models for this analyses, but did not achieve consistent convergence. As such, we opt...
Show MoreDear Dr. McGuine et al,
I'd like to commend you on running a large RCT on such an important topic (assessing the purported effectiveness of concussion-reduction technologies). Unfortunately I have some concerns about some aspects of your data and analysis, particularly the as-treated analysis in Table 4and your reported adherence numbers. I am hoping you can clarify these concerns and re-do parts of your analysis.
1. In the as-treated analysis you have a very strange result. Your multivariate risk ratio (which is actually a rate ratio) is 0.63 for everyone overall, 0.64 for females, and 0.93 for males. The result for everyone should be between the results for males and females. Can you please clarify how you got these results, including the exact model(s) you used and how you calculated the rate ratios? Did you use a group*sex interaction term to get the sex-specific results?
2. How you defined the as-treated group is concerning. You state that you only re-classified a subject if they spent >50% of their time in their non-assigned group OR if they were concussed while in their non-assigned group. This approach will bias the results of your as-treated analysis as you are deliberately misclassifying the AEs of people who do not get hurt and the non-concussed AEs of those who do. You need to classify every AE, rather than each athlete, as headgear or no headgear and repeat the as-treated analysis. Otherwise this analysis is highly questionable and...
Show MoreDear Dr. Anoop Balachandran
We would like to thank you for your insightful and interesting comment.
Regarding the first point, we presented the 28.5% to illustrate the relative difference in total absolute fat (kg) change between interventions, so the reader could have information about the relative difference between groups. We would like to highlight that it was only possible to perform this analysis using the within group changes, since the change between group analysis was showed in absolute values.
About the second point, it was not our purpose to analyse lean body mass; however, we agree that this topic is very important for health and athletic performance purposes. This is an unanswered question and we are performing studies to test the effects of interval training on lean body mass to help shedding light in the topic.
Best regards.
In this article the authors discuss their analysis of 21 female and 22 male athletic events. Testing all 43, they find 3 events significant with p<0.05. When testing 43 events, the expectation is that a well-calibrated statistical test will produce 2 false positives with random data, on average, due to the definition of the p-value. The odds of producing 3 false positives are also rather high; for normally distributed simulated data under the null, I found 3 or more false positives approximately 1/3 of the time such an analysis is performed, see here for a simulation notebook: https://github.com/davidasiegel/False-Positive-Rate-for-Multiple-Tests-i....
This is why adjustments for multiple comparisons needs to be performed. It was neglected in their initial study and neglected again in this study. In the 2017 study they state, "These different athletic events were considered as distinct independent analyses and adjustment for multiple comparisons was not required." This doesn't make sense to me; if the analyses are distinct, then all the more reason to correct for multiple comparisons. If a Bonferroni correction were performed, none of the p-values would test significant at the level of the study (p<0.05/43 = 0.001). Therefore I do not see why there is any reason to reject the null hypothesis for any of these results.
Perfor...
Show MorePages