’Types of studies

Eligible RCTs were identified from systematic reviews investigating the effects of exercise therapy and published in the Cochrane database of Systematic Reviews. RCTs, cluster-randomised trials and randomised crossover studies were included if they compared an exercise therapy intervention with a non-exercising control treatment.

Types of participants

Studies that included participants with or without a medical condition were eligible, except for participants receiving chemotherapy, as all or nearly all these participants are anticipated to experience adverse events. Otherwise, no studies were excluded based on specific characteristics of the participants.

Types of intervention

Exercise therapy was the main intervention and each exercise session had to include active exercise therapy for at least 50% of the total time. Furthermore, the exercise could not be combined with any pharmacological, surgical or electrotherapeutic intervention. Besides strength/resistance, aerobic and neuromuscular exercise (defined as exercise interventions targeting sensorimotor deficiencies and functional stability), the following active exercise interventions were also included: dancing, running, cycling, QiGong and Tai Chi. However, interventions like whole body vibration, facial exercises, yoga, stretching or range of motion exercises or bladder training were excluded. There were no restrictions on the setting in which the exercise therapy was performed, that is, classes, gymnasium, etc.

Types of control intervention

Studies with comparators such as a non-exercise therapy control group, usual care, attention intervention, etc were included, as were nutraceuticals, placebo and education (eg, back school and similar interventions). However, studies where the control group involved any exercise (including stretching), pharmacological, surgical or electrotherapeutic intervention were excluded.

Type of outcome measures

The outcomes of interest were measures of adverse events. As classifying adverse events as treatment-related is largely subjective, and with unknown validity, the current study was not focused on reported adverse effects, but on adverse events as any undesirable event occurring during the study, divided into serious and non-serious adverse events.’29

’Electronic searches

We […] searched the following databases from inception to 11 March 2016 without restrictions to language or publication status:

• Cochrane Central Register of Controlled Trials (CENTRAL, which includes the Cochrane Back and Neck group (CBN) trials register) (The Cochrane Library, 2016, Issue 2).

• MEDLINE (OvidSP, 1946 to March week 1 2016; Appendix 1).

• MEDLINE In-Process & Other Non-Indexed Citations (OvidSP, 10 March 2016),

• Embase (OvidSP, 1980 to 2016 week 10),

• Cumulative Index to Nursing and Allied Health Literature (CINAHL) (EBSCO, 1981 to 11 March 2016),

• PsycINFO (OvidSP, 2002 to March week 2 2016),

• Allied and Complementary Medicine Database (AMED) (OvidSP, 1985 to March 2016),

• CBN Trials Register (Cochrane Register of Studies (CRS))),

• Cochrane Complementary Medicine Field Trials Specialized Register (Cochrane Register of Studies Online (CRSO))),

• IndMED,

• PubMed,

• US National Institutes of Health ClinicalTrials.gov,

• World Health Organization (WHO) International Clinical Trials Registry Platform (ICTRP).

The searches were previously run in 2013 and 2014. In 2014, the ClinicalTrials.gov, WHO ICTRP and a supplementary search of the CBN Specialised Register in the CRS were added to the search strategy. In 2016, the PubMed search was revised to capture studies not in MEDLINE using the strategy recommended by Duffy 2014. The Information Specialist of the CBN conducted all searches except for the Cochrane Complementary Medicine Field Specialised Register, which we searched through the CRSO.

Searching other resources

We screened the reference lists of included studies and contacted experts in the field (eg, authors of included studies) for information on additional trials, including unpublished or ongoing studies.’33

Example: ‘We conducted a comprehensive database search using PubMed, EMBASE, Cochrane Library, SPORTDiscus and PEDro to search clinical practice guidelines (CPGs) that presented rehabilitation of ACL injuries. To search grey literature and CPG repositories we used the OpenGrey, National Guideline Clearinghouse of the Agency for Healthcare Research and Quality, Guidelines International Network and National Institute for Health and Care Excellence (NICE) databases. […] all searches up to the 30 September of 2018 […].’34

Example: ‘Twelve systematic searches covering diagnosis, prevention and treatment for each of the four sections (1: hamstring, 2: adductor, 3: rectus femoris/quadriceps and 4: calf) […]. No restrictions were applied concerning year of publication, however, only publications in English were included. We searched individual text words in title and abstract supplemented with Medical Subject Headings (MeSH) terms. We combined anatomical region of interest (eg, ‘Groin (MeSH)’ OR ‘adductor’ OR ‘groin’) AND type of injury (eg, ‘Athletic Injury (MeSH)’ OR ‘Strains and Sprains (MeSH)’ OR ‘strain*’ OR ‘injur*’ OR ‘re-injur*’ OR ‘reinjur*’) AND outcome for diagnosis and treatment domains (eg, ‘Diagnosis (MeSH)’ OR ‘exam*’ AND ‘Return To Sport’ OR ‘full training,’ respectively) OR intervention for prevention domains (eg, ‘Primary Prevention (MeSHs)’ OR ‘Reduc*’). […] A flow chart of searches and the complete search strategy for all searches and databases is available as supplemental.’38

Example: ‘The selection of studies was a three-stage process, with the identified citations independently evaluated for inclusion by two reviewers. The first stage was evaluation of titles selected with systematic searches described above. The article was included in this first screen if the title identified athletes and/or lumbar discectomy. We then reviewed the abstracts of all articles identified as meeting the search criteria. Full-text articles meeting criteria were retrieved and read independently by both reviewers and assessed for inclusion in the study. Disagreement was resolved by consensus between the two reviewers and a third reviewer if consensus could not be reached.’39

Example: ‘Two reviewers independently extracted data using a specifically designed standardised data extracting form (see study protocol), and afterwards the reviewers compared the extracted data for consistency. All inconsistencies between the two forms were resolved by discussion between the two data extractors. Any disagreement between the data extractors after the initial discussion related to inconsistencies between the two individual data extractions was to be solved by involving a third person. General study information, participants and intervention characteristics, compliance, adverse events, withdrawals and outcome measures were extracted. Where data were not available from tables or the results section, the authors of the study in question were contacted by email, with one reminder after 2 weeks, if they did not respond to the first email.’40

’Outcomes

The guideline panel identified outcomes of importance to patients; we used outcomes from the trials that most closely corresponded to those chosen by the patients (the systematic reviewers worked with 33 patient partners to identify and prioritise the outcomes of interest for the systematic review) (table 1). Because it was the most bothersome symptom for 86% of the patients who completed our survey, we considered pain as the most important outcome in this systematic review.

We defined serious harms as death, bleeding (uncontrolled or requiring transfusion), cardiac arrest requiring cardiopulmonary resuscitation, myocardial infarction, cerebrovascular accident, acute renal failure, unplanned intubation, requiring ventilator for >48 hours, deep infection (surgical site or organ/space) sepsis, septic shock, pneumonia, wound dehiscence, pulmonary embolism, deep vein thrombosis or peripheral nerve injury.

[…] We used a priori-defined decision rules for data extraction:

1. When trialists reported final values and change from baseline values for the same outcome, we extracted final values.

2. When trialists reported unadjusted and adjusted values for the same outcome, we extracted the unadjusted values.

3. When trialists reported data based on the intention-to-treat (ITT) sample and another sample (eg, per-protocol, as-treated), we extracted ITT-analysed data.

Data management

When trials used different outcome measures to evaluate the same construct, we chose the most common outcome measure as the index and transformed mean differences and SD of other outcome measures to the index instrument, and pooled the data using mean difference as the summary estimate […] For trials not reporting the index instrument, we followed a prespecified outcome hierarchy when deciding which data would be pooled (box 2).’41

Example of list of other variables for which data were sought:

‘We also extracted the following data: trial characteristics, patient demographic variables, diagnosis, treatment and data about trial methodology. Online supplementary appendix table 1 (box 3) presents a full list of extracted data items.’41

Example of explanation for assumptions made about any missing or unclear information:

‘If authors did not report relevant numeric outcome data in the text, we contacted the authors or, when available, extracted the data from figures and graphs’.41

Example: ‘We used the Risk of Bias 2 tool to assess risk of bias for each trial outcome. We assessed risk of bias on the basis of ‘assignment to intervention’ for all five domains: (1) randomisation process, (2) deviations from intended interventions, (3) missing outcome data, (4) outcome measurement and (5) selection of the reported result. An overall risk of bias judgement was made for each outcome and each time point as either ‘low risk’, ‘some concerns’ or ‘high risk’ of bias.

The assessment was performed independently by two reviewers […]. The reviewers did not perform risk of bias assessment or data extraction for publications in which they were involved as an author. Disagreements were resolved via consensus or by a third reviewer […] if necessary.’45

‘Synthesis of results

For the analysis on benefits, we calculated the effect sizes in the individual studies as standardised mean differences, allowing pooling and comparison of the various outcomes assessed in the individual trials. We estimated the standardised mean difference as the difference between the mean score of the intervention and control groups divided by the pooled SD of the final score. This estimate of the effect size using standardised mean difference has a slight bias overestimating the effect size, and we applied a correction factor to convert the effect size to Hedges’ g.

In the analysis on harms, we transformed the numbers of adverse events into log odds of events, allowing pooling of data from the individual studies. Results are reported as number of adverse events per 1000 procedures with 95% CIs.’46

Example: ‘We conducted meta-analyses, guided by considerations of bias. For the primary comparison, we pooled data from comparisons at low risk of bias. For the secondary comparison, we pooled data from comparisons irrespective of bias.

We assessed outcomes at 3 months, 6 months, 1 year, 2 years (for which we pooled data up to 3 years if no 2-year data were available), 5 years (we prioritised time points closest to 5 years) and >10 years following randomisation.’41

Example: ‘Studies were stratified by follow-up time categories: <2 years, 2 to 5 years, 5 to 10 years and >10 years […].

Predefined subgroups were (1) patients treated with ACL reconstruction compared with non-operative treatment and (2) skeletally immature patients compared with skeletally mature patients. We accepted skeletal immaturity as defined in the study […]. If skeletal immaturity was not defined in the study, we applied our definition of age under 16 years at injury for all patients.’47

Example: ‘In studies with no ORs presented, the data were transformed to ORs from standard mean difference of muscle strength between the group of participants who developed osteoarthritis and those who did not. Data from adjusted analyses were extracted if available.’48

Example: ‘When trials used different outcome measures to evaluate the same construct, we chose the most common outcome measure as the index and transformed mean differences and SDs of other outcome measures to the index instrument.’41

Example: ‘When not reported, the SD was estimated from the SE of the mean, 95% CI, p value or other methods suggested in the Cochrane Handbook. Means and SDs were estimated from median and range for two studies. Following Cochrane guidelines, for any study that included two different intervention groups (ie, cycling vs walking) and one control group, the sample size in the control group was evenly divided so a comparison could be made to each intervention. We imputed r=0.5 when the correlation of prescores and postscores was required and we performed sensitivity analyses using r values ranging from 0.1 to 0.9.’49

Example: ‘Injury risk proportions for individual studies and pooled estimates were summarised in forest plots for the following subgroups: woman, man and combined. A pooled estimate for the relative risk of ACL injury in women compared with men was calculated and summarised in a forest plot. Raw injury incidence rates for individual studies and pooled estimates were summarised in forest plots for the following groups and subgroups: woman, man and combined. Pooled incidence rate ratios for women compared with men were calculated and summarised in forest plots.’50

Example: ‘Random effect models were used as large heterogeneity was expected due to the different approaches used to compare (ie, controls or contralateral leg) and assess knee extensor muscle strength (ie, isometric or isokinetic) as well as different pain and function scores. A standard [Cochran] Q-test was used to test the heterogeneity between studies, and the I² statistic measuring the proportion of variance attributable to inconsistency was subsequently calculated. […] I² each to 100% indicate maximal inconsistency between individual study results. Furthermore, the τ² value expressing the between study variance was estimated.’51

Example: ‘Owing to expected clinical and methodological heterogeneity between the included studies, we used inverse variance random effects models to estimate relative risks with 95% CIs. […] We dealt with statistical heterogeneity using the I² statistic and prediction intervals. […] Analyses were conducted in either RevMan V.5.4 or Stata V.15.’52

Example: ‘We performed meta-analyses using a random-effects model as heterogeneity was expected in participant, intervention and outcome characteristics. […] A random effects meta-analysis was applied to estimate the overall relative risk of adverse events in the exercise therapy groups compared with comparator groups. Heterogeneity was examined as between-study variance and calculated as the I² statistic measuring the proportion of variation in the combined estimates due to study variance. An I² value of 0% indicates no inconsistency between the results of individual trials, and an I² value of 100% indicates maximal inconsistency. Meta-analyses were performed in STATA (V.16.1) using the ‘meta’ command.’53

Example: ‘Stratified analyses were performed for men and women. […] Our initial intention was to conduct subgroup analyses on patients with previous knee injury (ie, ACL injury), overweight, or malalignment. However, sufficient data for these analyses where not found, and thus not included in the present study.’48

Example: ‘We further explored between-study heterogeneity by comparing results from studies grouped according to several study level characteristics using stratified meta-analysis and meta-regression. Study level characteristics assessed were age, sex, MRI sequences employed, participation in weight-bearing sports, radiographic knee osteoarthritis, sample size and overall risk of bias. The prevalence estimates of primary compartment-specific outcomes (ie, tibiofemoral and patellofemoral cartilage defects, bone marrow lesions, osteophytes; medial and lateral meniscal tears) were pooled wherever reported and differences between compartments assessed with a two-proportion z-test.’55

Example: ‘Where sufficient trials were identified, meta-regression was undertaken using STATA (metareg command) to explore the impact of the following trial-level characteristics and whether they were associated with greater fall prevention effects:

1. Trial design: sample size: <20% missing outcome data; type of comparator intervention.

2. Participant characteristics: average age ≥75 years; control rate of falls; selected at high risk of falls.

3. Intervention components: included NICE-recommended components; actively provided treatment to address fall-related risk factors; whether adherence was assessed.’56

Example: ‘We planned a sensitivity analysis for pain at 1 year to assess the impact of attrition bias due to missing data […]. We also planned to assess small study bias by inspecting the distribution of funnel plots, but there were too few trials.’41

Example: ‘We controlled for smoking status and pre-existing diseases by performing additional sensitivity analyses. Analyses were restricted to never smokers, healthy participants, and healthy never smokers.’58

Example: ‘We examined the effects of methodological quality [bias] on the pooled estimate by removing studies that were at high or unclear risk of bias for the domains of blinding and incomplete outcome data. We had also intended to examine the effects of measurement device on the pooled estimate by removing studies that used pedometers, as previous studies suggest that these might be less accurate in detecting steps in people with COPD.’59

Example: ‘We planned to carry out sensitivity analyses on pain and function by examining the effects of: […]including trials with unclear allocation concealment (at risk of selection bias); including trials with an incomplete description of mild to moderate knee osteoarthritis; […] including trials at risk of detection bias (ie, unclear or no blinding of participant for participant-reported outcomes).’60

Example: ‘In 15 of 16 included studies, the effect sizes were not reported; instead, the authors were contacted and asked to provide raw data. Therefore, the risk of publication bias was expected to be minor. Nevertheless, publication bias was assessed visually on a funnel plot (the effect size by the inverse of its standard error) and statistically using Egger’s test, the Begg and Mazumdar rank correlation test and the trim-and-fill method. It should be mentioned, however, that other factors, such as study quality and true heterogeneity, can produce asymmetry in funnel plots.’61

Example: ‘Data were synthesised and the quality of evidence were evaluated using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) working group. […] Two authors assessed the quality of evidence for each outcome relating to diagnostic tests (eg, effectiveness), prevention (eg, risk of injury) and treatment (eg, time to return-to-play) according to the approach from the GRADE working group. Agreement was reached by consensus. The quality of evidence was graded as: (1) high, indicating that further research is unlikely to change the confidence in the estimate of effect, (2) moderate, indicating that further research is likely to have an important impact on confidence in the estimate of effect and may change the estimate, (3) low, indicating that further research is very likely have an important impact on the confidence in the estimate of effect and is likely to change the estimate or (4) very low, indicating high uncertainty about the estimate.

The starting quality of evidence was rated as ‘high’ when data were based on either RCTs for treatment and prevention purposes or rated as ‘low’ when based on observational studies. For diagnostic purposes, the starting quality of evidence was rated as high when based on cohort studies (prospective or cross-sectional). Subsequently, the quality of evidence could be downgraded one or two levels (eg, from high to moderate) for each of the following five domains of the GRADE approach: Study limitations (ie, serious risk of bias such as lack of blinding of outcome assessor or other concerns determined to influence the study result), inconsistency (ie, the heterogeneity of the results across studies if more than one study was included for the specific outcome), indirectness (ie, poor generalisability of the findings to the target population, eg, use groin injuries vs acute adductor injuries for prevention, and/or use of a clinically irrelevant outcome in relation to the question, eg, ‘time to end of treatment’ for ‘time to return-to-play’ outcomes), imprecision of the estimates (ie, wide CIs) and the risk of publication bias. Furthermore, the level of evidence for cohort studies could be upgraded due to a large effect, a dose–response relationship or if no effect was found and all plausible confounding factors identified in the study could be expected to increase the effect. An overview of the risk of bias and grading is provided as supplemental material.’38

Example: ‘The Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach was used to assess the strength of evidence. Studies were downgraded if there were issues with risk of bias, consistency, precision or directness of the outcomes. The reasons for downgrading the evidence are outlined in table 2.’64

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

PRISMA 2020 flow diagram template for systematic reviews (reproduced with permission). PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses.

Example: ‘Following deletion of duplicates, the literature search yielded a total of 616 abstracts. A total of 557 abstracts were immediately excluded based on the title and abstract screen; 59 articles were obtained in full text and the selection criteria applied. Fifty-one articles were excluded, as they did not compare outcomes between patients who had had revision and primary ACL reconstruction. Finally, eight studies were included for meta-analysis.’65

Example: ‘[…] the remaining 1378 were evaluated in full text. Of these, 161 were excluded due to exercise included in the control group, 139 for reporting results from the same population in another study, 79 for an additional therapy being delivered together with exercise or less than 50% of the intervention being exercise, 69 for the intervention involving vibration therapy, bladder training or other interventions not meeting the inclusion criteria for interventions. Further, 59 were excluded as they were reported in a language other than English, 45 because the full text was not accessible. Lastly, 8 were excluded because data were not extractable, 5 because the thesis was not available and 40 for other reasons.’29

Example: ‘The final analysis included a total of 555 youth (283 and 272 in concurrent exercise and aerobic exercise training group, respectively). Three studies included the same population, but analysed different parameters. Nine studies recruited obese youth exclusively; whereas the rest targeted both overweight and obese children. Most studies (n=8) included adolescents (aged 13–18 years), one included only children, and one enrolled both children and adolescents. All studies included boys and girls. Sample sizes across studies ranged from 30 to 150, with a mean of 55 participants.

The primary mode of the aerobic exercise training programmes were based on treadmills and cycle ergometers, elliptical trainers, walking and running programmes, and sports participation. The exercise intensity was monitored using either maximum heart rate, or peak oxygen uptake.

For resistance exercise, studies used body weight exercise, free weights, selectorised machines or circuit training. Interventions duration varied from 10 to 48 weeks, with a mean of 30 weeks’67 (figure 2 (labelled table 2 in the original publication) is an example of how one might present study characteristics).

• Download figure
• Open in new tab
• Download powerpoint

Figure 2

Example of study characteristics presented in a summary table (reproduced with permission).

Example: ‘Twenty-two trials (76%) were at high risk of bias. We had some concerns about bias in seven trials (24%). No trials were at low risk of bias. In studies that used both the VISA-A score and return to sport as outcome measures, there was no difference in risk of bias between the outcomes. In 48% of the trials, outcome measurement was a source of bias. All other sources of bias were also commonly judged as high risk: the randomisation procedure (21%), deviations from the intended intervention (28%), missing outcome data (28%) and selection of reported results (24%).’45

Example included in figure format: ‘One low quality study used x-ray to examine the pubic symphyses of athletes with hip/groin pain and those without pain. A reliable grading system quantified the abnormalities, which were present in all the hip/groin pain subjects (9/20 slight, 9/20 intermediate, 2/20 advanced). In contrast, the athletic control subjects had either no (3/20) or slight (17/20) abnormalities seen on x-ray. Another moderate quality study only reported the radiographic findings in the hip/groin pain subjects and not the control subjects. Synthesising the data on x-ray investigations of the pubic symphysis, there is currently limited evidence that x-ray findings differentiate athletes with hip/groin pain from athletes without pain.

Three moderate quality studies examined pubic bone oedema using MRI in athletes with hip/groin pain and controls. Dichotomous data for the presence or absence of bone oedema were extracted and pooled from these studies. The results indicated that there were high odds that participants with bone oedema on MRI would be in the hip/groin pain group with a large effect size; OR=41.63 (95% CI 1.6 to 1096.60). However, there was high heterogeneity demonstrated by this pooled result (I²=88%), and significant sensitivity to the data from the study of Cunningham et al (figure 3 (labelled figures 8 and 9 in the original publication)). The removal of this study data resulted in an OR=8.1 (95% CI 3.1 to 21.2) for the presence of bone oedema in subjects with hip/groin pain, representing moderate evidence, with a large effect size, that bone oedema in the pubic symphysis differentiates athletes with hip/groin pain from those without this pain.’68

• Download figure
• Open in new tab
• Download powerpoint

Figure 3

Example of how to present effect estimates in forest plots (reproduced with permission). M-H, Mantel-Haenszel.

’Risk of bias assessment

For the comparisons of subacromial decompression surgery versus placebo surgery, the risk of bias was low for all outcomes (figure 4). Due to detection bias (all studies), selection bias, attrition bias and selective reporting, for the comparisons of subacromial decompression surgery vs non-surgical treatment, the risk of bias was high for all outcomes except rotator cuff tears.

• Download figure
• Open in new tab
• Download powerpoint

Figure 4

Example of how to present domain-based risk of bias assessment results (reproduced with permission). ASAD, arthroscopic subacromial decompression.

Primary comparisons

Two trials were at low risk of bias and sufficiently clinically and methodologically homogeneous to allow pooling for the comparison of subacromial decompression surgery plus postoperative rehabilitation versus placebo surgery plus postoperative rehabilitation.’41

’Mental health symptoms and disorders among current elite athletes

Among those, 11 studies reported prevalence data on distress symptoms among 3335 male and female elite athletes (age ranging from 16 to 29 years) from team sports (eg, cricket, football, handball, ice hockey, rugby) and combined Olympic sports (eg, boxing, gymnastics, judo, rowing, swimming).

[…]

Mental health symptoms and disorders among former elite athletes

Among those, eight studies reported prevalence data on distress symptoms among 1686 former male and female elite athletes (age ranging from 34 to 62 years) from team sports (American football, cricket, football, ice hockey, rugby) and combined Olympic sports.’69

Example: ‘Three studies reported on previous history of stress fracture and its association with increased risk of future stress fracture. All three studies had similar findings in that athletes with a previous history of fracture were at increased risk of developing a future stress fracture with ORs ranging from 2.90 to 6.36. An exploratory meta-analysis confirmed the individual study results with runners with a previous history of stress fracture at five times higher risk of a future stress fracture (OR 4.99; 95% CI 2.91 to 8.56; p<0.001; I²=0%. […] Females were at 2.3 times higher risk compared with males (OR 2.31; 95% CI 1.24 to 4.29; p<0.008; I²=0%).’70

Example: ‘Stratified analyses for sex showed an increased risk in both men (OR 1.68, 95% CI 1.10, 2.58; I²=55.5%) and women (OR 1.59, 95% CI 0.94, 2.68; I²=54.3%). Differences in risk between men and women did not reach statistical significance (p=0.87).’48

Example: […] ‘The effect remained similar across all strata included in our prespecified…sensitivity analyses (Table 3) with the exception of the type of comparator intervention. […] The pooled estimates of effect remained similar in all prespecified…sensitivity analyses (figure 5 (labelled Table 3 in the original publication)).’56

• Download figure
• Open in new tab
• Download powerpoint

Figure 5

Example of how to present results of sensitivity analyses (reproduced with permission).

Example: ‘We assessed publication bias for this comparison. The funnel plot (figure 6) shows that small studies with negative effects are missing in the right lower quadrant, which indicates possible publication bias.’71

• Download figure
• Open in new tab
• Download powerpoint

Figure 6

Funnel plot for assessing publication bias (reproduced with permission). SE, standard error; SMD, standardised mean difference.

’Publication bias

The funnel plot indicated that almost all studies fell within the expected parameters, most with low SE indicating that most studies were large. A majority of studies reported that women had greater incidence proportion than men. The funnel plot for incidence rate ratio indicated that most studies fell within the expected parameters. Standard error was relatively low, indicating that studies were large, and a majority of studies reported that women were at increased risk of ACL injuries relative to men. The studies are not evenly distributed in the funnel, with studies missing from the lower left quadrant. Studies in the lower left quadrant would represent smaller studies that report a greater incidence proportion or incidence rate of ACL injuries in men compared with women.’50

Example of GRADE summary reporting and outcome reporting in table format: ‘A summary of the quality of evidence, based on risk of bias, study design, CIs and variability in results, has been collated using the GRADE approach (table 3).

All outcomes were rated as very low or low quality [we recommend systematic reviewers use the descriptor ‘certainty’ not ‘quality’] evidence demonstrating that the estimate of effect for those outcomes is uncertain.’64

Example: ‘There exist several other reviews, although previous reviews have focused on fewer interventions. The most important difference between our systematic review, and previously published reviews is that we have a more stringent assessment of the risk of bias and quality of included trials. This is important because the strength of recommendations (eg, in future guidelines) will be based on the quality of the evidence.

For exercise, our results are in line with the other reviews, with the exception that we concluded that there is only very low-quality evidence where other studies reported moderate or even high or strong evidence. Two reviews evaluated scapula-focused treatments, reporting moderate evidence, and significant but clinically not relevant effects; whereas we did not separately analyse the scapula-focused treatments.’25

Example: ‘Overall, there was a lack of consistent high-quality evidence to support nominating any particular movement quality outcome as a lower extremity injury risk factor due to inadequate reporting of concepts essential to establishing internal (how well an experiment was carried out) and external (can the results be applied to people and situations beyond the experiment) validity. The biggest threats to internal validity were related to the possibility of selection bias, and the reporting of, and adjustment for, potential influence of factors such as sex, injury history and training exposure.

Specifically, due to the lack of participant characteristic reporting, it was often difficult to determine if the individuals selected for a study differed systematically from those in the source population (selection bias). Equally important was the consistent omission of the characteristics of those lost to follow-up, which made it impossible to determine if participants lost to follow-up were systematically different from those retained in a study. The inability to determine selection bias not only questions the internal validity of several included studies, it impacts the degree to which the findings of these studies can be generalised to the larger population from which the samples were drawn (external validity).’72

Example: ‘We planned threshold analysis as a quantitative means to assess the robustness of network meta-analysis recommendations to potential limitations in the evidence. We were unable to use this approach because of substantial overlap in credible intervals from the network meta-analysis. Due to overlap in the intervals, no recommendations could be made, which is a fundamental prerequisite to performing a valid threshold analysis. To comply with our protocol, we report threshold results in (supplementary Web appendix), but chose to use GRADE to interpret the evidence. We were not able to evaluate small study bias due to too low number of trials. We found three completed trials in trial registers; two are under review, and the publication status of one trial is unknown.’45

Example: ‘On average, patients with subacromial pain syndrome reported reduced pain, and improved physical function and quality of life following both surgical and non-surgical treatment. However, at up to 5 years, irrespective of treatment, patients continued to report pain of an average of 1.5–3 on a scale of 0–10 points on a Visual Analogue Scale. Clinicians working in primary care who are treating patients with subacromial pain syndrome should be aware that some patients experience prolonged symptoms and consider care strategies to support coping.

A placebo control helps answer the research question ‘is there a benefit of subacromial decompression surgery?’ because it minimises the risk of detection and performance biases. Both sources of bias contribute to overestimation of treatment effects by up to 20%. The largely consistent findings of the unblinded studies leave little doubt of the inference that subacromial decompression surgery provides no important benefit to patients. The current evidence provides no support for subacromial decompression surgery as an intervention providing important benefit for patients with subacromial pain syndrome. High-quality evidence indicates that surgery versus placebo surgery confers no important benefit on pain and function—the outcomes most important to patients. Considering the body of evidence, further head-to-head comparisons of subacromial decompression surgery compared with placebo surgery or non-surgical management with the same population are unlikely to change the results. Policymakers, funders and clinicians should consider these results in their funding and clinical decisions regarding the management of patients with shoulder pain.

Our review was designed to assess the benefits and harms of subacromial decompression surgery for managing subacromial pain syndrome. To date, no trial has demonstrated benefit of surgery for any clinical subgroup. In the future, subgroup claims should be supported by data from well-conducted trials at low risk of bias and the use of established criteria for credibility of subgroup effects, ideally enhanced by IPD meta-analysis.

For harms we welcome well-performed observational studies that specifically report harms after subacromial decompression surgery separately from harms following other types of shoulder surgery. Although the finding of no important benefit of surgery is robust, the root cause of subacromial pain and the underlying pathological process remain uncertain, as does the possible best treatment—if such exists—for subacromial pain syndrome. Network meta-analysis might provide information on this question and provide hypotheses to be tested in future methodologically sound trials. Future triallists investigating any treatment for subacromial pain syndrome should adopt a common set of outcomes, and the outcome measures should be standardised.’41

Example: ‘The study was registered at PROSPERO (ID CRD42015024120).’40

Example: ‘[…] publicly available comprehensive study protocol including data extraction forms was uploaded at the following website: http://vbn.aau.dk/files/229186677/The_effect_of_the_FIFA_11_prevention_programmes_on_the_overall_injury_rate_in_football_a_systematic_review_and_meta_analysis_version1_1.pdf’40

Example: ‘This systematic review adhered to the […] review protocol, which was published prospectively.74 We prospectively registered this systematic review in the International Prospective Register of Systematic Reviews (PROSPERO registration number: CRD42016036788).’47

’Change between the protocol and published review

Following data extraction, as the majority of studies reported patient-reported pain separately from function, rather than reporting complete composite scores (eg, KOOS-5), we modified our analysis plan to more comprehensively report pain, function and quality of life outcomes separately. Similarly, the majority of studies reported either 6-month or 12-month outcome data (not both). Therefore, to provide a more complete picture of early, medium and long-term outcomes, we modified the time point at which outcomes were report to: under 6 months, 6–12 months and over 12 months, respectively. One included study did not perform MRI in all patients prior to randomisation .75

’Deviations from study registration and study protocol

‘Agreement by raters on risk of bias decisions for the included randomised controlled studies was calculated as a percentage of agreement and κ values, and included in the results. Since the secondary analysis concerning type of programme showed that only the FIFA 11+ prevention programme was effective in reducing injuries, all secondary outcomes concerning lower limb, hamstring, knee and ankle injuries were only analysed in relation to this programme.

Furthermore, a post hoc analysis on hip/groin injury in relation to this programme was also included. Preplanned secondary analyses on the incidence rate ratio in the following subgroups: gender (male and female), and mean age groups (youth (<19 years), seniors (19–30 years), old girls/boys (31–39 years) and veterans (>39 years)) were not conducted, as the included studies did not allow making meaningful comparisons with only six studies, where studies with male (n=3) and female participants (n=3) significantly differed in the age group they targeted. The predefined secondary analysis of compliance at team level was not performed as all team-level data could not be obtained from the corresponding authors of the included studies. Instead, the preplanned analysis of the association between prevention programme compliance and injury incidence was further supported by a post hoc analysis of the association between prevention programme compliance and the overall injury incidence rate ratio from each study to accommodate for the risk of substantial variance in injury incidence between studies due to other factors than the FIFA injury prevention programmes.’40

Example: ‘This research received no grant from any funding agency in the public, commercial or not-for-profit sectors. […] is supported and funded by the National Osteoporosis Society via the Linda Edwards Memorial PhD Studentship.’76

Example: ‘All authors have completed the ICMJE uniform disclosure form at (available on request from the corresponding author) and declare: […] has received personal fees from Össur, Flexion Therapeutics, Medivir, Teijin, MerckSerono, Allergan, and Galapagos and is editor-in-chief of Osteoarthritis and Cartilage; […] has received personal fees for lectures and royalties for books from Össur, Finnish Orthopedic Society, Studentlitteratur, and Munksgaard and is an associate editor of Osteoarthritis and Cartilage; no other relationships or activities that may appear to have influenced the submitted work.’46

Example: ‘…a publicly available comprehensive study protocol including data extraction forms was uploaded at the following website: http://vbn.aau.dk/files/229186677/The_effect_of_the_FIFA_11_prevention_programmes_on_the_overall_injury_rate_in_football_a_systematic_review_and_meta_analysis_version1_1.pdf’ 40

Example: ‘Data availability statement. Data are available in a public, open access repository: https://osf.io/q24xh’ 77 [the authors provide the dataset of included studies (with reasons for excluding studies from meta-analysis), and a list of excluded studies with reasons].