Statistics from Altmetric.com
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.
The last decades have seen an enormous shift in the practice of healthcare with widespread acceptance of the evidence-based practice paradigm.1 This change has been accompanied by an increase in the volume of clinical research2 and refinement of our understanding of how research should be conducted and reported.3 Concurrent with these changes, longitudinal analyses have identified improvements in the conduct and reporting of various types of research in many disciplines of healthcare.3–7 However, not all such analyses have identified improvements.8
A recent assessment of reports of randomised trials of physiotherapy interventions9 allows trial reports in sports physiotherapy to be compared to other clinical subdisciplines of physiotherapy, such as musculoskeletal, neurology or cardiorespiratory. Compared to all other physiotherapy trial reports, trials that enrolled sporting participants reported fewer key design features that reflect rigorous methods and allow readers to interpret the research fully. Therefore, the purpose of this article is to highlight the methodological features that are often lacking in sports physiotherapy trial reports, explain their significance and offer suggestions for improving the conduct and reporting of future trials.
Issues in sports physiotherapy trials
Moseley et al9 examined the conduct and reporting of almost 15 000 trial reports that were fully indexed on the Physiotherapy Evidence Database (PEDro; http://www.pedro.org.au). When the trial reports were evaluated using items on the PEDro Scale,10 sports physiotherapy trials performed less well on several items in comparison to other subdisciplines. The results are summarised in table 1 along with those for musculoskeletal trials and all trials, for the purposes of comparison.
In the following items, sports physiotherapy trial reports rated substantially lower than trial reports from other subdisciplines. For each item, we provided a short explanation and recommendations for improvement.
Eligibility criteria and source of participants
This item concerns a relatively simple issue of reporting clarity. The note for assessment of this PEDro item states that it is satisfied “if the report describes the source of subjects and a list of criteria used to determine who was eligible to participate in the study.” The population of interest and specific eligibility criteria for entry into the study should have been established as part of formulating the research question. Clearly stated, this information is critical in interpreting the study findings in terms of their generalisability. Achieving this item should be straightforward for researchers; it is a matter of specifying inclusion criteria in the study protocol and registration, then stating them explicitly in the published report of the trial, along with details of where and how the study participants were recruited.
Allocation concealment means that the researcher determining whether a potential participant should be included in the study is unaware of the group to which the participant would be assigned. If the researcher determining the inclusion knows whether each upcoming allocation is to the treatment or the control group, this could bias the decision to include or exclude some potential participants. There is empirical evidence that trials that do not have concealed allocation report larger between-group differences than those that do.11 Allocation can be concealed by placing each of the randomly ordered group allocations in a sealed, opaque and consecutively numbered envelope, and only opening the envelope after a participant has been enrolled in the study and allocated a study number. These envelopes obviously need to be prepared before recruitment begins, but this is a simple process that can be performed by a research collaborator at low cost. Another option is to separate inclusion from randomisation, so the two tasks are performed by different people in different places. Some trials achieve this separation by using a telephone, email or web-based randomisation service, which, while being more expensive than the envelope system, is relatively inexpensive (eg, http://www.ctc.usyd.edu.au/our-research/biostatistics/randomisation.aspx could randomise 200 participants using an interactive voice response telephone system for about $A2500).
Groups are similar at baseline
Groups with similar characteristics can be expected to have similar outcomes, so if only one of two similar groups receives an intervention in a trial, the difference in outcome between the groups can be attributed to that intervention. If there are differences between the groups before the intervention is applied, the effect of the intervention is less clear. Simply relying on the fact that there was random allocation to groups is not sufficient to demonstrate that the groups are similar.
As with the first item about eligibility criteria and source, this item can often be met by simply improving the clarity and completeness of the reporting of the study. The relevant information to show comparability of the groups includes baseline measures of simple demographics, any likely prognostic factors and outcome measures for each group. Summary data for each group can simply be presented in a table for comparison. Note that statistical comparisons of the groups at baseline are not recommended.12 ,13
Blinding of assessors
Blinding is often difficult to achieve in trials that evaluate the effects of complex interventions like physiotherapy. Blinding (sometimes called masking) means that the person specified (participant, therapist or assessor) does not know the group to which the participant has been allocated. While blinding participants and therapists to complex interventions can be difficult, blinding assessors (ie, the researchers who collect the outcome measures) should be possible in most circumstances. For participant-rated outcomes such as questions about symptom severity, researchers should strive to blind the investigator administering the questionnaire so that they cannot introduce bias by knowing the participant's group allocation. Where participants cannot be blinded, researchers should also consider including observer-scored (not self-rated by the participant) outcome measures in the study so that at least these outcomes can be assessor-blinded. Examples of the latter type of measures might be performance measures, imaging findings or work status as recorded by the employer or insurance provider.
Using blinded assessors can create logistical difficulties and potentially add to the expense of a study, simply because it requires someone other than the treating clinician(s) to collect the outcome measures. Solutions to this problem might be to add the cost of blinded outcome assessment into the research budget, to trade-off time between studies with other researchers or clinicians, and to use research students or interns for data collection.
Adequate follow-up rates (more than 85% follow-up)
Achieving this item requires that data for the key outcomes (eg, patellofemoral pain score) are obtained for at least 85% of the participants originally allocated to groups. This is, however, an arbitrary cut-off. The greater the proportion of missing data, the less reliable the results of a study will be. This is because participants who drop out of a study can introduce bias if they differ from those who stay in the study. A high dropout rate can reduce the effectiveness of randomisation and also affect the generalisability of the findings. A further consideration is that a high dropout rate might be an indication that many patients find the rigours, expectations or demands of the intervention too difficult, which again affects the generalisability.
Ensuring adequate follow-up rates, particularly where participants are to be followed up beyond the intervention period, is demanding and requires careful preparatory work. Strategies that can minimise the number of dropouts include: ensuring that the demand for outcome assessment is not too great, in terms of the number, difficulty or length of measures; providing alternative options for data collection, for example in person or by mail, email or telephone; and ensuring that there are enough researchers to collect the measures. Collecting a number of different forms of contact information (eg, home, work and mobile telephone numbers, email address and postal address) and supplementary contacts (eg, friends or family) is useful so that alternative points of contact are available if a participant moves or changes their email address. Providing incentives to participants can also be considered, but requires due regard to ethical considerations. It is important to ensure that participants completely and fully understand the demands on their time that completing the follow-ups will involve, which is something that needs to be discussed before enrolment in the study.
Almost inevitably, there are protocol violations in randomised controlled trials. Protocol violations may involve participants not receiving treatment as planned (including non-compliance) or receiving treatment when they should not have. When the data are analysed, each participant should be included in the analysis as though they had received the treatment or control condition as planned. This is usually referred to as ‘analysis by intention to treat’.
Not performing the analysis in this way reduces the effectiveness of the randomisation process and fails to account for the likelihood that patients cannot tolerate the intervention, thereby decreasing the generalisability of the findings. Intention-to-treat analysis gives a better estimate of how an intervention is likely to work in a pragmatic setting, rather than an idealised situation where everyone receives and complies with the treatment as planned.14 Researchers should clearly state that they intend to analyse by intention to treat in the registered trial protocol. Also, all investigators on the trial must understand that participants who do not receive their allocated treatment (whether due to non-compliance, intolerance or other reasons) should be followed up for all planned outcome measures if at all possible so that they can be included in the analysis. Obtaining statistical advice, either by contracting a statistician or by incorporating a statistician in the research team for the trial, may also be beneficial.
Improving the study's methodological quality demands detailed planning by the researchers and lays an often greater burden on researchers and participants. The benefit is more reliable results that are more likely to be published in good quality journals and to influence healthcare practice and policy.
The subdiscipline of sports physiotherapy can continue to improve the evidence generated by its clinical trials by embracing the need for the most rigorous research methods. Many of the issues identified can be remedied easily. From this report, the items eligibility criteria, source of participants, and similarity of groups at baseline simply require better reporting. Initiatives such as CONSORT3 offer useful guides in these cases. The items concealed allocation and intention-to-treat analysis mainly require careful planning. Methodical preparation and planning are assisted by trial registration and publication of a study protocol—steps that are becoming increasingly mandatory for publication of trial results.15 In addition to careful planning and clear reporting, meeting the requirements for blinding of assessors and adequate follow-up may require greater resources. An excellent study may therefore require extra time, personnel or money, which can be easily underestimated by researchers. This report serves to draw attention to design issues so that these important considerations are not overlooked.
Contributors SJK wrote the first draft of the manuscript. AMM and MRE provided comments and contributed to successive revisions of the manuscript. All the authors are guarantors.
Competing interests None.
Provenance and peer review Not commissioned; internally peer reviewed.