Article Text
PDF
Statistics from Altmetric.com
Do you have an idea for a large-scale, publicly funded randomised controlled trial (RCT)? Virtually all large public funding bodies now expect substantial work to have been done prior to them funding a large trial. Preliminary research suggests greater study feasibility and pilot data point to the potential for the investigators’ hypotheses to have merit. There is a new journal, Pilot and Feasibility Studies, which publishes the results of such studies. The purpose of this editorial is to encourage researchers to think about a pilot or feasibility study before the main trial and to publish the results.
Similar to a RCT, a pilot trial should have clear objectives, but in this case pertaining to feasibility and uncertainties in the main trial design, and a priori criteria for successful progression. It should be designed, conducted and reported using the same standards as any high-quality RCT. Pilot and feasibility studies are important in that they can ensure that scarce research money and researcher efforts are being invested in efficient trials that can provide definitive answers to important research questions. Guidelines for reporting pilot trials as part of an extension of the Consolidated Standards of Reporting Trials (CONSORT) statement have been published.1 2
A recent review of physical activity trials comments on some of the feasibility issues that are more common in this area.3 These include the challenge of blinding of group allocation, retaining interest of the comparison group, participant and instructor fidelity, and being able to recruit people who are not already active enough but interested enough in becoming active to sign up for the study. A second review in sports medicine reported on five journals which published over five articles which related to randomised pilot trials between 2012 and 2015.4 The authors found that reporting and understanding of pilot and feasibility studies in studies of physical activity were poor, with few providing sufficient explanation. Over half of the studies did not have feasibility objectives which corresponded with previous research findings, so that these findings were not being disseminated effectively to researchers in the field of physical activity. The low standard of reporting across most of the reviewed articles and the fact that the extended CONSORT 2010 statement was ignored by the journal editors highlight the need to actively disseminate these guidelines to ensure their impact. The authors also reported that journal editors in this area will rarely publish pilot and feasibility studies, but this might be because the studies are generally of poor quality. An objective of this paper was to encourage better design and reporting in the area of physical activity trials.
Current definitions
There are several papers that clarify the various distinctions in pilot and feasibility studies, and these are summarised in box 1.2 5 6
Feasibility and pilot studies
A feasibility study asks whether something can be done, should the investigator proceed with it and, if so, how.
A pilot study asks the same questions but also has a specific design feature. A pilot study replicates a future study or part of one conducted on a smaller scale.
An internal pilot study is simply the first part of the main trial. For an internal pilot trial, the design is fixed and decisions will be made (after a year say) as to whether recruitment to the full trial is possible in the time postulated.
In an external pilot trial, the subjects are not expected to be included in the main trial, and more radical changes to the design may be made, such as changing the outcome variable.
Feasibility studies include all preliminary work prior to a main study, and pilot studies are a subset of this.
To facilitate their identification, these studies should be clearly identified using the terms ‘feasibility’ or ‘pilot’ as appropriate, including feasibility studies that are largely qualitative.
Investigators should report appropriate objectives and methods related to feasibility; they should report whether their study is in preparation for a future randomised controlled trial.
Researchers should view feasibility as an overarching concept, with all studies done in preparation for a main study open to being called feasibility studies, and with pilot studies as a subset of feasibility studies.2 Authors should describe their feasibility objectives and methods well and in accordance with the guidelines,1 reporting clearly if their study is in preparation for a future RCT to evaluate the effect of an intervention or therapy. A pilot study should address the main uncertainties that have been identified in the development work in order to plan a main trial or other major piece of research.
The CONSORT statement and the Horne review1 4 only apply to randomised external pilot trials (see box 1). An internal pilot study will have a trial monitoring committee which will make suggestions as to whether the trial should be abandoned at that stage or extended either in length or eligibility of participants and possibly suggest revising the sample size upward to accommodate unanticipatedly large variability in the outcome variable. An external pilot trial may suggest, for example, that an individually randomised trial is not possible and so the team may consider a cluster trial as the main trial. The results from the pilot may suggest the main outcome variable may be difficult to measure accurately, so other outcome measures might be proposed. Note that changing the outcome variable from what is proposed in the protocol to that which appears in the main trial report often highlights major generalisability issues with the trial, and so it is better to identify the main outcome variable while planning the trial.7
Examples
Haines aimed to describe the feasibility of an exercise intervention (reduced-exertion, high-intensity interval training) in non-diabetic patients with hyperglycaemia.8 The study intended to recruit 40 participants but achieved only 6. It revealed several issues, including patient eligibility, challenges to recruitment, patient consent and poor clinician engagement. The process of accurately screening and case finding eligible patients was problematic. The author concluded that a trial in its current form is not feasible but made suggestions as to changes that could occur in future studies.
An excellent trial in sports medicine that, nevertheless, might have benefited from piloting is the Intervention Study on Prevention of Injuries in Runners at Erasmus MC or INSPIRE trial.9 This study with 2378 participants aimed to reduce running injuries in recreational runners by providing evidence-based online advice on modifying known risk factors. It not only failed to show an effect, but the point estimates were in the wrong direction to that expected. It also had a high drop-out rate (28% compared with 5% expected). A feasibility or pilot study might have highlighted the drop-out rate, giving the authors a chance to try and make completion of the outcome questionnaire more attractive. It may also have suggested that the intervention as it stands would be unlikely to be beneficial to the participants, leading to an opportunity for modifying the intervention. However, it is good that such a large, negative trial was published because a priori the hypothesis was worth testing.
Acknowledgments
We thank members of the CONSORT team on Pilot and Feasibility trials for helpful discussion (Sandra Eldridge, Christine Bond, Sally Hopewell, Claire Chan and Lehana Thabane).
Footnotes
Contributors MJC drafted the manuscript. MAM and GL suggested revisions. All authors read and approved the final manuscript.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests GL is the editor-in-chief of the journal Pilot and Feasibility Studies.
Patient consent for publication Not required.
Provenance and peer review Not commissioned; internally peer reviewed.