Article Text

Download PDFPDF
Are questionable research practices facilitating new discoveries in sport and exercise medicine? The proportion of supported hypotheses is implausibly high
  1. Fionn Büttner1,
  2. Elaine Toomey2,
  3. Shane McClean1,
  4. Mark Roe1,
  5. Eamonn Delahunt1,3
  1. 1 School of Public Health, Physiotherapy and Sports Science, University College Dublin—National University of Ireland, Dublin, Ireland
  2. 2 School of Allied Health, University of Limerick, Limerick, Ireland
  3. 3 Institute for Sport and Health, University College Dublin, Dublin, Ireland
  1. Correspondence to Mr Fionn Büttner, School of Public Health, Physiotherapy and Sports Science, University College Dublin—National University of Ireland, Dublin, Dublin 4, Ireland; fionn.cleirigh-buttner{at}ucdconnect.ie

Abstract

Questionable research practices (QRPs) are intentional and unintentional practices that can occur when designing, conducting, analysing, and reporting research, producing biased study results. Sport and exercise medicine (SEM) research is vulnerable to the same QRPs that pervade the biomedical and psychological sciences, producing false-positive results and inflated effect sizes. Approximately 90% of biomedical research reports supported study hypotheses, provoking suspicion about the field-wide presence of systematic biases to facilitate study findings that confirm researchers’ expectations. In this education review, we introduce three common QRPs (ie, HARKing, P-hacking and Cherry-picking), perform a cross-sectional study to assess the proportion of original SEM research that reports supported study hypotheses, and draw attention to existing solutions and resources to overcome QRPs that manifest in exploratory research. We hypothesised that ≥ 85% of original SEM research studies would report supported study hypotheses. Two independent assessors systematically identified, screened, included, and extracted study data from original research articles published between 1 January 2019 and 31 May 2019 in the British Journal of Sports Medicine, Sports Medicine, the American Journal of Sports Medicine, and the Journal of Orthopaedic & Sports Physical Therapy. We extracted data relating to whether studies reported that the primary hypothesis was supported or rejected by the results. Study hypotheses, methodologies, and analysis plans were preregistered at the Open Science Framework. One hundred and twenty-nine original research studies reported at least one study hypothesis, of which 106 (82.2%) reported hypotheses that were supported by study results. Of 106 studies reporting that primary hypotheses were supported by study results, 75 (70.8%) studies reported that the primary hypothesis was fully supported by study results. The primary study hypothesis was partially supported by study results in 28 (26.4%) studies. We detail open science practices and resources that aim to safe-guard against QRPs that bely the credibility and replicability of original research findings.

  • methodological
  • education
  • research
  • statistics
  • sport
View Full Text

Statistics from Altmetric.com

Footnotes

  • Twitter @peanutbuttner, @ElaineToomey1, @markroecoach, @EamonnDelahunt

  • Contributors FCB conceived the original idea, developed data extraction materials and composed the initial manuscript. FCB, ET, SMC and MR independently performed data extraction. ED arbitrated inter-rater disagreement. FCB, ET, SMC and ED provided comments on and contributed towards the revision of the final manuscript.

  • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

  • Competing interests None declared.

  • Patient consent for publication Not required.

  • Provenance and peer review Not commissioned; externally peer reviewed.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.