Article Text

PDF

The quality of research in sports journals
  1. C Bleakley1,
  2. D MacAuley2
  1. 1Institute of Postgraduate Medical and Health Science, University of Ulster, Jordanstown BT36 0QB, UK
  2. 2Department of Epidemiology, The Queen's University of Belfast, Mulhouse Building, Belfast, Northern Ireland, UK
  1. Correspondence to:
 Professor MacAuley, Department of Epidemiology, The Queen's University of Belfast, Mulhouse Building, Belfast, Northern Ireland, UK;
 dc.macauley{at}ulst.ac.uk

Abstract

Objective: To examine the evidence base of sports medicine research.

Methods: A sample of four major journals that present core research in sport and exercise medicine (British Journal of Sports Medicine, Medicine and Science in Sports and Exercise, Journal of Sports Medicine and Physical Fitness, and Physical Therapy) was examined using assessment criteria taken from the READER method.

Results: Randomised controlled trials comprised 10% or less of all original research articles. Observational/descriptive studies were the most commonly published study design. There was a highly significant difference (p<0.0001) in the contents of the four journals but when they were compared by categorising the better quality methods together (randomised control trial, case-control, and cohort studies), the difference was not significant (p = 0.09).

Conclusions: The overall pattern of publication type seems remarkably stable over medical journals, indicating that the quality of sports medicine research is comparable to that in other specialities.

  • research
  • quality
  • evidence basis
  • study types

Statistics from Altmetric.com

Much of sports medicine has developed empirically, and current practice often reflects practitioner experience rather than evidence from research findings. There is, however, considerable emphasis on evidence based medicine in other specialties. Few of us would argue with the importance of practising evidence based sports medicine, but to develop such a culture we must have quality research literature. Furthermore, as sports medicine develops and doctors with an interest seek formal specialist accreditation, there will be increasing professional and public interest in quality of research. The evolving specialty of sport and exercise medicine should therefore be prepared to measure itself against the criteria used in other branches of medicine, and one of the most important is the quality of the literature.

Healthcare interventions should be evaluated by the most scientifically sound and rigorous methods possible.1 There are a number of methods available, ranging from single patient case studies to the double blind, placebo controlled trial. Highlighting one particular study design as a panacea may be difficult. Randomised and observational studies have complementary roles,2 their value and applicability governed by the unique set of circumstances surrounding every research question. However, all things being equal, a well designed randomised controlled trial or a systematic review based on literature from all available randomised controlled trials in a particular field will provide the strongest evidence to support clinical practice and be the most likely to attribute effects to causes.2 Various scoring criteria are used to assess the quality of research based on their methods, and we used the assessment criteria taken from the READER method.3 This is a simple tool that provides a valid and reliable method of critically evaluating medical literature.4,5

Few studies have examined the quality of sports medicine literature. Thompson6 reviewed the contents of British Journal of Sports Medicine in the five year period from 1991 to 1995 inclusive. He found that a considerable number of original articles were observational/descriptive studies (41%) with few randomised controlled trials (3%). In this study we examine a convenient sample of four major journals that present core research in sport and exercise medicine: British Journal of Sports Medicine, Medicine and Science in Sports and Exercise (consistently ranked in the top three sports science journals between 1996 and 2000, according to the citation index), Journal of Sports Medicine and Physical Fitness, and Physical Therapy.

The aim of this study is to examine the evidence base of sports medicine research. The specific objectives are to identify the quality of the methods used in research published in four key sports related journals and to compare the journals.

METHOD

We examined the contents of the four journals using criteria modified from the appraisal component of the READER method.3 The study was restricted to original research articles published in these journals in the five year period 1996–2000 inclusive. We classified the studies according to the method. The first group comprised randomised controlled trials. A second group of quasi-experimental designs was divided into three categories: cohort studies, case-control studies, and single case repeated measures design. We included a further group of observational/descriptive studies and finally a group of case reports and studies using other methods. We compared the journals according to the number of studies in each group using κ2 analysis.

RESULTS

We identified 1051 original articles and classified them into the various groups (table 1). Randomised controlled trials comprised 10% or less of all original research articles, with a slightly greater proportion of cohort studies (10.5%). Observational/descriptive studies were the most commonly published study design, and the British Journal of Sports Medicine contained more case reports. In total, cohort, case-control, and single case study designs comprised nearly 46% of published work, across all four journals, with observational/descriptive and case studies making up the remaining 43%. We compared the journals according to the percentages of each study type, and there was a highly significant difference in their contents (p<0.0001) because of the high proportion of case studies in the British Journal of Sports Medicine and fewer case studies in Medicine and Science in Sports and Exercise. When the journals were compared by categorising the better quality methods together (randomised control trial, case-control, and cohort studies), the difference between the journals was not significant (p = 0.09), although there are a greater percentage of better quality articles in Medicine and Science in Sports and Exercise (37.5%) than in Physical Therapy (28.2%).

Table 1

Classification of study type in original papers published in the period 1996–2000 inclusive

DISCUSSION

Overall, the proportion of randomised controlled trials seems low (9.5%), but we can compare the proportion of such studies published in sports medicine with other specialties, albeit within slightly different time periods. In a study of three UK primary care journals, the British Journal of General Practice, Family Practice, and the British Medical Journal, more than 50% of studies published in a five year period were either qualitative studies or surveys of attitude and opinion. Overall, just 6% of studies were randomised controlled trials, although in the British Medical Journal the proportion was greater at 16% (42).7 Similarly, the proportion of randomised controlled trials published in US family medicine8 is relatively small at 3.4%. In a review of research published in nine general surgical journals, 46% were case series, with only 7% randomised controlled trials.9 A study of six community health journals found 4% randomised controlled trials and suggested that 42% of the remaining trials could have used a randomised study design.10 In a study of seven leading rheumatological journals, almost half of the published studies were descriptive, with 16% classified as randomised controlled trials.11 Therefore the proportion of studies identified in sports medicine in which high quality methods were used compares well with other medical specialities, in particular, general practice and surgery, but there were more high quality studies in rheumatology journals. Although we compared these journals directly, this may be unfair as contents are determined by both editorial policy and the quality of material submitted.

Notwithstanding these factors, the overall pattern of publication type seems remarkably stable and can be seen to reflect the overall quality of sports medicine research. Most encouraging is the finding that the proportion of randomised controlled trials published in the British Journal of Sports Medicine has increased from 3% to 7% in the two five year periods compared.

The importance of developing a high quality research base within sports medicine is twofold. Primarily, it will provide patients with diagnostic tests that are more reliable and interventions that are more efficacious. Furthermore undertaking and publishing high quality research studies will enhance the reputation of sport and exercise medicine as an academic discipline.12 Attracting funding for clinical research can be difficult, and trials of clinical interventions are complex and time consuming. The nature of sports medicine may also attract its own particular set of problems. Highly motivated athletes, for example, may be unhappy to be allocated to a control group so that recruitment of elite athletes, in particular, into randomised controlled trials may be problematic. Furthermore, sports medicine as a discipline relates to people across the sporting spectrum, from the recreational athlete to the highly paid professional. These population groups may differ greatly in the nature and intensity of physical activity and injury, and findings in one group may not be directly applicable to others.

Take home message

The number of randomised controlled trials published in the British Journal of Sports Medicine has increased over the past four years. The evidence base for sports medicine must continue to increase in terms of volume and quality, so that it can fully evolve from a clinical interest group to a recognised medical specialty.

REFERENCES

View Abstract

Footnotes

  • Conflict of interest: Domhnall MacAuley was the editor of the British Journal of Sports Medicine from 1996–2001.

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.