Basic statistical reporting for articles published in Biomedical Journals: The “Statistical Analyses and Methods in the Published Literature” or the SAMPL Guidelines

https://doi.org/10.1016/j.ijnurstu.2014.09.006Get rights and content

Introduction

The first major study of the quality of statistical reporting in the biomedical literature was published in 1966 (Schor and Karten, 1966). Since then, dozens of similar studies have been published, every one of which has found that large proportions of articles contain errors in the application, analysis, interpretation, or reporting of statistics or in the design or conduct of research (see, for example, Altman, 1991, Avram et al., 1985, Bakker and Wicherts, 2011, Gardner et al., 1983, Glantz, 1980, Godfrey, 1985, Gore et al., 1977, Kurichi and Sonnad, 2006, Lionel and Herxheimer, 1970, Murray, 1991, Nagele, 2003, Neville et al., 2006, Pocock et al., 1987, Scales et al., 2005, White, 1979, Yancy, 1990). Further, large proportions of these errors are serious enough to call the authors’ conclusions into question (Glantz, 1980, Murray, 1991, Yancy, 1990). The problem is made worse by the fact that most of these studies are of the world's leading peer-reviewed general medical and specialty journals.

Although errors have been found in more complex statistical procedures (Burton and Altman, 2004, Mackinnon, 2010, Schwarzer et al., 2000), paradoxically, many errors are in basic, not advanced, statistical methods (George, 1985). Perhaps advanced methods are suggested by consulting statisticians, who then competently perform the analyses, but it is also true that authors are far more likely to use only elementary statistical methods, if they use any at all (Emerson and Colditz, 1985, George, 1985, Golden et al., 1996, Lee et al., 2004). Still, articles with even major errors continue to pass editorial and peer review and to be published in leading journals.

The truth is that the problem of poor statistical reporting is long-standing, widespread, potentially serious, concerns mostly basic statistics, and yet is largely unsuspected by most readers of the biomedical literature (Lang and Secic, 2006).

More than 30 years ago, O’Fallon and colleagues recommended that “Standards governing the content and format of statistical aspects should be developed to guide authors in the preparation of manuscripts” (O’Fallon et al., 1978). Despite the fact that this call has since been echoed by several others (Altman and Bland, 1991, Altman et al., 1983, Hayden, 1983, Murray, 1991, Pocock et al., 1987, Shott, 1985), most journals have still not included in their Instructions for Authors more than a paragraph or two about reporting statistical methods (Bailar and Mosteller, 1988). However, given that many statistical errors concern basic statistics, a comprehensive — and comprehensible — set of reporting guidelines might improve how statistical analyses are documented.

In light of the above, we present here a set of statistical reporting guidelines suitable for medical journals to include in their Instructions for Authors. These guidelines tell authors, journal editors, and reviewers how to report basic statistical methods and results. Although these guidelines are limited to the most common statistical analyses, they are nevertheless sufficient to prevent most of the reporting deficiencies routinely found in scientific articles; they may also help to prevent some reporting errors by focusing attention on key points in the analyses.

Unlike many of other guidelines, the SAMPL guidelines were not developed by a formal consensus-building process, but they do draw considerably from published guidelines (Bond et al., 1995, Curran-Everett and Benos, 2004, Curran-Everett and Benos, 2007, Lang and Secic, 2006, Wilkinson and Task Force on Statistical Inference, 1999).

In addition, a comprehensive review of the literature on statistical reporting errors reveals near universal agreement on how to report the most common methods (Lang and Secic, 2006).

Statistical analyses are closely related to the design and activities of the research itself. However, our guidelines do not address the issues related to the design and conduct of research. Instead, we refer readers to the EQUATOR Network website (www.equator-network.org) where guidelines for reporting specific research designs can be found (for example, see the CONSORT (Moher et al., 2001), TREND (Des Jarlais et al., 2004), STROBE (von Elm et al., 2007)). These guidelines for reporting methodologies all include items on reporting statistics, but the guidelines presented here are more specific and complement, not duplicate, those in the methodology guidelines.

We welcome feedback and anticipate the need to update this guidance in due course.

Section snippets

Guiding principles for reporting statistical methods and results

Our first guiding principle for statistical reporting comes from The International Committee of Medical Journal Editors, whose Uniform Requirements for Manuscripts Submitted to Biomedical Journals include the following excellent statement about reporting statistical analyses:

Describe statistical methods with enough detail to enable a knowledgeable reader with access to the original data to verify the reported results. [Emphasis added.] When possible, quantify findings and present them with

Preliminary analyses

  • Identify any statistical procedures used to modify raw data before analysis. Examples include mathematically transforming continuous measurements to make distributions closer to the normal distribution, creating ratios or other derived variables, and collapsing continuous data into categorical data or combining categories.

Primary analyses

  • Describe the purpose of the analysis.

  • Identify the variables used in the analysis and summarize each with descriptive statistics.

  • When possible, identify the smallest difference

Reporting numbers and descriptive statistics

  • Report numbers — especially measurements — with an appropriate degree of precision. For ease of comprehension and simplicity, round to a reasonable extent. For example, mean age can often be rounded to the nearest year without compromising either the clinical or the statistical analysis. If the smallest meaningful difference on a scale is 5 points, scores can be reported as whole numbers; decimals are not necessary.

  • Report total sample and group sizes for each analysis.

  • Report numerators and

First page preview

First page preview
Click to open first page preview

References (42)

  • G.R. Bond et al.

    Statistical guidelines for the Archives of PM&R

    Arch. Phys. Med. Rehabil.

    (1995)
  • J.E. Kurichi et al.

    Statistical methods in the surgical literature

    J. Am. Coll. Surg.

    (2006)
  • P. Nagele

    Misuse of standard error of the mean (SEM) when reporting variability of a sample. A critical evaluation of four anaesthesia journals

    Br. J. Anaesth.

    (2003)
  • D.G. Altman

    Statistics in medical journals: developments in the 1980s

    Stat. Med.

    (1991)
  • D.G. Altman et al.

    Improving doctors’ understanding of statistics

    J. R. Stat. Soc. A

    (1991)
  • D.G. Altman et al.

    Statistical guidelines for contributors to medical journals

    BMJ

    (1983)
  • M.J. Avram et al.

    Statistical methods in anesthesia articles: an evaluation of two American journals during two six-month periods

    Anesth. Analg.

    (1985)
  • J.C. Bailar et al.

    Guidelines for statistical reporting in articles for medical journals. Amplifications and explanations

    Ann. Intern. Med.

    (1988)
  • M. Bakker et al.

    The (mis)reporting of statistical results in psychology journals

    Behav. Res.

    (2011)
  • A. Burton et al.

    Missing covariate data within cancer prognostic studies: a review of current reporting and proposed guidelines

    Br. J. Cancer

    (2004)
  • D. Curran-Everett et al.

    American Physiological Society Guidelines for reporting statistics in journals published by the American Physiological Society

    Am. J. Physiol. Endocrinol. Metab.

    (2004)
  • D. Curran-Everett et al.

    Guidelines for reporting statistics in journals published by the American Physiological Society: the sequel

    Adv. Physiol. Educ.

    (2007)
  • D.C. Des Jarlais et al.

    Improving the reporting quality of nonrandomized evaluations of behavioral and public health interventions: the TREND statement

    Am. J. Public Health

    (2004)
  • J.D. Emerson et al.

    The statistical content of published medical research: some implications for biomedical education

    Med. Educ.

    (1985)
  • J.E.D. Esquirol

    Cited in: Pearl R. Introduction to Medical Biometry and Statistics

    (1941)
  • M.J. Gardner et al.

    Is the statistical assessment of papers submitted to the British Medical Journal effective?

    BMJ

    (1983)
  • S.L. George

    Statistics in medical journals: a survey of current policies and proposals for editors

    Med. Pediatr. Oncol.

    (1985)
  • S.A. Glantz

    Biostatistics: how to detect, correct and prevent errors in the medical literature

    Circulation

    (1980)
  • K. Godfrey

    Comparing the means of several groups

    N. Engl. J. Med.

    (1985)
  • J. Golden et al.

    A review of the statistical analysis used in papers published in Clinical Radiology and British Journal of Radiology

    Clin. Radiol.

    (1996)
  • S.M. Gore et al.

    Misuse of statistical methods: critical assessment of articles in BMJ from January to March 1976

    BMJ

    (1977)
  • Cited by (254)

    View all citing articles on Scopus

    This paper was originally published in: Smart P, Maisonneuve H, Polderman A (eds). Science Editors’ Handbook, European Association of Science Editors, 2013. Reproduced with kind permission as part of a series of Classic Methods papers. An introductory Commentary is available as http://dx.doi.org/10.1016/j.ijnurstu.2014.09.007.

    View full text