Article Text
PDF
Statistics from Altmetric.com
Despite efforts to improve the statistical quality of research articles in medical journals, serious statistical errors or deficiencies in the design, analysis, reporting and interpretation still occur, even in highly ranked journals.1 Flawed statistics and methodology will negatively affect the study results and could consequently impact public health and patient care.2 Despite numerous educational papers on biostatistics as well as reporting guidelines including CONsolidated Standards Of Reporting Trials, STrengthening the Reporting of OBservational studies in Epidemiology, STAndards for the Reporting of Diagnostic accuracy studies, REporting recommandations for tumor MARKer prognostic studies, and Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (and others as listed in the Enhancing the QUAlity and Transparency Of health Research network; www.equator-network.org) endorsed by many journals, the methodological quality of medical publications still remains low.3 Editors and reviewers may not have expert knowledge of statistics, and worse, could remain unconvinced about the importance of solid methodology in medical research.4 Thus, a systematic approach to assess the methodological or statistical aspects of a scientific paper is needed.
Introducing the CHecklist for statistical Assessment of Medical Papers (CHAMP) statement
Although there are some excellent guidelines on reporting statistics in medical papers5 6 and further direction available from a small number of journals, a checklist for peer reviewers (and readers) to assess general statistical aspects in a research publication is lacking. In this paper, we present CHecklist for statistical Assessment of Medical Papers (CHAMP), which contains 30 items on general statistical aspects to assess during peer review of original papers (online supplemental appendix). The checklist includes considerations in the following sections: design and conduct (items 1–6), data analysis (items 7–16), reporting and presentation (items 17–23) and interpretation (items 24–30). A complete explanation and elaboration of the 30-item checklist with glossary of statistical terms is provided in a companion paper.7 The items in the checklist were selected based on a previous British Medical Journal checklist,8 literature review and experience of the author panel in reviewing the statistical content of numerous papers submitted to a variety of medical journals. The first author produced the checklist draft, the coauthors suggested addition or removal of the items and all authors approved the final version. Other colleagues provided extensive comments on the paper and are listed in the ‘Acknowledgments’ section of the explanation and elaboration paper.
Supplemental material
CHAMP does not cover all topics of medical statistics but focuses on important and common statistical issues that may generally arise. We appreciate that each type of study or statistical model such as a randomised trial or prediction model has specific issues which may not be covered in our checklist. We also note that for some items in the checklist there may be no decisive answer, and thus assessment of the methodology of a paper may involve some subjectivity. Moreover, the issues raised in the checklist are not equally important—for example, serious errors in design are irremediable regardless of how the data were analysed and problems of presentation are less important (as these can be easily fixed) than other statistical problems.
Applying CHAMP during peer review
Using CHAMP requires some elementary knowledge of statistics, as is also needed for the authors of scientific manuscripts. Further guidance on how to use the checklist can be found in the explanation and elaboration paper.7 Each item of the checklist is a reminder for the reviewer in formulating an overall assessment of the statistical analysis of the paper and perhaps in providing clarifying comments and revision requests to the authors. Future study of the CHAMP statement is needed to examine its utility and possibly establish a point system for rating the appropriateness of the statistical and methodological aspects of an original investigation.
In the interim, we hope CHAMP provides a useful tool in the editorial process for editors and referees for the statistical assessment of medical papers.
Ethics statements
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Twitter @RUNSAFE_Rasmus
Deceased Douglas G Altman’s deceased date: 3 June 2018
Contributors MAM, MN and DGA produced the first draft. GSC, RON, NPJ and MJC suggested revisions. All authors approved the final version.
Funding GSC was supported by the National Institute for Health Research Biomedical Research Centre, Oxford, and Cancer Research UK (programme grant: C49297/A27294).
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.