Abstract
OBJECTIVE: To describe the current policies regarding statistical review of clinical research in biomedical journals.
DESIGN: Cross-sectional survey.
PARTICIPANTS: Editors of biomedical journals that publish original clinical research.
MEASUREMENTS: General policies on statistical review, types of persons used for statistical reviewing, compensation of statistical reviewers, percentage of articles subject to such review, percentage of time statistical review makes an important difference, journal circulation, and selectivity.
MAIN RESULTS: Of 171 journals, 114 (67%) responded to the survey. About one third of journals had policies that guaranteed statistical review for all accepted manuscripts. In approximately half of the journals, articles were sent for statistical review at the discretion of the editor. There was some evidence that statistical review policies differed between journals of different circulation size. In journals in the top quartile of circulation (> 25,000) the probability of definitely having a statistical review before an acceptance decision was 52%, but it was only 27% in journals in the lower three quartiles (p=.09). The probability of a statistical consultant on staff ranged from 31% in the bottom quarter, to 58% in the middle two, to 82% in the highest quarter (p < .001). Editors judged that statistical review resulted in an important change in a manuscript about half of the time.
CONCLUSIONS: Except in the largest circulation medical journals, the probability of formal methodologic review of original clinical research is fairly low. As readers and researchers depend on the journals to assess the validity of the statistical methods and logic used in published reports, this is potentially a serious problem. This situation may exist because the cost of such statistical review can be considerable, and because finding appropriate reviewers can be difficult. It may also exist partly because editors or publishers may not regard such review as important. The professions of medical publishing, statistics, epidemiology, and other quantitative disciplines should work together to address this problem.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Altman DG, Bland JM. Improving doctors’ understanding of statistics. J R Stat Soc A. 1991;154:223–67.
Weiss ST, Samet JM. An assessment of physician knowledge of epidemiology and biostatistics. J Med Educ. 1980;55:692–7.
Wulff HR, Andersen B, Brandenhoff P, Guttler F. What do doctors know about statistics? Stat Med. 1987;6:3–10.
Borak J, Veiullieux S. Errors of intuitive logic among physicians. Soc Sci Med. 1982;16:1939–47.
Friedman SB, Phillips S. What’s the difference? Pediatric residents and their inaccurate concepts regarding statistics. Pediatrics. 1981;68:644–6.
Pocock SJ, Hughes MD, Lee RJ. Statistical problems in the reporting of clinical trials: a survey of three medical journals. N Engl J Med. 1987;317:426–32.
George SL. Statistics in medical journals: a survey of current policies and proposals for editors. Med Pediatr Oncol. 1985;13:109–12.
Garfield E. SCI Journal Citation Reports: A Bibliometric Analysis of Science Journals in the ISI Database. Philadelphia, Pa: Institute for Scientific Information; 1993.
Hansson S. Impact factor as a misleading tool in evaluation of medical journals. Lancet. 1995;346:906. Letter.
Dawson-Saunders B, Azen S, Greenberg RS, Reed AH. The instruction of biostatistics in medical schools. Am Statistician. 1987;41:263–6.
Hacker CS. Medical statistics at the Medical School of the University of Texas Health Center at Houston. Am Statistician. 1987;41:266–7.
Appleton D. What statistics should we teach medical undergraduates and graduates? Stat Med. 1990;9:1013–21.
Gore S, Jones IG, Rytter EC. Misuse of statistical methods: critical assessment of articles in BMJ from January to March 1976. BMJ. 1977;1:85–7.
Murray GD. The task of a statistical referee. Br J Surg. 1988;75:664–7.
McGuigan SM. The use of statistics in the British Journal of Psychiatry. Br J Psychiatry. 1995;167:685–8.
Gore SM, Jones G, Thompson SG. The Lancet’s statistical review process: areas for improvement by authors. Lancet. 1992;340:100–2.
Concato J, Feinstein AR, Holford TR. The risk of determining risk with multivariable models. Ann Intern Med. 1993;118:201–10.
Gardner MJ, Bond J. An exploratory study of statistical assessment of papers published in the British Medical Journal. JAMA. 1990;263:1355–8.
Welch GE II, Gabbe SG. Review of statistics usage in the American Journal of Obstetrics and Gynecology. Am J Obstet Gynecol. 1996;175:1138–41.
Mainland D. Statistical ritual in clinical journals: is there a cure?—II. BMJ (Clin Res Educ). 1984;288(6421):920–2.
Mainland D. Statistical ritual in clinical journals: is there a cure?—I. BMJ (Clin Res Educ). 1984;288(6420):841–3.
Avram MJ, Shanks CA, Dykes MHM, Ronai AK, Stiiers WM. Statistical methods in anesthesia articles: an evaluation of two American journals during two six-month periods. Anesth Analg. 1985;64:607–11.
White SJ. Statistical errors in papers in the British Journal of Psychiatry. Br J Psychiatry. 1979;135:336–42.
Vaisrub N. Manuscript review from a statistician’s perspective. JAMA. 1985;253:3145–7.
Goodman SN, Berlin JA, Fletcher SW, Fletcher RH. Manuscript quality before and after peer review and editing at Annals of Internal Medicine. Ann Intern Med. 1994;121:11–21.
Schor S, Karten I. Statistical evaluation of medical manuscripts. JAMA. 1966;195:1123–8.
Gardner MJ, Altman DG, Jones DR. Is the statistical assessment of papers submitted to the British Medical Journal effective? BMJ. 1983;286:1485–8.
Pitkin RM. Statistical evaluation of manuscripts: it’s all in the numbers. Obstet Gynecol. 1994;83(6):1043–4. Editorial.
Statistical review for journals. Lancet. 1991;337(8733):84. Editorial. See comments.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Goodman, S.N., Altman, D.G. & George, S.L. Statistical reviewing policies of medical journals. J GEN INTERN MED 13, 753–756 (1998). https://doi.org/10.1046/j.1525-1497.1998.00227.x
Issue Date:
DOI: https://doi.org/10.1046/j.1525-1497.1998.00227.x