ArticlesDoes quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses?
Introduction
The conduct of a meta-analysis is retrospective1 and is therefore susceptible to several sources of bias.2 Meta-analyses of randomised controlled trials (RCTs) include studies of variable methodological quality. Features of RCTs that confer the least biased estimates of treatment effect have been intensively studied lately. Differences in quality across trials may indicate that the results of some trials are more biased than others. Meta-analysts need to take this information into consideration to reduce or avoid bias whenever possible. Similarly, there are few data to guide reviewers as to whether any method of quality assessment provides a more biased estimate than any other. In this study, we addressed whether the method of quality assessment of RCTs by a validated scale approach rather than one involving individual components influences estimates of intervention efficacy.
Section snippets
Selection of meta-analyses
We randomly (random numbers table) selected 12 meta-analyses from our larger database of 491 meta-analyses of RCTs. Three inclusion criteria were used: that the report was published in English; that there was no formal incorporation of quality scores in the quantitative analysis; and that the outcomes were presented as binary data, reported as an overall quantitative summary result. Meta-analyses were excluded if the report did not provide references for the included trials. Nine of the
Trials
The 127 RCTs included in the 11 meta-analyses involved 10 492 patients. The 11 meta-analyses were published between 1988 and 1995 in ten journals or the Cochrane Database of Systematic Reviews. The trials on which they were based were published between 1960 and 1995, in 57 journals and three books. One study was unpublished. The majority of outcomes (15/22 [68%]) included can be defined as objective (eg, histological remission, major amputation, overall mortality, conception rate, smoking
Discussion
Assessment of the quality of reports of RCTs included in a meta-analysis adds another layer of complexity to the reviewing process. Our results suggest, however, that incorporation of an estimate of the quality of RCTs is important. We found a clinically important and statistically significant 30–50% exaggeration of treatment efficacy when results of lower-quality trials were pooled. Inflated estimates of treatment efficacy were found whether the trial quality assessments were made by a scale
References (30)
Bias in meta-analytic research
J Clin Epidemiol
(1992)- et al.
A meta-analysis of randomized trials of prenatal smoking cessation interventions
Am J Obstet Gynecol
(1994) - et al.
Assessing the quality of randomized clinical trials: is blinding necesssary?
Control Clin Trials
(1996) - et al.
Incorporating variations in the quality of individual randomized trials into meta-analysis
J Clin Epidemiol
(1992) Does blinding of readers affect the results of meta-analyses?
Lancet
(1997)- et al.
Assessing the quality of randomized controlled trials: an annotated bibliography of scales and checklists
Control Clin Trials
(1995) Problems induced by meta-analyses
Stat Med
(1991)- et al.
Rectal amniosalicylate therapy for distal ulcerative colitis: a meta-analysis
Aliment Pharmacol Ther
(1995) - et al.
Meta-analysis of the effect of placebo on the outcome of medically treated reflux esophagitis
Scand J Gastroenterol
(1995) - et al.
Sulfasalazine revisited: a meta-analysis of 5-aminosalicylic acid in the treatment of ulcerative colitis
Ann Intern Med
(1993)
Medical treatment of transient ischemic attacks: does it influence mortality?
Stroke
Treatment of deep venous thrombosis with low-molecular-weight heparins: a meta-analysis
Arch Intern Med
A meta-analysis of randomized placebo control trials in Fontaine stages III and IV peripheral occlusive arterial disease
Int Angiol
An overview of family interventions and relapse on schizophrenia: meta-analysis of research findings
Psychol Med
Continuation and maintenance therapy with antidepressive agents: meta-analysis of research
Pharmaceutisch Weekblad
Cited by (2812)
Instrumental assessment of sleep bruxism: A systematic review and meta-analysis
2024, Sleep Medicine ReviewsMaternal outcomes using delayed pushing versus immediate pushing in the second stage of labour: An umbrella review
2024, International Journal of Nursing Studies