Objectives: It has been shown that human clinical trials that lack randomization (RND) or blinding (BLD) often overestimate the magnitude of treatment effects. However, no studies have evaluated the effect of RND and BLD on animal research. The authors' objectives were to determine the proportion of animal studies presented at a national academic emergency medicine meeting that utilize randomization, blinding, or both; and to determine whether failure to employ these techniques changes the likelihood of observing a difference between treatment groups.
Methods: Two trained researchers reviewed abstracts presented at the 1997-2001 Society for Academic Emergency Medicine (SAEM) annual meetings using a standard data collection sheet. Studies that used an animal or cell line, compared two or more study groups, and measured an effect caused by the intervention or drugs were included. Studies were classified as randomized (RND+) if any part of the experiment involved random assignment of subjects to treatment groups, blinded (BLD+) if any assessment of the outcome was made by an investigator blinded to treatment group, and outcome-positive (Outcome+) if any difference between the study groups met the author's definition of significant. Following the initial review, differences in classification were resolved by consensus. The association between outcome and study methodology (RND, BLD or both) was measured using odds ratios (ORs) with 95% confidence intervals (95% CIs).
Results: A total of 2,592 studies were published as abstracts. Three hundred eighty-nine were animal studies, and 290 of these studies had two or more study groups. RND- and BLD- studies were more likely to be Outcome+ than RND+ or BLD+ studies (OR = 3.4; 95% CI = 1.7 to 6.9 and OR = 3.2; 95% CI = 1.3 to 7.7, respectively). When studies that used both RND and BND were compared with studies that used neither, the OR for a positive study was 5.2 (95% CI = 2.0 to 13.5).
Conclusions: These results suggest that animal studies that do not utilize RND and BLD are more likely to report a difference between study groups than studies that employ these methods.