Introduction
What is new?
- 1.
Interventions to change clinical practice should be evaluated using the most rigorous design feasible, considering the need for local vs. generalizable knowledge.
- 2.
This article summarizes the rationale and methods for evaluating knowledge translation interventions, considering the perspectives of managers and researchers.
- 3.
Managers should routinely incorporate simple evaluation designs into program implementation to increase the impact of their services.
- 4.
Researchers should contribute to the development of cumulative knowledge of effectiveness of interventions by conducting rigorous evaluations with rich descriptions of context and implementation, exploring reasons for success and failure.
Evidence based medicine should be complemented by evidence based implementation—Richard Grol
The field of knowledge translation (KT) promotes the uptake of evidence-based practices, but the methods used to promote these practices are often not themselves evidence based [1], [2]. There is a dearth of information about which interventions work and under which circumstances [3]. This may encourage people and thus pressure to attempt to improve quality of care using common sense, rather than evidence-based approaches [4]. The complexity of implementation research is daunting. It requires taking into account multiple levels: patients nested within a provider’s practice nested within a multidisciplinary team nested within a health facility nested in local and national health care systems. The conceptual and methodological challenges are significant. These challenges are likely why the average impact of implementation interventions has so far been found to be modest [5]. Furthermore, the conclusions one can draw about what are the most effective approaches and how they should be applied in a given setting are also limited [6]. For example, the Cochrane Collaboration has registered over 350,000 randomized controlled trials in clinical medicine [1] but only 2,400 experimental and quasi-experimental trials of interventions to improve health care delivery [7]. The complexity of changing the behavior of organizations and service providers and the potential benefits of doing this warrants a partial shift in effort from the development of new treatments to developing approaches to consistently implement treatments that are already known to work [8]. For instance, a recent study showed that a checklist of simple, routine, and evidence-based procedures for surgical safety had a profound effect on mortality and complications rates [9]. In addition to being successful, this intervention was extremely cost-effective and fairly simple to roll out. Shifting focus toward creating and evaluating interventions to implement evidence-based knowledge will be essential for improving health services.
In this article, we outline the need for evaluation of KT initiatives and efforts, describe methodologies for studies of effectiveness, and suggest how to choose amongst these and how to combine these with qualitative studies to explain mechanisms of effect.