Methodology
Methods of Formal Consensus in Classification/Diagnostic Criteria and Guideline Development

https://doi.org/10.1016/j.semarthrit.2010.12.001Get rights and content

Guidelines or diagnostic criteria in clinical practice assist physicians in their clinical decision-making and improve health outcomes for patients. Diagnostic and classification criteria should be based on evidence from rigorously conducted controlled studies. Formal group consensus methods have been developed to organize subjective judgments and to synthesize them with the available evidence. This review discusses 4 types of formal consensus methods used in the health field and their applications in rheumatology: the Delphi method, Nominal Group Technique, RAND/UCLA Appropriateness Method, and National Institutes of Health consensus development conference.

Section snippets

Defining Consensus

Consensus does not require full agreement among participants (7). A prespecified range is decided by the group running or leading the consensus methodology, according to the needs of the task to be performed. There are several methods for defining consensus with examples below (8):

  • A final vote determining percentage agreement (eg, 80%) among participants

  • Rating scale—a specified mean rating must be achieved on each topic for inclusion by group

  • A majority of participants must give a topic a

Participants

The composition of the group is important in determining the decision reached. Most agree that a participant should be an expert with credibility in the appropriate field (8, 9). An expert can be a clinician with vast clinical expertise, a researcher who is well versed with the current literature, or a layperson or patient who has experienced the impact of the disease or intervention or condition in question (4, 10). Ideal characteristics of a group participant would be researchers and

Importance of Evidence from Literature When Using Group Consensus

Although group consensus participants are generally recruited on the basis that they have superior knowledge of the published literature in the field, it is essential to supplement participants with up-to-date, evidence-based literature. When literature reviews are provided, the evidence is used both in the initial discussion and in later decision-making (14). A literature review (preferably a systematic review) provides a common starting point for the group and increases the perception that

Formal Consensus Methods

The focus will be on 4 types of formal consensus methods used in the health field: Delphi method, Nominal Group Technique, RAND/UCLA Appropriateness Method (RAM), and National Institutes of Health (NIH) consensus development conference methodology (Table 1). In practice, a combination of 2 formal consensus methods or their modifications can be used in a 2-step process, where 1 method is used for item generation or some initial consensus and then the other method is used for final consensus.

Grading of Recommendations Assessment, Development, and Evaluation (GRADE)

This review also discusses GRADE as it has been adopted by scientific world for the development of recommendations. GRADE is not a consensus methodology per se but uses consensus methods discussed above (especially Delphi and NGT) to assess quality and strength of a recommendation. GRADE methodology provides a systematic and transparent approach to rate the quality of evidence and grade the strength of recommendations for patient important outcomes. GRADE was developed by experts with a goal to

Limitations of Review

Although we present a comprehensive review of formal consensus methods, we did not conduct a systematic review. In our comprehensive review, the majority of the articles on formal consensus methodologies are implementations in diagnostic/classification criteria, guidelines, and response indices development.

Conclusion

Formal consensus methodologies have various applications in the medical literature from guideline development to development of criteria sets. These methods are being increasingly used in rheumatology. Each methodology has its unique attributes, and utilization of a particular methodology or combination of methodologies depends on the following: (1) clinical question; (2) audience; and (3) available resources.

When closely tied to evidence-based literature, these summarized statements can

Acknowledgments

We would like to acknowledge Drs. Carol Wallace and Gillian Hawker for providing constructive feedback during the writing of this manuscript.

References (67)

  • F. Zulian et al.

    The pediatric rheumatology European Society/American College of Rheumatology/European League Against Rheumatism provisional classification criteria for juvenile systemic sclerosis

    Arthritis Rheum

    (2007)
  • H.A. Linstone et al.

    The Delphi method: Techniques and applications

    (1975)
  • A. Fink et al.

    Consensus methods: Characteristics and guidelines for use

    Am J Public Health

    (1984)
  • J. Jones et al.

    Consensus methods for medical and health services research

    BMJ

    (1995)
  • K.G. Saag et al.

    American College of Rheumatology 2008 recommendations for the use of nonbiologic and biologic disease-modifying antirheumatic drugs in rheumatoid arthritis

    Arthritis Rheum

    (2008)
  • S. Jackson

    Team composition in organizational settings: Issues in managing an increasingly diverse work force

  • A. Vinokur et al.

    Group decision making by experts: Field study of panels evaluating medical technologies

    J Pers Soc Psychol

    (1985)
  • J.P. Kahan et al.

    Variations by specialty in physician ratings of the appropriateness and necessity of indications for procedures

    Med Care

    (1996)
  • I. Jacoby

    Evidence and consensus

    JAMA

    (1988)
  • H. Cruse et al.

    Quality and methods of developing practice guidelines

    BMC Health Serv Res

    (2002)
  • N.C. Dalkey

    Delphi

    (1967)
  • O. Helmer-Hirschberg

    The use of the Delphi technique in problems of educational innovations

    (1966)
  • N.C. Dalkey et al.

    An experimental application of the Delphi method to the use of experts

    (1962)
  • A.L. Delbecq et al.

    Group techniques for program planning: A guide to nominal group and delphi processes

    (1975)
  • M.K. Murphy et al.

    Consensus development methods, and their use in clinical guideline development

    Health Technol Assess

    (1998)
  • D. Khanna et al.

    Development of a provisional core set of response measures for clinical trials of systemic sclerosis

    Ann Rheum Dis

    (2008)
  • O. Distler et al.

    Defining appropriate outcome measures in pulmonary arterial hypertension related to systemic sclerosis: A Delphi consensus study with cluster analysis

    Arthritis Rheum

    (2008)
  • K.G. Loughlin et al.

    Using Delphi to achieve congruent objectives and activities in a pediatrics department

    J Med Educ

    (1979)
  • W.A. Thomson et al.

    Use of delphi methodology to generate a survey instrument to identify priorities for state allied health associations

    Allied Health Behav Sci

    (1979)
  • G. Bertsias et al.

    EULAR recommendations for the management of systemic lupus erythematosusReport of a task force of the EULAR standing committee for international clinical studies including therapeutics

    Ann Rheum Dis

    (2008)
  • H. Gazi et al.

    Outcome measurements in scleroderma: Results from a Delphi exercise

    J Rheumatol

    (2007)
  • C.A. Wallace et al.

    Preliminary criteria for clinical remission for select categories of juvenile idiopathic arthritis

    J Rheumatol

    (2004)
  • L.G. Rider et al.

    International consensus on preliminary definitions of improvement in adult and juvenile myositis

    Arthritis Rheum

    (2004)
  • Cited by (281)

    View all citing articles on Scopus

    Funding Source: No funding was obtained for the study design, collection, analysis, and interpretation of data; in the writing of the manuscript; and in the decision to submit the manuscript for publication. Dr. Khanna was supported by a National Institutes of Health Award (NIAMS K23 AR053858-04).

    None of the authors have competing interests related to the content of this manuscript.

    1

    Both authors contributed equally to the manuscript.

    View full text