RAND Methodological Guidance for Conducting and Critically Appraising Delphi Panels

The Delphi method is an iterative, anonymous, structured, group-based communication process and elicitation technique designed to help policymakers make decisions under conditions of uncertainty and incomplete information. It is based on the premise that asking a hand-picked group of anonymous experts the same questions several times and sharing the other experts’ answers will help objectively develop group consensus, which is used as a form of evidence. Although it was originally developed by RAND researchers as a forecasting methodology in military research, Delphi underwent many modifications and is now used by different disciplines, most notably by medicine, as a gold-standard approach for expert elicitation and stakeholder engagement. Researchers often rely on Delphi to estimate the probability of an event happening within a certain period of time, to forecast when an event is likely to occur, and to identify and prioritize key policy issues that need to be addressed.

Although the Delphi method is used by different disciplines and for different purposes, there is still a lack of methodological guidance for how to conduct rigorous Delphi studies. This manual addresses this critical gap by providing practical advice for when to use Delphi, what type of Delphi to choose, and what to consider when designing, implementing, and reporting Delphi panel results. The manual also includes the Delphi Critical Appraisal Tool (DCAT), which researchers can use both retrospectively to appraise the quality of an already conducted Delphi study and prospectively to design and implement a Delphi study.

Table of Contents

  • Chapter One


  • Chapter Two

    What Delphi Is

  • Chapter Three


  • Chapter Four

    Data Collection

  • Chapter Five

    Data Analysis

  • Chapter Six


  • Chapter Seven

    DCAT: Delphi Critical Appraisal Tool

  • Appendix A

    Summary of Recommendations

  • Appendix B

    DCAT Checklist

Research conducted by

Funding for this research was provided by gifts from RAND supporters and income from operations. The research was conducted by RAND Global and Emerging Risks.

This report is part of the RAND Corporation Tool series. RAND tools may include models, databases, calculators, computer code, GIS mapping tools, practitioner guidelines, web applications, and various toolkits. All RAND tools undergo rigorous peer review to ensure both high data standards and appropriate methodology in keeping with RAND’s commitment to quality and objectivity.

This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND’s publications do not necessarily reflect the opinions of its research clients and sponsors.

Download eBook for Free

Format File Size Notes
PDF file 0.8 MB

Use Adobe Acrobat Reader version 10 or higher for the best experience.

© 版权声明
点赞0 分享