Table 1

Preliminary thoughts: 10 challenges and potential solutions for more effective consensus meeting and report

Category	Suggested improvements
Who should be invited?	1. Organisers must ensure people with the appropriate expertise attend the meeting. Although ‘methodologists’ and ‘statisticians‘ are often omitted, having a statistician with expertise in randomised trials (or survival analysis) may remain insufficient for questions related to observational data (or recurrent events analysis).
	2. Organisers should choose participants that represent a wide range of perspectives, as recommended for most qualitative research projects trying to understand a phenomenon or beliefs. One challenge will be to differentiate diverse opinions that are reasonable, from diverse opinions that do not have any theoretical or empirical foundation.
	3. Although training future clinicians and researchers is important, students do not usually have the breadth of experience or knowledge that is required for broader discussions. Further, students often find it difficult to contradict or challenge their supervisor and other ‘leaders‘. Therefore, the organisers’ first priority should be to engage participants with the knowledge and experience to obtain the full range of reasonable opinions. Others may be invited as cost/feasibility permits.
What material should be distributed before meeting?	4. Organisers should gather and distribute relevant material prior to the meeting. For clinical treatment guidelines, this would often require systematic reviews of the effect of the different treatment options. For methodological or reporting guidelines, this would often require reviews explaining the advantages/disadvantages/underlying assumptions/requirements for different approaches to analyses for the particular topic. These reviews should be understandable to non-statisticians and methodologists if possible. If such reviews do not exist, the participating methodologists might consider writing up a short 1–2 paragraph ‘knowledge translation’ summary of the key points for distribution to other participants.
What material should be distributed before meeting?	5. Participants should also be encouraged to share articles that they believe are important to developing the consensus statement.
What is consensus?	6. Organisers need to define what will be considered criteria for consensus before the meeting. The required proportion may be context-dependent.
	7. For consensus statements that comprise texts from different subgroups of participants, all authors must have adequate time to review and provide feedback on the full consensus statement. This may take several iterations, but it is essential. Without this, the ‘consensus statement‘ is no more than a series of articles, each written by 3–4 people, regardless of the total number of authors on the paper. One challenge with this process is to produce a report within a short period of time.
	8. Authors need to report dissent among participants. This might just be the proportion who approve/disapprove, or an informative summary of the range of ‘ratings of agreement‘ that was obtained (eg, SD, quantiles). There will be implementation challenges. For example, obtaining votes on every word is infeasible. The consensus statement on femoroacetabular impingement syndrome18 included votes on final statements. This may represent a practical solution.
Reporting the consensus statement	9. Using an iterative feedback process should naturally identify those who agree and disagree with the final majority statement that is written by the lead author (or lead of a subgroup if applicable). Those who disagree with the final statement will need to choose who will lead the writing of the minority opinion. Having authors identify themselves with the majority or minority opinions is the most transparent method, but this may inhibit some participants from participating.
Reporting the consensus statement	10. The peer review process is challenging when consensus statements include a large group of experts in a small field. Although the Supreme Court model provides a form of peer review because authors are allowed to dissent, journal editors may need to engage experts from related fields (eg, statisticians, orthopaedic surgeons, rheumatologists not involved in sport and exercise medicine).