Article Text
PDF
Abstract
The purpose of this paper is to summarise the methodology for the 5th International Consensus Conference on Concussion in Sport. The 18 months of preparation included engagement of a scientific committee, an expert panel of 33 individuals in the field of concussion and a modified Delphi technique to determine the primary questions to be answered. The methodology also involved the writing of 12 systematic reviews to inform the consensus conference and submission and review of scientific abstracts. The meeting itself followed a 2-day open format, a 1-day closed expert panel meeting and two additional half day meetings to develop the Concussion Recognition Tool 5 (Pocket CRT5), Sport Concussion Assessment Tool 5 (SCAT5) and Child SCAT5.
- Concussion
- mild traumatic brain injury
- sports
- Delphi
- Consensus
Statistics from Altmetric.com
Introduction
The 1st International Conference on Concussion in Sport meeting in Vienna was held in 2001 to ‘provide recommendations for the improvement of safety and health of athletes who suffer concussive injuries in ice hockey, rugby, football (soccer) as well as other sports.’1 One catalyst for the meeting was the development of a greater understanding of concussion leading into the 2002 Winter Olympic Games and, paradoxically, occurred at a time when concussion in sport was not topical.
The output from that meeting was a summary and agreement statement of the 1st International Conference on Concussion in Sport, which contained a definition of concussion, clinical history, evaluation (including symptoms, signs and neuropsychological assessment) and initial management recommendations.1 There were several landmark components to this first statement. First, it served to establish a single concussion in sport definition, at a time when many disparate definitions existed. In this statement, the notion of a stepwise return-to-play approach was first presented.1
The authorship group of the first agreement statement (the ‘Concussion in Sport Group’) recognised the changing nature of the field and therefore planned the 2nd International Conference on Concussion in Sport in Prague in 2004. The output from that meeting was also a summary and agreement statement. In the second statement, the initial Sport Concussion Assessment Tool (SCAT) Card was first published.2
The 3rd International Consensus Conference on Concussion in Sport was held in Zurich in 2008 and was the first Consensus meeting of the Concussion in Sport group.3 This Conference followed a National Institutes of Health (NIH) Consensus development conference format (formerly, https://consensus.nih.gov). As part of this process, specific questions were prepared ahead of the meeting to define the scope of and guide the direction of the conference. In addition, several broad systematic reviews of the literature were prepared prior to the conference and circulated to the panel to assist in the answering of the conference questions. Lectures were then presented in a public session by a group of experts, followed by a discussion session. An expert panel was formed which then met in a closed session to prepare the consensus statement with the primary tasks of (1) elucidating a response to the questions and (2) drafting a consensus statement to serve as the scientific record of the conference.3 At the same time, the SCAT was revised and published as the SCAT2, along with a shortened version, called the ‘Pocket SCAT2’. That consensus statement was co-published in seven peer-reviewed journals.
The 4th International Consensus Conference on Concussion in Sport was held in Zurich in October of 2012 and followed a similar format to the 2008 meeting. This time, a separate systematic review was drafted for each specific consensus question in advance of the meeting.4 The NIH style of consensus development was again used with the goal of adhering to the highest scientific standard, with a structured format to achieve a clinically applicable outcome. Consistent with the goal of transparency, a full list of disclosures was also included with the consensus statement. The tools produced as part of this meeting included the Pocket Concussion Recognition Tool, the SCAT3 and the Child SCAT3, which were again co-published in multiple journals.
Around this time, an assessment of the processes behind the development of the 2008 documents was undertaken to help inform the 2012 document development process. This evaluation confirmed that the 2008 consensus statements were well regarded by independent appraisers. It also identified a need for more attention to issues relating to their implementation and relationship to health outcomes.5 This suggested that seeking the views of relevant experts before a concussion consensus meeting could help make the resulting statements more relevant to a larger stakeholder group.
The purpose of this current paper is to outline the methodology behind the planning and conduct for the 5th International Consensus Conference on Concussion in Sport and the resulting consensus statement.
Formation of the organising and scientific committee
Planning for the conference began following the 2012 conference, with active planning beginning in January of 2015. As part of this process, a series of steps was taken to build on the previous four consensus statements on Concussion in Sport. The initial organising group included the co-chairs of the past consensus meetings and the chief medical officers from each of the international sport bodies that provided financial support for the consensus process: FIFA, International Ice Hockey Federation, World Rugby, IOC and Fédération Equestre Internationale.
A scientific committee was formed in April of 2015. It consisted of experts who would take the lead in various topic areas. Next, a full expert panel was identified by the scientific committee, comprised of individuals with expertise in the field of concussion from various parts of the world. A literature search was also conducted to identify those who had been actively publishing in sport concussion during the previous 4 years. They were identified and selected based on their scientific, clinical and/or academic understanding on specific topics in the field of concussion. The expertise on the panel was broadened from past meetings to include additional healthcare professionals, organisations and experts in the field from both the sport concussion area and from related areas outside of sport, including international brain injury consortia.
The expert panel included 25 such invited panel members plus the scientific committee members. Of the 35 initially invited, all but two agreed to participate as expert panel members. The final expert panel included a variety of healthcare professionals and researchers with expertise in concussion, including sport medicine physicians, neurologists and neurosurgeons, a physiotherapist, neuropsychologists, athletic trainers, epidemiologists, neuropathologists and imaging scientists. These individuals were from Australia, Canada, England, Japan, the Netherlands, Norway, South Africa, Switzerland and the USA.
The following sections outline the steps taken to plan and prepare the meeting, the conduct of the Conference and the writing of the consensus statement.
Delphi method
A new addition to the 5th International Consensus Conference on Concussion in Sport was a formal process to identify the most important and relevant questions that the meeting should address. To do this, we used a modified Delphi technique.
This process involved two different groups reviewing the questions at different stages. This process occurred from 6 October 2015 to 20 January 2016. Five rounds were planned for question generation, feedback and revisions, engaging both the scientific committee (10 members) and the remaining expert panel (23 members).
Round 1
The modified Delphi method began with an open-ended question in which the scientific committee members were asked to suggest an unlimited number of questions to be addressed by the upcoming consensus statement. A list of 45 questions were suggested by the 10 contributors and subsequently divided into 11 main categories by a co-chair. Response rate was 90% (9/10).
Round 2
The second round sought rating of the list of 45 collated questions by the scientific committee. Participants rated these 45 questions on whether they were: ‘Essential’, ‘Desirable’ or ‘Not a priority’. Participants were also asked for suggestions regarding wording changes and other/additional questions. Based on the feedback from round 2, a co-chair of the conference distilled these questions into 14 main questions, with subquestions, which allowed retention of all suggested topics. These questions retained the initial 45 questions by identifying the key question with aligned subquestions. Response rate was 100% (10/10).
Round 3
The third round involved the additional expert panellists of 23 specialists in concussion (not including the scientific committee) who were asked to rank the list of 14 potential main questions (grouped with their subquestions) in order of importance. These expert panel members were also asked to submit any additional questions. Response rate was 100% (23/23).
Round 4
In the fourth round, the ranking of the questions from the third round were reviewed by the scientific committee. At this stage, the questions were reorganised and reduced to 12 questions. The two questions that were removed related to Paralympic athletes and policy/knowledge translation, which were decided to be outside of the scope of the conference. Response rate was 90% (9/10).
Round 5
The fifth and final round included the review of the final list of main questions (with subquestions) by the scientific committee. Minor wording changes only were made at this point. The list of final questions was also posted online at this time for public viewing on the British Journal of Sports Medicine website.
Systematic reviews
For each of the 12 final questions, a lead author was identified by the scientific committee. The lead author was selected for his or her scientific and clinical expertise and recognition as an expert in the field with respect to the specific question being addressed. A group of coauthors from the full expert panel, which included individuals with varying but complementary areas of expertise, was then assigned to each of the reviews. In some circumstances, additional authors were identified by the authorship groups to capture the expertise required to answer the primary questions and subquestions.
The systematic reviews were designed to answer each of the questions and subquestions in a standardised fashion. The literature was systematically searched, and a standardised process was developed using the Enhancing the QUAlity and Transparency Of health Research (EQUATOR) network, based on BJSM Author Guidelines and following the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guidelines6.
An initial search strategy was developed and peer reviewed by all authors to ensure that all relevant search terms were captured. The search strategies were reviewed for completeness and accuracy by a librarian who was knowledgeable in systematic reviews, using the Canadian Agency for Drugs and Technologies in Health Peer Review of Electronic Search Strategies Checklist.7 Searches were then revised and adapted by the librarian as required. The risk of bias in all included articles was assessed using the tool that best suited the literature retrieved in each systematic review. All reviews had a focus on sport-related concussion. However, in cases of sparse literature or literature of poor methodological quality, it was recommended that the discussion section of the papers could draw on other literature relating to other causes of mild traumatic brain injury. In total, approximately 60 000 published articles were initially screened by the authorship groups for the 12 systematic reviews.
All questions and subquestions were shared with the group of lead authors to facilitate discussion regarding potential overlap of the systematic reviews and to ensure that the reviews remained complementary. Seven teleconferences with the lead authors and scientific committee were held in the 5 months leading up to the meeting, during which time the drafts of manuscripts were completed.
Each lead author was asked to summarise the key messages from the review for presentation at the consensus meeting. The author group for systematic review #11 on long term effects met the day before the meeting to summarise the key messages from the systematic review of the literature. A draft of the remaining 11 systematic reviews were circulated to the expert panel prior to the meeting to ensure that all panellists were aware of the latest evidence. Following the consensus meeting, further editing was undertaken by the author groups prior to submission to BJSM. Systematic reviews were peer reviewed at BJSM where they were handled by the editor-in-chief and the associate editor responsible for systematic reviews. Systematic reviews were externally peer reviewed as per usual journal protocol.
Abstract submissions
Scientific abstracts relating to any of the 12 key topics for the conference were invited for submission. The expert panel was responsible for scoring the abstracts in their specific area. The scores were then collated and 202 abstracts were accepted for presentation; 178 abstracts as posters and 24 of the highest scoring abstracts were selected for an oral presentation. The objective of including the abstracts was to add to the systematic reviews by providing the latest evidence that had not yet been published. All abstracts were circulated to the expert panel members prior to the consensus meeting and were slated for publication in BJSM with the consensus statement and systematic reviews.
Consensus meeting
The 5th International Consensus Conference on Concussion in Sport was held on 27–28 October 2016 in Berlin, Germany. There was a total of 420 participants representing 24 countries. The format of the conference was a 2-day series of plenary sessions in which six of the predefined questions were presented and discussed each day. The conference was not designed to be a continuing medical education conference but was rather a forum for presenting a summary of the evidence for each of the questions.
Each 1-hour topic session consisted of between one and three short (5 min) abstract oral presentations, followed by a summary of the evidence from the systematic review (presented by the lead author) and a 20–25 min panel discussion that included the authorship group for each review, in which active audience participation was encouraged. A scribe was appointed to take notes of the audience discussion to ensure all audience comments were adequately captured and could be used to inform the closed expert panel meeting on the day after the main conference.
Expert panel meeting
A closed expert panel meeting occurred on 29 October 2016, the day following the consensus meeting. Individuals who were authors on the systematic reviews but were not expert panel members were invited to sit in the expert panel meeting as observers, but were not permitted to actively participate in the discussions. The objective of this meeting was to achieve consensus on answers to the 12 key questions through an interpretation of the scientific evidence and clinical relevance. This process included synthesis of the published evidence, consideration of new research presented in abstract form and a plenary session summary (including audience input). There was no requirement for unanimous agreement, and there was an option for dissenting or minority opinion. Any author who did not agree with the final consensus also had the option to withdraw from authorship. No dissenting opinion or withdrawal from the expert panel occurred during either this meeting or the subsequent review writing stages. During the day, each lead author presented a summary of findings from the systematic review and a summary of audience discussion from the consensus conference. A group discussion was then ensued. Following the discussion addressing each of the questions/subquestions, the panel voted on the summary statement for each topic and the expert panel moved on to the next question.
Concussion Recognition Tool (Pocket CRT5), Sport Concussion Assessment Tool (SCAT) 5 and Child SCAT5
The final day of meetings (30 October 2016) involved a subgroup of the expert panel meeting to discuss the development of tools arising from the meeting. For consistency, it was decided to use the number ‘5’ with each tool, to coincide with this 5th International Consensus Conference on Concussion in Sport. Thus, there is no SCAT 4, Child SCAT 4 or prior numbers associated with the CRT. The Concussion Recognition Tool (Pocket CRT5), SCAT5 and Child SCAT5 were based on the outcome of the expert panel meeting. The morning meeting was focused on development of the CRT5 and SCAT5. The afternoon meeting focused on the Child SCAT5, which incorporated relevant principles from the SCAT5 as appropriate. Once complete, the draft tools were circulated to working groups for feedback. When the final content was agreed on, the tools were sent for formatting prior to publication.
Writing of the consensus statement
During the expert panel closed session, each of the lead authors was asked to submit several PowerPoint slides with summary statements (and areas for future research) based on their systematic review, lecture and audience input. These were projected on a screen and edited with input from the entire expert panel. Notes were taken by a co-chair of the conference throughout the expert panel meeting and were used as a basis for drafting the overall consensus statement. The key statements that had been agreed on during the expert panel meeting were not altered during the drafting of the consensus statement. The end goal of the concussion statement was to provide a simple, clear message and tools that would equip the healthcare practitioner to diagnose and manage concussions in sport.
The consensus statement was initially drafted by the co-chairs of the scientific committee, which was then circulated for review and editing by the full scientific committee, followed by approval from the entire expert panel, who formed the author group of the final manuscript. Concurrently, the CRT5 and SCAT5 tools were drafted, as outlined in the accompanying manuscripts.
Summary
From the outset, the Concussion in Sport Group has had the goal of adhering to the highest scientific standards and transparency. The structure of the consensus conferences has evolved over time to incorporate new methods and broader input. It is intended that this manuscript will serve as a model for the planning for, and conduct of, the meetings over time. In doing so, this paper provides a template for others to follow and improve on in the future when organising future concussion and other sport medicine consensus conferences.
Footnotes
Contributor WHM, KJS, JD and PM contributed to the overall conceptualisation, writing and review of this manuscript.
OTO and CFF contributed to the Delphi technique, writing and review of the manuscript. AKH
contributed to the methodological concepts related to the systematic reviews, writing and review of this manuscript.
Competing interests WHM is the medical director of the National Hockey League. He has received research grant support through the University of Calgary from the Canadian Institutes of Health Research, Alberta Innovates Health Solutions, the FIFA, Alberta Children’s Hospital Research Institute and the Hotchkiss Brain Institute. He is the founding chair of the Sport Injury Prevention Research Centre, which is one of the International Research Centres for Prevention of Injury and Protection of Athlete Health supported by the IOC. He has a clinical consulting practice in sport medicine at the University of Calgary Sport Medicine Centre. He is an expert group member of the IOC Medical Commission and has received travel funding in that capacity. KJS is a physiotherapy consultant at Evidence Sport and Spinal Therapy in Calgary, Alberta, Canada. She has received honoraria for speaking at various conferences. She has received research grant support through the University of Calgary from the Alberta Children’s Hospital Research Institute, Hotchkiss Brain Institute and the Canada Foundation for Innovation (including matching funds through the Province of Alberta). She is a faculty member at the Sport Injury Prevention Research Centre, within is one of the International Research Centres for Prevention of Injury and Protection of Athlete Health supported by the IOC. CFF is a member of World Rugby’s Independent Expert Panel on Concussion. She was not involved in any other aspects of the International Consensus Conference on Concussion in Sport except for the planning for the Delphi study and review and editing of the manuscript. PM is a coinvestigator, collaborator, or consultant on grants relating to mild TBI funded by several governmental organisations. He is directly employed by the National Health and Medical Research Council of Australia and is based at the Florey Institute of Neuroscience and Mental Health. He is co-chair of the Australian Centre for Research into Sports Injury and its Prevention (ACRISP), which is one of the International University Research Centres for Prevention of Injury and Protection of Athlete Health supported by the IOC. He is co-chair of the International Concussion in Sport Group. He has a clinical and consulting practise in general and sports neurology. He receives book royalties from McGraw-Hill and was employed in an editorial capacity by the British Medical Journal Publishing Group from 2001 to 2008. He has been reimbursed by the government, professional scientific bodies and sporting bodies for travel costs related to presenting research on mild TBI and sport-related concussion at meetings, scientific conferences and symposiums. He received consultancy fees in 2010 from Axon Sports (US) for the development of educational material (which was not renewed) and has received research funding since 2001 from Cogstate Inc. He has not received any research funding, salary or other monies from the Australian Football League, FIFA or the NFL. The Australian Football League funds research at the Florey Institute under a legal memorandum and PM does not receive any money from this industry-funded research. PM is a cofounder and shareholder in two biomedical companies (involved in eHealth and Compression garment technologies) but does not hold any individual shares in any company related to concussion or brain injury assessment or technology.
Provenance and peer review Not commissioned; externally peer reviewed