You are here

Article Text

Article menu

Download PDFPDF

Original articles
Concussion information online: evaluation of information quality, content and readability of concussion-related websites
  1. Osman H Ahmed1,
  2. S John Sullivan1,
  3. Anthony G Schneiders1,
  4. Paul R McCrory2
  1. 1Centre for Physiotherapy Research, School of Physiotherapy, University of Otago, Dunedin, New Zealand
  2. 2Centre for Health, Exercise and Sports Medicine, University of Melbourne, Melbourne, Australia
  1. Correspondence to Professor S John Sullivan, Centre for Physiotherapy Research, University of Otago, PO Box 56, Dunedin 9054, New Zealand; sjohn.sullivan{at}otago.ac.nz

Abstract

Background The internet plays an important role in the dissemination of health information to the general public. Information on orthopaedic sports medicine websites has been shown to be of a varying standard, and to date there has been no evaluation of the overall quality of concussion-related websites.

Methods A four-stage methodological sampling technique was used to identify concussion-related websites. Websites were assessed for the presence of a quality standard (the HONcode), their adherence to current expert concussion knowledge using a custom-developed concussion checklist ('CONcheck'), and their readability using the established Flesch Reading Ease (FRE) and Flesch–Kincaid Grade Level (FKGL).

Results 43 Websites were identified from the search strategy with the majority (70%) not HONcode certified. A wide distribution of scores was seen for the CONcheck (0–22), FRE (16.3–77.4) and FKGL (6–17.8). Statistical analysis using independent t tests between websites with the HONcode and websites without the HONcode showed no significant differences between the two groups for CONcheck (t41=0.571, p=0.571), FRE (t41=0.808, p=0.424) and FKGL(t41=−0.964, p=0.341) scores.

Conclusions The variability in the standard of concussion-related websites highlights the need for sports medicine website providers to consider the delivery, content and readability of information to the public.

https://doi.org/10.1136/bjsm.2010.081620

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Recent advances in concussion knowledge1 ,2 have been widely disseminated and referenced in the scientific literature.3 Recommendations from the 2008 Concussion in Sport (CIS) statement2 highlight the importance of knowledge transfer,4 which is defined in the health setting as 'the process of exposing new knowledge to the frontline healthcare provider'.5 Traditional methods of transferring knowledge face-to-face and through leaflet-based approaches6,–,8 are now being supplemented by websites which provide concussion information and advice.9 ,10

    A major factor in the shift to online knowledge transfer is the continuing expansion of the internet into everyday life, with recent studies suggesting as many as 61% of American adults regularly look online for health information.11 The emerging role, and growing acceptance, of the internet in sports medicine has led to the creation of numerous websites which offer advice on injury management,12 ,13 injury prevention14 ,15 and training/rehabilitation programmes.16 ,17 Websites for many medical conditions18,–,20 have been evaluated and critiqued in the literature, however to date, there are relatively few studies that have examined the overall quality of sports medicine websites.21 A recent study by Starman et al (2010)22 demonstrated 'a poor overall quality' of orthopaedic sports medicine websites, and recommended patient access and use for only well-known websites from reputable sources.

    Online health information has been shown to contain inaccuracies and inconsistencies,23 ,24 indicating a need to evaluate the standard of the information provided.25 ,26 Three factors crucial to the integrity of a website are the information quality, the content of the information and its readability. The widely-used benchmark for information quality on health websites is the HONcode,27 created and maintained by the Health on the Net (HON) Foundation to provide a recognised ethical standard. Web publishers can apply to have the HON Foundation evaluate their website, and websites that comply with the HONcode principles are permitted to display the HONcode validation certificate.28 ,29 Currently, there is no generic tool for evaluating the content of all health websites. Several condition-specific instruments have been created,18 ,19 however, there is no tool which specifically assesses online brain injury or concussion information. The third identified factor relevant to the integrity of a website is its readability, and this is commonly assessed using readability utilities such as the Flesch Reading Ease Score (FRES) and Flesch–Kincaid Grade Level (FKGL).30 These tools have been primarily utilised for the printed word (including brain injury information leaflets),31 and have also been used to assess the readability of healthcare websites.32 ,33

    Numerous organisations, both government and private, provide websites which deliver educational advice and information relating to concussive brain injury.34,–,37 The importance of ensuring the provision of accurate online information is not disputed, however to date, there has been no investigation of the standard of this information in the field of concussion education and management. The purpose of this study was to evaluate the information quality, content and readability of a range of selected websites providing information on/related to concussion or sports concussion in particular.

    Methodology

    A four-stage sequential strategy was undertaken, incorporating both a rigorous non-exhaustive systematic search and expert knowledge to ensure a comprehensive list of relevant websites for examination. This study utilised information that is freely available in the public domain, and ethical approval was not required or sought.

    Search strategy

    Stage 1: Google Search

    The World Wide Web was systematically searched for websites relating to concussion using the search terms 'concussion' and 'sports concussion' in order to identify the websites that members of the public would most typically locate. The search was undertaken on 6 September 2010 using Google, the most-used search engine on the web.38 To improve the sensitivity of the search, Google sites from major English-speaking countries were included: http://www.google.com (USA), http://www.google.co.uk (UK), http://www.google.co.nz (New Zealand), http://www.google.ca (Canada), http://www.google.co.za (South Africa) and http://www.google.com.au (Australia). For pragmatic reasons, the first 10 hits identified from each of these Google websites (ie, those shown on the first search page) were retrieved, combined in a spread sheet and duplicates removed manually.

    Stage 2: Expert Knowledge

    The websites retrieved from stage 1 ('Google Search') were supplemented by websites suggested by members of the research team in consultation with international colleagues. This included websites that are well-known providers of health information.

    Stage 3: International Sport Federations

    A sample of websites hosted by international sporting federations including, but not limited to, those federations sponsoring the international CIS meetings were included. The websites of selected international sporting federations whose sports are known to have a high incidence of concussion/head injury were also included.

    Stage 4: Industry Sample

    Websites which offered concussion-related services, primarily neuropsychological testing services and products, were identified by the research team from their prior knowledge.

    Inclusion criteria for all four stages of the search strategy required that each website be in English and freely available to the public (ie, not require any membership or registration). Websites were excluded from the study if they were irrelevant to the topic of concussion/sports concussion, or were commercially sponsored links or news stories. As this study focused on written information only, links to video (eg, YouTube) or audio (eg, podcasts) were excluded.

    Information quality (HONcode)

    The HONcode is the most established and widely-used instrument for evaluating health information on the internet, and has been awarded to 7300 certified websites on more than 10 million pages in 102 countries.27 The HON Foundation awards the HONcode, a quality standard, to websites which follow the HONcode principles (listed in table 1) with the HONcode logo displayed on the pages of these websites. For the purposes of this study, information quality was designated by the presence of the HONcode logo on the website.

    View this table:
    Table 1

    HONcode principles*

    Content assessment (CONcheck)

    Due to the lack of a validated concussion evaluation instrument, a custom-designed 11-point checklist (a concussion information checklist called 'CONcheck') was created by the research team. Face validity of CONcheck was established by the input from a member (PM) of the group who drafted the 2008 CIS Statement.2 CONcheck was developed via a reiterative process of extracting core material from the 2008 CIS Statement,3 and refining this into a series of short statements which reflected the key messages(see table 2). CONcheck was created with a focus towards the general public searching for concussion information. A consequence of this is that some specialised return-to-play information from the 2008 CIS Statement2 targeted towards the management of elite athletes was intentionally omitted from CONcheck.

    View this table:
    Table 2

    CONcheck items

    CONcheck was reviewed and modified by the research team until a final version was agreed upon, with external input sought (from WM, one of the authors of the 200139 and 20041 Summary/Agreement Statements and the 2008 Consensus Statement)2 and a simple descriptive coding scheme developed. Each of the items was scored as follows: 'Present and Complete' (2), 'Present but incomplete' (1), 'Absent' (0) or 'Inaccurate' (*). For each CONcheck item, a website was coded as 'Inaccurate' (*) if there was any information present which was contrary to that item of CONcheck (see examples in table 5). Those items classified as 'Inaccurate' were excluded when calculating the total CONcheck score. These results were tabulated to provide a total score, with the maximum score being 22.

    Readability assessment

    An essential feature of any website is its ability to communicate to the reader, and a key aspect of this is the capacity to deliver information at a level where it is understood. The FRES and FKGL30 are among the most widely accepted readability tools.31 The FRES uses the length of sentences and the number of polysyllabic words to determine scores from 0 to 100.32 A score of under 30 represents text that is very difficult to read, a standard score is between 61 and 70, while a score of over 91 would indicate that the text is very easy to read.31 The FKGL utilises the mean sentence and word length to calculate reading level. FKGL scores up to grade 12 (final year of high school) correspond to US educational school grades, with scores higher than 12 indicative of college level and domain-specific experts.32

    The inbuilt readability statistics feature of Microsoft Word 200732 was used to determine the FRES and FKGL for each website. Approximately 1 A4 page (400–500 words) of text was selected at random from each of the websites identified and analysed.

    Procedure

    Prior to the content assessment, the PI (OA) and two research colleagues (SA and HPL) reviewed the CONcheck a priori rules and discussed coding procedures. These three researchers were well informed of the content of the 2008 CIS Statement2 upon which CONcheck was based and each researcher had several years of experience in working with concussed sports persons and in the field of sports concussion research. Pilot testing was undertaken (by the PI) on two websites relating to concussion which did not appear in the first 10 Google hits in stage 1 of the search strategy. In the same 24-h period, (00:00–24:00 7 September 2010) each researcher independently reviewed each website using CONcheck.

    Hyperlinks within the body of text that navigated to information within each website were followed and included in the analysis; however, hyperlinks leading to external websites were not followed. It was permitted to follow hyperlinks which navigated to a PDF within the same website, and the information from these PDFs was included for analysis. Once all researchers had completed CONcheck, results were pooled and where differences in the scoring existed, agreement was reached via consensus. It was noted if the information was contained on a PDF within a website, and also if the website made reference to either of the CIS Group statements.1 ,2

    Statistical analysis

    Descriptive statistics (mean±SD) were computed for all variables (overall CONcheck score, FRE and FKGL) and frequencies were determined for each CONcheck item. Independent t tests were performed to analyse for differences between those websites that obtained the HONcode and those that did not. An a priori level of significance was set at p≤0.05 for all analysis. Descriptive statistics were calculated using SPSS (V.16.0; SPSS, Chicago, Illinois, USA). Where inaccurate information was identified on a website, quotes were extracted to provide examples of this.

    Results

    Implementation of the search strategy resulted in the identification of 43 websites (figure 1). The majority (30/43 or 70%) of websites were not HON certified (table 3). PDFs containing concussion-related information were found on 10 websites (23%), and five of these PDFs were either the 2008 CIS consensus statement2 or the 2004 CIS summary and agreement statement.1 Of these five websites referring to the 2004/2008 statements, only one was HON certified.

    Figure 1
    Figure 1

    Search strategy and identification of websites.

    View this table:
    Table 3

    Website details, readability and HONcode

    CONcheck items 2 and 3 scored highest (ie, had the highest frequency of '2-Present and Complete' scores) with 74% (N=32) and 72% (N=31) of websites, respectively (table 4). Those items of the CONcheck where websites scored the weakest (ie, had the highest frequency of '0-Absent' scores) were items 4, 9 and 11. Six websites (14%) achieved the maximum CONcheck score (22 out of a possible 22) and seven websites (16%) obtained the lowest possible score (zero). Readability scores ranged from 16.3 to 77.4 (mean=46.95, SD=14.23) for FRE and from 6 to 17.8 (mean=11.17, SD=2.72) for FKGL.

    View this table:
    Table 4

    CONcheck and readability summary

    No significant differences were seen when comparing websites meeting the HONcode criteria (N=13) to those websites not meeting the HONcode criteria (N=30). Websites without the HONcode had slightly higher CONcheck scores than those with the HONcode (12.3±7.9 compared to 10.8±7.2, t41=0.571, p=0.571), and were also marginally easier to read as determined by the FRE (48.1±15.3 without HONcode, 44.3±11.5 with HONcode, t41=0.808, p=0.424) and the FKGL (10.9±2.9 without HONcode, 11.8±2.4 with HONcode, t41=−0.964, p=0.341).

    Examples of incorrect information are displayed in table 5 along with extracted supporting quotes. The CONcheck item which was most commonly incorrect (by six websites) was item 5, 'Consultation with a doctor is essential after suspected concussion'.

    View this table:
    Table 5

    Examples of incorrect information

    Discussion

    This initial evaluation of online concussion information shows a wide variation in the overall quality of websites. The HONcode was selected as the measure of information quality in this study to provide an ethical standard in line with the HON Principles (table 1), and the presence of the HONcode allows the reader to be sure of the 'reliability and credibility' of the information they are reading.27 The absence of the HONcode in more than two-thirds of the websites in this study is similar to that in a recent review of orthopaedic sports medicine websites.22 There are several possible reasons for the lack of wider HON certification. Websites may: not be aware of the HONcode and its potential value, have unsuccessfully applied for HONcode certification, or elected not to apply for the HONcode as they do not perceive it as important or may feel their site would not get accredited. It is important to note that while the HONcode requires websites to state when they were last updated, this does not indicate the currency of the information content. Information relating to return-to-play best practice following concussion has evolved over recent years and websites need to reflect such changes by ensuring that information provided is in line with current concussion knowledge.

    The CONcheck instrument used in this study was created to assess which websites provide concussion information in accordance with the key aspects of the 2008 CIS consensus statement, which is currently considered the 'gold standard' for concussion information.3 All websites that contained links to PDFs containing the CIS meeting statements registered the maximum possible CONcheck score, however, other websites had a total absence of concussion information. With the exception of the International Ice Hockey Federation43 and postdata collection the International Rugby Board,40 sporting federation websites were lacking in specific concussion information. In a practical sense, this may not be problematic as individuals may seek concussion information from more medically-focused websites. It can be argued, however, as these sporting organisations are contributing members towards the CIS group and figureheads of their respective sports, they could be expected to be more proactive in providing concussion education resources to both their members and the medical personnel responsible for their care.

    Literature from the US has shown that a significant proportion of the general public struggle with comprehending healthcare information in the printed format,41 with the recommendation made that health information should be targeted towards a FKGL of around 6.42 The websites included in this study had a mean FRE of 46.95 and FKGL of 11.17 suggesting that existing online concussion information might not be at a level where the majority of the population would be able to fully comprehend and use it. In contrast, a random sample of concussion-related 'tweets' (short posts placed on the microblogging site Twitter)43 analysed by our research team had an FKGL of 2.9 and an FRE of 85.6, suggesting that information via this medium or language level, albeit short, could be more comprehensible to the majority of the population.

    As well as being readable, it is critical that the knowledge transferred is accurate. The high proportion (17/43 or 40%) of websites in this study which contained elements of incorrect information, although lower than other healthcare websites previously evaluated,40 suggests that there is potential for the mismanagement of concussion through erroneous or inaccurate advice. The CONcheck item which was most frequently incorrect related to the need to seek consultation with a doctor after concussion. Seeking medical consultation postconcussion is accepted as an essential aspect of concussion management2 ,3 and therefore it was of concern that many of the websites neglected to offer this key information.

    The evolution of online health information encourages the use of innovative methods in the transfer of concussion knowledge.44 ,45 The 2008 CIS Statement (developed for physicians, athletic trainers and sports medicine professionals among others)2 was widely disseminated through the academic literature3 but this information may not necessarily filter down to grassroots level, namely to concussed players and their team-mates and those who care for them. There is a clear need to supply critical information such as the 2008 CIS Statement2 to the widest audience in as clear terms as possible, and in a format whereby individuals and their families can understand it. Future versions of the consensus statement should consider a 'plain language' summary statement targeted towards the general public to maintain knowledge transfer fidelity, in a format which can be easily posted and marketed. Recent studies have demonstrated the emerging role of Social Networking Sites (SNSs) such as Facebook46 and Twitter47 in relation to concussion information, and these SNSs have the potential to significantly facilitate knowledge transfer.

    Due to the large number of medical/health websites, a selective and non-exhaustive website identification approach was taken in this study which meant that some relevant websites (eg, snowsports websites) may not have been captured. A consensus model added strength to the study.

    Further studies are required to continue to validate the CONcheck to ensure that it becomes a recognised tool for use by healthcare providers, who may be looking to direct consumers (patients and players) towards online concussion information. Due to the importance of ensuring accurate online information, similar studies should be conducted in selected areas of sports medicine to evaluate the overall quality of information on the internet for the most common sporting injuries/conditions (eg, anterior cruciate ligament injury, Achilles tendinitis etc).

    Conclusion

    This study has shown that the overall quality of online concussion information is varied. Many of the websites identified by the search strategy were missing key facts relating to concussion management and were not written at a level where the majority of people would be able to understand them. Although the knowledge base behind current concussion management practices is sound, steps need to be taken to ensure that the information is disseminated in a manner which will be accessible to all parties. SNSs (including Facebook and Twitter) have the potential to play a role in the future of concussion knowledge transfer.

    Acknowledgments

    The authors thank Sridhar Alla and Hopin Lee for undertaking the CONcheck coding, and Professor Willem Meeuwisse for his input in the development of CONcheck.

    References

    1. ↵
      1. McCrory P,
      2. Johnston K,
      3. Meeuwisse W,
      4. et al
      . Summary and agreement statement of the 2nd International Conference on Concussion in Sport, Prague 2004. Br J Sports Med 2005;39:196–204.
      OpenUrlAbstract/FREE Full Text
    2. ↵
      1. McCrory P,
      2. Meeuwisse W,
      3. Johnston K,
      4. et al
      . Consensus Statement on Concussion in Sport: the 3rd International Conference on Concussion in Sport held in Zurich, November 2008. Br J Sports Med 2009;43(Suppl 1):i76–90.
      OpenUrlFREE Full Text
    3. ↵
      1. Alla S,
      2. Sullivan SJ,
      3. McCrory P,
      4. et al
      . Spreading the word on sports concussion: citation analysis of summary and agreement, position and consensus statements on sports concussion. Br J Sports Med 2011;45:132–5.
      OpenUrlAbstract/FREE Full Text
    4. ↵
      1. Provvidenza CF,
      2. Johnston KM
      . Knowledge transfer principles as applied to sport concussion education. Br J Sports Med 2009;43(Suppl 1):i68–75.
      OpenUrlAbstract/FREE Full Text
    5. ↵
      1. Pappano D,
      2. Conners G,
      3. McIntosh S,
      4. et al
      . Sources of knowledge transfer among primary care pediatric health care providers. Clin Pediatr (Phila) 2008;47:930–4.
      OpenUrlAbstract/FREE Full Text
    6. ↵
      1. Gronwall D
      . Minor head injury. Neuropsychology 1991;5:253–65.
      OpenUrlCrossRef
    7. ↵
      1. Moore C,
      2. Leathem J
      . Information provision after mild traumatic brain injury (MTBI): a survey of general practitioners and hospitals in New Zealand. N Z Med J 2004;117:1–9.
      OpenUrl
    8. ↵
      1. Gianotti S,
      2. Hume PA
      . Concussion sideline management intervention for rugby union leads to reduced concussion claims. NeuroRehabilitation 2007;22:181–9.
      OpenUrlPubMedWeb of Science
    9. ↵
      Think First Canada [Internet]. http://www.thinkfirst.ca/programs/concussion.aspx (accessed 17 Jun 2010).
    10. ↵
      Brainline [Internet]. http://www.brainline.org (accessed 6 October 2010).
    11. ↵
      The social life of health information [Internet] http://www.pewinternet.org/Reports/2009/8-The-Social-Life-of-Health-information (accessed 11 Aug 2010).
    12. ↵
      The Virtual Sports Injury Clinic [Internet] http://www.sportsinjuryclinic.net/ (accessed 9 Oct 2010).
    13. ↵
      ElastoPlastSport.com: Injury Management and Prevention [Internet] http://www.elastoplastsport.com.au/Injury/Default.aspx (accessed 10 Oct 2010).
    14. ↵
      Oslo Sports Trauma Research Centre [Internet] http://www.klokavskade.no/en/ (accessed 30 Sep 2010).
    15. ↵
      About.com: How to Prevent Sports Injuries [Internet] http://sportsmedicine.about.com/cs/injuryprevention/a/aa101801a.htm (accessed 10 October 2010).
    16. ↵
      Sports Injury Info [Internet] http://www.sports-injury-info.com/sports-injury-rehab.html (accessed 30 Sep 2010).
    17. ↵
      StrengthCoach.Com: Strength and Conditioning [Internet] http://www.sportspecific.com/ (accessed 10 Oct 2010).
    18. ↵
      1. Lewiecki EM,
      2. Rudolph LA,
      3. Kiebzak GM,
      4. et al
      . Assessment of osteoporosis-website quality. Osteoporos Int 2006;17:741–52.
      OpenUrlCrossRefPubMedWeb of Science
    19. ↵
      1. Harland J,
      2. Bath P
      . Assessing the quality of websites providing information on multiple sclerosis: evaluating tools and comparing sites. Health Informatics J 2007;13:207–21.
      OpenUrlAbstract/FREE Full Text
    20. ↵
      1. Burneo JG
      . An evaluation of the quality of epilepsy education on the Canadian World Wide Web. Epilepsy Behav 2006;8:299–302.
      OpenUrlCrossRefPubMedWeb of Science
    21. ↵
      1. Sambandam SN,
      2. Ramasamy V,
      3. Priyanka P,
      4. et al
      . Quality analysis of patient information about knee arthroscopy on the World Wide Web. Arthroscopy 2007;23:509–13.e2.
      OpenUrlPubMedWeb of Science
    22. ↵
      1. Starman JS,
      2. Gettys FK,
      3. Capo JA,
      4. et al
      . Quality and content of Internet-based information for ten common orthopaedic sports medicine diagnoses. J Bone Joint Surg Am 2010;92:1612–18.
      OpenUrlCrossRefPubMed
    23. ↵
      1. Purcell GP,
      2. Wilson P,
      3. Delamothe T
      . The quality of health information on the internet. BMJ 2002;324:557–8.
      OpenUrlFREE Full Text
    24. ↵
      1. Kunst H,
      2. Groot D,
      3. Latthe PM,
      4. et al
      . Accuracy of information on apparently credible websites: survey of five common health topics. BMJ 2002;324:581–2.
      OpenUrlFREE Full Text
    25. ↵
      1. Hanif F,
      2. Read JC,
      3. Goodacre JA,
      4. et al
      . The role of quality tools in assessing reliability of the internet for health information. Inform Health Soc Care 2009;34:231–43.
      OpenUrlCrossRefPubMedWeb of Science
    26. ↵
      1. Gustafson DH,
      2. Wyatt JC
      . Evaluation of ehealth systems and services. BMJ 2004;328:1150.
      OpenUrlFREE Full Text
    27. ↵
      Health On The Net Foundation [Internet] http://www.hon.ch/home1.html (accessed 17 Jun 2010).
    28. ↵
      PubMed [Internet] http://www.ncbi.nlm.nih.gov/pubmed (accessed 24 Sep 2010).
    29. ↵
      World Health Organisation [Internet] http://www.who.int/en/ (accessed 10 Oct 2010).
    30. ↵
      1. Farr JN,
      2. Jenkins JJ,
      3. Paterson DG
      . Simplification of Flesch Reading Ease Formula. J Appl Psychol 1951;35:333–7.
      OpenUrlCrossRefWeb of Science
    31. ↵
      1. Macdonald S,
      2. McMillan TM,
      3. Kerr J
      . Readability of information leaflets given to attenders at hospital with a head injury. Emerg Med J 2010;27:279–82.
      OpenUrlAbstract/FREE Full Text
    32. ↵
      1. D'Alessandro DM,
      2. Kingsley P,
      3. Johnson-West J
      . The readability of pediatric patient education materials on the World Wide Web. Arch Pediatr Adolesc Med 2001;155:807–12.
      OpenUrlCrossRefPubMedWeb of Science
    33. ↵
      1. van der Marel S,
      2. Duijvestein M,
      3. Hardwick JC,
      4. et al
      . Quality of web-based information on inflammatory bowel diseases. Inflamm Bowel Dis 2009;15:1891–6.
      OpenUrlCrossRefPubMedWeb of Science
    34. ↵
      Centres for Disease Control and Prevention [Internet] http://www.cdc.gov/ (accessed 24 Sep 2010).
    35. ↵
      ACC Homepage (The Accident Compensation Corporation) [Internet] http://www.acc.co.nz/index.htm (accessed 30 Sep 2010).
    36. ↵
      Mayo Clinic [Internet] http://www.mayoclinic.com/ (accessed 30 Sep 2010).
    37. ↵
      WebMD [Internet] http://www.webmd.com/default.htm (accessed 30 Sep 2010).
    38. ↵
      HTMLsource [Internet] http://www.yourhtmlsource.com/promotion/searchengines.html (accessed 2 Jun 2010).
    39. ↵
      1. Aubry M,
      2. Cantu R,
      3. Dvorak J,
      4. et al
      . Summary and agreement statement of the First International Conference on Concussion in Sport, Vienna 2001. Recommendations for the improvement of safety and health of athletes who may suffer concussive injuries. Br J Sports Med 2002;36:6–10.
      OpenUrlFREE Full Text
    40. ↵
      IRB: International Rugby Board [Internet] http://www.irb.com/index.html (accessed 5 Nov 2010).
    41. ↵
      1. Clauson KA,
      2. Zeng-Treitler Q,
      3. Kandula S
      . Readability of patient and health care professional targeted dietary supplement leaflets used for diabetes and chronic fatigue syndrome. J Altern Complement Med 2010;16:119–24.
      OpenUrlCrossRefPubMedWeb of Science
    42. ↵
      1. Vallance JK,
      2. Taylor LM,
      3. Lavallee C
      . Suitability and readability assessment of educational print resources related to physical activity: implications and recommendations for practice. Patient Educ Couns 2008;72:342–9.
      OpenUrlCrossRefPubMedWeb of Science
    43. ↵
      Twitter [Internet] http://www.twitter.com (accessed 14 Oct 2010).
    44. ↵
      1. Eysenbach G
      . Medicine 2.0: social networking, collaboration, participation, apomediation, and openness. J Med Internet Res 2008;10:e22.
      OpenUrlCrossRefPubMed
    45. ↵
      1. Steininger K,
      2. Rückel D,
      3. Dannerer E,
      4. et al
      . Healthcare knowledge transfer through a web 2.0 portal: An Austrian approach. Int J Healthc Tech Manag 2010;11:13–30.
      OpenUrlCrossRef
    46. ↵
      1. Ahmed OH,
      2. Sullivan SJ,
      3. Schneiders AG,
      4. et al
      . iSupport: do social networking sites have a role to play in concussion awareness? Disabil Rehabil 2010;32:1877–83.
      OpenUrlCrossRefPubMedWeb of Science
    47. ↵
      1. Sullivan SJ,
      2. Schneiders AG,
      3. Cheang CW,
      4. et al
      . What's happening? A content analysis of concussion-related traffic on Twitter. Br J Sports Med 2011; Published Online First: 15 March 2011 doi:doi:10.1136/bjsm.2010.080341.
      OpenUrlAbstract/FREE Full Text

    Footnotes

    • Funding This study was facilitated by funding from the International Rugby Board.

    • Competing interests None.

    • Provenance and peer review Not commissioned; externally peer reviewed.