Article Text

Download PDFPDF

Quality of reporting in sports injury prevention abstracts according to the CONSORT and STROBE criteria: an analysis of the World Congress of Sports Injury Prevention in 2005 and 2008
  1. Uzung Yoon,
  2. Karsten Knobloch
  1. Plastic, Hand and Reconstructive Surgery, Hannover Medical School, Hannover, Germany
  1. Correspondence to Dr Karsten Knobloch, Plastic, Hand and Reconstructive Surgery, Hannover Medical School, Carl-Neuberg-Street 1, 30625 Hannover, Germany; kknobi{at}yahoo.com

Abstract

Objective The quality of reporting in congress abstracts is likely to influence clinical decision-making.

Hypothesis The quality of reporting in sports injury prevention abstracts has increased over the last 3 years, as did the number of randomised controlled trials (RCT).

Methods 154 abstracts from the 2005 and 186 abstracts from 2008 World Conferences on Sports Injury Prevention in Norway were analysed.

Main outcome measures Scores of 17 Consolidated Standards of Reporting Trials (CONSORT) criteria for RCT, or 22 Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) criteria for observational studies were determined.

Results Improvement in reporting was evident in RCT (CONSORT score 5.8±0.9 vs 8.6±2.9, p=0.001, CI −4.29 to −1.43) as well as for observational studies (STROBE score 7.9±1.6 vs 9.9±1.7, p<0.001, CI −2.34 to −1.53) between 2005 and 2008. RCTs were published in 9.1% in 2005 versus 10.2% in 2008 (p=0.727). RCT demonstrated a significant improvement in main outcome (0% vs 57.9%, p<0.001). For observational studies, a significant improvement was reported in rationale (53.5% vs 98.2%, p<0.001), objectives (82.2% vs 95.1%, p=0.012), study design (25.2% vs 65%, p<0.001), setting (43.7% vs 65.6%, p=0.002), variables (20.7% vs 74.2%, p<0.001), participants (0.7% vs 10.4%, p=0.001) and funding (0% vs 5.5%, p=0.006).

Conclusion While the percentage of published RCTs in abstracts at the World Congress of Sports Injury Prevention remained unchanged, an improvement in reporting of abstracts was evident from 2005 to 2008, as determined by CONSORT/STROBE criteria. However, substantial and comprehensive use of the CONSORT and STROBE criteria might further increase the quality of reporting of sports injury conference abstracts in the future.

View Full Text

Statistics from Altmetric.com

Introduction

A large amount of healthcare research is first presented at conferences and meetings, and published as an abstract in the proceedings. The fact that reporting of trials presented in conference abstracts needs to be improved has been stated before.1 Among all randomised trials submitted to the 1992 American Society of Clinical Oncology conference analysed with a 10-year follow-up in CENTRAL and PubMed, the lack of information was a major problem in assessing trial quality: no abstracts reported on allocation concealment, 16% reported on blinding, and 14% reported intention to treat analysis. The authors concluded that the information given for trials in conference proceedings can be unstable, especially for trials presenting early or preliminary results, and needs to be improved.

In 1996, a group from the Department of Epidemiology and Biostatistics of the Memorial Sloan Kettering Cancer Center in New York City published their first report entitled: ‘Improving the quality of reporting of randomized controlled trials. The CONSORT statement.2 In 2001, Consolidated Standards of Reporting Trials (CONSORT) have been revised as a set of guidelines to improve quality in reporting in randomised controlled trials (RCTs) by use of a checklist and flow diagram.3 The revised checklist includes 22 items selected because empirical evidence indicates that not reporting this information is associated with biased estimates of treatment effect, or because the information is essential to judge the reliability or relevance of the findings. The revised flow diagram depicts information from four stages of a trial (enrolment, intervention allocation, follow-up and analysis). The diagram explicitly shows the number of participants, for each intervention group, included in the primary data analysis. Inclusion of these numbers allows the reader to judge whether the authors have done an intention-to-treat analysis. In sum, the CONSORT statement is intended to improve the reporting of an RCT, enabling readers to understand a trial's conduct and to assess the validity of its results.

The CONSORT group published recommendations for reporting of randomised trials in journal and conference abstracts recently4 (table 1). The checklist for conference abstracts is strongly recommended by the CONSORT group to gain clarity and strength in the conference abstracts.

Table 1

Study type of the presented abstract at the World Congress of Sports Injury Prevention in Oslo 2005 and Tromsø 2008

Somewhat similar to the aforementioned CONSORT criteria for RCT, the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement has been instituted for observational studies. A checklist of 22 items was presented, addressing three main study designs of analytical epidemiology: cohort, case–control and cross-sectional studies. Twenty-three individuals attended a meeting in September 2004, Bristol, UK, including editorial staff from Annals of Internal Medicine, BMJ, Bulletin of the World Health Organization, International Journal of Epidemiology, JAMA, Preventive Medicine and The Lancet, epidemiologists, methodologists, statisticians and practitioners from Europe and North America. They identified items deemed to be important to include in checklists for each type of study and decided on the strategy for finalising and disseminating the STROBE Statement.5

Based on these recommendations (CONSORT for RCT and STROBE for observational studies) we thought to evaluate the quality of conference abstracts at the first and second World Congress of Sports Injury Prevention in Oslo 2005 and Tromsø 2008 regarding these criteria. We hypothesise that the quality of reporting in conference abstracts improved from 2005 to 2008.

Methods

A total number of 340 abstracts presented at the first and second World Congress of Sports Injury Prevention in 2005 in Oslo (154 abstracts) and 2008 in Tromsø (186 abstracts), which were published in the British Journal of Sports Medicine in 2005 and 2008, were assessed regarding the quality of reporting according to the CONSORT and STROBE criteria by two researchers independently (UY and KK). The mean score of both investigators was taken. The inter-rater variability was 6%, which has been determined in an unpublished prehoc analysis of 250 sports medicine conference abstracts. Clinical studies with the known study design according to the level of evidence were discriminated from experimental studies involving no humans as participants, such as laboratory animal experiments.

CONSORT criteria were applied for all RCT and STROBE for all observational studies. Scores were calculated for 17 CONSORT and 22 STROBE criteria for all abstracts of the 2005 and 2008 meetings depending on whether the abstract met the items proposed in the criteria. Experimental studies were not analysed in 2005 and 2008 due to the lack of reporting quality publications.

Outcome measures

The study design (RCT, non-randomised trial, experimental), 17 CONSORT and 22 STROBE criteria for reporting in conference abstracts were determined.

Statistics

Descriptive statistics involved the calculation of frequencies and percentages. CONSORT and STROBE scores for clinical studies were calculated as the sum of individual scores for each of the criteria (1=present, 0=absent). A t test was used to compare CONSORT and STROBE Scores from 2005 and 2008. Data were analysed using the SPSS statistical software package V.14.0 (SPSS, Chicago, Illinois).

Results

Study design of conference abstracts

The vast majority of presented abstracts were non-randomised clinical trials in 2005 (88%) and 2008 (87%, p=0.727, table 2). RCT were described in 9.1% in 2005 versus 10.2% in 2008 (p=0.727) of the abstracts. From the non-randomised clinical trials, most were cohort study designs, with 5.8% versus 21% (p<0.001), followed by case–control studies in 1.3% versus 3.2% (p=0.243) and cross-sectional studies in 0% versus 7.5% (p=0.001). Overall, prospective trials were published in 71.4% versus 72% (p=0.494).

Table 2

Quality of reporting of abstracts at the World Congress of Sports Injury Prevention meetings in 2005 in Oslo and 2008 in Tromsø according to the Consolidated Standards of Reporting Trials criteria for randomised control trials (2005, N=14; 2008, N=19)

CONSORT criteria for reporting RCT in conference abstracts

A significant improvement in reporting RCT according to the CONSORT criteria in conference abstracts was evident with a score for CONSORT of 5.8±0.9 versus 8.6±2.9 (p=0.001, CI −4.29 to −1.43, score range 1–17) from the 2005 conference in Oslo to the 2008 conference in Tromsø (figure 1). A detailed description of main outcomes demonstrated the most significant improvement in reporting in the abstracts of RCT. The study design mentioned in the title, trial design, participants, blinding, number analysed, outcome and funding did not improve significantly in the reporting RCT in conference abstracts. No improvement at all over the 3 years was noted for reporting quality of the title randomised, authors, intervention, objectives/hypothesis, randomisation, numbers randomised, recruitment and number analysed, harms and trial registration.

Figure 1

Score of randomised controlled trials according to the Consolidated Standards of Reporting Trials (CONSORT) criteria (black columns) and the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) criteria for observational studies (grey columns) for conference abstracts at the first and second World Congress of Sports Injury Prevention in Oslo 2005 and Tromsö 2008.

STROBE criteria for reporting observational studies

The improvement of the score for STROBE was 7.9±1.6 in 2005 versus 9.9±1.7 in 2008 conference abstracts reporting results of observational studies (p<0.001, CI −2.34 to −1.53, score range 1–22) from 2005 to 2008 (table 3). Applying the STROBE criteria for observational studies, significant improvements were reported for background/rationale (53.5% vs 98.2%, p<0.001), objectives (82.2% vs 95.1%, p=0.012, p<0.001), study design (25.2% vs 65%, p<0.001), setting (43.7% vs 65.6%, p=0.002), variables (20.7% vs 74.2%, p<0.001), participants (0.7% vs 10.4%, p=0.001) and funding (0% vs 5.5% p=0.006). No significant improvement was evident in title and abstract, data sources/measurement, statistical methods, outcome data, main result, other analysis and interpretation for observational studies. No improvement was reported either in participants, bias, study size, quantitative variables, descriptive data, other analysis, limitation and generalisability.

Table 3

Quality of reporting of abstracts at the World Congress of Sports Injury Prevention meetings in 2005 in Oslo and 2008 in Tromsø according to the Strengthening the Reporting of Observational Studies in Epidemiology criteria for observational studies (2005, N=135; 2008, N=164)

Geographical distribution of the presented abstracts

Most of the abstracts, presented at the World Congress of Sports Injury Prevention in 2005 and 2008, were from Europe (52.6%, 47.3%), followed by Oceania and North America in 2005 and North America and Asia in 2008. A detailed distribution is displayed in figure 2.

Figure 2

Continental distribution (%) of published abstracts at the World Conferences of Sports Injury Prevention in Oslo 2005 and Tromsö 2008.

Discussion

The major finding of this work is: comparing the quality of conference abstracts published at the 2005 and the 2008 World Congress of Sports Injury Prevention, an improved quality of reporting is evident, as determined by the CONSORT and STROBE criteria. While the percentage of published RCT in abstracts at the World Congress of Sports Injury Prevention remained unchanged at about 10%, a significant improvement was evident in reporting abstracts in RCT as well as for observational studies.

A recent analysis studying the quality of reporting of randomised clinical trials in abstracts of the 2005 annual meeting of the American College of Rheumatology found that the quality was suboptimal in many RCT abstracts.6 A more structural approach that ensures more detailed reporting of eligibility criteria, active and comparator interventions flow of participants, and adequate summary and precision of result is warranted. In this respect, the CONSORT statement sets out to recommend what information should be reported when describing a RCT. They strongly recommend the use of structured abstracts for reporting RCTs in both the abstracts of journal articles and conference proceedings. Given the fact that the CONSORT criteria were primarily designed for RCTs and showing discrepancies in matching with all abstracts, the STROBE Statement was used for observational designs. Currently, there are no published data on the conference abstract quality regarding a score based on the CONSORT statement for RCT or based on the STROBE Statement for observational studies.

The STROBE Statement was developed to assist authors when writing up analytical observational studies, to support editors and reviewers when considering such articles form publication. In contrast to the CONSORT statement, the STROBE criteria focused on reporting observational studies.

The STROBE criteria are currently not limited on the abstract, but for the full manuscript. The criteria bias, study size, quantitative variables, descriptive data, limitation and generalisability demonstrate some inconsistencies in matching to the abstract. Some of these criteria were reported in the abstract, and some criteria were generally not described in an abstract because of space limitations. In addition, the STROBE statement should not be interpreted as an attempt to prescribe the reporting of observational research in a rigid format. The checklist items should be addressed in sufficient detail and with clarity somewhere in an article, but the order and format for presenting information depend on author preferences, journal style and the traditions of the research field.

Clear, transparent and accurate reporting or research is important because it enables readers to understand what was done and hence to evaluate the reliability and relevance of the findings. The extension of CONSORT and STROBE Statement aims to improve the reporting in RCT and observational studies, respectively. However, we believe, beyond the primary scope of CONSORT for RCT and STROBE for observational studies, that even other clinical trials will benefit from the implementation of the aforementioned CONSORT and STROBE criteria to improve the reporting quality, both in the abstract and in the full manuscript.

As far as the minimum requirements for future conference abstracts are concerned, it is rather difficult to propose detail suggestions. Following the CONSORT and STROBE criteria, in our view it would be crucial in future to clearly note the study design, details on recruitment of participants, main outcome scores, harms and especially potential funding. These items would allow the abstract reader to assess the conference abstract in a more comprehensive way. However, limited available space has to be considered in this regard. Therefore, author guidelines might be adjusted in a way to enhance the information content, even for conference abstracts such as for the third World Conference on Sports Injury Prevention

Word count limitation in conference abstracts has to be determined in this regard. To obtain a more comprehensive reporting quality of conference abstracts in clinical trials, a modified abstract structure is to be considered in the future. A more comprehensive structure of RCT according to the CONSORT and observational studies according to the STROBE statement will need a further subdivision of abstract categories introduction, methods, results and conclusion. However, we calculated that between 275 and 300 words would be necessary to obtain and address all issues raised in the CONSORT or STROBE statements. The American Society for Plastic Surgeons has introduced the need for additional conference abstract information during abstract submission. The following subheadings are to be addressed:

  • type of clinical question;

  • study population;

  • intervention/exposure of interest;

  • outcome;

  • research design;

  • methods;

  • data;

  • conclusion.

For future conference organisation teams, it might be worth while considering subheadings for abstract submission. External funding as well as harms should be addressed in future conference abstracts in much more detail. For RCT details on the randomisation process, the type of analysis (intention to treat), and the trial registration number are reasonable issues to be addressed.

Limitations

Given the suggestions of the organising committee regarding the abstract format and the restricted word count, one has to take into account that the implementation of a more comprehensive reporting in abstracts might be hampered by the abstract requirements by the organisers. Furthermore, the CONSORT criteria were suggested primarily for RCT, which take only about 10% of study designs in conference abstracts. We also acknowledge that STROBE is currently limited to three main observational study designs: cohort, case–control and cross-sectional studies. No statements or checklists for experimental or other study designs were available at the time of the preparation of this manuscript.

What is already known on this topic

  • Conference abstracts are a widely used source for medical information.

  • Consolidated Standards of Reporting Trials (CONSORT) criteria for randomised controlled trials (RCT) and Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) for observational studies are suggested to improve the quality of reporting of clinical trials.

  • CONSORT criteria for the improvement of the quality of reporting in conference abstracts are published.

What this study adds

  • Quality of reporting in conference abstracts still needs to be improved, albeit the recent improvements regarding RCT according to the CONSORT criteria and observational studies according to the STROBE criteria in abstracts in sports injury prevention abstracts.

  • In particular, details on randomisation, numbers randomised, intention-to-treat analyses and harms as well as trial registration for RCT are to be implemented in future conference abstracts.

Conclusion

While the percentage of published RCT in abstracts at the World Congress of Sports Injury Prevention remained unchanged, improvement in reporting of abstracts was evident from 2005 to 2008, as determined by CONSORT/STROBE criteria in both RCT and observational studies. However, substantial and comprehensive use of the CONSORT and STROBE criteria might further increase the quality of reporting of sports-injury conference abstracts in the future.

References

View Abstract

Footnotes

  • Competing interests None.

  • Provenance and peer review Not commissioned; externally peer reviewed.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.