Statistics from Altmetric.com
The media play a vital role in science communication, and media coverage of scientific articles is influential and encouraged. Altmetric scores are ‘alternative metrics’ that complement traditional citation-based metrics. They are an extensive group of metrics based on scientific articles’ mainstream media coverage or mentions on social media networks, among other factors.1 In sports science journals, Altmetric scores have a stronger relationship with citations than journal impact factor or open access status.2 Funding bodies value citations creating an incentive for academics to promote their work in the media, but this could result in high-quality research competing with media agency priorities.
Journalists play an essential role in disseminating research findings to a wider audience.3 They have a difficult job translating complex messages under deadline into stories that people will both read and understand.4 It is a delicate situation whereby journalists play a vital role in shaping healthcare utilisation and beliefs3 but do not always have the time and skills to interpret research findings or critically report their flaws. Unfortunately, it is not uncommon that weaker studies, preliminary findings and publications with positive results may attract headlines and higher Altmetric scores, whereas studies with a robust design but a null effect may not.5 6
This commentary aims to examine the Altmetric score of two recently published studies that investigated the relationship between exercise and COVID-19 severe clinical outcomes. This insight could help readers understand why something has a substantial impact—it may not be the research quality alone, but rather other factors such as the timing or the country of origin of the publication that may drive the Altmetric.
A study by Sallis et al 7 was published first in April 2021. It was a retrospective observational study that compared hospitalisation rates, intensive care unit admissions and mortality among patients with COVID-19 with varying adherence to physical activity guidelines. They found that ‘physical inactivity is associated with a higher risk of severe COVID-19’ and recommended ‘efforts to promote physical activity be prioritised by public health agencies and incorporated into routine medical care’. Following publication and a BMJ press release, 284 news outlets promoted the study using attention-grabbing headlines such as: ‘Physically active persons can fight off COVID-19 more easily’ and ‘Regular physical activity a strong protection against COVID-19’.
The second study by Lee et al 8 was published just 3 months later, in July 2021, also in the BJSM. It was also an observational cohort that investigated the potential associations between physical activity and risk of SARS-CoV-2 infection, severe illness from COVID-19 and COVID-19-related death. Of note, this study was conducted with robust methods including a theoretical basis for adjusting for confounders based on a causal Directed Acyclic Graph (cDAG).9 cDAGs provide researchers with a blueprint of the exposure and outcome relationship and the other variables that play a role in that causal question, and help researchers to identify biases that can affect the validity of causal inference in observational studies.9 The authors reached a similar conclusion to that of Sallis et al but were more cautious in reporting it: ‘our findings suggest that engaging in physical activity has substantial public health value and demonstrates potential benefits to combat COVID-19.’
The first study (Sallis et al) had a massive impact; its Altmetric score is currently 8774. The second study (Lee et al), published later, currently has an Altmetric score of 889, and its trajectory suggests that it is not likely to reach the heights achieved by Sallis et al. In table 1, we show the number of times each study was downloaded in the month published, and its current Altmetric score.
Both papers provide important insights regarding the relationship between physical activity and COVID-19 clinical outcomes. Journalists promoted the Sallis et al publication intensively, but unfortunately not the Lee et al publication. A qualitative study investigating journalist views on media coverage of medical tests and overdiagnosis provides a template to understand media coverage from the perspective of journalists.10 The interviewed journalists stated that they would promote research findings that they thought were topical and that recommended implementable changes in the management of a common condition. Both Sallis et al and Lee et al satisfy this criterion with suggestions for minimising the severity of a COVID-19 infection. The journalists also stated that peer-reviewed research was a prerequisite for reporting; again, both Sallis et al and Lee et al published in the same reputable, peer-reviewed journal. Neither of these criteria seems to explain the massive difference in Altmetric between the studies.
Other significant differences that may explain the discrepancy were the location (the USA vs South Korea), the level of promotion on personal or university social media channels, and, the timing of the release (April 2021 vs July 2021). Journalists acknowledge that attempts to keep content interesting can impact media coverage.10 Lee et al used additional important research methods, but to publish second, was ‘old news’ and just bad luck?
Journals that prepare press releases should exercise caution and be explicit with their message as media agencies are not qualified or motivated to identify gaps in research methods. A media advocacy strategy is one option to support journalists accurately relating research findings. For example, researchers, journals, policy consultants and journalists funded a strategy to accompany a series of papers on low back pain published in The Lancet in 2018. This strategy improved the accuracy of how low back pain was reported in the media, increasing from 35.9% of stories being at least very accurate before the promotion to 100% being completely accurate during the promotion.6
We recognise a media advocacy strategy may not be feasible for all publications. Meanwhile, the onus is on the research community to support our colleagues to conduct and promote high-quality research. Scientists and journalists should work together to ensure that high-quality work is not overlooked and, in doing so, provide access to the best available evidence to the public—no matter the timing or the country of origin.
Patient consent for publication
Twitter @EdelOH, @MaryOKeeffe007
Contributors EO'H conceived the study. RRNR provided methodological expertise. MO'K provided content area expertise. All authors read, contributed to and approved the final version of the manuscript.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.