Accuracy of ECG interpretation in competitive athletes: the impact of using standised ECG criteria
- Jonathan A Drezner1,
- Irfan M Asif1,
- David S Owens2,
- Jordan M Prutkin2,
- Jack C Salerno3,
- Robyn Fean1,
- Ashwin L Rao1,
- Karen Stout2,
- Kimberly G Harmon1
- 1Department of Family Medicine, University of Washington, Seattle, Washington, USA
- 2Division of Ciology, University of Washington, Seattle, Washington, USA
- 3Division of Ciology, Seattle Children's Hospital, Seattle, Washington, USA
- Correspondence to Jonathan A Drezner, MD, Department of Family Medicine, 4060 East Stevens Way, University of Washington, Box 354410, Seattle, WA 98195, USA;
Contributors All authors contributed to the study design, data analysis, writing and/or review of this manuscript.
- Received 21 September 2011
- Accepted 6 January 2012
- Published Online First 5 February 2012
Background Interpretation of ECGs in athletes is complicated by physiological changes related to training. The purpose of this study was to determine the accuracy of ECG interpretation in athletes among different physician specialties, with and without use of a standised ECG criteria tool.
Methods Physicians were asked to interpret 40 ECGs (28 normal ECGs from college athletes randomised with 12 abnormal ECGs from individuals with known ciovascular pathology) and classify each ECG as (1) ‘normal or variant – no further evaluation and testing needed’ or (2) ‘abnormal – further evaluation and testing needed.’ After reading the ECGs, participants received a two-page ECG criteria tool to guide interpretation of the ECGs again.
Results A total of 60 physicians participated: 22 primary care (PC) residents, 16 PC attending physicians, 12 sports medicine (SM) physicians and 10 ciologists. At baseline, the total number of ECGs correctly interpreted was PC residents 73%, PC attendings 73%, SM physicians 78% and ciologists 85%. With use of the ECG criteria tool, all physician groups significantly improved their accuracy (p<0.0001): PC residents 92%, PC attendings 90%, SM physicians 91% and ciologists 96%. With use of the ECG criteria tool, specificity improved from 70% to 91%, sensitivity improved from 89% to 94% and there was no difference comparing ciologists versus all other physicians (p=0.053).
Conclusions Providing standised criteria to assist ECG interpretation in athletes significantly improves the ability to accurately distinguish normal from abnormal findings across physician specialties, even in physicians with little or no experience.
The interpretation of a resting 12-lead ECG in athletes is challenging whether performed during preparticipation ciovascular screening or for diagnostic purposes in the setting of concerning symptoms or family history. Ciac adaptation and remodelling from regular athletic training produces common ECG alterations that may be considered abnormal in other settings. Over the past decade, criteria to differentiate normal, physiological ECG changes in athletes from ECG findings suggestive of underlying ciovascular pathology have been refined.1,–,8 The use of modern, standised ECG criteria has decreased the false-positive rate during preparticipation screening from 7–15%1 9 to 2–5%7 10 11 when performed by physicians experienced in ECG interpretation.
The European Society of Ciology (ESC) and the International Olympic Committee (IOC) recommend a resting ECG in addition to a stand history and physical evaluation during preparticipation screening, while the American Heart Association (AHA) supports performing an ECG in the setting of concerning symptoms, family history or physical examination findings.3 12 13 Whether obtained for screening or diagnostic purposes, it is critical that ECGs in athletes are correctly interpreted to avoid missing potentially dangerous ciovascular conditions or ordering unnecessary follow-up testing. The purpose of this study was to assess the accuracy of ECG interpretation in athletes among different physician specialties, with and without use of a standised ECG criteria tool.
An online ECG interpretation exercise was designed consisting of 40 ECGs: 28 normal ECGs acquired from Division I college athletes randomised together with 12 abnormal ECGs from individuals with known ciovascular pathology. The abnormal ECGs represented the most common causes of sudden ciac death (SCD) in young athletes including five ECGs with changes consistent with hypertrophic ciomyopathy (HCM), two demonstrating long QT syndrome, two with Wolff-Parkinson-White syndrome, and one each with arrhythmogenic right ventricular ciomyopathy, left ventricular non-compaction and Brugada syndrome. The normal ECGs came from National Collegiate Athletic Association (NCAA) Division I collegiate football and basketball athletes and demonstrated common ECG changes consistent with physiological adaptations of training such as sinus bradycia, sinus arrhythmia, early repolarisation and isolated increases in QRS voltage which are considered normal based on modern criteria. The ECGs selected for inclusion were agreed upon by a panel of physicians composed of four ciologist including a paediatric and adult electrophysiologist and two ciomyopathy experts and three sports medicine physicians, all with experience in interpreting ECGs in athletes.
All identifying patient information and computer-generated interpretations were removed from the ECGs; however, interval values and axis measurements were left in place. The ECG interpretations provided by commercially available ECG systems are often inaccurate when applied in young athletes. These ECG systems are programmed with adult and paediatric norms that do not account for the ECG changes that correspond with normal, physiological ciac remodelling in trained athletes. Thus, computer-generated ECG interpretations will frequently list ECG findings as ‘abnormal’ when they are actually normal variants in athletes. Therefore, the device-generated ECG interpretations were removed to avoid biasing study participants.
Physicians from academic and community practice settings were recruited to participate from different specialties including primary-care residents, primary-care attending physicians, primary-care sports medicine attending physicians and ciologists, none of which had any specialty training or experience in ECG interpretation in athletes. Participants were asked to assume that the ECGs were from asymptomatic athletes between the ages of 14 and 35 and to classify each ECG as either ‘normal or variant – no further evaluation or testing needed’ or ‘abnormal – further evaluation and testing needed’. After reviewing and classifying the 40 ECGs, physicians received a two-page ECG criteria tool to guide their interpretations while reviewing the 40 ECGs a second time (see Web only data supplement file). The ECG criteria tool was based on the 2010 ESC consensus statement for interpretation of the 12-lead ECG in athletes, as well as other research and input from experts familiar with ciac screening in athletes.2 4 8 The first page of the criteria tool consisted of two tables. The first table listed each individual ECG criteria that should be considered abnormal, unrelated to athletic training, and warrant additional investigation (table 1). A second table listed the normal, physiological ECG changes commonly found in trained athletes that should not trigger additional testing (table 2). The second page of the criteria tool consisted of several figures serving as examples of uncommon ECG patterns such as a variant pattern of repolarisation in African–American athletes, complete bundle branch blocks, ventricular pre-excitation, calculation of the QT interval corrected for heart rate, Brugada pattern ECG and Epsilon waves. The criteria tool could not be downloaded until the first round reviewing ECGs was completed. During the second round of ECG review with access to the criteria tool, if the ECG was classified as abnormal, the participants were additionally asked to check which individual criterion was present that categorised the ECG as abnormal.
Paired t tests were used to compare the proportion of ECGs correctly categorised (normal vs abnormal) before and after use of the ECG criteria guideline. Student t test was used to compare the proportion of ECGs correctly categorised between different physician specialty groups; α=0.05 and ß=0.20 (power=0.80) were assumed with at least 10 physicians in each specialty group needed to achieve adequate power (80%) to detect a statistically significant difference (p<0.05) in correctly classifying an ECG between physician specialty groups. The study was not designed to compare the correct classification of ECGs for a specific diagnosis among different physician specialties or to compare the correct classification of normal ECGs based on different findings considered normal variants. Descriptive statistics were used to analyse the classification of normal and abnormal ECGs representing a specific diagnosis by the group as a whole. The study was approved by the Human Subjects Division at the University of Washington.
Sixty physicians completed the exercise: 22 primary-care residents, 16 primary-care attending physicians, 12 sports medicine attending physicians and 10 ciologists. None of the physicians had any specialty training in ECG interpretation in athletes. At baseline, primary-care residents accurately categorised 73% of ECGs as normal (no further testing needed) or abnormal (further testing needed). Primary-care attending physicians correctly classified 73%, sports medicine attending physicians 78% and ciologists 85%. Ciologists were significantly better than all primary-care physicians at interpreting the ECGs before the use of the ECG criteria tool (p<0.001).
With use of the ECG criteria tool, all physician groups significantly improved their ability to correctly classify ECGs (p<0.0001) (figure 1). Primary-care residents increased their accuracy to 92%, primary-care attending physicians to 90%, sports medicine attending physicians to 91% and ciologists to 96%. With use of the ECG criteria tool, there was no statistical difference in the accuracy of ECG interpretation between ciologists (96%) and all other physicians (91%) (p=0.053).
The specificity of the whole group (normal ECGs correctly classified as normal) improved from 70% to 91%, and the sensitivity (abnormal ECGs correctly classified as abnormal) improved from 89% to 94% with use of the ECG criteria tool. Similarly, the positive-predictive value improved from 69% to 91%, and the negative-predictive value improved from 86% to 94% with use of a standised criteria tool (table 3).
The normal ECGs were most often incorrectly categorised. Without use of the criteria tool, a false-positive interpretation was provided in 30% of the normal ECGs. After use of the criteria tool, the false-positive rate decreased to 9%. The specific criteria marked as abnormal when the ECG was actually normal varied widely. The proportion of ECGs representing a specific diagnosis that was correctly classified before and after use of the criteria tool is shown in figure 2.
SCD is the leading cause of death in young athletes on the playing field and typically the result of undiagnosed structural or electrical ciovascular disease.14,–,16 The prevalence of ciovascular disorders known to cause SCD in young athletes is approximately 0.3% or 3 athletes in 1000.1 10 13 17,–,20 A recent study found the incidence of SCD in college athletes to be four to five times higher than prior estimates, affecting 1 in 43 000 athletes per year.14 With each tragic death of a young athlete during sports, intense public and medical scrutiny is raised reging the adequacy of preparticipation screening programmes and current strategies for prevention.
Ciovascular screening in athletes is routinely practiced and endorsed by most major sporting and medical associations including the AHA, ESC and IOC.3 12 13 However, universal agreement on a single screening protocol to identify athletes at risk for SCD remains elusive and a topic of considerable debate. The screening controversy is centred upon the inclusion (or not) of a resting 12-lead ECG in addition to a history and physical examination during the preparticipation evaluation. While ECG increases the sensitivity to detect silent ciac conditions in athletes, widespread use of ECG has raised concerns reging false-positive results, cost-effectiveness, physician infrastructure and healthcare resources.21
ECG is also commonly obtained in athletes with ciovascular-related symptoms, concerning medical history or physical examination findings. Whether performed for diagnostic or screening purposes as part of the ciac evaluation in athletes, quality ECG interpretations are crucial to identify athletes that require additional evaluation and to limit the number of false-positive analyses that lead to expensive secondary investigations.
It is imperative to recognise that the total-positive and false-positive rate for ECG interpretation are significantly affected by the criteria chosen to define ‘abnormal.’ Many ECG changes once referred to as ‘abnormal’ are now recognised as physiological and part of benign ciac adaptations in athletes (so-called athlete's heart). Physicians interpreting ECGs in athletes should be familiar with common training-related ECG alterations that are normal variants. In contrast, training-unrelated ECG changes suggest the possibility of underlying pathology, require further diagnostic investigation and should be considered abnormal. In 2010, the ESC Section on Sports Ciology published an international position statement summarising modern recommendations to distinguish pathological ECG abnormalities from physiological ECG alterations in athletes.4 In 2011, a US led statement on ECG interpretation in athletes provides additional detail and a description of contemporary ECG criteria and recommendations for secondary evaluations of abnormal findings.8
The most significant change from past ECG guidelines is the elimination of isolated QRS voltage criteria for left ventricular hypertrophy (LVH) as a cause for further evaluation. Isolated voltage criterion for LVH is an insensitive marker for LVH, found in up to 40% of highly trained athletes but in less than 2% of patients with HCM.22 Isolated increases in QRS amplitude are common in trained athletes but have not been found to be associated with a diagnosis of HCM in young adults undergoing preparticipation screening.23 In contrast, non-voltage criteria for LVH such as atrial enlargement, left axis deviation, a ‘strain’ pattern of repolarisation, ST-segment depression, T-wave inversion or pathological Q waves are considered abnormal and require further evaluation.4
Despite the publication and promotion of consensus guidelines for ECG interpretation, the impact of providing standised criteria during ECG interpretation has not been previously studied. ECG training is a core part of medical education; however, ECG training specific to athletes is rare, and stands to optimise training and the best way to achieve and maintain competency are not known and largely consensus-based.24 25
This study showed that simply providing physicians with a standised criteria tool, with no other training or information, significantly improves the ability for physicians to accurately distinguish normal from abnormal ECG findings in competitive athletes. Physicians were asked to correctly classify an ECG as normal (requiring no further testing) or abnormal (requiring further testing). Improvements in ECG interpretation with use of a criteria tool were seen across physician specialties despite these physicians having little or no experience in ECG interpretation in athletes. Physicians were not asked to make a specific diagnosis using only the ECG, although in some cases of ion channel disorders or ventricular pre-excitation this may have been possible. In the setting of abnormal findings suggestive of ciomyopathy, a definitive diagnosis is rarely made on the basis of ECG alone and usually requires additional investigations and collaborating history.
Hill et al26 examined the accuracy of paediatric ciologists in interpreting an athlete's ECGs. Fifty-three participants reviewed 18 ECGs, 8 from patients with normal hearts and 10 from patients with conditions known to cause SCD. Participants were asked to fill in the blank and provide the diagnosis as their primary measure of accuracy. Only 69% of ECGs were given the correct diagnosis leading study authors to conclude that mistakes in ECG interpretation could lead to high rates of inappropriate diagnostic testing and sports restriction.26 However, certain diagnoses, such as HCM and myocitis, cannot be made on ECG alone, and this may have biased the results tows a lower accuracy. In fact, in the case of ECGs representing HCM, 80% of the participants would have restricted these players from play and 85% would have ordered an echociogram based on the ECG, even though only 59% could provide the correct diagnosis based on ECG alone.26 The correct follow-up testing recommended by participants in the Hill study is similar to the baseline accuracy of ciologists (85%) in our study before use of the criteria tool. Thus, the ‘accuracy rate’ referred to in the study by Hill et al may be an inaccurate representation of a physician's ability to interpret an athlete's ECG. A physician may not be able to correctly identify a specific diagnosis on ECG but may still realise that additional testing or consultation is warranted.
This study and the study by Hill et al demonstrated the need for improvement in ECG interpretation even among experts and the importance of using standised criteria to guide the distinction between physiological changes and findings suggestive of pathology. It is not surprising that without a reference range or framework to judge an athlete's ECG that correct interpretation is challenging, similar to the difficulty in interpreting an unfamiliar laboratory value without having the reference range of normal values. ECG interpretation in athletes is complicated by physiological ECG changes that overlap with abnormal findings found in some ciovascular disorders. In this study, the most common ECGs that were incorrectly classified were normal or normal variant ECGs from competitive athletes. Thirty per cent of normal ECGs were incorrectly classified as abnormal without use of the criteria tool. After provision of standised criteria and with no additional teaching, normal ECGs that were incorrectly interpreted decreased to 9%.
The debate reging ECG screening in young athletes remains quite polarised. Major criticisms against ECG screening have been that testing cannot be done in large scale and that the false-positive rate will be too high (in the range of 10–40%) leading to unnecessary diagnostic testing and restriction from athletic participation. An initial screening study performed in the USA over two decades ago reported a false-positive rate of 15%.9 However, more recent studies applying modern ECG criteria to screen athletes have resulted in substantially lower false-positive rates ranging as low as 2–5%.7 10 11
The cost-effectiveness of ECG screening is also affected by the use of modern ECG criteria to guide interpretation. In a study of ciovascular screening in college athletes, the overall cost per diagnosis including all secondary evaluations was similar between history and physical compared with ECG.27 However, ECG interpretation was based on 2005 guidelines.3 If the study applied modern ECG criteria to assist in distinguishing physiological adaptations of an athlete's heart from findings suggestive of a pathological ciac condition,4 it is possible the ECG false-positive rate would be reduced by 85% (from 19% to <3%) with a correlating cost reduction simply by excluding voltage only and incomplete right bundle branch block findings as indications for additional investigation.
Similarly, in a study by Baggish et al, 510 college athletes were screened with history, physical examination, ECG and echociography.17 ECG improved the sensitivity for detecting important ciac abnormalities from 45% to 91%, but the study also reported a high false-positive ECG rate of 16%.17 However, application of the 2010 ESC criteria reduced the number of abnormal ECG findings from 16% to 9%.28 The improved specificity (reduction in false-positive rate) was largely driven by the reclassification of isolated QRS voltage criteria for LVH from abnormal to normal.
This study is limited by the small physician sample that interpreted the ECGs and that there was no method to ensure that the criteria tool was being used appropriately or referred to when needed. This study was purposefully conducted in physicians with little or no experience in ECG interpretation in athletes, and it is possible that a similar exercise in experienced physicians would yield greater accuracy. The study also does not account for testing behaviour and the possibility that physicians may interpret ECGs differently in a clinical setting where consequences exist if a clinically relevant finding is missed. Although physicians reviewed the same set of ECGs twice, there is little reason to believe that improvements in accuracy were related to a testing effect when the only change was the availability of a standised ECG criteria tool. In addition, this study had a very high proportion of abnormal ECGs (12/40), and the sensitivity and specificity may be different in a clinical setting with a lower frequency of true disease.
This study demonstrates that without further education, the baseline ability of physicians with little or no experience in interpreting an athlete's ECG is relatively poor and would lead to an unacceptable amount of secondary evaluations at a significant cost. However, simply providing physicians with standised criteria set with which to evaluate an ECG significantly improves accuracy. While this is only the first step in understanding the needs and potential for larger infrastructure development for the interpretation of ECGs in athletes, this study establishes that physician education in ECG interpretation is feasible and accompanied by significant improvements in all statistical measures of performance when a reference stand is used to guide interpretation.
What is already known on this topic
▶ ECG interpretation in athletes is complicated by ECG changes from physiological ciac adaptations that overlap with abnormal ECG findings found in some ciovascular disorders.
▶ Modern stands for ECG interpretation exist to help distinguish normal ECG changes in athletes from abnormalities suggestive of underlying pathology.
What this study adds
▶ In physicians with little experience in ECG interpretation in athletes, a large proportion of normal ECGs in athletes are incorrectly classified as abnormal.
▶ Use of standised criteria to assist ECG interpretation significantly improves physician accuracy to distinguish normal from abnormal findings across physician specialties.
▶ Physician education in modern stands for ECG interpretation in athletes may improve the ciovascular care of athletes by limiting false-positive results and enhancing disease detection.
Prevention of SCD in athletes remains a highly visible topic in ciology and undoubtedly more research on ciovascular screening and ECG interpretation in young athletes is needed. It is critical that physicians of any specialty be guided by stands that improve disease detection and limit false-positive results. A simple two-page criteria tool using modern stands for ECG interpretation significantly improves accuracy, even in physicians with little or no experience. Understanding the value of using a reference stand is only the first step tows improved ECG interpretation. Greater efforts tows physician education and formal training are needed to produce even better results leading to improved care of athletes.
Competing interests None.
Ethics approval University of Washington Division of Human Subjects.
Provenance and peer review Not commissioned; externally peer reviewed.