Article Text
PDF
Abstract
Background/aim The purpose of this study was to develop a subjective, self-report, sleep-screening questionnaire for elite athletes. This paper describes the development of the Athlete Sleep Screening Questionnaire (ASSQ).
Methods A convenience sample of 60 elite athletes was randomly distributed into two groups; 30 athletes completed a survey composed of current psychometric tools, and 30 athletes completed a revised survey and a sleep specialist structured clinical interview. An item analysis was performed on the revised survey with comparison to clinical decisions regarding appropriate intervention based on a sleep specialist assessment.
Results A comparison of existing sleep-screening tools with determination of clinical need from a sleep specialist showed low consistency, indicating that current sleep-screening tools are unsuitable for assessing athlete sleep. A new 15-item tool was developed (ASSQ) by selecting items from existing tools that more closely associated with the sleep specialist's reviews. Based on test-retest percentage agreement and the κ-statistic, we found good internal consistency and reliability of the ASSQ. To date, 349 athletes have been screened, and 46 (13.2%) identified as requiring follow-up consultation with a sleep specialist. Results from the follow-up consultations demonstrated that those athletes identified by the ASSQ as abnormal sleepers have required intervention.
Conclusions The research developed a new athlete-specific sleep-screening questionnaire. Our findings suggest that existing sleep-screening tools are unsuitable for assessing sleep in elite athletes. The ASSQ appears to be more accurate in assessing athlete sleep (based on comparison with expert clinical assessment). The ASSQ can be deployed online and provides clinical cut-off scores associated with specific clinical interventions to guide management of athletes’ sleep disturbance. The next phase of the research is to conduct a series of studies comparing results from the ASSQ to blinded clinical reviews and to data from objective sleep monitoring to further establish the validity of the ASSQ as a reliable sleep screening tool for elite athletes.
- Sleep
- Recovery
- Questionnaire
- Athlete
Statistics from Altmetric.com
Introduction
The relationship between sleep, postexercise recovery and performance in elite athletes, has become a topic of great interest in the sport science and medicine community. A growing body of scientific evidence has confirmed that there is a critical link between sleep and circadian rhythm factors, cognitive processes and metabolic function.1–5
There is interest within the sport science community in understanding the effect of sleep and circadian rhythms on athletic performance, recovery and regeneration.6 Sleep extension and circadian rhythm research have provided objective evidence that supports this focus.7–15 The amount of sleep, the quality of the sleep, as well as the timing of the sleep period, are considered to be the key factors that play a role in an athlete's ability to train, maximise the training response, recover and perform.8 ,9 ,11 ,16–18
Despite what is now understood about the importance of sleep, circadian factors and recovery, the issue remains that little is known about the sleep behaviours of athletes from an epidemiological perspective. There are no valid and reliable methods for screening athletes for sleep disturbance and sleep related impairment, and, most importantly, it is extremely difficult to screen the sleep behaviours of athletes given their lifestyle, the demands of training and travel associated with international competition. For instance, in-lab polysomnography, the gold standard diagnostic test for primary sleep disorders, and clinical interviews by sleep specialists, are too resource-dependent to be considered efficient and economical for use as screening or diagnostic tools for athletes.9 Significant advances have been made measuring athlete sleep using wrist actigraphy,8–11 however, this method is costly and time consuming when large numbers of athletes are being assessed for sleep disturbance. The current sleep screening tools and questionnaires that might be employed remotely are inexpensive and straightforward to implement, however, they may be inadequate for this specialised population. Standard sleep screening questionnaires are designed for the general population and do not take into account the impact of frequent international travel on long-term sleep behaviour. These questionnaires do not consider the impact of environmental factors such as heavy training loads and stress associated with competition over the course of a year, which are critical factors that affect an athlete's sleep.17 ,18
In 2008, Samuels conducted two pilot studies to assess the degree of severity of sleep dysfunction and sleep-related impairment in adolescent and adult Canadian athletes using the Pittsburgh Sleep Quality Index (PSQI).19 Using a conservative cut-off score ≥8, the findings indicated higher than expected numbers of athletes with poor sleep quality among the two studies.16 This research raised two important questions:
What is the true prevalence of sleep disturbance and sleep related impairment in a population of elite athletes?
Is an athlete-specific sleep-screening tool required for accurate identification of clinically significant sleep disturbance and sleep related impairment?
The present study was designed to address these questions by developing a self-reported sleep-screening questionnaire for elite athletes—the ASSQ.
Methods
To begin the process of developing and testing a new athlete-specific sleep-screening questionnaire, a convenience sample of 60 athletes was randomly selected from 300 eligible athletes who train out of the Canadian Sport Centre—Calgary. Fifty-eight of these athletes provided consent and were randomised into the study (the remaining 2 athletes declined to participate without penalty). From these 58 athletes, 30 were assigned to phase I (9 females and 21 males, mean age 25.0(SD=3.9)), and 28 were assigned to phase II (12 females and 16 males, mean age 26.8(SD=3.3)). All study procedures were carried out in accordance with ethical approval from the University of Calgary Conjoint Health Research Ethics Board (Ethics ID E-21109).
ASSQ development—phase I
Athletes completed a self-report sleep screen composed of current standard self-report psychometric tools including the PSQI,19 which is the most commonly used and widely cited psychometric sleep quality screening tool in sleep medicine and research; the Adjusted Neck Circumference,20 which is a pretest probability screening tool for obstructive sleep apnoea; and the Athlete Morningness/Eveningness Scale, which at the time of the research was the only athlete specific psychometric tool for determining endogenous sleep phase or chronotype.21 These questionnaires were selected on the basis of publication, overall use and subject-matter expert preference. Each athlete subsequently had a clinical interview conducted by an expert sleep specialist (who has extensive experience in assessing athlete sleep problems) within 30 days of completing the self-report psychometric tools. The interview was structured and based on the domains of the PSQI (bedtime, sleep onset latency, wake time, total sleep time, night-time awakenings and disturbance, sleep quality, sleep medication use, daytime functioning). The information gathered was then used to inform and guide the development of the ASSQ.
ASSQ development—phase II
Following the assessment of data from phase I, the second group of athletes completed a revised 51 question self-report screen, which included the Insomnia Severity Index (ISI), a valid and reliable psychometric screening tool, used to assess and monitor the severity of an individual's insomnia symptoms and response to treatment, whereas the AMES and ANC were replaced with the Composite Scale of Morningness (CSM), a well validated chronotyping questionnaire that is easy to implement, and the Maislin Apnea Risk Index (MAP), a multivariate sleep apnoea screening tool that also includes items used to assess sleep-related impairment.22 The athletes repeated the revised self-report screen 24 h later to provide test-retest correlation. Each athlete had a structured clinical interview by the expert sleep specialist. Clinical outcomes (none, mild, moderate, or severe clinical problem) were generated from the results of the revised self-report screen and clinical interview.
ASSQ development—questionnaire assembly
The 51 questions from the revised self-report screen were reduced to 15 and grouped to represent the critical domains used to assess athlete sleep and circadian factors: Total Sleep Time, Insomnia, Sleep Quality and Chronotype. Modifiers that are critical to sleep disturbance in athletes were also included: Travel Disturbance (sleep disturbance and performance impairment associated with travel) and Sleep Disordered Breathing (snoring, gasping). The identification of the critical domains and modifiers was accomplished through a thorough review of the revised self-report screen questions that indicated good concordance with clinical interview and best practices in clinical management of elite athletes’ sleep issues.
An aggregate ‘Sleep Difficulty’ score based on responses from the four domains (Total Sleep Time, Insomnia, Sleep Quality, Chronotype) was developed to discriminate the athletes into two groups: (1) those athletes who require further assessment (with a Sleep Difficulty Score of >12); and (2) those athletes who do not require further assessment (with a Sleep Difficulty Score of ≤12). The selection of >12 as the cut-off criteria was clinically based on early review of the data by the primary investigator in consultation with the statistician. A score of 12 was determined to be a moderate threshold for identifying athletes in need of help because of the ethical and clinical responsibility to identify and treat athletes who are struggling with their sleep.i However, this cut-off point was designed to be subject to adjustment in future studies of sensitivity and specificity.
Athletes scoring ≤12 are further distributed into two categories: no clinical problem and mild clinical problem. Athletes with no clinical problem receive standardised sleep education offered by the team support staff, who are trained to provide this level of education. Athletes with a mild clinical problem receive basic sleep monitoring by their support staff. Athletes scoring >12 are further distributed into two categories: moderate and severe clinical problem. Athletes with a moderate clinical problem are referred to a sports medicine physician for further assessment, and athletes with a severe clinical problem are referred to a sleep physician.ii
In addition to the Sleep Difficulty Score there are two other routes to referral—Travel Disturbance and Sleep Disordered Breathing (SDB). As figure 1 demonstrates, an athlete who snores and/or gasps in his or her sleep would be referred for further evaluation, even if his or her Sleep Difficulty Score is ≤12. Within figure 1, this is demonstrated by the SDB route to referral: Snore=No, Gasp=Yes (OR) Snore=Yes, Gasp=No (OR) Snore=Yes, Gasp=Yes. For example, a bobsledder who snores and/or gasps in his sleep needs to be evaluated for sleep apnoea through a sleep physician assessment and a sleep study. Similarly, an athlete who has both difficulty sleeping and daytime dysfunction when travelling for his or her sport would be referred for further evaluation, again, even if the Sleep Difficulty Score is ≤12 (within the figure, this is demonstrated by the Travel Disturbance route to referral: Sleep Disturbance=Yes, Daytime Dysfunction=Yes). For example, a national team downhill skier who identifies herself as having trouble with sleep and performance when travelling, will require further assessment and specific assistance with jet lag and travel fatigue.
Routes to referral: (1) Sleep Difficulty Score >12=referral; (2) Sleep Disordered Breathing (SDB) with snoring and/or gasping in sleep=referral; and (3) Travel Disturbance with a combination of difficulty sleeping and daytime dysfunction when travelling for a sport=referral.
Finally, demographic information is used to address the following critical factors that affect sleep in athletes, including: age, gender, able-bodied athlete versus para-Olympic athlete, and season, that is, precompetition, competition, off-season.
Statistical analysis
Analyses were performed using R with the psych package.23 Test-retest agreement was estimated with Pearson’s r. Intra-rater agreement, and agreement between the sleep specialist and the questionnaires were estimated with Cohen’s κ-statistic.24 Items were brought forward for consideration for use in the final survey instrument if (1) they were consistent with clinical assessment, (2) they demonstrated a positive point-biserial with the overall score, and (3) there was an association between the item and one of the four categories of sleep difficulties (sleep quality, sleep disturbance, insomnia and chronotype). To determine if there was an association, simple correlations (Pearson’s r) between the instrument and outcomes (0=no referral, education only, 1=no referral, education and monitoring, 2=referral to a Sport MD and 3=referral to a Sleep MD) were used. Logistical regression was then applied to the items and items were considered if the p values were significant (p<0.05). To estimate the internal consistency of the final questionnaire, Cronbach’s α was utilised.
Results
ASSQ development—phase I
Findings from phase I supported a concordance rate of 57% and 53% between the self-report sleep screen (made up of the PSQI, the ANC and the AMES), and the clinical interview by the sleep specialist that included a retest for the AMES and PSQI, respectively. This finding is close to chance—indicating that these measures are inadequate for subjective sleep screening of an elite athlete population.
ASSQ development—phase II
Test-retest correlations of data from Phase II were high (0.94 on average) for the PSQI, ISI, MAP, and the CSM (making up the revised self-report sleep screen). For the four outcomes (1=None, 2=Mild, 3=Moderate and 4=Severe), there was good agreement from day 1 to day 2 for PSQI, ISI, chronotype, apnoea index, sleep quality, sleep latency and sleep disturbances (83%, 90%, 90%, 90%, 77%, 80% and 90%, respectively). Cohen’s κ-statistic for the same indices were, respectively, 0.69, 0.72, 0.67, 0.75, 0.56, 0.69 and 0.53. Based on test-retest percentage agreement and the κ-statistic, there appeared to be good internal consistency and reliability of the survey. All items advanced for consideration in the ASSQ had significant p values (p<0.01).
ASSQ development—questionnaire assembly
A detailed review of the questionnaire items was performed to determine which items had the strongest association with the subject-matter expert's determination of clinical need. An enhanced and shorter 15-item questionnaire was the result. The questionnaire is structured around the key sleep factors of interest: ‘Total Sleep Time’ (night sleep, nap frequency and nap duration), ‘Insomnia‘ (time to fall asleep, trouble staying asleep, sleep medication usage), ‘Sleep Quality’ (satisfaction with sleep rated on a five point scale), ‘Chronotype’ (preferred bedtime, preferred wake time, sleep inertia on awakening, morningness/eveningness rating), ‘Sleep Disordered Breathing’ (snoring, gasping while sleeping) and ‘Travel Disturbance’ (sleep disturbance while travelling, performance disturbance while travelling).
The Sleep Difficulty score, which is used to guide intervention, is calculated from Total Sleep Time, Insomnia, Sleep Quality and Chronotype. However, as figure 1 demonstrates, combinations of responses to questions corresponding to sleep disordered breathing or sleep disturbance due to travel can result in a direct recommendation for referral, as described above.
Preliminary evidence of ASSQ validity
Since the ASSQ's development, 349 athletes have been screened. The findings from these results are currently being prepared for publication in a separate manuscript. However, a key finding is relevant to the current paper. Forty-six athletes (13.2%) had a sleep difficulty score above the 12-point cut-off, and were thus referred for follow-up consultation. Results from follow-up consultations demonstrated that athletes identified by the ASSQ as abnormal sleepers did require intervention. This provides preliminary evidence for the ASSQ's validity, although more work is required to validate the questionnaire.
Discussion
The results from previous studies using the PSQI to screen athletes16 seemed to indicate a high prevalence of poor sleep quality in the younger athlete population. This was contrary to the expectation that poor sleep quality would not be likely in a young, healthy athletic population. Our findings indicated the weakness of using standard sleep screening methods in a specialised (athlete) population. For this reason, we developed a subjective sleep-screening questionnaire specifically for elite athletes—the Athlete Sleep Screening Questionnaire (ASSQ).
Key results and potential implications
Key results and potential implications from the development of the ASSQ are as follows:
A battery of existing sleep screening questionnaires including the Pittsburgh Sleep Quality Index (PSQI), Athlete Morningness/Eveningness Scale (AMES) and Adjusted Neck Circumference (ANC) demonstrated, with clinical assessment by an expert, a concordance rate of 57%, 53% and overall poor applicability to the population, respectively—indicating that these measures are inadequate for subjective sleep screening of an elite athlete population.
The newly developed ASSQ had good internal consistency and reliability, based on test-retest percentage agreement and the κ-statistic.
Of the 349 athletes screened using the ASSQ to date, 46 (13.2%) were identified as requiring follow-up consultation with a sleep specialist, and results from follow-up consultations demonstrated that athletes identified by the ASSQ as abnormal sleepers did require intervention.
The goal of this research is to use information gathered by the ASSQ to improve training and recovery strategies throughout the life cycle of an athlete thereby enhancing the process of long-term athlete development.
Future directions
Although the data provide preliminary support for the validity of the questionnaire from a psychometric perspective, the survey needs further testing before it can be said to be clinically valid. We anticipate two distinct types of validation will occur during the next stages of the research.
First, clinical validity will be established by: (1) exposing a larger group of elite athletes to the survey; (2) randomly selecting a sample from each of the four-categories of clinical interest (none, mild, moderate and severe); (3) conducting a clinical interview by a sleep expert who is blinded to the athletes’ sleep difficulty ratings; and (4) determining if the survey can actually place athletes correctly into the four categories. If the survey is correct in its placement of athletes, then it will be clinically valid and will be able to function as a proxy for more expensive screening procedures.
Second, construct validity will be established by selecting a sample of athletes to participate in: (1) an actigraphy study where the athletes wear sleep-monitoring devices on their wrists 24/7 to get a profile of their sleep patterns; and (2) a polysomnography study where a smaller sample of athletes visit a sleep centre and are monitored using the gold standard of objective sleep measurement. In each of these studies, athletes would also be screened using the ASSQ. If the survey is accurate in its identification of sleep disorders (insomnia, sleep disordered breathing, etc), that will be further evidence of its validity as a sleep-screening tool. If that is the case, the widespread adoption of the ASSQ will allow for accurate and cost-effective sleep screening of elite athletes. The implications of this for athletic performance and recovery are considerable.
What are the new findings?
A battery of existing sleep screening questionnaires including the Pittsburgh Sleep Quality Index (PSQI), Athlete Morningness/Eveningness Scale (AMES) and Adjusted Neck Circumference (ANC), demonstrated, with clinical assessment by an expert, a concordance rate of 57%, 53% and overall poor applicability to the population, respectively—indicating that these measures are inadequate for subjective sleep screening of an elite athlete population.
The newly developed ASSQ had good internal consistency and reliability, based on test-retest percentage agreement and the κ-statistic.
Using the ASSQ, of the 349 athletes screened to date, 46 (13.2%) were identified as requiring follow-up consultation with a sleep specialist, and results from follow-up consultations demonstrated that athletes identified by the ASSQ as abnormal sleepers did require intervention.
How might it impact on clinical practice in the near future?
Current sleep screening tools/questionnaires are likely to be inadequate for screening an athlete population because this specialised cohort is younger, healthier and fitter than the general population. They also may have different sleep requirements than the general population due to the volume, frequency and intensity of their training regimen, and the onerous nature of frequent transcontinental and international travel. The present study developed the Athlete Sleep Screening Questionnaire (ASSQ) to address this gap and provide clinicians with an athlete specific sleep-screening questionnaire.
The goal of this research is to use information gathered by the ASSQ to improve training and recovery strategies throughout the life cycle of an athlete, thereby enhancing the process of long-term athlete development.
Acknowledgments
The authors would like to acknowledge the support and contribution of Dr Stephen Norris and Dr David Smith to the development of a sleep and human performance initiative at the University of Calgary Sport Medicine Centre, and Canadian Sport Centre Calgary.
References
Footnotes
Contributors CS is a clinical assistant professor of medicine and adjunct professor in the faculty of Kinesiology at the University of Calgary. CS is the medical director of the Sleep Centre for Human Performance in Calgary. He is the Principal Investigator of the research and is responsible for all elements of its execution from planning to publishing. LJ is the research coordinator for the Sleep Centre for Human Performance. She wrote the manuscript being presented for publication (with guidance from CS). DL provided methodological advice and conducted the statistical analyses throughout all phases of the research. WM is the anchor author on the manuscript, and has been responsible for the inception of the project, facilitating the recruitment of athletes, implementing the dissemination of the ASSQ and providing editorial advice for the manuscript.
Funding This study has been provided by the Canadian Olympic Committee and Own the Podium.
Competing interests None declared.
Ethics approval University of Calgary Conjoint Health Research Ethics Board (Ethics ID E-21109).
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement We will make data directly related to the analyses reported on in this manuscript available to those who wish to publish jointly. The corresponding author may be contacted at dr.samuels@centreforsleep.com for more information.
↵i Future research is being undertaken to further refine the cut-off score based on sensitivity and specificity testing that will focus on the clinical and construct validity of the ASSQ.
↵ii These groupings are based on clinical decisions made by the principle investigator and direct the intervention.