Article Text

other Versions

Practice management of musculoskeletal injuries in active children
  1. Mathieu Boudier-Revéret1,2,3,
  2. Barbara Mazer1,2,3,
  3. Debbie Ehrmann Feldman2,4,
  4. Ian Shrier5
  1. 1School of Physical and Occupational Therapy, McGill University, Montreal, Quebec, Canada
  2. 2Centre de Recherche Interdisciplinaire en Réadaptation du Montréal Métropolitain, Montréal, Quebec, Canada
  3. 3Jewish Rehabilitation Hospital, Laval, Quebec, Canada
  4. 4École de réadaptation, Université de Montréal, Montreal, Quebec, Canada
  5. 5Centre for Clinical Epidemiology and Community Studies, Lady Davis Institute for Medical Research, Jewish General Hospital, Montreal, Quebec, Canada
  1. Correspondence to Dr Ian Shrier, Department of Family Medicine, Centre for Clinical Epidemiology and Community Studies, McGill University, SMBD-Jewish General Hospital, 3755 Cote Ste-Catherine Road, Montreal, QC H3T 1E2, Canada; ian.shrier{at}


Background Although increasing participation in physical activities has significant health benefits, there are no guidelines to help professionals decide when it is safe to return to activity after injury.

Objective To examine the specific criteria (eg, strength, pain) that expert sport medicine clinicians use for return to activity decisions in children with musculoskeletal injuries.

Methods The authors conducted an online cross-sectional survey of certified Canadian sport medicine doctors (MDs) and sport rehabilitation specialists (physiotherapists (PTs) or athletic therapists (ATs)). The authors asked how they would measure each of the following signs in the context of a knee injury: sport-specific skills, pain, swelling, strength, range of motion (ROM) and balance. Clinicians also ranked the importance of each sign with respect to influencing their recommendations for each of five clinical vignettes.

Results The overall response rate was 33.6% (464/1380) with similar rates for each profession. For each clinical sign, all three professions preferred the same measure to determine readiness to return to play: standardised testing for sport-specific skills, impact on function for pain, palpation for swelling, manual muscle testing for strength, visual inspection for ROM and standing on one leg with eyes closed for balance. Regarding importance of specific signs for return to activity, all professions had similar responses for one vignette, but MDs differed from PTs and ATs for the remaining four. Finally, pain was ranked as the no 1 or 2 most important sign in all five vignettes by 41.0% of MDs, 18.1% of ATs and 11.3% of PTs, whereas sport-specific skills was chosen by 9.6% MDs, 12.0% ATs and 16.1% PTs.

Conclusion Our results provide the foundation for future work leading towards the development of interdisciplinary consensus guidelines.

Statistics from


Increasing participation in physical activities is associated with an increase in quality of life,1 enhanced academic performance,2,,4 a positive impact on behaviour,5,,7 better mental8 and physical health,9 10 and decreased obesity.11 In addition, children who are physically active are more likely to remain active in adulthood.12 13

Despite its benefits, an increase in physical activity is also associated with an increase in the risk of injury. An estimated 20.6 million children are injured each year in the USA,14 and 559 000 in Canada.15 According to national emergency department data, reinjury rates in children vary between 15%16 and 32%,17 and reinjury rates increase with age.15

When a child is injured, clinicians are often asked when it would be safe to return to activity, but research is limited, and no guidelines currently exist. The first step towards prioritising areas for research and developing guidelines is to understand current practice. As part of a larger study examining the current practice of healthcare professionals with expertise in sports injuries, our objective was to examine the specific types of criteria (eg, strength, pain) that sport medicine practitioners use for return to activity decisions.

Materials and methods

The current study is part of a larger programme and reports only the data relevant to our specific objective. Ethical approval was obtained from the Centre for Interdisciplinary Research in Rehabilitation of Greater Montreal (CRIR) Ethics Committee. In brief, we conducted a cross-sectional survey of Canadian sport medicine (medical doctors (MDs) certified by the Canadian Academy of Sport Medicine) and sport rehabilitation specialists (physiotherapists (PTs) certified by the Canadian Physiotherapy Association and athletic therapists (ATs) certified by the Canadian Athletic Therapists Association). The survey was developed after focus-group discussions with members of each profession and pilot-tested on a separate group of members.

The current manuscript focuses on the choice of measure for each of the following signs in general (appendix 1): sport-specific skills, pain, swelling, strength, range of motion (ROM) and proprioception/balance (for brevity, ‘balance’). For each clinical sign, respondents had to give a return to activity cut-off value. To determine if context is important, clinicians read a brief clinical vignette (appendix 2) and ranked the importance of each sign within that context. Clinical vignettes are used to isolate clinician decision-making from other factors affecting management,18 19 and are a valid method of measurement of the quality of physician practice.20


We classified respondents according to their preference when they were certified by more than one organisation (n=15). When clinicians chose ‘other’ to indicate the type of preferred test and then simply indicated a combination of factors, we recategorised them using the measure with the highest level of objectivity and reliability (ordering provided in table 1). For cut-off values, we standardised those measurement approaches that used different scales and units to consistently indicate normal, mild, moderate–severe according to the following a priori algorithm based on literature and our clinical experience:

  • Sport-specific: when answers were expressed as a percentage of normal, we considered 100% as normal, 80–99% as mild and <80% as moderate–severe.

  • Pain: for visual analogue scales, we considered 0 as normal, 1–4 as mild and 5–10 as moderate–severe.21 22 In cases where the scale was reversed (eg, 100% meant no pain), we used 60% as the cut-off between mild and moderate–severe. As sensitivity analyses, we also used 80% as the cut-off between mild and moderate–severe, and recategorised pain as minimal for scores 1–2 and mild for scores 3–4.

  • Swelling: when expressed as differences in centimetres, we considered 0 cm as normal, ≤1 cm as mild and ≥1 cm as moderate–severe. When answers were expressed as a percentage of normal, we considered 100% as normal, 80–99% as mild and <80% as moderate–severe.

  • Strength endurance: when measured as the number of repetitions until fatigue, we arbitrarily considered ≥30 as normal, 15–29 as mild and <15 as moderate–severe.

  • ROM: We considered 100% as normal, 90–99% as mild restriction and <90% as moderate–severe restriction. When expressed as degrees of knee flexion, our standardisation process was limited to choosing one of the various recommended normal values (range 130–150°).23 We considered 150° as normal (because children have less muscle bulk), 130–149° as mild restriction and ≤129° as moderate–severe restriction (position of hip was not specified in survey).

  • Balance: we arbitrarily considered ≥30 s as normal, 15–29 s as mild and <15 s as moderate–severe.

Table 1

Proportion of respondents who chose a particular measure for each clinical sign

To compare the ranking of the different signs for a particular vignette across professions, we report the percentage of subjects within each profession who ranked the sign as number 1 or 2 for that vignette. To determine which signs were considered important regardless of the clinical context, we calculated the percent of clinicians within each profession who considered the sign important (ranked 1 or 2) across all vignettes. For these analyses, we only included clinicians who answered all five vignettes (n=311). When clinicians (n=26) had two consistent clinical signs (always ranked either 1 or 2) across all five vignettes, both were included in the results.

We present categorical variables as percentages. We used the χ2 test (or Fisher exact test where appropriate) for comparisons with categorical exposures and outcomes. We considered p<0.05 as statistically significant for overall comparisons of the three professions. To determine which professional group was different from the others (ie, AT, PT or MD), we conducted post hoc testing of each pairwise comparison using Fisher exact test and considered p<0.1 as suggesting a difference between groups that should be explored in future studies because of the smaller sample sizes.


Of the 1439 email invitations sent, 49 were returned as undeliverable, and 10 people replied that they were not currently working in the field. Of the remaining 1380 potential subjects, 464 (33.6%) responded to the survey (430 English, 34 French), including 265 ATs (33.2% of eligible ATs), 123 MDs (31.5% of eligible MDs) and 76 PTs (39.8% of eligible PTs). A summary of their demographic and clinical practice information is presented in table 2. Overall, compared with MDs and PTs, ATs were younger (75.6% in the 25–35 category vs 8.3% and 35.5% respectively) and had fewer years in practice (52.3% in the ≤5 years category vs 9.0% and 15.8% respectively). However, ATs worked at more events per year. As anticipated, children represented a small percentage of the clientele for all three professions (<30% of client-visits).

Table 2

Demographic and clinical practice information

Table 1 shows that the most common method used for each clinical sign was the same across professions. For example, standardised testing is the most frequently used method for sport-specific skills across all professions, as is impact on function for pain, palpation for swelling, manual muscle testing (MMT) for strength, visual inspection for ROM and standing on one-leg with eyes closed for balance. Despite these similarities, the overall pattern of responses is significantly different between the three professions for each of the six clinical signs (p ranges from <0.001 to 0.027). More specifically, post hoc comparisons suggested that MDs' preferences were most different for sport-specific skills (eg, subjective report), pain (eg, visual/verbal analogue scale) and swelling (eg, visual inspection) (p ranged from 0.0001 to 0.06 for MD vs AT or MD vs PT, p ranged from 0.085 to 0.82 for AT vs PT). PTs were different (p<0.01) from MDs and ATs for strength (p=0.17 for MD vs AT) and ROM (p=0.99 for MD vs AT), and all three groups differed from each other for balance (p<0.001 for each pairwise comparison).

For each clinical sign, respondents indicated the cut-off value (either normal, mild restriction or moderate–severe restriction) they would employ to determine a return to activity without restriction for the measure they most frequently used. Subjects who answered ‘do not use’ for that clinical sign were excluded (table 3). Again, the most frequent responses were similar across professions, and the moderate–severe category was rarely chosen (max 14.3%). In addition, the pattern of responses across professions was similar for pain, strength, ROM or balance (p ranges from 0.35 to 0.94). For sport-specific skills (overall p=0.016), MDs appeared to require higher function compared with ATs and PTs (p≤0.02 for both comparisons), whereas all three groups were likely different from each other for swelling (overall p=0.0046, individual comparison p ranged from <0.001 to 0.069).

Table 3

Proportion of respondents* who chose particular cut-off values to determine return to activity without restriction

For pain, we considered mild as 1–4 on a 0–10 visual analogue scale.22 Others have suggested minimal is 1–2 and mild 3–4.24 To explore the effect, we recategorised those who used a continuous scale (n=82); 54.9% chose none/minimal, 29.3% chose mild, and 15.9% chose moderate–severe. However, 17% of those using the continuous measures chose none compared with approximately 6% of the entire group, suggesting this subgroup of respondents was more conservative (would be less likely to recommend return to activity) than the respondents as a whole.

When a continuous measure was the preferred method (handheld dynamometer and dynamometer for strength, visual inspection/approximation, goniometer and inclinometer for ROM, and one-leg eyes closed, one-leg eyes open and balance device for balance), respondents generally used the contralateral side for comparison (80.0% for strength (n=20), 72.9% for ROM (n=369), 82.9% for balance (n=199)), and normative standards or ipsilateral antagonist muscle were mentioned less frequently.

For our analysis examining the importance of context (clinical vignettes), we included only the 311/464 (166 for ATs, 83 for MDs and 62 for PTs) subjects who answered all five vignettes. Table 4 shows the frequency of respondents who considered a particular sign as either first or second ranking, stratified by vignette and by profession. Overall, pain was the most frequent choice by MDs in each vignette. Balance was infrequently ranked among the most important signs by each of the professions, with the exception of PTs in vignette 1 (33.9%). The pattern of responses were dissimilar for vignettes 1, 2, 3 and 5 (p ranged from <0.001 to 0.0034) but not for vignette 4 (p=0.17). When groups were compared separately, ATs and PTs were more similar (p values 0.044, 0.22, 0.24 and 0.29) than MDs versus PTs, or MDs versus ATs (p ranges from <0.001 to 0.009).

Table 4

Proportion of respondents* who ranked each clinical sign fi rst or second for each vignette (n=311)

Figure 1 presents the percentage of respondents in each category that ranked a particular sign on the x-axis as 1 or 2 in each of the five vignettes. Pain appeared to be the most consistently important sign for MDs (41.0%) and ATs (18.1%). PTs most consistently used sport-specific skills (16.1%). The overall pattern of responses was significantly different between the three professions (overall p=0.007). For individual comparisons, ATs and PTs were similar (p=0.32), but MDs were different from the two other groups (p<0.01 for both comparisons).

Figure 1

Proportion of subjects who ranked a particular clinical sign 1 or 2 across all five vignettes (denominator is only respondents who answered all five vignettes, athletic therapists (ATs)=166, medicine doctors (MDs)=83, physiotherapists (PTs)=62). Because each subject had two answers (ie, 1 and 2), the total possible proportion if one sums all categories together is 200%. ROM, range of motion.


Overall, MDs, PTs and ATs preferred a measure for each clinical sign that does not require specialised devices: standardised testing for sport-specific skills, impact on function for pain, palpation for swelling, MMT for strength, visual inspection for ROM and standing on one leg with eyes closed for balance. Similarly, the patterns of responses for cut-off values for the tests were qualitatively similar; the ‘normal’ category was the majority's choice for every clinical sign except pain and swelling (both mild restriction).

Pain was considered the most important clinical sign to determine return to activity across all vignettes, and ‘impact on function’ was the most frequently chosen measure of pain. Although we do not know the psychometric properties of ‘impact on function’ by itself, it is included as part of standardised pain questionnaires such as the Brief Pain Inventory25 and the Glasgow Pain Questionnaire26 (eg, how pain interferes with walking or housework). By comparison, categorical and visual/verbal analogue scales are both considered appropriate for osteoarthritis pain27 but were less frequently chosen in our study. Paediatric and non-paediatric sport medicine guidelines recommend a ‘minimal’ level of pain in order to return to activity,28,,30 but ‘minimal’ is not defined. In our exploratory analysis categorising pain as minimal, we still found 45% of respondents willing to return a child to activity if there was more than minimal pain. Older athletes in elite sports often return to activity if in pain,31 and more research is required to determine appropriate levels for children.

The preferred measure for sport-specific skills (second most common choice) across all professions was standardised testing (eg, sport-specific drills, shuttle run), which is the most objective measure for this sign. Although standardised tests usually require considerable time to administer, they are often used in preseason evaluations, and this familiarity may be why they were ranked high. Most respondents required the child to have no restrictions before returning to activity, but a significant proportion chose mild restriction. Published guidelines appear vague, and the only references we could find suggested a level of ‘skills regained’29 or ‘ability to perform the specific motions and actions required for your sport effectively’ before returning to activity.30

For ROM, most of our respondents chose visual inspection, which is consistent with preferences already in the published literature.32 Although more objective measures may be more accurate, clinicians may not consider the smaller differences clinically relevant. The majority of our respondents required full ROM to return to activity, which corresponds to the literature.28,,30 However, ROM was generally chosen as the first or second important sign by less than 50% of respondents in each clinical vignette, and therefore many clinicians would likely allow a return to activity, even if there were some restrictions.

For strength, all three professions preferred MMT. Although MMT requires training to reach an acceptable level of reliability,33 it can have diagnostic potential and acceptable inter-rater reliability when assessing symptomatic or asymptomatic upper limb muscles.34 With respect to return to play, most sources recommend a minimum strength of 90% compared with the uninjured side.28,,30 In our sample, 53.2–57.4% of respondents believed that 90% was not enough and favoured a completely normal strength before returning a child to activity.

We could not find any literature comparing the reliability or preference for different measures of swelling in a clinical situation. The preferred method of measure for the knee (the joint used in our survey) was palpation, which is simple and can differentiate effusion from oedema. Whether palpation would be the preferred measure for other joints (eg, ankle) or for other areas (eg, calf) needs to be further explored. For return to activity, the majority of ATs and PTs answered ‘mild restriction,’ which is the level generally accepted.29 30 However, most MDs were more cautious and suggested returning a child to activity only when swelling was absent. An aspect of swelling that was not assessed in the survey is its duration. Some guidelines specify that if swelling is persistent and recurrent, the acceptable level for return to play should be even closer to normal.29

For balance, our respondents preferred a simple one-leg stand eyes-closed test.35 Although a timed dynamic one-leg eyes-closed test is more reliable in adolescents,36 it requires that a special foam pad be available in all examining rooms. The only published guideline with respect to return to play was ‘good’ balance and proprioception, and this was not child-specific.29 Although approximately half of each profession who consider balance important required normal balance before returning the child to activity, it was ranked with the lowest priority of all signs, and most children are likely allowed to return despite deficits in this area.

Determining what is ‘normal’ is difficult, especially in the absence of preseason testing and in growing adolescents: children mature at different speeds for their physical, visual, physiological and psychological developments.37 Our respondents overwhelmingly considered the opposite limb as the standard for normal (range 72.9–82.9%), followed by norms (range 10–27%), which is consistent with the literature in cases of single-limb injury.28 However, clinicians should be aware that injury may decrease the ability to recruit or fully activate the contralateral muscle,38 and so ‘normal’ may not be identical to ‘preinjury’ status.

The pattern of responses for the most important clinical signs in the vignettes (table 4) suggested that MDs were generally different from PTs and ATs (four out of five vignettes) and that ATs and PTs were always similar. The similarity between PTs and ATs is not surprising given their comparable training for musculoskeletal injuries. The marked difference for MDs may reflect a different philosophy, training or focus on health issues and needs to be explored in future studies if interdisciplinary recommendations/guidelines are ever to be developed.

Limitations of study

We surveyed all certified Canadian sport medicine professionals and obtained a response rate of 33.6%; recent surveys into current practice have had response rates ranging from 25% to 40% for several conditions (transfusion, pain management, health practice) to 50–75% for more life-threatening outcomes or those conducted by the Food and Drug Administration.39,,42 Some of our respondents had children representing less than 10% of their client-visits, and it is possible that perceptions of individuals who mostly treat children are different from those that treat children less frequently. Therefore, our results should be interpreted as a general overview of current practice management among all types of sport medicine health professionals.

What is known on this topic

  • Although there are limited guidelines on return to play criteria in adults, there are no guidelines for return to play in children.

  • In addition, there are no studies describing current practice management of child musculoskeletal injuries, which is a necessary first step towards developing consensus and furthering research in this area.

What this study adds

  • The results of this study describe areas of consensus and disagreement among different sport medicine specialists (medicine doctors, physiotherapists (PTs) and athletic therapists (ATs)) with respect to return to play criteria in children.

  • There was consensus across professions for the preferred measure of each of the following clinical signs: sport-specific skills, pain, swelling, strength, range of motion, balance.

  • Although the importance of each clinical sign was dependent on the clinical context, more physicians (compared with ATs and PTs) considered pain as one of the two most important signs, and balance was infrequently mentioned by any profession.

To minimise respondents' time and maximise participation, we asked respondents to consider a knee injury when describing which tests they would generally use to assess return to activity. We recognise that other tests may be used for other anatomical locations, as suggested by the variability in our results for the ranking of specific tests across the five different clinical vignettes. Future research should explore whether answers do indeed differ by anatomical location and injury type.

Although we provided several choices for each clinical sign, some respondents routinely chose ‘other’ and listed a combination of signs. These were recategorised based on our assessment of objectivity and reliability, and others may have recategorised them differently. Similarly, we recategorised cut-off values to normal, mild restriction and moderate–severe restriction for each test so that we could compare across tests with different units; others may have used different categorisations. We also believe a patient's age and physical maturity may influence the cut-off value, and this should be explored in future studies. Finally, as in most studies, all respondents received the same version of the survey. It is possible that some individuals reflexively chose the first option for each question because of the order in which they were presented (occurred for 4/6 questions).


The results of this study suggest areas of consensus and disagreement among different sport medicine specialists (MDs, PTs and ATs) with respect to criteria used to decide on return to sport following musculoskeletal injuries in children. For example, it appears that the best test has to be feasible within the confines of a clinical practice and adapted to children. Lastly, these results, which should not be interpreted as the current standard of care, provide the foundation for future work leading towards the development of interdisciplinary consensus guidelines.


Thank you to J Reichman, for creating the online survey, and to I Sebestyen, for managing the data collection. The authors are grateful for the collaboration of the focus-group participants who contributed to the development of the questionnaire and vignettes.

Appendix 1 Questions regarding measures for specific clinical signs

Each of the following questions asks you how you measure a particular clinical sign (pain, swelling, strength, range of motion, proprioception, sport-specific skills) in the clinic. Please choose the measure you most commonly use in your own clinical practice, to evaluate the different clinical signs. You will then be asked to indicate the criteria you use to determine when you would recommend that a child return to activity without restriction. For the purposes of this survey, we are interested in the general principles you use to determine when you would recommend return to activity. Because your practice may vary for different injuries, for this section of the survey, respond for how you would approach a knee injury or an injury to a muscle around the knee.

Which of the following measures of sport-specific skills would you use as a criterion to recommend return to activity?

  • subjective report of game performance (eg, how is your game);

  • observed field test;

  • standardised testing (eg, shuttle run, sport-specific drills);

  • I do not use sport-specific skills as a criterion for return to activity;

  • other (please specify);

Which of the following measures of pain would you use as a criteria to recommend return to activity?

  • impact of pain on function (eg, impairs daily activities, impairs sport performance, does not affect function);

  • categorical scale (eg, none, mild, moderate, severe);

  • visual/verbal analogue scale (eg, 0–10, 10 being most severe);

  • standardised pain questionnaire;

  • I do not use pain as a criterion for return to activity;

  • other (please specify).

Which of the following measures of swelling would you use as a criterion to recommend return to activity?

  • palpation (eg, none, mild, moderate, severe);

  • visual inspection/approximation (eg, none, mild, moderate, severe);

  • girth measurement-tape measure (eg, relative to contralateral side);

  • I do not use swelling as a criterion for return to activity;

  • other (please specify).

Which of the following measures of strength would you use as a criterion to recommend return to activity?

  • manual muscle testing (eg, 5/5 normal, 3/5 lift against gravity);

  • handheld dynamometer/portable myometer (eg, microfet, Nicholas MMT);

  • dynamometer (eg, Cybex/Kincom/Biodex);

  • endurance (eg, number of repetitions);

  • I do not use strength as a criterion for return to activity;

  • other (please specify).

Which of the following measures of range of motion would you use as a criterion to recommend return to activity?

  • visual inspection/approximation (eg, compared with contralateral/normal);

  • goniometer;

  • inclinometer (ie, gravity-dependent goniometer);

  • I do not use range of motion as a criterion for return to activity;

  • other (please specify).

Which of the following measures of proprioception/balance would you use as a criterion to recommend return to activity?

  • one-leg eyes open;

  • one-leg eyes closed;

  • walking on balance beam;

  • foam cushion;

  • subjective complaint (eg, I feel unsteady);

  • I do not use proprioception/balance as a criterion for return to activity;

  • other (please specify).

Appendix 2 Clinical vignettes

Vignette 1 Sixteen-year-old male house league hockey player (forward) was wrestling with a friend and had an isolated second-degree medial collateral ligament sprain. There is no evidence of fracture on x-ray, and the physes are closed. Two weeks later, he returns to see you. There is no effusion, laxity remains, and he reports having only mild pain after jogging 10 min.

Vignette 2 Nine-year-old female recreational gymnast (mostly floor exercises) with 3-month history of gradual onset lumbar back pain. Bone scan is normal. She is pain-free at rest and during most gymnastic activities but is very sore after practising back-walk-overs (lumbar hyperextension). Neurological signs are all normal.

Vignette 3 During the semifinals of a provincial tennis tournament, a 13-year-old female felt a pop in her left midcalf. Your on-court examination strongly suggests an isolated gastrocnemius strain and an intact Achilles tendon. She walks without a limp, it is painful to run, but she feels she can finish the tournament.

Vignette 4 Twelve-year-old male with 2-month history of Osgood–Schlatter Disease (mild separation of the tibial apophysis on x-ray with open physes). He has a mild limp if he is very active on any given day, but this occurs only once or twice a week. Symptoms increased last week during ‘spring break’ when he increased his snowboarding. His mother has kept him out of activities.

Vignette 5 At a regional competition, a 9-year-old male competitive diver with hopes and talent to eventually make the Olympic team complains of hurting his wrist during a 3 m dive 15 min ago. There is mild pain at the end of extension and flexion. There is no concern of a scaphoid or other fracture. He has pain when he dives if he ‘misses’ the dive. He wants to continue the competition.


View Abstract


  • Funding This study was funded by the Réseau provincial de recherche en adaptation-réadaptation. IS is funded by a Senior Clinical Investigator Award from the Fonds de la Recherche en Santé du Québec (FRSQ). DEF is currently funded by FRSQ Junior II programme.

  • Competing interests None.

  • Ethics approval Ethics approval was provided by the Centre for Interdisciplinary Research in Rehabilitation of Greater Montreal (CRIR) Ethics Committee.

  • Provenance and peer review Not commissioned; externally peer reviewed.

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.