Article Text
Abstract
Objectives: To investigate issues of curriculum in the context of a postgraduate sports medicine training programme, specifically in the field of clinical biochemistry and haematology.
Methods: Following the Delphi methodology, a series of sequential questionnaires was administered to curriculum developers, clinical teachers, examiners, and registrars.
Results: Agreement on a core syllabus for this subject with an indication of the core aims and objectives of teaching and learning in this field and the associated required skills and competencies. An indication of current and ideal teaching and learning methods and reasons for these preferences. A consensus of key features of a teaching module for this field and of appropriate methods of examination.
Conclusions: The data derived from this study, as well as the experience of engaging in it, will better inform curriculum developers, teachers, and students of one another’s perceptions as to what is important in and appropriate to teaching and learning in this field of sports medicine. Engagement in the exercise and broader consideration of the outcomes by those who develop the curriculum, teach, and formulate the examination process will facilitate attainment of the ideal of well aligned teaching, learning, and examination in this specific field.
- Delphi
- curriculum
- alignment
- biochemistry
- haematology
Statistics from Altmetric.com
Sports medicine is a relatively new specialty in which a comprehensive knowledge of internal medicine is required. Curricula for the training of sports medicine practitioners have been developed in a number of nations,1–4 but little has been published about the methods by which these have been developed. Many of these curricula are only broadly defined, and discussion relating to the field of clinical haematology and biochemistry appears to be scant indeed.
The British Joint Committee for Higher Medical Training employs an advisory committee of haematology experts to oversee training in haematology and presumably monitor its curriculum.5 Their curriculum is well defined in terms of knowledge, skills, attitudes, learning methods, and assessment, but the method of determination of the curriculum is unclear. Similarly the US Accreditation Council for Graduate Medical Education publishes programme requirements for advanced training in both haematology and sports medicine, which include general aspects of the educational programmes, specific programme content (which includes clinical experiences and technical and other skills), and formal instruction but, again, the method of determination of the curriculum is not stated.6
Three studies are available in the literature in which the process for determining aspects of the sports medicine curriculum is described.7–9 In the first, a modified Delphi approach was used to determine which aspects of sports medicine were of importance in emergency medicine.7 The second study, using a two round Delphi technique, sought to define the roles and responsibilities of the sports medicine specialist in the United Kingdom.8 The third study, using a single questionnaire, sought to develop learning outcomes for an ideal MSc course in sports and exercise medicine.9 In all of these studies, it could be argued that the approach to the design of the curriculum was notable for the absence of any overarching guiding principle or framework.
A key curriculum design principle that has recently achieved prominence is constructive alignment whereby learning activities (what the student does), learning objectives (intended learning outcomes), and assessment (what is graded) act to reinforce one another.10 In constructive alignment, teaching is considered to be the support of learning; the focus shifts to what students do rather than what teachers do. In such a system, students become more responsible for constructing their own meaning based on what they do—that is, they engage in their own learning.
The aim of this study was to investigate issues of curriculum to form a basis for improved curriculum alignment in the context of the postgraduate sports medicine training programme conducted by the Australian College of Sports Physicians. In particular, given that different staff act as curriculum designers, teachers, and assessors, a Delphi methodology was chosen for appraisal of its potential simultaneously to explore similarities and differences held between these groups of staff, and to enhance alignment and coherence across the groups.
METHOD
The Delphi technique is a research method that can be used not only to investigate the range of opinions and other attributes held by participants, but also to gain consensus. It often involves administering a series of questionnaires in a series of iterations. Each version of the questionnaire is refined and updated by taking account of the responses from the previous round.11,12
In the medical sphere, this methodology has been used to identify a medical curriculum,13 evaluate the self identified levels of clinical skills in newly graduated doctors,14 define the sports medicine specialist,15 identify appropriate tasks for interns,16 identify educational requirements for general practitioners,17 develop clinical guidelines for management,18 design emergency medical service systems,19 and determine topics important to readers of a general medical journal.20
In this study, after seeking permission from the President of the Australian College of Sports Physicians, three groups of fellows (two curriculum designers, six teaching fellows, and six examiners) and six registrars from that College consented to participate in a three round Delphi investigation. Each of these participants was sent a letter informing them of the nature of the study and a series of three questionnaires by mail. The researchers were blind to the identities of individual respondents, and were only able to identify responses by group.
The initial questionnaire firstly asked participants to rank, in order of importance, the 10 most important topics from a list of 24 which had been compiled by an expert in the area (KF). They were also invited to list other topics thought to be important but not included in the list. Scores (10 for the response ranked highest to 1 for the response ranked lowest) were awarded for each response, and totals were used to define the 10 most important topics. The second question asked participants to identify the main aims and objectives of learning in this area, and in the third question participants were asked to indicate up to five skills or competencies that they believed to be the most relevant in this area of study.
In the second iteration, the participants were informed of the top 10 topics collectively thought to be most important. They were invited to indicate any topic not in the top 10 that they felt was vital for inclusion. The participants were then informed of the top seven aims and objectives collectively thought to be most important. They were invited to indicate any topic not in the top seven that they felt was vital for inclusion. The participants were then informed of the top eight skills and competencies collectively thought to be most important. One of these, “management of iron related problems”, was selected, and the participants were asked to indicate five or more specific actions that were felt to be important in fulfilling this competency.
Participants were then given a list of teaching methods and asked, firstly, to rank their top six preferred current methods of teaching (learning in the case of registrars) in clinical haematology and biochemistry in sport and, secondly, what they believed were the top six optimal methods of teaching or learning in this field. Participants were then asked to outline three reasons for their first choice of current practice and their first choice of optimal practice.
Participants were then presented with a list of 11 content items or associated features of a potential teaching/learning module and asked to rank, in order, the six items felt to be most important for inclusion in such a module. Scores were awarded, and the combined totals were used to define the six components of the teaching module collectively felt to be most important.
The third iteration started with feedback about the five actions collectively felt to be most important in the competency “management of iron related problems”. Lists were then presented (in rank order) of the five most often used current teaching methods and (in random order) of the methods felt to be optimal. Participants were asked to rank the optimal methods in order of preference. As tutorials prepared by registrars was the most popular teaching/learning strategy, participants were asked “what do you believe to be the advantages of this method?”.
Problem based learning (PBL) appeared in the top five optimal methods but was placed 6th in the “currently used” list. In the third iteration, this technique was further explored by asking “how would you describe PBL?” and then “if you consider PBL has specific advantages in learning and teaching, please specify them.”
Feedback was then presented on the list of 11 content items or associated features of a potential teaching/learning module suggested in the second iteration. The most popular five items were presented in random order, and participants were asked to rank them 1 to 5 in order of preference.
As the final two questions related to two aspects of curriculum presented for the first time in the third iteration, the method used for them was a survey.
The next area for exploration concerned how teachers and registrars determine topics that should be taught or learnt. The participants were asked “if you are or were to be teaching or learning in the area of clinical haematology and biochemistry in sport, how do or would you determine which topics to teach or learn?”.
In the final question in the third iteration, the issue of assessment was addressed. Participants were presented with a list of 10 potential methods for examination of registrars in this area and were also invited to specify other methods if they felt so inclined. Scores were awarded, and totals were used to define the five components of the assessment felt to be most appropriate.
Table 1 summarises the main topics covered in the respective questionnaires and the methods used to assess them.
RESULTS
The response rate was 90% (18/20) overall for each of the three questionnaires. Those who responded to the first responded to all three. The response rate for individual groups was curriculum developers 100% (2/2), registrars 100% (6/6), examiners 83% (5/6), and teachers 83% (5/6).
Table 2 shows the 10 topics considered most important by the overall group and the rankings of these topics by individual groups and the overall teaching and examining staff.
Table 3 shows the most often named aims and objectives suggested, and table 4 the most often named skills and competencies.
Table 5 gives the most common responses to the question related to specific actions in management of iron related problems. The first three responses were well represented in each group but the fourth was poorly represented in the registrar group with only one registrar suggesting that taking an appropriate history was important.
Table 6 shows the results for current and optimal teaching/learning methods. When, in the third iteration, respondents were asked to rank the top five teaching or learning methods, both the overall group and the non-registrar group indicated the following were given, in order of preference:
-
Tutorials prepared by registrars
-
Problem based learning
-
Tutorials prepared by teachers
-
In clinic teaching
-
Book learning
Tutorials prepared by registrars was the most often mentioned current teaching or learning practice. The most commonly expressed reason for this choice was that “student based learning is more effective”. Other reasons included:
-
Group discussion is encouraged
-
Encourages self directed learning by students
-
Group discussion of management of individual cases is facilitated
-
Habit
-
Easy to provide feedback
-
Time constraints/time available
-
A good environment for teaching.
Opinion on the advantages of this method was specifically asked for in the third iteration. Two responses were clearly the most common:
-
Preparation is the best way of learning (nine responses)
-
Facilitates group discussion (seven responses).
Book learning was highly ranked by the registrar group, with 50% indicating that it was the best method. Their reasons included:
-
Time constraints
-
Lack of availability of tutors
-
Easily available information
-
Lack of relevant patients
-
Self directed topics
-
The only option
PBL scored highly as an optimal method. The most often expressed reasons for this choice were that it is easier to remember from examples and it was more practical. Other reasons included:
-
The literature is better understood if read after problem solving
-
It is easier to cover important topics
-
Convenient
-
It is the best way to learn
-
A better way of determining students’ understanding
Two questions were asked about PBL in the third iteration. The first asked for a description of problem based learning. By far the most common response was:
-
Focuses on a case rather than a disease (11 responses)
The second question asked about the specific advantages of this method. Eighteen different responses were offered. The most common responses were:
-
Immediate relevance of the specific problem (seven responses)
-
Tests practical application of knowledge (four responses)
-
Integrates basic sciences and clinical aspects, improves retention, uses real-life patients (three responses each)
Two respondents suggested that PBL had no advantages.
Table 7 gives the responses to the final question in the second iteration which referred to the content and associated features of a teaching module in this field of study.
In the third iteration the participants were asked to rank the top five features of a teaching module as indicated by their responses to the second iteration. Table 8 shows the results.
When it was asked in the third iteration if any other items should be have been included in the top five, three of 18 (17% of respondents) indicated in the affirmative. The topics suggested for inclusion (both suggested twice) were sample multiple choice questions and sample short answer questions.
When the participants were asked about how they determined what to teach or learn in this area, three answers were clearly most common:
-
Personal experience (13 responses)
-
Consult the written syllabus (12 responses)
-
Concentrate on current topics in the literature (nine responses).
Four respondents each indicated that they would consult with colleagues or work from previous exam questions.
Finally, when the participants were asked to suggest, in rank order, the examination method most suited to this field of medicine, the responses were as shown in table 9.
DISCUSSION
In this study, the Delphi process has been used for two distinct purposes. On one hand, it served as a practical survey device to canvass similarities and differences on aspects of curriculum held by key players in the teaching and learning process. On the other hand, by iteratively working toward a consensus, it offered a relatively benign mechanism to initiate a process of reconciliation of those differences that were unearthed. The scope of the study was both an advantage and a disadvantage. In attempting to cover all aspects of the curriculum in one investigation and being mindful of the issue of participant fatigue, only three questionnaires were used. This allowed only two Delphi iterations to be achieved for most of the areas covered and required the use of a survey methodology for the final two curriculum components.
In its capacity as a survey approach, the Delphi process has delivered a series of findings that are of benefit to those involved in curriculum development in this specific area of sports medicine. Firstly, it has delineated the 10 agreed most important topics in the area of clinical biochemistry and haematology in sports medicine in a group of fellows and registrars of the Australian College of Sports Physicians (table 1). This set of 10 topics strongly circumscribes the written (intended) curriculum.
On the pivotal issue of determining what to teach or learn, respondents suggested that personal experience was important but that many would consult the written syllabus for guidance. This indicates that the written syllabus should explicitly point out the important topics that need to be covered. This may seem obvious, but currently available syllabuses may not be as explicit as those seeking guidance appear to require. This study identifies an appropriate topic list for the field of clinical haematology and biochemistry in sports medicine.
Secondly, anaemia/iron deficiency, investigations in the tired athlete, and the effects of training on haematological and biochemical variables featured prominently in answers related to aims and objectives and skills and competences. Although some overlap in responses was experienced, this enabled useful triangulation, adding to the overall validity.21 That there was overlap of this sort does, however, suggest that some respondents had difficulty in discerning the difference between aims, objectives, skills, and competencies and that this is an issue that requires further clarification in future studies.
Thirdly, preferred methods of teaching and learning, both those currently used and those considered to be optimal, were remarkably similar across the different educational staff groups. The favoured responses are, encouragingly, in accord with prevailing thinking in support of student centred learning.22 Book learning defined as “students learning from text books and journal articles sourced by themselves” was highly ranked by 50% of the registrar group, with some of their reasons for this preference suggesting that lack of time, patients, and tutors forced them into this perhaps less than ideal method of learning. Examiners and teaching fellows scored PBL highly as a teaching method and showed a good awareness of this popular, but controversial,23 method of learning.
The responses to the questions related to features of a teaching module tended to confirm the tutorial or PBL formats as popular, with discussion groups and case studies ranking second and third. Interestingly the more didactic, less student centred, “spoon feeding” approach was supported by the overall group and registrars’ first place ranking for topic summaries, presumably produced by the course coordinator and involving little student input or effort.
For many teachers, the concept of a curriculum appears to be little more than the syllabus of topics that need to be covered (and subsequently mastered by students), although the extent to which this was prevalent within or across our participant groups is debatable. More formally, the concept has been considered as encompassing all of the learning experiences provided or enabled by an institution or course of study. This has been termed the “curriculum-in-theory” or the “intended curriculum”, and is to be distinguished from the “curriculum-in-action”.14 The intended curriculum comprises four key elements: content, teaching and learning strategies, assessment procedures, and an evaluation process.24 This study has explored aspects of the first three of these elements, concentrating particularly on content as well as “actual” and “optimal” teaching and learning strategies—that is, strategies belonging to the curriculum-in-action and the intended curriculum.
Despite the comparatively superficial consideration of approaches to and methods of assessment in this particular study, the pivotal nature of assessment in the curriculum mandates some comment. In aligned teaching, the assessment reinforces learning and is the senior partner in learning and teaching. Get it wrong and the rest collapses.10 The nature of the knowledge and skills to be assessed therefore needs to be delineated by the assessors in consultation with the curriculum developers and teachers.
Further exploration of assessment was, however, beyond the scope of this first exploratory examination of the utility of the Delphi methodology for appraising curriculum alignment in the context of multiple staff cohorts responsible for curriculum design, teaching, and assessment in clinical haematology and biochemistry.
What is already known on this topic
-
There are no current reports that address determination of multiple aspects of a curriculum for clinical haematology and biochemistry in sports medicine
What this study adds
-
Demonstration of a methodology that could be useful in determining a curriculum in all fields of sports medicine
-
A consensus derived from four groups of stakeholders (curriculum developers, teaching fellows, examiners, and registrars) on major aspects of a curriculum for clinical haematology and biochemistry in sports medicine
Our study, which focused on horizontal (between groups) alignment, has not only revealed those substantive aspects of the curriculum that were already largely in alignment, but forms a basis for greater alignment and coherence across those areas that initially featured a degree of discrepancy among staff groups responsible for the written syllabus, methods of teaching and learning, and assessment. Despite the magnitude of the task, if we are to achieve optimum outcomes in teaching and learning, both horizontal and vertical alignment should be pursued across a broader range of curricula. The next challenge, based on data such as ours, is to research and develop vertical alignment between the various components of the curriculum.
REFERENCES
Footnotes
-
Competing interests: none declared
-
Formal ethics committee approval was not sought for this study. The data arose from a minor term project which was part of one subject in a Graduate Certificate in Higher Education. At the onset of the project there was no indication nor expectation that the data may be significant enough to publish. Despite this the authors, before the start of the study, contacted the then President of the Australian College of Sports Physicians for his approval to conduct the study and to survey fellows and registrars of that College. Each of the participants was sent a letter which explained the nature of the study. Their consent to participate was indicated by their response or non-response. The researchers were blinded to the identities of the respondents and no individual can be identified from the data.