Objective: Reliable data allows for the generalisation of study findings to the wider population. The aim of this study was to assess the reliability of using community team based primary data collectors for the collection of exposure and protective equipment use data.
Methods: Nine clubs (23 teams) from a metropolitan Australian Football league in Victoria each provided one primary data collector to monitor exposure and protective equipment use over a regular playing season. Four random audits of this data collection for each team were conducted throughout the regular playing season. The audits were compared with data collected by the club data collectors and the level of agreement assessed.
Results: Although exposure data agreement was higher during competition and protective equipment use agreement higher during training, there was no significant difference in data collected by the primary data collectors and the random audits.
Conclusions: The use of trained data collectors associated with Australian Football teams provides reliable information about player exposure and protective equipment use in community intervention studies.
Statistics from Altmetric.com
Epidemiological monitoring studies, and formal evaluations of injury prevention measures, in community sports rely on the collection of reliable data.1,2 Unlike studies in more professional or elite athletes, community based sporting teams rarely have medically trained personnel associated with teams or in attendance at games or training sessions. This means that relevant data need to be collected by either paid research staff or by volunteers associated with the teams themselves. For logistic and funding reasons, the use of team based primary data collectors (PDCs) is often the most feasible option. However, the reliability of data collection by PDCs has not previously been reported. The aim of this paper is to describe the reliability of using PDCs to collect exposure and protective equipment use data during a community based sports injury randomised controlled trial.
The Australian Football Injury Prevention Project (AFIPP) was a randomised controlled trial to assess the effectiveness of headgear and mouthguards in community level Australian Rules Football. Nine metropolitan football clubs in Victoria volunteered 23 senior/junior teams for the project. Over the duration of one playing season, every player involved in AFIPP was monitored during all games and training sessions for their exposure (that is, amount of participation time) and their protective equipment behaviours.
Each club nominated a PDC to be responsible for the collection of exposure and protective equipment use data. The PDCs were formally trained and received a procedures manual and data collection forms to ensure standardised data collection processes. Exposure data forms required PDCs to place a tick next to each player’s name during training and games, if they were present, and a cross if they were absent. Similarly, compliance data forms were used to indicate whether players wore headgear and/or mouthguards. Overall, there were nine trained data collectors (one from each club) who were responsible for collecting exposure and protective equipment use data over the regular playing season. All PDCs were parents of players, team managers, or club presidents. All data collectors were instructed to be present for the entirety of training and games.
A random audit of the collection of the exposure and protective equipment use data by the PDCs was undertaken by the researchers. Each team was audited twice at training and twice during competition (that is, 23 teams were each audited four times for a total of 92 audits). Exposure and protective equipment use information was collected by the auditors using the same protocol and data collection forms as those used by the PDCs. All auditors and PDCs worked from the sideline of the playing field. The audits were conducted to assess the reliability of primary data collection in the field and did not consider the validity of the information collected.
The average percent agreement between the audits and the PDCs for both exposure and protective equipment use during both training and games was calculated. The two exposure measurements were compared by the kappa statistic and McNemar tests were used to compare protective equipment use (%) data. Kappa values were interpreted using the strength of agreement scale as reported by Feinstein.3
Ethics approval was obtained from the Deakin University and Monash University Human Research Ethics Committees.
The overall average percent agreement, across all teams, for the exposure and protective equipment use data is shown in Table 1.
Generally, the percent agreement in exposure data was highest for games with the overall strength of agreement falling in the “substantial” category. There was also a high level of agreement in the protective equipment use data. The McNemar tests indicated no significant differences between the PDCs and the random audits for either mouthguard or headgear use.
Reliability of measurement is important for the interpretation and generalisation of research findings.4 This is the first community based Australian Football injury study to assess the reliability of its field based data collection procedures for the collection of exposure and protective equipment use data. Overall, there was a high level of agreement between the team based PDCs and the independent audits. This suggests that the use of PDCs provides a reliable method for collecting this data.
The level of agreement in the exposure data was higher during competition than training. Competitive games occur in a structured session and this may make it easier to define players as either in attendance or not in attendance. In contrast, during training sessions, there are often more players on the ground doing many different tasks at any one time. The sessions are also less structured and not all players may be present for the whole time.
Although average agreement for exposure was high during a match (90.9%), one reason for it not being 100% was that players, although allocated playing numbers, may not necessarily remain in their numbered jumper for the entirety of the match or season (for reasons such as injury or lack of player jumpers). Identification, therefore, may be difficult as jumper number is the most convenient way, from the sideline, to identify players.
The reliability of the protective equipment use data was (slightly) higher during training than games. Previous studies have found that community level Australian Footballers are less likely to wear mouthguards/headgear during training.5 From a data collector’s perspective, it may be easier to record compliance data during training, as studies show that fewer players are likely to use protective equipment during this time.5 Unlike headgear, mouthguards can be difficult to see from a distance and, during competition, accurate monitoring of mouthguard use can only take place during the quarter time breaks. Even then, visibility of mouthguards can be hard to monitor as players often take their mouthguards out of their mouth and place them down their socks or in their shorts during the breaks. This could be one of the reasons why data agreement was higher during training for this piece of equipment.
Although there was some variability in the measurements recorded by the PDCs and the independent auditors, there is substantial agreement in exposure and protective equipment use data from both sources. This means that trained PDCs based in teams from community sports clubs can collect reliable exposure and protective equipment use data. They are therefore a reliable alternative source of data collection in community based studies of sports safety behaviours.
Take home message
The use of trained primary data collectors is a reliable way to collect data in community sports injury projects.
Reliable data is important for a study to be generalised to the broad population. In large scale community research projects, often it is difficult, from a logistic and funding perspective, to have the researcher collect the data. Information about the reliability of team based data collectors for the collection of data in community based research is limited.
This study demonstrated that with standardised training, team based primary data collectors are able to collect reliable data in large scale community interventions.
The Eastern Football League and the clubs, teams, primary data collectors, and players are thanked for their contribution to this project.
AFIPP was funded by the Victorian Health Promotion Foundation. Rebecca Braham was supported by a National Health and Medical Research Council research scholarship and a small grant provided by the Victorian Health Promotion Foundation.
Conflict of interest: none declared.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.