Background Benchmarking is an established means of identifying levels of specialist practice and competence-based interviews are a tool used to facilitate this. The London 2012 Olympic and Paralympic Games (LOCOG) provided an opportunity to introduce a pragmatic approach to assess expert behaviour in large numbers of volunteer physiotherapists.

Aim To test inter-tester repeatability of an assessment matrix used to score clinical expertise in a sporting context, followed by reporting on the findings for physiotherapy selection.

Methods Four volunteers were videoed answering an identical sport-specific clinical scenario. Recordings were initially scored by two experienced assessors (gold standard). Subsequently, nine assessors scored the same videos then compared with the gold standard. 602 physiotherapists were assessed during volunteer interviews for LOCOG. Scores were compared to those based on self-reported clinical experience. Cross-tabulation was used to determine levels of agreement for the initial scoring matrix and the relationship between the two scoring systems was analysed.

Results Levels of agreement ranged from 22% to 88%. A good correlation (r=0.754 p>0.001) was found between self-reported clinical experience and scenario scores. Marginal data points in the correlational analysis indicated that only 52% of participants attained the same score across assessments.

Conclusions A tool to facilitate assessment for large scale selection could be used for Host Nation physiotherapist selection for Major Games. There is a need to train assessors in this environment, as well as provide detailed matrices, specific to each context being assessed, to ensure that observations can be made on domain-specific and general, non-technical aspects.