Article Text
Abstract
Objective To develop and evaluate a modified version of the Victorian Institute of Sport Assessment-Achilles (VISA-A) questionnaire, for use in sedentary patients with Achilles tendinopathy, using the Consensus-based Standards for the selection of health Measurement Instruments recommendations.
Methods Twenty-two sedentary patients with Achilles tendinopathy completed the VISA-A and provided feedback regarding the relevance, comprehensiveness and comprehensibility of each item, response options and instructions. Patient and professional feedback was used to develop the VISA-A (sedentary) questionnaire. Reliability, validity and responsiveness of the VISA-A (sedentary) was evaluated in 51 sedentary patients with Achilles tendinopathy: 47.1% women, mean age 64.8 (SD 11.24).
Results Factor analysis identified two dimensions (symptoms and activity) for the VISA-A (sedentary). Test–retest reliability was excellent for symptoms (intraclass correlation coefficient, ICC=0.991) and activity (ICC=0.999). Repeatability was 1.647 for symptoms and 0.549 for activity. There was a significant difference between the VISA-A and VISA-A (sedentary) scores both pretreatment and post-treatment. There was stronger correlation between the pretreatment to post-treatment change in the VISA-A (sedentary) scores (r=0.420 for symptoms, r=0.407 for activity) and the global rating of change than the VISA-A scores (r=0.253 for symptoms, r=0.186 for activity).
Conclusion The VISA-A (sedentary) demonstrates adequate reliability, validity and responsiveness in sedentary patients with Achilles tendinopathy. The VISA-A (sedentary) is a more appropriate measure than the VISA-A for this cohort and is recommended for clinical and research purposes.
- Achilles
- Tendinopathy
Data availability statement
Data are available on reasonable request.
Statistics from Altmetric.com
WHAT IS ALREADY KNOWN ON THIS TOPIC
The Victorian Institute of Sport Assessment-Achilles (VISA-A) is the most widely used patient-reported outcome measure (PROM) for Achilles tendinopathy (AT) but the psychometric properties of the questionnaire in sedentary individuals are unknown.
WHAT THIS STUDY ADDS
The VISA-A (sedentary) demonstrated adequate reliability, validity and responsiveness in sedentary patients with AT, whereas the VISA-A lacked structural validity, was less responsive and demonstrated a floor effect in this cohort.
HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY
The VISA-A (sedentary) represents a more appropriate PROM for sedentary patients and will better enable clinicians and researchers to assess the impact of AT and efficacy of specific interventions.
Introduction
Achilles tendinopathy (AT) is the preferred term for persistent Achilles tendon pain and loss of function related to mechanical loading.1 AT is frequently seen in runners2 but only 35% of patients presenting to general practice describe symptoms related to sporting activity.3 Athletic and sedentary patients may, therefore, represent subgroups of AT, with differing aetiologies4 and varying degrees of impact on physical activity, function and quality of life.5 6
Patient-reported outcome measures (PROMs) quantify an individual’s perception of the impact of a pathology that cannot be captured with clinical tests or diagnostic imaging.7 The Victorian Institute of Sport Assessment-Achilles (VISA-A) questionnaire is one of the most widely used PROMs for patients with AT,8 covering the domains of symptoms, function and physical activity.9 However, the VISA-A was developed with an athletic population and the psychometric properties of the questionnaire in sedentary individuals are unknown.10 Athletic and sedentary patients are likely to value different outcomes and PROMs must, therefore, reflect the issues that are important to these specific populations.6 Without appropriate measurement properties, it is difficult to determine the impact of a pathology or efficacy of specific interventions using a PROM.5
The aim of this study was to develop and evaluate a modified version of the VISA-A that can be used in sedentary patients with AT. This questionnaire, the VISA-A (sedentary), was developed to measure the severity of AT using the Consensus-based Standards for the selection of health Measurement Instruments (COSMIN) recommendations.11 12 We hypothesised that there would be a significantly larger change in pretreatment to post-treatment scores for the VISA-A (sedentary), and a stronger correlation between the change in VISA-A (sedentary) and global rating of change (GROC) scores.
Methods
This was a prospective study involving patients referred to two National Health Service (NHS) Foundation Trusts.
Equity, diversity and inclusion statement
Patients were included if they were aged 18 years or older with a clinical diagnosis of AT but did not participate in Achilles tendon loading sports, inclusive of all genders, race/ethnicities, socioeconomic levels and occurrence in a marginalised community. Patients were excluded if they were unable to understand the English language or complete the questionnaires, thus findings may not be generalisable to these cohorts. Our author team consisted of two women and four men, with junior, mid-career and senior physiotherapy clinician/researchers, four from the UK and two from Australia.
Development of the VISA-A (sedentary)
Consecutive patients presenting to an outpatient’s physiotherapy department for non-surgical management of AT were used as the study population for the development of the VISA-A (sedentary) questionnaire. All patients were referred with clinically and MRI-confirmed AT by four lower-limb orthopaedic surgeons working in a secondary care musculoskeletal outpatient’s clinic. The clinical criteria used to diagnose AT were localised Achilles tendon pain during loading (eg, calf raise) and palpation. This patient sample was representative of the target population in which the PROM was to be administered and evaluated (table 1), with regard to the referral pathway to physiotherapy.
Patient and public involvement
Eligible patients completed a paper copy of the VISA-A at their initial physiotherapy assessment and were asked to write comments regarding the relevance, comprehensiveness and comprehensibility of each item, response options and instructions. Patients were then encouraged to offer alternative suggestions (item generation) based on their perception of their condition during a one-to-one, informal interview conducted by one of the authors (RN). Patient feedback was discussed with the referring orthopods and expert panel (RN, JLC, JEG and SO’N) to determine whether this was representative of other sedentary AT patients. The expert panel consisted of four physiotherapists, all of whom have a special interest in tendinopathy and multiple publications on the topic in peer-reviewed journals. These professionals were also asked to provide input regarding item relevance and comprehensiveness of the VISA-A for this patient cohort based on clinical experience and the existing literature. A provisional version of the VISA-A (sedentary) was created, evaluated and adapted until no further changes were recommended by patients or professionals (figure 1). In total, 30 individuals (22 patients and 8 professionals) were consulted during the development process; points raised are listed in online supplemental table 1.
Supplemental material
The VISA-A (sedentary) was developed using a reflective model where all items of the PROM are intended to be a manifestation of the same underlying construct. The construct to be measured by the VISA-A (sedentary) was the severity of AT in sedentary patients, with eight questions covering symptoms and their impact on activity (online supplemental appendix A). The structure and item weighting remained consistent with the VISA-A with lower scores indicating greater severity of AT.9 Based on feedback during the development phase, the scoring scale was reversed so ‘no pain’ was positioned to the left of the scale; the VISA-A scale was also reversed for the evaluation phase to ensure any differences in VISA scores could not be attributed to scale reversal (online supplemental appendix B). Scale reversal has been used in previous studies to avoid patient misinterpretation and support self-administration of the VISA-A.13 The intended application of the PROM is for clinical and research purposes, with the PROM administered on paper.
Supplemental material
Supplemental material
Evaluation of psychometric properties
For factor analysis to be considered ‘adequate’, the COSMIN checklist recommends a sample size of at least five times the number of items and >100 patients, or at least six times the number of items but <100 patients. The VISA questionnaires contain eight items, therefore, a minimum sample size of 48 is recommended but for other elements on the COSMIN checklist to be deemed ‘adequate’, a minimum sample size of 50 was required. To account for potential dropouts, a sample size of 55 was chosen to adequately evaluate the psychometric properties of the VISA-A (sedentary).
Patients were recruited, assessed and treated by three of the authors (JR, SO’N and RN) with each patient completing paper copies of the VISA-A and VISA-A (sedentary) questionnaires at their initial face-to-face appointment (table 1). The VISA-A (sedentary) was repeated 3 days later via telephone call, by the treating physiotherapist who was blinded to the initial results, before commencing treatment. This time frame was considered suitable as the interval between measures should be long enough to prevent recall, but short enough to ensure the patient’s presentation remains stable. The median duration of symptoms prior to treatment was 24 weeks, with a maximum of 520 weeks. Thirty-seven patients presented with first-episode AT, with the remaining 14 reporting recurrent symptoms. No modification to the treatment/intervention was made as part of this study.
The VISA-A and VISA-A (sedentary) were administered in paper format at discharge to determine the pretreatment to post-treatment change in scores. The GROC questionnaire (online supplemental appendix C), which is a valid, reliable and clinically relevant measure that is not condition specific,14 was also administered at discharge to determine the patient’s overall perception of change following treatment. Patients were instructed to complete the GROC with specific reference to their Achilles tendon to ensure responses were relevant.14 The median time to discharge was 12.0 weeks (IQR 6.0 (10–16)). Patients with incomplete data sets were removed from the study.
Supplemental material
Statistical analysis
Data analysis was performed by two of the authors (TM and RN). All statistical analyses were performed using SPSS V.25.0 (SPSS) and R (R V.3.2.0, The R Foundation for Statistical Computing). Continuous variables were assessed for their distributions using graphical analysis (construction of histograms and normal Q-Q plots) and through the Kolmogorov-Smirnov and Shapiro-Wilk tests. Statistical analysis and presentation are consistent with the CHecklist for statistical Assessment of Medical Papers.15
Internal consistency
Internal consistency of the VISA-A (sedentary) was determined by calculating Cronbach’s alpha (α), including the change to α when items were deleted. Internal consistency was considered acceptable for Cronbach’s α coefficients between 0.70 and 0.95.16 Kaiser-Meyer-Olkin (KMO) and Bartlett’s Test of Sphericity (BTS) were calculated and considered acceptable if KMO values were above 0.5 and BTS p<0.05. Exploratory factor analysis (EFA), using principal component analysis based on the correlation matrix with varimax rotation, was performed to assess the structural validity of the VISA-A (sedentary). Eigenvalues were used to identify underlying factors; if more than one factor with an eigenvalue >1 was identified, the questionnaire was reviewed and split into subscales that only loaded onto one factor with an eigenvalue >1.
Test-retest reliability
Three-day test–retest reliability was assessed through calculation of the intraclass correlation coefficient (ICC) with 95% CI, using a two-way mixed-effect model (with raters considered fixed and participants random) for absolute agreement based on single ratings (ICC 3.1),17 and Bland-Altman plots. Proportional bias was assessed through linear regression of the difference in scores on the mean of scores, with the null hypothesis that the slope of this line equals zero.
Measurement error
Measurement error was expressed as within-subject standard deviation (sw) and calculated as sw 2 = Σ d i 2 where n is the number of subjects and di is the difference between an individual’s pretreatment and 3-day retest VISA-A (sedentary) scores.18 Repeatability was calculated as sw x 1.96× √ 2 (18). The minimal clinically important difference (MCID) was determined using distribution (0.5 × SD of the mean difference in the pre–post treatment scores) and anchor-based analyses (difference between the change in scores of responders and non-responders on the GROC).19
Construct validity and responsiveness
The difference between VISA-A and VISA-A (sedentary) scores, both pretreatment and post-treatment, was evaluated using the Wilcoxon signed rank test. Agreement between the VISA-A and VISA-A (sedentary) scores, both pretreatment and post-treatment, was assessed with Bland-Altman analysis and through calculation of the ICC using a two-way random-effect model (with raters and participants considered random) for absolute agreement based on single ratings (ICC (2.1)).
The correlation between the pretreatment to post-treatment change in VISA-A and VISA-A (sedentary) scores and the GROC was calculated using Spearman’s rank correlation coefficient. Effect size and standardised response means were calculated based on responders (4 to 7 on GROC) and non-responders (−7 to 3 on GROC). Where relevant, a p<0.05 was considered significant for all analyses.
Floor and ceiling effects were determined by calculating the percentage of patients recording the highest or lowest possible scores at baseline or discharge. If more than 15% of patients achieved these scores, floor or ceiling effects were considered present.20
Results
Fifty-five patients were recruited to evaluate the psychometric properties of the VISA-A (sedentary). Four patients were excluded due to incomplete data, therefore, full data sets were available for 51 patients (figure 2), which meets the COSMIN requirements for adequate evaluation.11
The mean pretreatment to post-treatment change in scores were 14.24 for the VISA-A and 31.04 for the VISA-A (sedentary) (table 2). The mean GROC at discharge was 4.75 (SD 2.11).
Internal consistency
The overall Cronbach’s α for the VISA-A (sedentary) was 0.724 (table 3). All inter-item correlations for the questions were <0.70, with all items having at least one inter-item correlation >0.3. The questionnaire’s corrected item-total correlations ranged from 0.27 to 0.65, with Cronbach’s α decreasing to 0.719 or less with the removal of any item, indicating all items should be retained.
EFA indicated that the VISA-A (sedentary) has a multidimensional structure with two factors demonstrating an eigenvalue >1. The questionnaire was split with items 1–5 (symptoms) loading onto one factor and items 6–8 (activity) loading onto the other (table 4 and online supplemental figure 1). For Q1-5 and Q6-8, KMO values were 0.627 and 0.652 respectively. For Q1-5 and Q6-8, BTS values were both p<0.001.
Cronbach’s α for symptoms and activity was 0.663 and 0.563, respectively (table 3); the α value did not increase with removal of any item. For both subscales, the inter-item correlations were <0.70, with all items having at least one inter-item correlation >0.3. Corrected item-total correlations ranged from 0.31 to 0.58 for symptoms and 0.42 to 0.46 for activity.
The overall Cronbach’s α for the VISA-A (Q1-8) was 0.705, with α values increasing to 0.720 when items seven or eight were removed, indicating these items should not be retained. Factor analysis was not possible for the VISA-A, therefore, subsequent analyses were performed using the same dimensions as the VISA-A (sedentary). Cronbach’s α for Q1–5 was 0.724, increasing to 0.800 with the removal of item five, indicating this item should not be retained. Cronbach’s α was not calculable for Q6–8 as all patients scored zero for Q7 and Q8.
Test–retest reliability
The ICC for agreement between the 3-day retest and pretreatment VISA-A (sedentary) scores was excellent for symptoms and activity (table 3). There was no significant difference between the repeated measures for symptoms (p=0.99) or activity (p=0.32). Bland-Altman analysis (online supplemental figure 2) showed no bias of 0.0 (95% CI −0.24 to 0.24) for symptoms, with narrow limits of agreement from −1.66 to 1.66 and no significant evidence of proportional bias (p=0.66). There was a very small mean bias of 0.039 (95% CI −0.040 to 0.117) for activity with the CI crossing zero, narrow limits of agreement from −0.51 to 0.59 and no significant evidence of proportional bias (p=0.33).
Measurement error
The sw for symptoms and activity was 0.594 and 0.198, respectively. Repeatability was 1.647 for symptoms and 0.549 for activity (table 3). The MCID using a distribution-based analysis was 3.31 for symptoms and 5.03 for activity; using an anchor-based analysis the MCID was 4.33 and 4.88.
Construct validity and responsiveness
There was a significant difference between the VISA-A and VISA-A (sedentary) scores, with the VISA-A scores being significantly lower both pretreatment (p<0.001 for symptoms and activity) and post-treatment (p=0.022 for symptoms, p<0.001 for activity). Bland-Altman analysis (online supplemental figures 3 and 4) showed proportional bias between the questionnaires at both time points, with greater differences observed for higher scores both pretreatment and post-treatment. The ICC for agreement between scores was moderate to excellent for symptoms but poor for activity at both time points (table 5).
There was stronger correlation between the pretreatment to post-treatment change in the VISA-A (sedentary) scores (r=0.420 (95% CI 0.163 to 0.623) for symptoms, r=0.407 (95% CI 0.148 to 0.614) for activity) and the GROC than the VISA-A scores (r=0.253 (95% CI −0.02 to 0.494) for symptoms, r=0.186 for activity (95% CI −0.09 to 0.431)). Effect size and standardised response means for symptoms and activity are presented in online supplemental table 2.
Floor and ceiling effects
One patient (1.96%) achieved a maximum score for VISA-A (sedentary) symptoms post-treatment, while three patients (5.88%) obtained maximum scores for activity, therefore the VISA-A (sedentary) demonstrated no floor or ceiling effect. Twenty patients (39.22%) recorded minimum scores for VISA-A activity pretreatment and 10 (19.61%) post-treatment, indicating a floor effect for activity (Q6–8).
Discussion
The VISA-A (sedentary) demonstrated excellent test–retest reliability with very high ICC values and narrow 95% CIs. There was stronger correlation between the VISA-A (sedentary) and the GROC than the VISA-A, although the correlations were moderate. The VISA-A (sedentary) demonstrates no floor or ceiling effect, adequate reliability, validity and responsiveness and is recommended for use in this cohort.
Since PROMs measure constructs that can only be reported by patients themselves, no gold standard exists for these measures.21 The VISA-A is one of the most widely used PROMs in AT studies, but recent publications have questioned its validity and responsiveness.22 23 It is felt that internal validity of the VISA-A was not adequately investigated in the original study,9 likely due to the fact that it was developed before the COSMIN guidelines were published. However, recent systematic reviews conclude that while the VISA-A demonstrates insufficient evidence for measurement error there is sufficient evidence for reliability, construct validity and responsiveness.24 25
In accordance with the COSMIN checklist,11 the VISA-A (sedentary) was developed using an adequate number of symptomatic patients and professionals from relevant disciplines, indicating that the questionnaire has content validity. Patient feedback and measures of internal consistency indicate that items 6–8 of the VISA-A are not relevant to sedentary individuals, with patients unable to score higher than 60/100 on this PROM. Measures of construct validity and responsiveness also demonstrate statistically significant differences between questionnaires for the activity dimension (Q6–8), which is not unexpected as the VISA-A was developed using an active population and is recommended for use in homogeneous groups.9 10
Clinical implications
EFA identified the VISA-A (sedentary) as a two-dimensional PROM; therefore, each dimension should be scored out of 50 points (online supplemental appendix D). The MCID values were similar using distribution and anchor-based analyses, indicating that a change of 5 points for symptoms and 5 points for activity is clinically relevant for sedentary individuals with AT. Only 27/51 patients (52.9%) achieved the MCID of 14 points on the VISA-A,26 despite 49/51 (96.1%) patients recording a perceived improvement on the GROC.
Supplemental material
Limitations
It was not possible to perform an EFA for the VISA-A, indicating a lack of structural validity in this cohort, therefore the VISA-A was split into the same dimensions as the VISA-A (sedentary) to allow adequate analyses, as recommended by Comins et al.23 Although Cronbach’s α was acceptable (>0.70) for the VISA-A (sedentary) overall, the lower limit of the 95% CI was 0.593, and the α value was 0.563 for the activity dimension with wide confidence intervals. This may reflect the different scoring format and weighting of response options for items 7–8. Future studies should investigate whether a more consistent and evenly weighted scoring system is warranted or improves the internal consistency of the VISA-A (sedentary).
The test–retest reliability of the VISA-A (sedentary) was excellent. Baseline questionnaires were administered face-to-face in paper format, with follow-up scores obtained 3 days later via phone call. Although the environment (hospital vs home) and administration (paper vs telephone call) of the two questionnaires were different, this method was employed to minimise patient inconvenience and represents a pragmatic approach given the inherent difficulties in conducting research of this nature amid the COVID-19 pandemic.
To promote methodological quality of the study, recommendations from the COSMIN checklist were adhered to where possible. Data from 51 patients were available for this study, indicating adequate sample size. To be categorised as ‘very good’, at least 100 patients are required, therefore it is recommended that the VISA-A (sedentary) is further investigated in studies with larger sample sizes. The study sample contained a comparable number of male and female patients, but all individuals were over the age of 40, which limits the generalisability of findings to younger sedentary patients.
Several recent publications have identified an association between psychosocial variables and outcomes in tendinopathy.27 28 Although limited evidence exists for AT, and specific psychosocial variables (eg, kinesiophobia) were not voluntarily reported by patients in this study, the importance of including a psychosocial domain may become more evident as our understanding of AT evolves.
Conclusion
The VISA-A (sedentary) demonstrates adequate reliability, validity and responsiveness in sedentary patients with AT. The VISA-A (sedentary) represents a more appropriate measure than the VISA-A for this cohort and is recommended for clinical and research purposes, with each dimension scored out of 50 points.
Data availability statement
Data are available on reasonable request.
Ethics statements
Patient consent for publication
Ethics approval
The study was approved by the University and NHS ethics committees and conducted in accordance with the ethical standards of the World Medical Association Declaration of Helsinki (2002).
Acknowledgments
We would like to thank the patients and staff at Warrington and Halton Hospitals, NHS Foundation Trust and University of Leicester NHS Trust.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Twitter @thekneeresource, @tendonresearch, @sethoneill
Contributors RN and SO'N contributed to the study design, development of the VISA-A (sedentary), data collection, data analysis and writing up of the study. JLC and JEG contributed to the study design and development of the VISA-A (sedentary). JR contributed to data collection. TM contributed to data analysis and writing up of the study. RN is responsible for the overall content as guarantor.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.