Article Text

Challenges with cost-utility analyses of behavioural interventions among older adults at risk for dementia
1. Jennifer C Davis1,2,3,
2. Stirling Bryan1,2,
3. Carlo A Marra4,5,
4. Ging-Yuek R Hsiung6,7,
5. Teresa Liu-Ambrose3,7,8
1. 1Centre for Clinical Epidemiology and Evaluation, Vancouver Coastal Health Research Institute, Vancouver, Canada
2. 2School of Population and Public Health, University of British Columbia, Vancouver, Canada
3. 3Aging, Mobility, and Cognitive Neuroscience Laboratory, Department of Physical Therapy, University of British Columbia, Vancouver, Canada
4. 4Faculty of Pharmaceutical Sciences, University of British Columbia, Vancouver, Canada
5. 5Centre for Health Evaluation and Outcome Sciences, Providence Health Care Research Institute, Vancouver, Canada
6. 6Division of Neurology, Department of Medicine, University of British Columbia, Vancouver, Canada
7. 7Brain Research Centre, Vancouver Coastal Research Institute, University of British Columbia, Vancouver, Canada
8. 8Centre for Hip Health and Mobility, University of British Columbia, Vancouver Coastal Research Institute, Vancouver, Canada
1. Correspondence to Dr Jennifer C Davis, Centre for Clinical Epidemiology and Evaluation, Vancouver Costal Health Research Institute|The University of British Columbia, 7th floor, 828 West 10th Avenue, Vancouver, BC, Canada V5Z 1M9; Jennifer.davis{at}ubc.ca

## Abstract

Background Cognitive decline is one of the most prominent healthcare issues of the 21st century. Within the context of combating cognitive decline through behavioural interventions, physical activity is a promising approach. There is a dearth of health economic data in the area of behavioural interventions for dementia prevention. Yet, economic evaluations are essential for providing information to policy makers for resource allocation. It is essential we first address population and intervention-specific methodological challenges prior to building a larger evidence base. We use a cost-utility analysis conducted alongside the exercise for cognition and everyday living (EXCEL) study to illustrate methodological challenges specific to assessing the cost-effectiveness of behavioural interventions aimed at older adults at risk of cognitive decline.

Methods A cost-utility analysis conducted concurrently with a 6-month, three-arm randomised controlled trial (ie, the EXCEL study) was used as an example to identify and discuss methodological challenges.

Results Both the aerobic training and resistance training interventions were less costly than twice weekly balance and tone classes. In critically evaluating the economic evaluation of the EXCEL study we identified four category-specific challenges: (1) analysing costs; (2) assessing quality-adjusted life-years; (3) Incomplete data; and (4) ‘Intervention’ activities of the control group.

Conclusions Resistance training and aerobic training resulted in healthcare cost saving and were equally effective to balance and tone classes after only 6 months of intervention. To ensure this population is treated fairly in terms of claims on resources, we first need to identify areas for methodological improvement.

• Exercise
• Physical activity and exercise methodology
• Physical activity promotion in primary care
• Walking
• Weight lifting

## Introduction

In our current landscape of finite and limited resources, policy makers must make informed decisions. This paper considers the general area of economic evaluation of behavioural interventions targeting older adults at risk of dementia, and discusses some of the particular challenges associated with cost-utility analysis in this clinical field.

Cognitive decline is one of the most prominent healthcare issues of the 21st century. By 2040, over 80 million people will have dementia.1 Dementia is associated with increased healthcare resource utilisation.2 ,3 In 2010, the worldwide cost of dementia was US$604 billion with 70% of these costs arising from North America and Western Europe.3 Thus, the societal value of identifying cost-effective intervention strategies for dementia is considerable, especially given the scarcity and finite nature of healthcare resources.4 ### Treatment options exist—pharmacological treatment The causes of dementia are numerous; Alzheimer's disease and vascular dementia are the two most common.5 Using Alzheimer's disease as an example to discuss the cost-effectiveness evidence currently available from pharmacological treatment, we know that a number of economic evaluations have been conducted. Specifically, a recent systematic review of the cost-effectiveness of pharmacological interventions for individuals with Alzheimer's disease demonstrated wide divergence ranging from concluding cost-effectiveness to not being cost-effective for the same treatment.6 For example, the incremental cost-effectiveness ratio (ICER) for patients treated with donepezil versus no drug treatment ranged from a dominant value (less money and improved outcomes) to €281 416.13 per quality-adjusted life-year (QALY).6 The sheer quantity of estimates for pharmacological treatment provides is an essential starting platform of evidence; however, reasons for divergence in the ICERs warrant further exploration. ### Behavioural interventions Physical activity is a potentially inexpensive treatment that could have substantial preventative and restorative properties for cognitive function.7 Evidence from randomised controlled trials (RCT) demonstrates that targeted exercise training promotes cognitive function in older adults.8–12 Specifically, three meta-analyses of RCTs of exercise concluded that targeted exercise training has cognitive benefits.13–15 Further, recent RCTs provide preliminary evidence that the cognitive benefits of targeted exercise training extend to older adults with mild cognitive impairment (MCI)—a largely understudied population but an ideal target population for prevention.11 ,12 ,16 To our knowledge, there are only four studies to date that examined the efficacy of different types of targeted exercise training on cognitive function in older adults with MCI.12 ,17–19 An essential counterpart to such clinical research is economic evaluation. ### Economic evaluations—absent in RCTs of behavioural interventions promoting cognitive function—essential tool informing policy decisions Economic evaluations guide essential health policy decisions. Critically relevant to the understated need of economic evaluations, is the fact that very few RCTs in the important field—behavioural interventions promoting cognitive functionhave included economic analysis as part of the protocol. Further, given the plethora of economic data available for pharmacological interventions, there is a pressing need to generate economic data for potentially equally promising intervention strategies that may act as complements or substitutes to pharmacological interventions—behavioural (eg, exercise) interventions. Building the health economic data evidence base for behavioural interventions is an essential component to promote fair claims on relevant resources for this specific population. However, improvement in economic evaluation methodology relating specifically to older adults at risk of cognitive decline is needed before claims on resources can be addressed in an appropriate way. ### Challenges in conducting a concurrent economic evaluation within an RCT in a population of older adults at risk of dementia Trials with older adults with any degree of cognitive decline carry the inherent challenge of defining a valid outcome measurement and ensuring adherence with the intervention of choice (ie, exercise).20 The additional component—an economic evaluation—includes a key element essential for healthcare decision-making. We use the cost-utility analysis presented below as a case study to examine the unique methodological challenges related to conducting economic evaluations alongside clinical trials among older adults at risk of dementia. We provide a brief description of the cost-utility analysis used as a case study for this paper. We designed a concurrent, prospective economic analysis using individual-level data on cost and effectiveness outcomes as part of the exercise for cognition and everyday living (EXCEL) study; a three arm RCT.18 The main results have been reported elsewhere.18 Our primary objective for the economic evaluation was to determine the incremental cost-utility ratio (cost per QALY gained) of twice weekly resistance training (RT) or twice weekly aerobic training (AT) as compared to twice weekly balance and tone (BAT) classes (comparator). The comparator programme is not specifically designed to combat cognitive decline. First, we provide a brief overview of the EXCEL study. ## Methods Ethical approval was obtained from the Vancouver Coastal Health Research Institute and the University of British Columbia's Clinical Research Ethics Board. All participants provided written informed consent. ### Overview of economic evaluation We used a Canadian healthcare system perspective in our cost-effectiveness and cost-utility analyses and a 6-month time horizon for the economic evaluation. The study sample included 86 community dwelling women aged 70–80 years. The interventions for the EXCEL study included three participant groups: twice weekly RT, twice weekly AT and the comparator (control) group, twice weekly BAT classes. ### Costs Briefly, we used a self-report questionnaire to track total healthcare resource utilisation prospectively for each participant for 6 months.21 Further detail on the collection of cost data were previously reported.22 ### Effectiveness We calculated the total QALYs lost or gained at 6 months for each of the three participant groups using the EQ-5D administered at baseline and trial completion (6 months). We used linear regression to calculate the incremental QALYs for each participant adjusted for baseline utility score. All statistical analyses were carried out using STATA V.11.0. ### Handling missing data For all discrete time points, we used a combination of multiple imputation and bootstrappin g to estimate uncertainty caused by missing values.23–,26 We imputed missing EQ-5D and healthcare resource use. ### Cost-utility analysis We also calculated the incremental cost per QALY for twice-weekly RT and twice weekly AT compared with twice weekly BAT classes (comparator). We used 5000 bootstrapped replications of mean difference in cost and mean QALY differences.26 ## Results ### Healthcare use and costs Unit costs for healthcare cost items are provided in the online supplementary appendix 1. The mean total healthcare costs were notably lower in the cost-saving region of the cost-effectiveness plane for the AT and RT groups compared with BAT (p<0.05; see table 1 and figure 1). Table 1 Results of base case analysis Figure 1 Cost-effective plane depicting the 95% confidence ellipses of incremental cost and effectiveness (quality-adjusted life-years (QALYs) for comparison between aerobic training and balance and tone classes (comparator). ### Health outcomes Mean QALYs and adjusted mean QALYs calculated from the EQ-5D are provided in table 1. ### Adjusting QALYs for baseline utility in each group The incremental QALYs after 6 months calculated using the EQ-5D and adjusted for baseline EQ-5D levels, were 0.000 for the AT group and −0.006 for the RT group compared with BAT classes (table 1). ### Cost-utility analysis Based on the bootstrapped estimates from the cost-effectiveness plane (figures 1 and 2), we found that RT was less costly and equally effective than BAT classes. Figures 1 and 2 demonstrates that for AT and RT compared with BAT classes, all of the bootstrapped cycles (nearly 100% of the 5000 cycles) were represented in the south quadrant (ie, cost-saving region). This indicates that AT and RT are less costly and equally effective to the comparator (BAT classes). Figure 2 Cost-effective plane depicting the 95% confidence ellipses of incremental cost and effectiveness, (QALYs) for comparison between resistance training and balance and tone classes (comparator). ## Discussion Before conclusions can be made from this study, we first need to discuss the challenges that are specific to conducting economic evaluations alongside behavioural interventions among older adults at risk of dementia. An improvement in methodology is needed to facilitate the appropriate allocation of resources. To our knowledge no relevant methodological discussions relating to conducting concurrent economic evaluations alongside clinical trials exist for cognitive impairment and dementia. Below, we categorise these intervention population-specific challenges into four themes: (1) analysing costs; (2) assessing QALYs: Is there a most appropriate method; (3) incomplete data: follow-up time, changes in cognitive status and censoring; and (4) ‘Intervention’ activities of the control group. ### Analysing costs There are unique challenges faced when valuing medical expenditure related costs as well as costs related to loss of productivity among individuals with dementia. A recent study articulating the cost of dementia highlighted that the yearly monetary cost per person that was attributable to dementia ranged from CAD$41 689 to CAD\$56 290.27 The monetary value calculated was dependent on the method used to value informal care.27 Such a wide variation in cost valuations can have substantial impacts on the results of a cost-utility analysis.

These data highlight the need to consider methodology for accurately articulating dementia related costs. For economic evaluations in the EXCEL study, rather than defining dementia-related costs, we chose to report more broadly total health resource utilisation costs. A potential benefit of choosing option one is that it eliminates the concern over accurately defining dementia-related costs. Further, the use of cost diaries (ie, self-reporting) to quantify health resource utilisation is well supported in the literature as a valid method among a general population.21 A few limitations of this approach are as follows (1) wider variation in total health resource utilisation costs; (2) larger sample size required due to wider variation; and (3) may fail to see the true impact of the intervention on incremental costs if the dementia-specific incremental costs are smaller than the observed variation.

An alternative method would be to develop a consensus on what cost items are directly attributable to dementia. Given that individuals with dementia often have other comorbidities—most commonly stroke or depression28—defining dementia-related costs is a significant challenge.27 Another important set of cost items important to value in this population is unpaid time of family or friends spent helping an affected individual achieve their activities of daily living.29 Of note, this perspective was not valued in the EXCEL study. However, equally important is the issue of how to accurately assign a monetary cost to an informal caregivers time. Specifically, the following items need to be considered (1) unit cost of caregivers time; (2) loss of work productivity of caregiver; and (3) time (travel time and time spent with the individual).

Methodological challenges relating to how to value resources remains a long-standing issue.30 The recommendations to date are broad; researchers should be specific in the reporting and analysis of their cost items and different valuations and inclusions of cost items should be explored in sensitivity analyses.30 Specific differences in the incremental costs calculated in a systematic review of over 3000 cost-utility ratios indicated differences in costing methodology among the following (1) disclosure of study perspective; (2) statement of time horizon; (3) use of discounting for future costs and QALYs; (4) statement of year of currency; (5) reporting of sensitivity analysis; and (6) use of incremental analysis.

In summary, moving forward with economic evaluations alongside clinical trials among individuals with dementia, future research needs to focus on what cost items should be included, accurate valuation of cost items and mode of data collection for such cost items.

### Assessing QALYs: Is there a most appropriate method?

When dealing with a clinical population with dementia or at high risk of cognitive decline, assessing QALYs becomes a particularly challenging task due to three methodological issues, including response rate, recall bias and the validity of a proxy.31–33 For example, the EQ-5D (also used in the EXCEL study) is a widely used instrument in cost-utility analyses across the globe because of its demonstrated feasibility and reliability. We chose the EQ-5D in the EXCEL study because it contained the appropriate domains we expected the intervention to impact and because of its feasibility. However, these are not adequate reasons to use an instrument and we need to consider potential challenges of such instruments.

Cognitive function is a spectrum and thus an important question to consider is: At what threshold of cognitive function is an individual considered impaired and unable to accurately value their health state utility values?

First, when considering establishing thresholds of cognitive function, it is crucial to consider individuals cognitive function trajectories over time. Evidence indicated that health-related quality of life (HRQoL) is relatively stable among Canadians until age 70 at which point it begins to decline.34 From recently published HRQoL trajectories for Canadians,34 we speculate that a 6-month time horizon is not long enough to observe any significant incremental changes in QALYs between the AT or RT groups compared with BAT classes.35 However, a counter argument may be that we would expect people at risk for dementia to decline at a greater rate; therefore, shorter time horizons may be advisable when not using a proxy.

Second, in the EXCEL study, our study population consisted of individuals with probable MCI. We did not make use of a proxy; however, this may be acceptable for the EXCEL study because the range and mean Montreal Cognitive Assessment scores across the three groups indicate the distributions were comparable across the three participant groups. As such, we did not anticipate that our mean incremental QALY estimates would be largely affected. However, we were unable to ascertain the validity of the mean QALYs. A key challenge this limitation highlights is what ‘gold standard’ should be used to ascertain the validity of participant self-reported health state utility values.

#### Response rate

Among individuals with at high risk of cognitive decline or with dementia, achieving a high response rate is a key challenge to consider. For example, one study that included 390 patients and their caregivers used the EQ-5D to assess patients’ health state utility values. Data demonstrated that the response rate of caregivers and of patients was higher for those with mild dementia compared with those of moderate dementia.31

#### Recall bias

Three commonly used instruments to assess HRQoL are the SF-36, EQ-5D and Health Utilities Index36–39 each assume a different recall period ranging from an individual’s perceived health status today, in the past 1 week or in the past 4 weeks. Differences in these instruments and the implied recall period are an integral consideration among individuals with dementia or MCI because these individuals may be unable to accurately recall information for such periods. Hence, it is necessary to determine which recall periods are most appropriate to administer along the cognitive status spectrum.40

#### Use of a proxy

Participants in the EXCEL study had MCI. Hence, it is possible that they rated their health state better and thus we may observe a ceiling effect rendering it more difficult to estimate any incremental QALY differences.32 In considering solutions for this issue, we explore the evidence surrounding the use of proxies. One study examined the inter-rater agreement of patient and proxy completion of the EuroQol EQ-5D instrument among individuals with dementia.32 Both caregivers and physicians were compared with patient self-report. The results indicated that across all three raters, there was a high degree of variation that did not lead to support of any method.32 However, the data provided support for interviewer administration of the EQ-5D.32 Another study supported the use of the EQ-5D by patients and the inter-rater reliability between patients and proxies.31 Given that there are differences in assessment of HRQoL between people with dementia and proxies, a justified decision should be made at study outset about which perspective to take.

In summary, moving forward with economic evaluations alongside clinical trials among individuals with dementia or MCI, future research needs to explore cognitive status thresholds for when a proxy should be considered, what combination of proxies should be used to ensure validity and optimal response rate, and instrument selection based on target intervention domains and recall bias.

### Incomplete data: follow-up time, changes in cognitive status, censoring

A general challenge of economic evaluations that are concurrent with clinical trials is that the follow-up time usually coincides with the duration of the trial. This follow-up time for the economic evaluation is often too short to assess full health gains or losses.

Further, censored observations are a key challenge when dealing with frail elderly populations at high risk of cognitive decline. The frequency of censored observations among individuals in this population is often high due to death or loss of competence to provide further data. This argument further supports the use of a shorter time horizon for individuals at high risk of dementia. To our knowledge, this challenge has not been discussed in the economic of economic evaluations among individuals with cognitive impairment.

In summary, moving forward with economic evaluations alongside clinical trials among individuals with dementia or MCI, future research needs to explore methods for determining feasible and appropriate follow-up times for economic evaluations that may extend beyond the clinical trial and ascertain appropriate methods for handling increased censoring observations.

### ‘Intervention’ activities of the control group

The activities prescribed for the control group is a particularly important methodological issue for behavioural interventions. In the EXCEL study the control (comparator) group was assigned to active treatment. An active treatment group was thought necessary to prevent contamination (ie, control participants may feel ‘left out’ and start doing other exercises on their own (ie, their own intervention) or increased attrition. Hence, there is a justified need for an active control group (ie, the BAT classes group) for behavioural interventions. However, in real life, these people may be ‘doing nothing’. This issue is particularly relevant to behavioural interventions in comparison with pharmaceutical interventions where participants can remain blinded due to a placebo. Not all RCTs are justified in providing some active control both in terms of ethics and usual practice. This issue does affect the primary outcomes of economic evaluations often making the ICER an overly conservative estimate. In the case of the EXCEL study, the BAT classes were designed to specifically include exercises and not to combat cognitive decline. However, benefits of social group interaction may lead to an overly conservative estimate of health outcome benefit. As such, it may be advisable for economic evaluations to model a ‘do-nothing’ alternative.

## Conclusions

This critically important population that cannot be ignored for resource allocation. To ensure this population is treated fairly in terms of claims on resources, we identified areas for methodological improvement. Most methodological challenges discussed relate specifically to population-specific challenges; however, a few examples of behavioural intervention-specific challenges. The EXCEL study provided a platform to discuss population and intervention-specific methodological challenges in conducting cost-utility analyses alongside clinical trials.

### What are the new findings

• Aerobic training and resistance training interventions were less costly than twice-weekly balance and tone classes.

• This study sheds light on to population (ie, individuals at high risk of dementia) and behavioural intervention-specific methodological challenges. We discuss four category-specific methodological challenges of conducting behavioural interventions among older adults at risk of dementia (1) analysing costs, (2) assessing quality-adjusted life-years (3) incomplete data and (4) ‘Intervention’ activities of the control group.

• Identifying and discussing such methodological challenges is a key first step to ensure this population is treated fairly in terms of claims on finite resources.

• Future research needs to explore cognitive status thresholds for when a proxy should be considered and what combination of proxies should be used to ensure validity and optimal response rate.

### How might it impact on clinical practice

• Cognitive decline is one of the most prominent healthcare issues of the 21st century.

• Treatment options for dementia exist—pharmacological and behavioural interventions—yet, there is a dearth of health economic data in the area of behavioural interventions for dementia prevention.

• The societal value of identifying cost-effective intervention strategies for dementia is considerable, especially given the scarcity and finite nature of healthcare resources.

## Acknowledgments

The authors would like to thank the instructors for their commitment to the participants’ health and safety. TL-A is a Canada Research Chair in Physical Activity, Mobility and Cognitive Neuroscience and a Michael Smith Foundation for Health Research (MSFHR) Scholar. JCD is a Michael Smith Foundation for Health Research and Canadian Institutes of Health Research Postdoctoral Fellow. CAM is a Canada Research Chair in Pharmaceutical Outcomes.

• ## Supplementary Data

This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

Files in this Data Supplement:

## Footnotes

• Contributors JCD and TL-A were involved in study concept and design, acquisition of data, analysis and interpretation of data, preparation of manuscript, and critical review of the manuscript. SB and CAM were involved in study concept and design, analysis and interpretation of data, preparation of manuscript, and critical review of the manuscript. G-YRH was involved in interpretation of data and critical review of the manuscript

• Competing interests TL-A received funding for this study from the Pacific Alzheimer's Research Foundation.

• Ethics approval Obtained from the Vancouver Coastal Health Research Institute and the University of British Columbia's Clinical Research Ethics Board.

• Provenance and peer review Not commissioned; externally peer reviewed.

• Data sharing statement JCD and TLA had full access to all of the data (including statistical reports and tables) in the study and can take responsibility for the integrity of the data and the accuracy of the data analysis.

## Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.