The anchor-based minimal important change, based on receiver operating characteristic analysis or predictive modeling, may need to be adjusted for the proportion of improved patients

Abstract

Objectives

Patients have their individual minimal important changes (iMICs) as their personal benchmarks to determine whether a perceived health-related quality of life (HRQOL) change constitutes a (minimally) important change for them. We denote the mean iMIC in a group of patients as the “genuine MIC” (gMIC). The aims of this paper are (1) to examine the relationship between the gMIC and the anchor-based minimal important change (MIC), determined by receiver operating characteristic analysis or by predictive modeling; (2) to examine the impact of the proportion of improved patients on these MICs; and (3) to explore the possibility to adjust the MIC for the influence of the proportion of improved patients.

Study Design and Setting

Multiple simulations of patient samples involved in anchor-based MIC studies with different characteristics of HRQOL (change) scores and distributions of iMICs. In addition, a real data set is analyzed for illustration.

Results

The receiver operating characteristic–based and predictive modeling MICs equal the gMIC when the proportion of improved patients equals 0.5. The MIC is estimated higher than the gMIC when the proportion improved is greater than 0.5, and the MIC is estimated lower than the gMIC when the proportion improved is less than 0.5. Using an equation including the predictive modeling MIC, the log-odds of improvement, the standard deviation of the HRQOL change score, and the correlation between the HRQOL change score and the anchor results in an adjusted MIC reflecting the gMIC irrespective of the proportion of improved patients.

Conclusion

Adjusting the predictive modeling MIC for the proportion of improved patients assures that the adjusted MIC reflects the gMIC. Limitations: We assumed normal distributions and global perceived change scores that were independent on the follow-up score. Additionally, floor and ceiling effects were not taken into account.

Introduction

Health-related quality of life (HRQOL) has become an important outcome in current studies evaluating the benefit and harm of treatments for various medical conditions. However, questionnaires designed to measure HRQOL provide scores that lack intrinsic meaning. Similarly, changes in such scores offer no obvious interpretation of the importance of those changes. This is where the concept of the “minimal important change” (MIC; also called “minimal clinically important change” or “minimal (clinically) important difference”) comes in [1]. The MIC is defined as the minimal amount of change in an HRQOL score that is perceived as “important” [2], [3].

MICs can be determined in different ways. Two broad approaches include distribution-based methods and anchor-based methods [4]. Distribution-based methods relate HRQOL change scores to the distribution of change scores or the probability that a change score might be attributed to measurement error. Anchor-based methods relate HRQOL change scores to an external criterion (the anchor) of what constitutes the smallest HRQOL change that is deemed important [5]. Often, the patient's “global perceived change” (GPC) is used as such an anchor—and for good reasons. By providing the anchor for the MIC, patients directly provide the standard by which to measure the benefits and harms of their treatments.

HRQOL changes can be important in two opposite directions: improvement and deterioration. For the sake of simplicity, however, we will limit our discussion to the direction of improvement. Everything that is true for the MIC for improvement applies—although reciprocally—to the MIC for deterioration. In the Section 8, we will offer specific suggestions on how to apply the methodology elaborated in this paper to the MIC for deterioration.

Furthermore, we will focus on two specific anchor-based approaches: the receiver operating characteristic (ROC) method and the predictive modeling method. A popular anchor-based MIC method uses ROC analysis to determine the change score that is optimally discriminating between importantly improved patients and not importantly improved patients [6]. Recently, we have introduced a novel method to determine an anchor-based MIC based on predictive modeling [7]. In this approach, the MIC is related to the change score-specific likelihood ratio (i.e., the ratio of the likelihood of a specific change score in the importantly improved group to the likelihood of that specific change score in the not importantly improved group). The predictive modeling-based MIC (in short: “predictive MIC”) is defined as the change score for which the likelihood ratio equals 1 [7]. We have demonstrated that the predictive MIC method and the ROC-based method provide identical MIC values when the HRQOL change scores are normally distributed, and their variances are equal across the improved and not improved groups. In addition, the predictive MIC was more precise than the ROC-based MIC, and its 95% confidence interval (CI) was easier to calculate [7].

The ROC-based or predictive MIC is based on the analysis of HRQOL change scores in relation to patient-rated GPC scores. Apparently, patients are able to rate their GPC in HRQOL. We assume that they do so by comparing their perceived HRQOL improvement with their individual minimal important change (iMIC). If patients' perceived HRQOL improvement exceeds their personal iMIC, they will rate themselves as “improved,” and otherwise, they will rate themselves as “not improved.” We assume that each patient has their own iMIC as a personal benchmark of what constitutes a minimal important HRQOL improvement. As the MIC is intended to reflect the minimal improvement that is important to (a group of) patients, it seems desirable that the MIC reflects the mean of the iMICs in a group of patients. So, the mean iMIC should constitute the gold standard to compare the MIC with. We propose to denote the mean iMIC as the “genuine MIC” (gMIC) because it is the amount of improvement that the “average” patient values as (minimally) important. The gMIC (i.e., the mean iMIC) of a group of patients is, assumingly, relatively invariant. Unfortunately, iMICs and their mean, the gMIC, are not directly observable and measurable. The gMIC is a theoretical construct, put in place to explain how patients respond to GPC questions in an anchor-based MIC study.

So, how does the ROC-based or predictive MIC relate to the gMIC? Although iMICs escape direct observation, the relationship between the MIC and a set of iMICs can be studied using simulation techniques. Through simulations, unobservable variables can be created and controlled to enable the exploration of relationships between hidden phenomena (e.g., iMICs) and observable variables (e.g., MICs). Moreover, it is easy to simulate large samples with specified characteristics. The leading question in this paper is “How does the MIC, as determined by ROC analysis or predictive modeling, compare to the gMIC?”.

We have extensively explored the relationship between ROC based or predictive MICs and gMICs using simulations, and we will present some interesting findings. In particular, we will examine:

• To what extent the ROC based or predictive MIC equals the gMIC when the proportion of improved patients (hereafter: “proportion improved”) is 0.5 (Section 3);

• To what extent the MIC equals the gMIC when the proportion improved is smaller or greater than 0.5 (Section 4);

• If it is possible to adjust the MIC for the proportion improved bias (Sections 5 Adjusting the MIC for the proportion improved, 6 Validation of the adjusted MIC formula);

• The application of the MIC adjustment in a real data set (Section 7).

Before we continue with these findings, we will describe our simulation method in the next section.