Repeatability and responsiveness of exercise tests in pulmonary arterial hypertension

Introduction

Pulmonary arterial hypertension (PAH) is characterised by the progressive increase in pulmonary vascular resistance leading to right heart failure, poor exercise tolerance and early death. Because of its relationship with functional class and survival, exercise capacity is generally considered as a surrogate marker of disease severity in PAH [1]. As a result, the 6-min walk test (6MWT) has been the primary end-point in the majority of the clinical trials [2].

In PAH, the 6MWT has been assumed to be repeatable and more responsive to clinical changes than the incremental cardiopulmonary exercise test (CPET) on ergocycle [3]. Based on these results, the 6MWT has become, by far, the most popular exercise testing modality in PAH [2]. However, the repeatability of the 6MWT has never been formally assessed in PAH, and its responsiveness to therapeutic intervention is questionable. Indeed, recent meta-analyses documented that the increase in 6MWT following monotherapy and combination therapy averaged 35 m [2] and 22 m [4], respectively, corresponding to ∼10% and ∼7% of baseline values. Moreover, the 6MWT may suffer from a ceiling effect in patients with mild PAH [5]. These observations suggest that the responsiveness of the 6MWT to interventions may be insufficient in PAH, leading to underestimation of the real treatment benefit and potentially to impeding drug development in this disease. The validation of a reliable exercise testing modality that is repeatable and responsive to interventions is therefore clearly needed in PAH. The endurance shuttle walk test (ESWT) and the cycle endurance test (CET) have been used in other chronic diseases, such as chronic obstructive pulmonary disease (COPD), and were shown to be more sensitive to clinical changes than the 6MWT or the CPET [6, 7]. CET is now the primary end-point in most controlled trials evaluating bronchodilators and exercise training in COPD. Conversely, endurance exercise tests have been rarely used in PAH and their responsiveness remains unknown.

The objective of this study was to compare the test–retest repeatability and responsiveness of the 6MWT, the ESWT and the CET in PAH. We hypothesised that the endurance tests would be as repeatable as the 6MWT, whereas their responsiveness following the addition of sildenafil citrate over initial monotherapy would be superior to that of the 6MWT.

RESULTS

Patients’ characteristics

A total of 22 patients were recruited from June 2009 to August 2011. We excluded one patient during the familiarisation phase due to a significant ST segment depression during exercise and another patient withdrew consent during the second treatment phase (fig. 2). Baseline characteristics of the 20 PAH patients who completed the entire protocol are shown in table 1. Most patients had idiopathic PAH or PAH associated with connective tissue disease and were in functional class II (75%). They were treated with bosentan (n = 15), ambrisentan (n = 3) and epoprostenol i.v. (n = 2) for 22±18 months.

• Download figure
• Open in new tab
• Download powerpoint

Figure 2–

Study flow chart.

View this table:

• View inline
• View popup

Table 1– Patient characteristics

Test–retest repeatability

Test–retest repeatabilities and corresponding Bland–Altman plots are shown in figure 3. Following 4 weeks of placebo, the mean±sd differences in the duration/length of the 6MWT, the ESWT and the CET were -5±22 m, +19±144 s and -14±35 s, respectively. The 6MWT showed less variation than the ESWT and the CET, and appeared to be the most repeatable test, as assessed by the coefficients of variation (table 2), the Pearson correlations and the modified Bland–Altman analysis. The heart rate, oxygen uptake and the oxygen saturation by pulse oximetry were the most repeatable physiological parameters (table 2).

• Download figure
• Open in new tab
• Download powerpoint

Figure 3–

Test–retest repeatabilities of duration/length of a) the 6-min walk test (6MWT), b) the endurance shuttle walk test (ESWT) and c) the cycle endurance test (CET) are shown. The duration/length of post-placebo tests significantly correlated with pre-placebo tests for the 6MWT, the ESWT and the CET. The Pearson correlations (solid lines) and the identity lines (dotted lines) are shown. d–f) Modified Bland–Altman plot showing the relative change between pre- and post-placebo tests according to the mean duration/length of d) the 6MWT, e) the ESWT and f) the CET. The solid line represents the mean relative difference between the two tests, whereas the dotted lines correspond to the coefficients of variation (CV). The CV were similar for patients below and above the median value at baseline for the 6MWT (4±3% versus 2±2%), the ESWT (12±13% versus 15±10%) and the CET (16±16 versus 11±8%). The limits of agreement (1.96 sd) were -48–38 m, -263–301 s and -83–56 s for the 6MWT, the ESWT and the CET, respectively.

View this table:

• View inline
• View popup

Table 2– Test–retest repeatability of the physiological parameters

Responsiveness of exercise tests

Responsiveness of exercise tests is shown in figure 4. The placebo-corrected changes in exercise capacity following the addition of sildenafil citrate to the baseline therapy were 18±25 m (p = 0.02), 58±235 s (p = 0.58) and 29±77 s (p = 0.09) for the 6MWT, the ESWT and the CET, respectively. Despite relative increases in 6MWT being lower compared with endurance tests (+4%, +15% and +12% from baseline), the standardised response mean (95% CI) values were 0.72 (0.23–1.21), 0.25 (-0.20–0.70) and 0.38 (0.07–0.83) for the 6MWT, ESWT and CET, respectively. Cardiac and ventilatory responses at peak exercise were unchanged following the addition of sildenafil citrate (data not shown). The mean±sd Likert scores following sildenafil citrate and placebo were 0.5±1.4 and 0.4±1.1, p = 0.77, respectively. The Likert scores correlated with the objective changes in duration/length for the 6MWT (r = 0.32; p = 0.05), the ESWT (r = 0.52; p = 0.01) and the CET (r = 0.35; p = 0.04), respectively. The areas under the ROC curve to detect a clinically perceptible improvement in exercise capacity (Likert score ≥1) were 0.63, 0.74 and 0.63 for the 6MWT, the ESWT and the CET, respectively. The increase in exercise capacity associated with a true positive rate (specificity) of 90% to be associated with a Likert score ≥1 were 32 m, 66 s and 53 s for the 6MWT, the ESWT and the CET, respectively.

• Download figure
• Open in new tab
• Download powerpoint

Figure 4–

Responsiveness of a) the 6-min walk test (6MWT), b) the endurance shuttle walk test (ESWT) and c) the cycle endurance test (CET). Data are presented as mean change in duration/length following add-on sildenafil citrate and placebo, and the mean (95% CI) placebo-corrected changes are given. d) The standardised response mean of each test. ^#: p = 0.02.

Discussion

This study documented that among exercise tests currently used in chronic respiratory disorders, the 6MWT was the most likely to capture changes in exercise capacity induced by the addition of sildenafil citrate over baseline monotherapy in PAH. The enhanced responsiveness of the 6MWT was related to its better repeatability compared with the CET and the ESWT, supporting the use of this functional exercise test as an appropriate and clinically relevant end-point in randomised controlled trials.

Exercise capacity is generally considered to be a surrogate marker of disease severity in PAH [1]. Actually, the 6MWT and the CPET have excellent discriminative properties in this disease, and are closely correlated with functional class, disease severity and survival especially for idiopathic PAH patients [1, 16]. However, their responsiveness has been questioned recently. Indeed, previous randomised controlled trials failed to show improvement in exercise capacity when the peak oxygen uptake assessed by CPET was the primary end-point [3] or when the 6MWT was the outcome measure in less severely impaired patients [17].

Previous studies concluded that the ESWT [7] or the CET [6] were more responsive than the 6MWT to bronchodilators in COPD. In PAH, the relative increase in exercise tolerance was enhanced when assessed using the CET compared with the 6MWT following exercise training [18, 19]. In the present study, the relative increase in exercise capacity following the addition of sildenafil citrate was also higher for the ESWT and the CET compared with the 6MWT. However, only the increase in the 6MWT was statistically significant following the addition of sildenafil citrate to the baseline monotherapy. Using the standardised response mean, we also found that the ability to capture changes in exercise capacity was greater for the 6MWT than for the ESWT or the CET. Indeed, the standardised response mean for the 6MWT was close to 0.8, a threshold considered “large” compared with values between 0.2 and 0.5, which were considered to be small to moderate for the ESWT and the CET, respectively [20]. This parameter was chosen rather than the t-statistic, which provides no indication of random variation between tests, or the Cohen's effect size, which fails to recognise the relationship between placebo and treatment values. The standardised response mean is thus one of the most appropriate measures of the responsiveness of a measuring tool [21]. It has direct implication in the determination of sample size for clinical trials: the larger the standardised response mean, the smaller the sample size needed to demonstrate a treatment effect. The better repeatability of the 6MWT may account for its increased ability to capture changes in exercise capacity following the addition of sildenafil citrate. Indeed, the coefficient of variation of the 6MWT distance was well within 10%, an arbitrary value considered to be optimal for this sort of assessment tool [14]. It is noteworthy that the enhanced repeatability of the 6MWT might have been influenced by the fact that this test is repeatedly performed by PAH patients as part of their routine follow-up. Indeed, exercise tests are associated with a learning effect [9]. However, special attention was made to ensure that any variability between the tests was related to random errors instead of systematic bias as patients underwent practice tests to minimise any learning effect. Also, the order of the tests for each subject was randomly determined. Moreover, systematic bias was excluded using Bland–Altman analysis and crossover statistical analyses did not reveal any sequence, visit or time effect. As the majority of patients were in WHO functional class II, it is also possible that most patients exercised near their maximal walking speed, which may have increased the 6MWT repeatability. Therefore, whether such repeatability is representative of its reliability in the setting of a multicentre study, with more severe patients and with a larger cohort of patients, remains unknown. However, the capacity of the 6MWT to document clinical improvement in previous randomised controlled trials supports this concept [2].

Very few studies have assessed the effect of a combination therapy on exercise capacity in PAH. When the previous randomised placebo-controlled trials were pooled in a recent meta-analysis, the mean increase in 6MWT was 22 m when a second PAH therapy was added to baseline therapy compared with placebo [4]. Only one study has assessed the effect of sildenafil citrate 80 mg three times daily in addition to baseline epoprostenol i.v.; it found an improvement of 28.8 (95% CI 13.9–43.8) m in the 6MWT distance with the combination therapy compared with epoprostenol i.v. alone [22]. In the present study, a relatively low dose of sildenafil citrate was used for a shorter period of time (4 versus 16 weeks) to assess responsiveness of exercise tests in PAH, for the following reasons: 1) only sildenafil citrate 20 mg three times daily is commercialised for the treatment of PAH in Canada; 2) in the pivotal randomised controlled trial evaluating the effect of sildenafil citrate monotherapy [23], most of the improvement in exercise capacity occurred during the first 4 weeks of treatment with no dose–effect relationship; and 3) 4 weeks of placebo was deemed to represent an appropriate period to assess mid-term repeatability of exercise testing while minimising the risk of clinical deterioration on placebo. Finally, a majority of our patients were in WHO functional class II. Therefore, the study design, potential drug–drug interaction between bosentan and sildenafil citrate [24] and patients’ characteristics may have negatively affected the increase in exercise capacity observed in our study.

Previous studies have suggested that the minimal detectable difference (assessed using distribution-based method) [25] and the minimal important difference (assessed using anchor-based method) [26] of the 6MWT distance following the initiation of PAH therapy were 30–40 m. Although significant, the improvement in walking distance observed following the combination of sildenafil citrate to baseline monotherapy was thus modest and below the suggested minimal important difference for this variable and for the perception threshold. However, changes in exercise capacity significantly correlated with the subjective perception of change in clinical status as assessed by a Likert scale. In an attempt to assess the minimal increase in exercise capacity that was perceptible by patients in our cohort, a ROC curve was constructed. Changes in 6MWT distance and in ESWT that had >90% chance (specificity) of being perceived by the patient were close to the minimal important difference of the 6MWT previously suggested in PAH [26], as well as previously reported in COPD for the ESWT [27]. Conversely, the perception threshold for the CET was far from the proposed minimal important difference of 105 s reported for CET in COPD [28]. One potential explanation is that our CET was too short in duration. We arbitrarily chose to set the intensity at 80% of the peak workload to try to minimise the high variability that has been seen in COPD at lower relative workloads [29]. However, according to the American Thoracic Society, a constant workload test should last a minimum of 360 s [9], which was not the case in our study. The relatively high intensity might have induced early peripheral muscle acidosis and fatigue. Therefore, the CET might have been less susceptible to track a beneficial effect on pulmonary haemodynamics following sildenafil citrate administration. This also reinforces the fact that minimal important difference estimates obtained in one disease cannot necessarily be extrapolated to other conditions and that the minimal important difference might be disease, severity and intervention specific [30].