Abstract
We compared the test characteristics of the shock index (SI) and the simplified pulmonary embolism severity index (sPESI) for predicting 30-day outcomes in a cohort of 1,206 patients with objectively confirmed pulmonary embolism (PE).
The primary outcome of the study was all-cause mortality. The secondary outcome was nonfatal symptomatic recurrent venous thromboembolism (VTE) or nonfatal major bleeding.
Overall, 119 (9.9%) out of 1,206 patients died (95% CI 8.2–11.5%) during the first month of follow-up. The sPESI classified fewer patients as low-risk (369 (31%) out of 1,206 patients, 95% CI 28–33%) compared to the SI (1,024 (85%) out of 1,206 patients, 95% CI 83–87%) (p<0.001). Low-risk patients based on the sPESI had a lower 30-day mortality than those based on the SI (1.6% (95% CI 0.3–2.9%) versus 8.3% (95% CI 6.6–10.0%)), while the 30-day rate of nonfatal recurrent VTE or major bleeding was similar (2.2% (95%CI 0.7–3.6%) versus 3.3% (95%CI 2.2–4.4%)). The net reclassification improvement with the sPESI was 13.4% (p = 0.07). The integrated discrimination improvement was estimated as 1.8% (p<0.001).
The sPESI quantified the prognosis of patients with PE better than the SI.
Recent guidelines emphasise the importance of early risk stratification of patients with acute pulmonary embolism (PE) 1. Risk stratification of patients with PE may identify patients at high risk of early death, who may benefit from more intensive surveillance or aggressive therapy 2, 3. Alternatively, patients deemed to be at low risk for early complications (i.e. death, recurrent venous thromboembolism (VTE) and major bleeding) might be considered for partial or complete outpatient treatment of their PE 4, 5.
The pulmonary embolism severity index (PESI) was developed to estimate 30-day mortality in patients with acute PE. The PESI used objective clinical items to produce a risk stratification score. Some investigators have used the PESI to identify patients with a low mortality risk who may be suitable for home management 6–8. Use of the PESI may not be practical for routine application in busy hospital emergency departments, because it requires computation of a score based on 11 different variables, and each variable has a different weight. Recently, Jiménez et al. 9 developed a simplified version of the PESI (i.e. simplified PESI (sPESI)). The sPESI included the variables of age >80 yrs, history of cancer, history of chronic cardiopulmonary disease, heart rate ≥110 beats·min−1, systolic blood pressure <100 mmHg and arterial oxygen saturation (Sa,O2) <90%. Patients with none of the variables present are categorised as low-risk and those with any variable present are categorised as high-risk. The study showed that the sPESI successfully predicts 30-day mortality after acute symptomatic PE and reduces the complexity of the original prediction rule.
The shock index (SI), defined as heart rate divided by systolic blood pressure, is an independent predictor of 30-day mortality in patients with acute PE 10. In a novel management strategy, SI was used to accelerate the triage of patients with suspected acute PE. Those with SI ≥1 and right ventricular (RV) dysfunction on echocardiogram received early reperfusion therapy to avoid lengthy imaging tests 11. Alternatively, SI showed a high sensitivity to identify a subgroup of patients at low risk of death in a multinational cohort (Computerised Registry of Patients with Venous Thromboembolism (RIETE)) of patients with acute PE 10.
Since the sPESI and the SI are easily measurable and might be accurate for identifying low-risk patients with acute PE in real-world clinical situations, the present study aimed to compare both prediction rules in a large sample of ambulatory patients with acute symptomatic PE. The study also assessed the ability of the prediction rules to identify low-risk patients with acute PE who might be candidates for treatment in the outpatient setting.
METHODS
Study design
Using prospectively collected baseline data at the time of PE diagnosis and outcome data from this cohort, we retrospectively assessed the test characteristics of the SI and the sPESI for predicting 30-day mortality, nonfatal recurrent VTE and nonfatal major bleeding. We then assessed the ability of the prediction rules to identify low-risk patients with acute PE who could be candidates for treatment in the outpatient setting. All patients provided informed consent for their participation in the prospective registry in accordance with the requirements of the ethics committee of the hospital (Ramón y Cajal Hospital, Madrid, Spain) and this study was approved by the ethics committee.
Patients, setting and eligibility criteria
Patients were recruited from the Emergency Department of Ramón y Cajal Hospital between January 1, 2003 and December 31, 2009. Eligible patients were required to have acute symptomatic PE confirmed by objective testing. A diagnosis of PE was confirmed by either a high-probability ventilation–perfusion scan result (according to the criteria of the Prospective Investigation of Pulmonary Embolism Diagnosis 12), a lower limb venous compression ultrasonography positive for a proximal deep vein thrombosis (DVT) in patients with inconclusive ventilation–perfusion scans 13 or previously described criteria to detect acute pulmonary embolism on contrast-enhanced PE-protocol helical chest computed tomography (CT) 14.
Study outcomes
The primary outcome used to validate the prediction rules was all-cause mortality 30 days after diagnosis of acute symptomatic PE. The secondary outcome was objectively confirmed nonfatal symptomatic recurrent VTE or nonfatal major bleeding. We assessed mortality using patient or proxy interviews, and/or hospital chart review. One investigator (D. Jiménez Castro, Respiratory Dept, Ramón y Cajal Hospital and Medicine Dept, Alcalá de Henares University, Instituto Ramón y Cajal de Investigación Sanitaria, Madrid, Spain) adjudicated the cause of all deaths as 1) definite fatal PE, 2) possible fatal PE or 3) death from other causes. Death was judged to be definite fatal PE if it was confirmed by autopsy or if death followed a clinically severe PE, either initially or shortly after an objectively confirmed recurrent event, in the absence of any alternative diagnosis. Possible fatal PE consisted of death in a patient who died suddenly or unexpectedly. Patients with symptoms or signs of recurrent VTE were evaluated with objective tests. Recurrent DVT was diagnosed by the appearance of a new noncompressible vein segment, a ≥4-mm increase in the diameter of a thrombus on venous ultrasound, or a new intraluminal filling defect or extension of a previous filling defect on a venogram 15. Recurrent PE was confirmed either by a new perfusion scan defect involving ≥75% of a lung segment, the presence of a new intraluminal filling defect or an extension of a previous filling defect on helical chest CT 14. Trained attending radiologists blinded to patient clinical information assessed the imaging studies. Bleeding complications were classified as major if they were overt and were either associated with a decrease in haemoglobin level of ≥2.0 g·dL−1, required a transfusion of ≥2 units blood, or were retroperitoneal or intracranial.
Treatment
Patients were initially hospitalised and treated with therapeutic doses of parenteral anticoagulants (intravenous unfractionated heparin or weight-based doses of subcutaneous low molecular weight heparin (enoxaparin)) while they were converted to oral vitamin K antagonist therapy. Thrombolytic treatment was instituted in patients with confirmed PE and haemodynamic impairment as deemed appropriate by the attending physician. After the initial “overlap” treatment period, patients were continued on dose-adjusted oral vitamin K antagonist therapy (acenocoumarol; target international normalised ratio (INR) of 2.5 (therapeutic range 2.0–3.0)). The INR was usually monitored daily until the therapeutic range had been achieved, then twice or three times weekly for the first weeks, and then once a week to once a month, depending on the stability of the results. Patients that developed contraindications to anticoagulant therapy had an inferior vena cava filter placed and the anticoagulant discontinued.
Statistical analysis
Baseline characteristics are presented as mean±sd for continuous data and n (%) for categorical data. We calculated the sPESI score for each patient. Patients were classified as low- (0 points) or high-risk (≥1 point) 9. Patients with an SI (heart rate/systolic blood pressure) of <1 were defined as low-risk whereas those with n SI ≥1 were considered high-risk. We calculated the proportion of low- versus high-risk patients based on each prognostic model and determined the proportion of patients with 30-day adverse outcomes (all cause mortality or nonfatal recurrent VTE and nonfatal major bleeding) among low- versus high-risk patients. Proportions of patients with adverse events among groups were compared using the Chi-squared test with Yates correction or Fisher's Exact Test, and the McNemar's test. To assess test accuracy, we estimated sensitivity, specificity, positive and negative predictive values, and likelihood ratios for each model. We compared the discriminatory power of the two models by comparing the area under each receiver operating characteristic curve (C statistic). We examined the proportion of patients who would be reclassified into high- or low-risk categories between the SI and sPESI, and calculated the values of the net reclassification improvement (NRI) and the integrated discrimination improvement (IDI) comparing both prognostic models 16. 95% confidence intervals were computed from the binomial distribution by using the Statistical Package for Social Sciences (version 15.0; SPSS Inc., Chicago, IL, USA).
RESULTS
Of the 5,213 patients evaluated for possible acute symptomatic PE during the study period, 1,248 (24%) patients had objectively confirmed PE. Of these, 13 (1.0%) patients refused to give informed consent and 29 (2.3%) patients were lost to follow-up, leading to a final study sample of 1,206 patients. Of these, 1,145 patients had a positive helical CT, a high-probability ventilation–perfusion lung scan or both, and 61 had a nondiagnostic ventilation–perfusion lung scan and a proximal DVT documented by compression ultrasonography. A fraction of patients (n = 995) was included in a previous study that developed the sPESI clinical prediction rule 9.
Table 1 shows the patients’ clinical symptoms, predisposing conditions and relevant findings at presentation. The overall number of patients treated with inferior vena cava filters was small (22 (1.8%) out of 1,206 patients). 38 (3%) out of 1,206 patients were treated with thrombolytic therapy. We obtained echocardiographic data for 675 patients. RV dysfunction was present in 268 (39.7%) out of 675 patients. Of these, 30 (11.2%) patients (95% CI 7.4–15.0%) received thrombolytic therapy. Of the 1,206 patients in the study sample, 119 (9.9%) patients (95% CI 8.2–11.5) died within 30 days of presentation. Overall, 58 (48.7%) patients died from definite or possible PE, seven (5.9%) patients from bleeding and 54 (45.4%) patients from other causes. Among the seven fatal bleeding events, three were intracranial, two retroperitoneal and two gastrointestinal. 42 patients reached the secondary end-point; 10 patients had an episode of objectively confirmed nonfatal symptomatic recurrent VTE, 28 patients had an episode of nonfatal major bleeding, and four patients both bled and recurred.
For 30-day all-cause mortality, survival was lower in the sPESI high-risk group compared to the sPESI low-risk group (log rank p<0.001; fig. 1). For 30-day all-cause mortality, survival was lower in the SI high-risk group compared to the SI low-risk group (log rank p<0.001; fig. 2). Using patients in this study cohort, the sPESI classified a significantly lower proportion of patients as low-risk (369 (31%) out of 1,206 patients, 95% CI 28–33%) than the SI score (1,024 (85%) out of 1,206 patients, 95% CI 83–87%) (p<0.001). Compared with the SI low-risk patients, the sPESI low-risk patients had a lower mortality (85 (8.3%) out of 1,024 patients versus six (1.6%) out of 369 patients) (table 2). The sPESI high-risk patients had a slightly lower mortality (113 (13.5%) out of 837 patients) compared to the SI high-risk patients (34 (18.7%) out of 182 patients). The 30-day rate of nonfatal recurrent VTE or major bleeding was similar for both prediction rules’ low-risk strata (2.2% (95% CI 0.7–3.6%) versus 3.3% (95% CI 2.2–4.4%)).
The sPESI had a substantially higher sensitivity and negative predictive value, and a lower negative likelihood ratio than the SI for predicting 30-day mortality in the study cohort (table 3). The sPESI (C statistic 0.71, 95% CI 0.68–0.73) had a better discriminatory power than the SI (C statistic 0.63, 95% CI 0.61–0.66) to predict 30-day mortality (p = 0.018). When 30-day adverse events were considered, the negative predictive value for the sPESI low-risk strata was 98%, compared with 97% for the SI (table 4). When 30-day overall mortality was considered in the subgroup of 1,103 normotensive patients (i.e. systolic blood pressure ≥100 mmHg) with acute PE, the negative predictive value for the sPESI low-risk strata was 98.4%, compared with 92.0% for the SI. When 30-day adverse events were considered, the negative predictive value for the sPESI low-risk strata was 97.8%, compared with 96.6% for the SI.
For the 79 patients in our cohort who died, reclassification was more accurate when the sPESI was used and for no participants did it become less accurate. Among the subjects who did not die, five patients were reclassified in a lower risk category and 581 were reclassified in a higher risk category. The NRI was estimated at 0.13 (p = 0.07) with the sPESI, resulting from a net 66.4% increase in nonsurvivors correctly identified as being at high risk but a net 53.0% increase in survivors incorrectly identified as being at high risk. The IDI was estimated as 0.018 (p<0.001). Compared with the SI, a history of cancer resulted in a net 49% increase in nonsurvivors correctly identified as being at high risk. Age >80 yrs resulted in a 36% increase, Sa,O2 <90% resulted in a 34% increase and chronic cardiopulmonary disease resulted in a 24% increase.
DISCUSSION
This study shows that the sPESI successfully predicted 30-day mortality after acute symptomatic PE. Compared to the SI, the sPESI had better prognostic accuracy. Our findings further validate a previously developed simple prognostic score (i.e. sPESI) for patients with acute symptomatic PE.
Clinical prognostic models were developed to identify low-risk patients with PE who may be candidates for outpatient care or a shorter hospital stay 16, 17. The most widely validated prognostic model is the PESI, which accurately stratifies patients into five risk classes (I–V) with increasing risk of short-term mortality, ranging from 1.1% in class I to 24.5% in class V 17. However, the original PESI uses 11 clinical parameters, with differently assigned weights, and its scoring depends on calculations that may be difficult to apply in the clinical setting. Both the sPESI and the SI might be suitable for use in busy emergency departments, but their prognostic accuracy had not previously been assessed in a head-to-head comparison.
In our sample, the sPESI and SI models showed a large discrepancy in the classification of low-risk patients. The two models demonstrated different rates of outcomes for the low-risk groups. The sPESI demonstrated a higher discriminatory power for predicting 30-day mortality than the SI. Low-risk patients identified using the sPESI had a lower mortality and a higher negative predictive value for death than low-risk patients identified using the SI. Low-risk patients identified by the sPESI showed a negligible 30-day risk of death, as shown by a negative predictive value of 98%. The negative predictive value of 98% for 30-day adverse events in the low-risk categories makes this score very useful for selecting patients for outpatient treatment. Because the sPESI was specifically developed to identify low-risk patients with PE, the positive predictive values (11–16%) and the positive likelihood ratios (1.34–1.51) for predicting mortality were low.
The C statistic is the most commonly used method of determining model discrimination; that is, how well the model can discriminate between persons in whom the outcome of interest will develop and those who will not. In the present study, we also evaluated recently described measures of model discrimination (NRI and IDI), which appear to be more sensitive tests of improvement in model discrimination than the C statistic analyses 18. In our study, the sPESI reclassified 66% of nonsurvivors as high-risk patients. Moreover, an IDI of 0.018 suggests that the sPESI identifies low-risk patients with acute PE significantly better than the SI.
In our study, only half of the deaths were due to PE. This proportion of PE-related deaths is similar to that of the International Cooperative Pulmonary Embolism Registry 19 and the RIETE registry 20. The variables that are used to calculate the SI (heart rate and systolic blood pressure) express the cardiopulmonary consequences of PE. We believe that the sPESI is more useful because it adds one domain that quantifies the age of the patients and two domains that capture coexisting illness (cancer and chronic cardiopulmonary disease).
The present study has limitations. First, the main limitation is the larger part of the study cohort (82%) was used to derive the sPESI. Thus, we actually expect that the sPESI outperforms any other prognostic model in this cohort. However, the study nevertheless demonstrates the suboptimal performance of the SI to identify low-risk patients with acute symptomatic PE. Secondly, although we prospectively collected clinical data in the study cohort, we performed a retrospective analysis, which can be subject to various biases. Thirdly, we could not estimate the potential impact of treatments on patient outcomes, because this information was not consistently available in our sample.
In summary, the study demonstrates that the sPESI has better prognostic accuracy than the SI. Our study did not directly assess the impact of the sPESI on the management of PE. Future studies should address whether this clinical prediction rule may help identify low-risk patients and, thus, possible candidates for home treatment.
Footnotes
Support Statement
The study was supported in part by grants FIS 08/0200 and SEPAR 2008.
Statement of Interest
None declared.
- Received May 5, 2010.
- Accepted June 28, 2010.
- ©2011 ERS