Prognostic score systems and community-acquired bacteraemic pneumococcal pneumonia

Severity scores

PSI, CURB-65 and mATS scores were, with one exception, calculated according to the original publications 5, 7, 15. In the emergency room of the Karolinska University Hospital, serum creatinine, rather than serum urea, is the commonly used test of renal function. Serum creatinine has also been used as an alternative to urea as a predictor for severe disease 16, a procedure that was recommended in the 2001 ATS guidelines 17. However, the cut-off values for serum urea used in PSI and CURB-65 are not the same (>10.7 mM and >7 mM, respectively). The current authors therefore chose to use a serum creatinine value of ≥130 mM, which corresponds to the higher of the two urea levels, for both scores. For calculating severity scores, all missing data were considered normal. There was no difference in the percentage of missing data between patients included prospectively or retrospectively (data not shown).

High-risk patients were defined as those having on admission a PSI risk class IV–V, a CURB-65 risk class 3–5, or mATS with >1 minor or ≥1 major criteria present. As a comparative baseline of severity, Acute Physiologic and Chronic Health Evaluation (APACHE)-II scores were also calculated 18. The APACHE-II score is probably the best-established severity score overall, and although quite laborious to use, it has also been shown to correlate well with mortality in IPD 13, 14.

Statistics

Fisher's exact test or the unpaired t-test were used when appropriate. Multiple logistic regression models were used to analyse independent risk factors for the ICU treatment need and death.

Pneumonia-specific severity scores

The need for ICU treatment was significantly higher (p<0.0001) in high-risk than in low-risk patients for all three severity scores: 19 out of 53 (35.8%) versus one out of 61 (1.6%) for PSI; 12 out of 22 (54.5%) versus eight out of 92 (8.7%) for CURB-65; and 18 out of 27 (66.7%) versus two out of 87 (2.3%) for mATS (fig. 1⇑). Similarly, a significantly higher mortality (p<0.01) was seen in high-risk than in low-risk patients: 13 out of 53 (24.5%) versus 0 out of 61 for PSI; eight out of 22 (36.4%) versus five out of 92 (5.4%) for CURB-65; and 11 out of 27 (40.7%) versus two out of 87 (2.3%) for mATS (fig. 2⇑).

As can be seen in figure 1⇑, there was a gradual increase in the need for ICU treatment with increasing CURB-65 severity scores, similar to that seen with APACHE-II scores, while for PSI all patients but one in need of ICU treatment had a score ≥IV. Similarly, with two exceptions, ICU treatment was only needed, in patients with one major, or two or more minor, mATS criteria. As can be seen in figure 2⇑, a similar pattern was seen for the correlation between severity score and mortality. The only exception was for CURB-65 where the single patient in risk class 5 was treated in the ICU, but survived.

Sensitivity, specificity, positive and negative predictive values

All three scoring systems had a good accuracy for predicting a fatal outcome, with an area under the receiver operating characteristics curve between 0.83 and 0.87 (fig. 3⇓). Using the high-risk definition, a cut-off for PSI of ≥IV, for CURB-65 of ≥3, and for mATS of >1 minor or ≥1 major, the sensitivity/specificity of the three tests for predicting death were 100/60%, 62/86% and 85/84%, respectively (table 2⇓). By lowering the cut-off for severe pneumonia for CURB-65 to ≥2, the sensitivity and negative predictive values for this score increased to 92 and 99%, respectively, while the specificity and positive predictive value decreased to 67 and 27%, respectively.

• Download figure
• Open in new tab
• Download powerpoint

Fig. 3—

Receiver operating characteristic (ROC) curves for pneumonia severity index (PSI; ♦); modified American Thoracic Society (mATS; ▴) rule; and CURB-65 (confusion; urea >7 mM; respiratory rate ≥30 breaths·min^-1; blood pressure <90 mmHg systolic or ≤60 mmHg diastolic; aged ≥65 yrs old; ▪). Area under ROC curves: PSI 0.85; CURB-65 0.84; and mATS 0.83.

View this table:

• View inline
• View popup

Table 2—

Sensitivity and specificity rates for the three tested score systems with regards to severe bacteraemic pneumococcal pneumonia

Antibiotic treatment

The initial antibiotic treatments used were second- or third-generation cephalosporins (n = 57 out of 114; 13 in combination with other antibiotics); a penicillin (mostly benzyl-penicillin) in monotherapy (n = 42); carbapenems (n = 6); fourth-generation cephalosporins (n = 3); clindamycin (n = 3); and n = 1 each of piperacillin-tazobactam, ciprofloxacin and erythromycin. The 13 fatal cases were initially treated with: 1.5g cefuroxime three times·day^-1 (n = 8; one in combination with aminoglycoside); carbapenems (n = 2); clindamycin (n = 1); erythromycin (n = 1; switched to clindamycin on day 2); and ciprofloxacin (n = 1; switched to clindamycin on day 3).

Resistance

Most (77 %) of the pneumococcal strains were fully susceptible to all tested antibiotics. Resistance was most commonly found for trimethoprim-sulfamethoxazole (16%). Only 4.4% of the strains had decreased susceptibility to penicillin, with minimum inhibitory concentration (E-test) ranging 0.25–2.0 mg·L^-1. Macrolide resistance was seen in 7% of the strains.

Serotypes

A specific pneumococcal serotype could be determined in 109 out of 114 patients. The most common serotypes were types 1 (16%), 14 (13%), 9V (10%) and 4 (8%). Fatal cases were seen especially among patients infected with serotypes 3 (two deaths out of four infected), 6A (two out of five), and 19F (two out of seven).

Multivariate analysis of risk factors for death

For APACHE-II and each of the three pneumonia-specific severity scores studied (PSI, CURB-65 and mATS), a logistic regression analysis was performed, using the severity score as the continuous variable. The only two factors that turned out to be independently correlated to death in each analysis were the respective severity score and treatment in OD than DID. For APACHE-II the odds ratio (OR; 95% confidence interval (CI)) for a fatal outcome was 1.24 (1.07–1.44) for each incremental score point. The corresponding ORs per point were 1.05 (1.02–1.08) for PSI (score I–V); 2.9 (1.44–5.74) for CURB-65 (score 0–5); and 10.25 (2.72–38.66) for mATS (score 0, 1 minor or (>1 minor or ≥1 major)).

In the logistic regression analyses for APACHE II, PSI, CURB-65 and mATS, the ORs (95% CI) for risk of death in those being treated in OD, as compared with DID, were 10.85 (1.81–64.94), 9.86 (1.50–64.87), 11.57 (2.17–61.58) and 14.87 (2.12–104.44), respectively.

Patients treated elsewhere than DID

A greater proportion of patients in OD, as compared with DID, had severe underlying conditions such as cancer (57.7 versus 9.1%; p<0.001) and immunosuppressive treatment (42.3 versus 6.8%; p<0.001), while other conditions such as chronic heart failure or lung disorders were equally distributed between the groups. According to the severity scores, patients treated in OD more frequently presented with a severe disease on admission, although the absolute number of patients with severe disease was 1.5–2 times as great in DID than in OD (table 3⇓). There was also a greater tendency for patients in OD to have presented with septic shock (19 versus 11%; nonsignificant), and required treatment in the ICU (31% versus 14%; p = 0.07), than those treated in DID. Serotype distribution also differed between the two groups, with serotypes 9V (19%), 6A (12%) and 19F (12%) being the most common in OD, compared with serotypes 1 (19%), 14 (15%) and 4 (9%) among patients treated in DID. No major differences between the two groups were seen in the antibiotic resistance pattern or in the empiric antibiotic treatment used. Unsurprisingly, patients treated in OD had a higher case-fatality rate than those treated in DID: 34.6 (nine out of 26) versus 4.5% (four out of 88; p<0.0001). However, this difference remained significant after stratification for severity (table 3⇓). Among patients with the most severe disease (PSI IV–V, CURB-65 3–5 or mATS with >1 minor or ≥1 major criterion), the case-fatality rates were about three times higher in patients treated in OD than in those treated in DID.

Case-fatality in bacteraemic pneumococcal pneumonia

Earlier studies of bacteraemic pneumococcal pneumonia in Stockholm (1977–1995) have demonstrated similar case-fatality rates (7–11%) 13, 19 to that in the present study (11%; 13 out of 114). In contrast, case-fatality rates in studies of IPD from other centres, both inside and outside Sweden, have varied considerably, from ∼3 to 25%, or higher 10, 14, 20–29. These varying case-fatality rates are likely to be due to differences between study populations concerning demographics, underlying health characteristics or severity of illness on admission to hospital 12, 13. Therefore, it must be of great importance to stratify patients according to severity of illness when comparing results between different sites. This strategy was applied in a recent in study where the Pitt bacteraemia score was used for defining patients who were critically ill (Pitt score >4) among 844 patients with invasive pneumococcal disease from 10 countries 14. A Pitt score >4 had a predictive value for mortality of 81%, and was also shown to be highly correlated (p = 0.0001) with the APACHE-II score.

Bacteraemic pneumococcal pneumonia and severity scores

In the present study, the APACHE-II score was used as a reference for the pneumonia-specific scores, since it has been shown to be highly correlated to the outcome of invasive pneumococcal disease 13, 14. In the study by Kalin et al. 13, the case-fatality rate was only 2.6% in patients with an APACHE-II score <12 (the median value), compared with 21% for those with a score of ≥12. This in accordance with the present study, in which no patient with an APACHE-II score <14 died, compared with 22% of those with a score ≥14. Approximately the same “cut-off level” was found in two other studies, on bacteraemic pneumococcal pneumonia 26 and on severe all-cause pneumonia 30, respectively, which both demonstrated that an APACHE-II score >15 was associated with a poor outcome.

The pneumonia-specific severity scores were as effective as the APACHE-II score in identifying patients at high risk of death. The PSI, with a cut-off of ≥IV, had the highest sensitivity and was the score that best identified patients with a low risk of death. Conversely, PSI had the lowest specificity: only 10% of patients with a score of ≥IV actually died, compared with 40% of patients who had >1 minor or ≥1 major criteria according to the mATS score. Furthermore, to calculate the PSI score, a large number of variables, including results of laboratory analyses and chest radiography, have to be provided, making this prognostic help difficult to implement in the routine clinical setting. (APACHE-II also suffers from this problem). The CURB-65 score has the obvious advantage of being very easy to use, requiring only five variables. With a cut-off “high-risk” score of ≥2, CURB-65 performed similarly to PSI (cut-off ≥IV), with a high sensitivity (92%) and a low specificity (67%). Conversely, a CURB-65 cut-off of ≥3 resulted in a rather low sensitivity (62%) but a moderately good specificity (86%). mATS was the best system in terms of specificity and positive predictive value, and had only a slightly lower sensitivity. It should be taken into account that this is the only one of the tested score systems that is primarily aimed at finding the patients with the most severe pneumonia. However, the mATS rule can difficult to interpret in terms of ICU care, since the major criterion “need for mechanical ventilation” can only be obtained in an ICU at the Karolinska University Hospital, thus creating a circular argument.

The case-fatality rate in the present study was much higher than in the derivation and validation cohorts of CURB-65 7 (27 versus 16% for a cut-off of ≥2; 36 versus 22–23% for a cut-off of ≥3). Also, using PSI (scores of IV–V) or mATS scores, the patients in the present study with bacteraemic pneumococcal pneumonia had higher case-fatality rates than in studies of all-cause CAP 31–35. One reason for this could be that invasive pneumococcal pneumonia is a more severe disease in itself than “all-cause pneumonia”. Another possibility is that patients with invasive pneumococcal pneumonia more often have severe underlying conditions, but that these illnesses carry a low weight according to the severity scores. For CURB- 65, there could have been a third reason, namely that the current authors used a cut-off value for S-creatinine that corresponded to a serum urea concentration of 11 mM instead of the 7 mM used in the original publication 7. However, this made little difference to the case-fatality rate. If the lower serum urea level was applied, 28 instead of 22 patients would have had a “high-risk” pneumonia (score of ≥3), with a case-fatality rate of 10 out of 28 (35.7%), as compared to eight out of 22 (36.4%) when the cut-off for serum urea concentration was 11 mM.

One strength of the current study is that all patients with invasive pneumococcal pneumonia who fulfilled the inclusion criteria at the study site during a 2-yr period were included, thus removing any sampling errors. There are also limitations to the current study. First, the number of patients treated in OD was small and the patients were all identified retrospectively, while all but two of those treated in DID were identified prospectively. However, exactly the same procedure in handling the data was used, with an identical standardised case-report form, and no difference in the number of missing variables between the two groups was seen. For all patients, both prospectively and retrospectively included, the data used for severity-score calculations were mostly obtained from medical records and laboratory databases compiled by the medical staff in charge of the patient during the first 24 h from admission. It is therefore highly unlikely that the retrospective analysis of patients' data would produce a less reliable result than the prospectively assembled data set. It is also unlikely that the retrospective analysis of medical records would misclassify severe underlying conditions, such as cancer or immunosuppressive treatment, or outcome parameters, such as intensive care treatment and death. Secondly, the time from admission until antibiotic treatment is initiated has been shown to have a certain impact on mortality 36, but such data were not available for the patients in the current study. Thirdly, it could be argued that the current authors have strayed from the original publications when using creatinine instead of urea in calculations for PSI and CURB-65. However, the results for APACHE-II and mATS scores, where urea is not included, were quite consistent with those for PSI and CURB-65.

Different case-fatality rate in a subgroup of patients despite stratification for severity of illness

Patients admitted to OD had a higher proportion of severe underlying diseases than those treated in DID, were more frequently severely ill on admission according to severity scores and, accordingly, had a higher case-fatality rate. However, the difference in mortality remained even when patients belonging to identical severity score strata were compared. One probable reason for this difference is that the presence of a severe underlying disease, such as cancer, is not included in the CURB-65 or mATS score calculations, and may have an unwarrantedly low impact on the PSI and APACHE-II scores. Further, it is also possible that the pneumococcal disease in itself may differ between patients who are previously healthy and those who have an underlying severe illness. The dominating pneumococcal serotypes were 1, 14 and 4 in DID (where there were a low percentage of patients with severe underlying diseases) and 9V, 6A and 19F in OD (where severe underlying disease was very frequent). This is in accordance with the findings of a study where the current authors and others demonstrated that some serotypes, e.g. 1 and 7F, have a high invasive potential and behave like primary pathogens, mostly infecting previously healthy people 37. In contrast, other sereotypes, e.g. 3, 4, 6A, 6B, 8, 9V, 14, 18C, 19F and 23F, seemed to be less invasive and behave like opportunistic pathogens infecting mostly people with chronic underlying disease.

The difference in case-fatality rates between the two groups indicates that neither APACHE-II nor the specific pneumonia severity scores are able to fully adjust for important differences between study populations, e.g. concerning severe underlying conditions. This, in turn, can create a risk of significant biases in noncontrolled multicentre studies of the prognosis of IPD, if sites with different population mixes are compared with each other. From a clinical viewpoint, this difference in mortality also enhances the importance of a thorough clinical assessment on admission, in addition to calculating severity scores, for the adequate management of and prognostic estimation in CAP patients.

In conclusion, the present study demonstrated that score systems developed for the assessment of community-acquired pneumonia could be used in defining severity of disease in pneumococcal pneumonia. CURB-65 is the most versatile score for use in a routine clinical setting and should be applied to all patients with community-acquired pneumonia in the emergency room. However, the current authors have also demonstrated that mortality within a certain score stratum may differ 3–4-fold depending on the patient group studied.