Abstract
The current authors developed a new prediction rule based on the five risk classes defined by the Pneumonia Severity Index to identify allocation of inpatient care in community-acquired pneumonia.
The decision to hospitalise in low-risk classes (I−III) was unquestionable, if the presence of one or more of the following were evident: arterial oxygen tension <8.0 kPa (60 mmHg), shock, decompensated coexisting illnesses, pleural effusion, inability to maintain oral intake, social problem, and lack of response to previous adequate empirical antibiotic therapy.
The results at 18 months after implementation of this new prediction rule are reported in a series of 616 patients. The mortality rate was 0.5% in 221 patients treated as outpatients versus 8.9% in 395 patients treated as inpatients. Specific additional criteria for hospitalisation included in the prediction rule were present in 106 of the 178 low-risk patients treated as inpatients, whereas in the remaining 72, the decision to hospitalise was apparently unjustified by the prediction rule. These 72 patients showed a better outcome (significantly shorter hospitalisation, days on intravenous antibiotics, mortality, and complicated course) than high-risk patients and low-risk patients who met the additional specific criteria for deciding hospital admission.
Therefore, admission in these low-risk patients might have been avoided by strict adherence to the new prediction rule. Another relevant finding was that the Pneumonia Severity Index alone did not identify all patients who needed to be admitted to the hospital.
Community-acquired pneumonia (CAP) remains a common infection with significant morbidity, mortality and a large economic burden 1. However, despite being a common disease with a well-characterised natural history and effective antibiotic therapy, an important variability in the process of healthcare has been documented 4. One of the key decisions is whether to treat patients as outpatients or as inpatients. Hospitalisation has a direct bearing on diagnostic testing, antibiotic therapy, intensity of daily control measures and costs of treating this illness. The admission decision is based on the presence of various risk factors for either death or for a complicated course of illness, although psychosocial considerations, characteristics of the healthcare system, and patient preferences may also enter into the decision to hospitalise 6. In the end, however, determination of the initial site of care remains an “art of medicine” decision 8.
In the last years, multiple studies have used multivariate analysis to develop prediction rules for outcome of CAP that could be used to assist in the initial admission decision. One approach developed by the British Thoracic Society (BTS) Research Committee, was aimed at identifying high-risk patients who not only usually require admission, but also often require admission to the intensive care unit (ICU) 9. The approach developed by the Pneumonia Patient Outcome Research Team (PORT), which may be considered complementary to the BTS risk-scoring system, was based on a Pneumonia Severity Index (PSI) that stratified patients into five risk classes. This categorisation has been extrapolated as a guide for decisions regarding hospitalisation 11. However, the utility of this mortality prediction rule in supporting admission decision has not been sufficiently assessed in prospective studies.
The purpose of this study was to validate the mortality predictive model of Fine et al. 11 in the present authors' local setting, and to evaluate outcomes after the application of the PORT prediction rule for defining inpatient care in patients with CAP. This new prediction rule includes specific additional factors available at the time of the decision to hospitalise that increase the likelihood of a poor prognosis in PSI low-risk classes. Therefore, the present authors' prediction rule is focused on defining need for hospitalisation in low-risk patients classified according to conventional PSI score, so as not to underestimate the severity of their illness.
The current study reports results at 18 months after implementation of this new prediction rule.
Methods
Setting
A prospective study was carried out in Galdakao Hospital, a 400-bed university hospital in the province of Bizkaia, Basque Country (northern Spain) serving a population of 300,000 inhabitants. This medical institution belongs to the network of public hospitals of the Basque Healthcare Service, which provides free unrestricted care to nearly 100% of the population.
Study population
The study population was comprised of a consecutive cohort of adult patients (≥18 yrs of age) with CAP admitted to the emergency service of the hospital from March 1, 2000 to September 30, 2001. Pneumonia was defined as pulmonary infiltrates on a chest radiograph not known to be old and symptoms consistent with pneumonia, including cough, dyspnoea, fever, and/or pleuritic chest pain. Patients with pneumonia were excluded if they were known to be positive for human immunodeficiency virus, chronically immunosuppressed (defined as immunosuppression for solid organ transplantation, postsplenectomy, taking ≥10 mg·day−1 of prednisone or similar drugs for >30 days, treatment with other immunosuppressive agents, or neutropaenic, i.e. <1.0×109 L−1 neutrophils), or were hospitalised during the previous 14 days.
During the study period, 616 patients were entered in the database. As an additional quality control study, possible missing cases were checked in the system, and then a retrospective review was performed by searching all cases who had a discharge diagnosis of CAP during the same period (International Classification of Diseases-ninth revision-clinical modification, codes 480.0–480.9, 481, 482.0–482.9, 483.0–483.8, 485, 486, 487.0, 507.0). Case confirmation required that the patient had an initial diagnosis of pneumonia at the emergency service. Twenty-two patients (3.4%) fulfiled these criteria and were not included in the authors' database or followed in the study. Nevertheless, demographic variables, coexisting conditions and severity characteristics, measured by PSI, of these patients, were similar to the study sample.
Criteria for admission decision
Over a 6-month period, a multidisciplinary team that included staff physicians of the service of pneumology (n=2) and the emergency department (n=2), nurses (n=3) and experts in the quality of care (n=1) were responsible for developing explicit criteria for management of CAP, including the decision to hospitalise a patient. One of the key purposes was to develop specific criteria for the treatment of CAP, in particular the decision to hospitalise. A main goal of the project was to establish a prediction rule for admission decision-making based on risk classes defined by Fine et al. 11 according to the PSI risk-scoring system. This clinical prediction rule assigns a score based on 20 items that includes demographic factors, coexisting illnesses, physical examination findings, and laboratory and radiographic findings. Five severity classes are defined; higher scores indicate more severe pneumonia. Patients with scores of ≤70 points (classes I and II) were allocated to treatment in the outpatient setting, whereas patients with scores of ≥90 points (classes IV and V) were allocated to inpatient care. In patients with scores 71−90 (class III), the admission decision remained open although outpatient therapy after an observation period at the emergency department was recommended.
The research team considered it appropriate to incorporate an additional series of criteria into the admission decision rule, so that the decision to hospitalise in low-risk classes (I–III) was unquestionable in the presence of one or more of the following factors: arterial oxygen tension (Pa,O2) <8.0 kPa (60 mmHg), shock (systolic blood pressure <12.0 kPa (90 mmHg) and requirement for vasopressors), evidence of decompensated coexisting illnesses, pleural effusion (encapsulated or layering to a depth of ≥2 cm on lateral decubitus chest radiograph), chest radiograph showing bilateral involvement, inability to maintain oral intake, social problem (patient dependent and no caregiver available), and lack of response to previous antibiotic therapy (no defervescence <38°C within 72 h of adequate empirical antimicrobial treatment). These additional criteria were selected according to a critical literature review and the authors' best judgment.
Study intervention
The protocol was developed with the participation of members of the emergency department and the pneumology service and was presented in detail to staff physicians prior to the start of the study and to resident physicians at the start of their emergency department rotation period. In order to facilitate ease of use for routine patient care, a computer program was developed in which after entering risk factors defined by Fine et al. 11, the patient was assigned to the risk class (I–V). According to the risk class and information about the presence of additional features, the resulting recommendation for the site of care decision and antibiotic therapy was generated by the program. The time required to enter information during the patient's encounter was ∼3 min. Staff and resident physicians were trained to use the computer program in the clinical setting. Antibiotic therapy followed the recommendations of the Spanish Society of Pneumology and Thoracic Surgery 12, which are similar to those of the American Thoracic Society 13. Microbiological studies to identify a specific aetiological diagnosis were also considered. The study intervention was approved by the Hospital de Galdakao ethics review board. A follow-up committee assessed the results at monthy intervals and according to data obtained, continuation of the project was recommended.
Assessment of outcome
For patients initially treated as outpatients, the research team provided a telephone number to contact a study nurse or physician and a list of primary-care providers. A follow-up telephone call from study personnel was made within 3–5 days of the emergency department visit to assess persistence of fever, respiratory symptoms, adherence to the treatment plan, ability to maintain oral intake, and daily life activities. Subsequent hospital admission and vital status within 4 weeks was recorded. For hospitalised patients, the following information was recorded: vital status at 4 weeks, ICU admission, medical complications (any clinically relevant event related to pneumonia and unrelated to medication occurring during hospitalisation), and hospital readmission within 4 weeks due to pneumonia-related complications. A “complicated course” was defined as the presence of ICU admission and/or evidence of pneumonia-related complications and/or readmission to the hospital. Length of stay (calculated as the discharge date minus the admission date), duration of intravenous antibiotic therapy, and discharge status were also recorded. An appointment was made for a control visit 30 days after hospital discharge. Clinical outcome was considered satisfactory in patients with absence of fever for the past 10 days, absence or minimal respiratory symptoms, disappearance of >75% of signs of pneumonia from the chest radiograph images, and return to work and normal daily life activities. During the first year, patients were also surveyed by telephone at 4 weeks when they were asked about satisfaction with care received and information provided using a five-point category scaling from 1 (bad) to 5 (excellent). Patients treated in the outpatient setting were asked about preference for hospital-based care (“Looking back to the time when the decision was made about where to treat you for the pneumonia, would you have preferred to be hospitalised rather than treating you as an outpatient?”).
Statistical analysis
Descriptive statistics of sociodemographic and clinical variables included frequencies, percentages, means and sds. Data from the PORT study 11 were included for comparison purposes. Comparisons of differences in variables between patient subgroups were performed using the Chi-squared test or Fisher's exact probability test for categorical variables and the analysis of variance with the Scheffe's method for multiple comparisons or the Kruskal-Wallis test for continuous variables. In order to validate the PSI score in the local setting the present authors followed recomendations made by others authors 14. This implies the use of a logistic regression model and a receiver-operating characteristic (ROC) curve to evaluate how well the model distinguished survivors from patients who died. A two-tailed p-value <0.05 was considered statistically significant.
Results
A total of 616 patients were included prospectively in the study. Demographic variables, coexisting conditions, physical examination findings, laboratory and radiological findings, and PSI risk class in these patients compared with data from the PORT study cohort 11 are shown in table 1⇓. Both cohorts showed similar characteristics, although patients in the present study were older, with a greater predominance of females, and a higher proportion of patients with Pa,O2 <8.0 kPa. The mean PSI score was 79.3 (99.5 in patients admitted to the hospital and 43.1 in patients treated as outpatients).
Demographic and clinical characteristics of patients in this study compared with the Pneumonia Patient Outcome Research Team (PORT) cohort study#
Suitability of the predictive rule developed by Fine et al. 11 was tested in the study population; W statistic of the area under the ROC curve was 0.881, indicating that the model discriminated well between survivors and patients who died. Moreover, the goodness-of-fit Chi-squared test indicated that the logistic model fitted the data adequately (p=0.493). As shown in figure 1⇓, there was a close agreement between observed and expected frequencies over the 10 deciles of risk, indicating a high calibration level between model performance and actual outcome.
Differences between observed mortality (□) and predicted mortality (└) according to the pneumonia severity index (PSI) risk class from Fine et al. 11.
A total of 36 patients died, with an overall mortality of 5.8%. There was an increase in mortality according to PSI risk classes. Among low-risk classes (I−III), only two class III patients died as a result of neurological coma and massive haemoptysis, respectively. Of the 221 (35.9%) patients who were initially treated in the outpatient setting (table 2⇓), one (0.5%) patient died (a 92-yr-old male with poor clinical condition and PSI class IV, who was transferred home upon family request). In addition, five (2.3%) patients were subsequently readmitted to the hospital, although no complicated course or mortality within 4 weeks was recorded. The mortality rate in 395 (64.1%) patients allocated to inpatient care was 8.9% (35 patients). In this subgroup of inpatient care, mortality in high-risk classes was significantly higher than in low-risk classes (p<0.001). Thirty-eight (9.6%) patients presented a complicated course, without significant differences according to risk classes.
Outcomes according to risk class
Of the 395 patients admitted to the hospital, 178 (45%) were assigned to low-risk classes (I−III). Inpatient care was justified in 106 (60%) of patients because criteria included in the prediction rule for admission decision were present. In the remaining 72 (40%) patients, although specific criteria for hospitalisation were absent, they were hospitalised according to the personal judgment of the emergency physicians (table 3⇓). As shown in table 4⇓, mortality rate was significantly higher in high-risk classes (IV and V) compared with low-risk classes (I−III) independently of whether or not specific criteria for deciding hospitalisation were present. However, in the low-risk subgroup in which specific criteria for inpatient care were present, the percentages of patients with complicated course, the duration of intravenous antibiotic therapy, and the length of stay were similar to high-risk patients. In contrast, low-risk patients for whom specific criteria for inpatient care were absent showed significant differences (p<0.05) compared with both high-risk classes and low-risk classes with the specific additional criteria for deciding hospital admission, with respect to length of stay, days on intravenous antibiotic therapy, complicated course, and pneumonia-related complications (table 4⇓). Clinical outcome at 30 days was considered satisfactory for 99.1% of patients treated in the outpatient setting and for 89.6% of patients allocated to inpatient care.
Criteria for deciding inpatient care in low-risk class patients
Outcomes in patients admitted to the hospital according to presence of absence of specific criteria to hospitalise
At the 4-week telephone survey, data from 307 (78.3%) patients were available. Care received and information provided was rated as “very good” or “excellent” by 75.6% of the patients, without significant differences between patients treated as inpatients or outpatients. Conversely, when 130 patients treated in the outpatient setting were asked about preference for hospital-based care, 84.2% of the 120 respondents preferred to be treated as outpatients.
Discussion
The current study, like others 16–20, shows the suitability of the risk factors for death of the PSI-scoring system in the present authors' local setting. Other authors have shown that pre-defined PSI scores have a good discriminant capacity but risk of death is overestimated 21. In order to solve the problem, study-specific recalibration of the PSI-scoring system has been proposed, which was not necessary in the current study.
The present results also show that strict use of the PSI-scoring system as a prediction rule for deciding inpatient care would not allow the identification of a subset of low-risk patients (I–III) that presented a complicated clinical course. Including additional specific criteria based on adverse circumstances for low-risk classes in the prediction rule reported here, allowed allocation of a group of patients to be treated as inpatients, although they had a low mortality rate, developed important pneumonia-related complications.
The admission decision for high-risk class patients (IV and V) seems justified due to the severity of illness and the estimated mortality of >5% 22. In this cohort, all these patients were admitted except one patient who refused, and the mortality rate was 15.4%. The low-risk class patients have a lower mortality, but according to data from different studies, this subgroup of patients accounts for ∼40% of all hospital admissions in patients with CAP 4. This indicates that there are additional reasons other than variables included in the PSI-scoring system for deciding inpatient care. There are some authors who justify these admissions by a retrospective review of the circumstances of patients at the time of admission decision-making 16. In the new decision rule reported here, prospective specific criteria for deciding site of care were included. Some of these criteria, such as the inability to maintain oral intake, social problem, a decompensated coexisting condition, or shock, can directly justify treatment in the inpatient setting, whereas prognostic implications of other features, such as pleural effusion, chest radiograph showing bilateral involvement, or Pa,O2 <8.0 kPa have been extensively documented 13.
The possibility, suggested by Fine et al. 11 of treating low-risk patients in the outpatient setting was subsequently confirmed in an intervention trial 26, in which the use of the PSI score coupled with enhanced outpatient services effectively identified low-risk patients with CAP in the emergency department, and safely increased the proportion initially treated as outpatients. In this study, however, patients aged ≥85 yrs, unable to take oral medication, or with a Pa,O2 <8.0 kPa were not eligible, but hospital admission could have been justified in the presence of any of these factors even in low-risk patients. In the present study, there were 72 low-risk patients, none of whom died or presented a complicated course, who were treated as inpatients without any apparent reason. It is likely that this group of patients could have been treated in the outpatient setting, which in turn may indicate that hospital admission in these patients might have been avoided by strict use of the new prediction rule. Accordingly, the percentage treated as inpatients initially decreased from 64.1% to 52.4% in the total series of patients, and from 45% to 35.8% in the low-risk subgroup. In this respect, theoretically, the present results would be consistent with an 18% decrease in the admission of low-risk patients reported by Marrie et al. 27 after implementation of a critical pathway at the institutional level. The current authors' consider this an area of improvement.
Operativity is another important aspect of a prediction rule. Variables included in the PSI-scoring system are clearly defined and can be easily assessed and available at the time of the admission decision. The usefulness of computers and clinical information systems to support clinicians in real-time during the decision-making process has already been emphasised 28. In the computer program developed in the present study, the required information can be entered in ∼3 min; the program then assigns the patient a risk class, selects the appropriate antibiotic regimen, and allocates an optimal site of care. However, successful guideline implementation in a clinical setting includes strategies that target not only the disease, but also include other forces that significantly influence the admission decision. In a study that measured adherence with an actively implemented guideline to reduce hospitalisations for patients coming to the emergency department with CAP 7, overall nonadherence with the guideline was 43.6%. Lack of adherence was associated with a variety of patient, system, and physician factors 7. In the present study, 3.4% of patients were excluded from the study because specific criteria for deciding inpatient care were not applied, and 11.7% of all patients included in the study were admitted to hospital without any apparent reason and without fulfilment of the guideline recommendations.
Several limitations are worth noting. First, there are a number of prediction rules or guidelines for admission decision-making. In the new prediction rule desribed here, the criteria proposed by Fine et al. 11 was used because the predictor variables are all explicitly defined and can be readily assessed at the time of patient examination. However, up to the present time, there is no evidence regarding the superiority of one prediction rule over the others. Second, the patient's (or family relatives) preference in the triage decision was not considered. Prediction rules neglect the importance of patient preferences in clinical decision-making. This point is highlighted by the observations that the vast majority of low-risk patients with CAP do not have their preferences for site of care solicited, even though many have a strong preference for outpatient care 29. In the present study, 84.2% of patients treated in the outpatient setting, surveyed at 30 days, stated they would have preferred home care. However, preferences expressed during the period of recuperation may be different from those expressed at the time of diagnosis or during the acute phase of treatment. Third, mortality was measured at 30 days, similar to other reports, although recent studies have found an association of mortality and CAP within 45 days of presentation 30. Finally, the special additional criteria included in the new prediction rule to identify inpatient care in low-risk (I–III) patients do not represent an exhaustive list. Other clinical factors not included in the present guidelines may also have important contraindications to outpatient care but, in the present authors' opinion, the most frequent and the most relevant factors have been considered when developing the prediction rule used in the current study. Even so, future studies should evaluate if some of those additional criteria are needed or, if others should be included.
The present authors conclude that prediction of severity of illness using the model developed by Fine et al. 11 was consistent with the results found in the present authors' healthcare system, which supports the use of this prediction rule in the authors' environment. This prediction rule based on the incorporation to the Pneumonia Severity Index-scoring system of specific criteria available at the time of admission decision-making, proved to be appropriate to identify the need for inpatient care for a subgroup of low-risk patients (I–III), with unfavourable prognosis to be managed in the outpatient setting. An important aspect of this study, is that 106 of 178 patients, who are defined as low risk by the Pneumonia Severity Index, actually needed admission to the hospital. However, in the absence of any of these specific additional risk factors, hospital admission for the management of low-risk patients does not seem to be justified. The usefulness of this new prediction rule in different patient populations and healthcare systems warrants further investigation.
Acknowledgments
The authors would like to thank A. Chueca for data collection, E. Puerta for assistance in the development of the guideline, and M. Pulido for editorial assistance. Cooperation of residents and staff physicians from the emergency department and the pneumology service is greatly acknowledged.
- Received July 3, 2002.
- Accepted October 28, 2002.
- © ERS Journals Ltd
References
