A new conceptual framework for ICU performance appraisal and improvement☆,☆☆

https://doi.org/10.1053/jcrc.2002.33033Get rights and content

Abstract

Purpose: This study examined the use of outcomes for the purposes of ICU evaluation and improvement. We reviewed the strengths and weaknesses of an outcomes-centered approach to intensive care unit (ICU) evaluation and present a more comprehensive conceptual framework for ICU evaluation and improvement. Materials and Methods: Data was collected from 2 sources: (1) a structured review of the literature, with relevant articles identified using Medline, and (2) 85 semistructured interviews of health care professionals (eg, physicians) and health care administrators (eg, chief executive officer). The interviewees came from 4 institutions: a 900-bed East Coast teaching medical center, a 600-bed East Coast teaching medical center, a 590-bed East Coast teaching medical center, and a 435-bed West Coast private community hospital. A nonrandomized, purposeful sample was used. Results: A conceptual framework for ICU evaluation is presented that identifies and defines 3 different types of variables: performance (eg, appropriateness of care, effectiveness of care), outcome (eg, resource use, mortality), and process (eg, timeliness of treatment, work environment).The framework emphasizes performance variables and the relationships between performance, outcome, and process of care variables, as a logical focus for ICU evaluation and improvement. Conclusions: Performance variables offer distinct advantages over outcome variables for ICU evaluation. Their use, however, will require additional development of current evaluation tools and methods. They provide the ability to identify the value an ICU adds to patient care in a hospital or to an episode of illness, and to evaluate integrated systems for providing care. Copyright 2002, Elsevier Science (USA). All rights reserved.

Section snippets

Study overview

The study progressed over 2 distinct phases: (1) use of literature review data to construct the 3 sets of variables described earlier, and (2) interviews of health care professionals and administrators to further refine and validate these variables and their organizing structure.

Literature extraction

Identification of the initial performance, outcome, and process variable sets was based on: (1) a Medline-directed literature review; and (2) seminal works in health care performance and quality measurement theory. A

A multi-attribute definition of performance

When evaluating an ICU, a hospital, or a health care network, no single performance variable can adequately characterize all important aspects of performance. To be able to meet the diverse needs of the many constituents that are involved in evaluating the delivery of health services, a multi-attribute definition of performance is required. Table 1 presents a minimum set of unique performance variables and provides an example of an approach to the measurement of each: (1) appropriateness of

Discussion

Variables that measure performance have appeared in the literature, though they are not always labeled as such. One example was presented as a measure of performance from an employer's perspective. This was defined as the sum of patient health status impact plus customer service quality, divided by the total costs of care and illness.37 This performance variable is a ratio of 3 concepts, each of which must be defined using numerous outcomes. Another performance variable was presented as a ratio

Conclusion

The most common current method to evaluate an ICU is to measure and compare individual outcomes. However, an unresolved issue is how information provided by the numerous outcomes that are commonly measured can be used effectively for this purpose. This is an important issue, especially when one considers that the rankings of ICUs, based on individual outcomes, can change dramatically depending on which outcome is compared. That is, some ICUs are better if judged by one out-come and worse if

References (68)

  • JK Iglehart

    Health policy report: The American health care system

    N Engl J Med

    (1992)
  • ME Campion

    The Olmsted County Benchmark Project: Primary study findings and potential implications for corporate America

    Mayo Clin Proc

    (1992)
  • KM Eisenhardt

    Building theories from case study re-search

    Acad Manage Rev

    (1989)
  • A Donabedian

    The quality of care: How can it be assessed?

    JAMA

    (1988)
  • BD Browne

    The rigors and rewards of clinical evaluation in an alliance of academic health systems

    Qual Manage Health Care

    (1996)
  • HA Miles

    Qualitative Data Analysis: An Expanded Sourcebook

    (1994)
  • GS Shaller

    Setting benchmarks for cost-effective care

    Business Health

    (1986)
  • P Jacobs et al.

    National estimates of intensive care utilization and costs

    Crit Care Med

    (1990)
  • JR Lave et al.

    The economics of intensive care units, in Anonymous medical aspects of critical care

  • JL Bams et al.

    Outcome and costs of intensive care

    Intensive Care Medicine

    (1985)
  • WJ Sibbald et al.

    How can new technology be introduced, evaluated and financed in critical care?

    Clin Chem

    (1990)
  • PMD Kalb

    Utilization strategies for intensive care units

    JAMA

    (1989)
  • DC Angus et al.

    Organization and management of critical care systems in unconventional situations

    Crit Care Clin

    (1993)
  • JE Zimmerman et al.

    Improving intensive care: Observations based on organizational case studies in nine intensive care units: A prospective, multi-center study

    Crit Care Med

    (1993)
  • JR Le Gall et al.

    A new Simplified Acute Physiology Score (SAPS II) based on a European/North American multicenter study

    JAMA

    (1993)
  • D Teres et al.

    Using severity measures to describe high performance intensive care units

    Crit Care Clin

    (1993)
  • LI Iezzoni

    Judging hospitals by severity-adjusted mortality rates: The influence of the severity adjustment method

    Am J Public Health

    (1996)
  • M Shwartz

    Do severity measures explain differences in length of hospital stay?

    Health Serv Res

    (1996)
  • JW Thomas

    Does risk-adjusted readmission rate provide valid information on hospital quality?

    Inquiry

    (1996)
  • A Donabedian

    The epidemiology of quality

    Inquiry

    (1985)
  • DM Berwick

    Health services research and quality of care: Assignments for the 1990's

    Med Care

    (1989)
  • A Donabedian

    Explorations in Quality Assessment and Monitoring: The Definition of Quality and Approaches to its Assessment

    (1980)
  • CS Cleeland et al.

    A model for the treatment of cancer pain

    J Pain Symptom Manage

    (1986)
  • S Katz

    The Portugal Conference: Measuring quality of life and functional status in clinical epidemiological research

    J Chronic Dis

    (1987)
  • JE Ware et al.

    The MOS 36-item short-form health survey (SF-36). I. Conceptual framework and item selection

    Med Care

    (1992)
  • P Suter et al.

    Predicting out-come in ICU patients

    Intensive Care Med

    (1994)
  • DS Salkever et al.

    Episodes of illness and access to care in the inner city: A comparison of HMO and non-HMO populations

    Health Serv Res

    (1976)
  • JA Solon et al.

    Delineating episodes of medical care

    Am J Public Health

    (1967)
  • J Yergan

    Health status as a measure of need for medical care: A critique

    Med Care

    (1981)
  • LA Aday

    Health Care in the US: Equitable for Whom?

    (1980)
  • WC Richardson

    Measuring the urban poor's use of physician services in response to illness episodes

    Med Care

    (1970)
  • JM Krumberger

    Linking critical care family research to quality assurance

    AACN Clin Issues Crit Care Nursing

    (1991)
  • JE Turnbull

    Consumer information, patient satisfaction surveys, and public reports

    Am J Med Qual

    (1996)
  • MR Gold

    Health maintenance organizations: Structure, performance, and current issues for employee health benefits design

    J Occup Med

    (1991)

    • Cited by (16)

    • Barriers and supportive conditions to improve quality of care for critically ill patients: A team approach to quality improvement

      2015, Journal of Critical Care
      Citation Excerpt :

      Different frameworks focusing on the successful implementation of research into practice have been developed. Some have focused on QI in specific areas (eg, the ICU), whereas others have aimed to facilitate different kinds of dissemination methods (eg, guideline dissemination using QI collaboratives [30–33]). However, no frameworks have yet focused specifically on the potential determinants of team success in QI collaboratives [11].

    • Application of analytic hierarchy process for measuring and comparing the global performance of intensive care units

      2005, Journal of Critical Care

    • Monitoring outcomes with relational databases: Does it improve quality of care?

      2004, Journal of Critical Care

    • Critical care performance measurement: The time has come for all

      2006, Critical Care Medicine

    • Low risk monitoring in neurocritical care

      2018, Frontiers in Neurology

    • Why try to predict ICU outcomes?

      2014, Current Opinion in Critical Care
    View all citing articles on Scopus
    ☆

    This work was performed at the University of Pittsburgh.

    ☆☆

    Address reprint requests to Armando J. Rotondi, PhD, University of Pittsburgh, Room 612B Scaife Hall, CCM, 3550 Terrace St, Pittsburgh, PA 15261.

    View full text
    Copyright © 2002 Elsevier Science (USA). All rights reserved.