Review Article
Application of Rasch analysis in health care is increasing and is applied for variable reasons in mobility instruments

https://doi.org/10.1016/j.jclinepi.2010.02.012Get rights and content

Abstract

Objective

To identify the frequency of Rasch analysis use in health instrument development or refinement and the characteristics of Rasch application in mobility scales.

Study Design and Setting

The entire databases of Medline, CINAHL, PEDro, EMBASE, Cochrane Central Register of Controlled Trials, and Cochrane Database of Systematic Reviews were searched until January 2009. Articles that reported the development or refinement of health instruments using Rasch analysis were included. Of the 234 articles that met inclusion, 10 were categorized as “mobility” instruments. Data were extracted relating to each instrument and the use of Rasch analysis in the development or refinement of the instruments.

Results

The number of articles reporting the use of Rasch analysis of health instruments is increasing, from 1 article in 1987 to 48 articles in 2007. Of the 10 mobility instruments examined, the primary reason Rasch was used varied. Reasons included assessing instrument unidimensionality, differential item functioning, rating categories, item hierarchy, and redundant items.

Conclusion

The application of Rasch analysis in health instrument development has markedly increased in recent years. However, few mobility instruments have been developed or refined using Rasch analysis. The reasons that the Rasch model was used varied across mobility instruments.

Introduction

What is new?

Key finding:

  1. Rasch analysis offers many features that facilitate the development and refinement of health instruments. The use of Rasch improves the confidence in instrument utility.

  2. The number of articles reporting the use of Rasch analysis in health instruments is increasing.

What this adds to what is known?
  1. Rasch analysis has been applied in the development or refinement of 10 mobility instruments.

What is the implication, what should change now?
  1. The key features of the Rasch model have been explored to facilitate future test developers in developing high-quality health outcome measures.

A primary goal of scale development is to create a valid measure of an underlying construct [1]. In the context of health, outcomes are measured to assist decision making in patients’ clinical management. Outcome measures can be used to predict which patients will benefit from a particular intervention and to document where the patient improves or declines after time or after an intervention is applied [2].

The development of a high-quality instrument involves multiple important steps. The first step is to develop a precise and detailed notion of the target construct [1]. Once the construct of interest is clearly defined, the next step is to create an item pool. The fundamental goal of this process is to systematically sample all contents that may be potentially relevant to the target construct. The logic being that psychometric analysis can identify poor quality items that should be removed from the scale, but it cannot identify items that should have been included but were not [1]. After the initial item pool is complete, these items need to be pilot tested and any items that prove to have practical limitations removed. The items are then tested in a larger sample population. Analysis of the clinimetric properties of the data is then undertaken, assessing validity and reliability.

Two currently available methods for assessing instrument unidimensionality are Rasch analysis, based on a modern test theory approach, and factor analysis, based on a classical test theory approach. The advantages of Rasch analysis have been well documented in the literature [3].

The Rasch model was developed by Georg Rasch for the investigation of reading ability in 1952 [4]. The Rasch model is a probabilistic model that states that an item response is a result of an interaction between person ability (e.g., level of mobility) and item difficulty (e.g., difficulty of mobility task) [5]. If data fit the model, the scale is defined as being unidimensional.

Alternatively, if data do not fit the model, this can be for a range of reasons. A common cause is that instrument items may be measuring another construct, and Rasch analysis allows these items to be identified. When scales are multidimensional, summing of item scores may cause misleading assumptions to be made. The Barthel Index, for example, is a multidimensional scale, as Rasch analysis has identified that both mobility and continence items exist [6].

Rasch analysis allows estimation of the intervals between ordinal items. Fitting of data to the Rasch model places item and person parameters on the same logit scale, providing a linear transformation of the raw score [7] facilitating interpretation of change scores. Some ordinal scales approximate interval scales as the relationship between the raw score and the Rasch-converted measure is almost linear, but this relationship often deteriorates toward the extremes of the scales [8]. The clinical implication of this being that it can be more or less difficult to achieve a change in score on an ordinal scale depending on which part of the scale spectrum the person commenced. Therefore, another advantage of Rasch analysis is that it provides clinicians with a more accurate level of measurement.

Rasch analysis also assists in the development and refinement of scales by identifying the persons with ability located above the hardest item or below the easiest item on the logit scale (i.e., identification of floor and ceiling effects). This is important as floor and ceiling effects are common in instruments that measure mobility [9]. In addition, the Rasch item hierarchy can also identify items of similar difficulty (item clustering), which can then be removed. Item removal, however, must be approached with caution as argued by Bohlig et al. [10].

Rasch analysis also facilitates the assessment of differential item functioning (DIF). DIF occurs when persons of the same ability have items that operate differently based on another variable, such as age or gender. Assessment of DIF is important as it improves generalizability of the instrument by testing that item response patterns are similar across, for example, different genders, age groups, or times of assessment. For instance, if men and women respond systematically differently on a particular item, this influences the ability to compare the total scores on the construct of interest across genders. Although the ability to assess DIF is not unique to only Rasch analysis, it is nevertheless a very useful feature with important clinical implications.

Rasch analysis also facilitates the investigation of item thresholds. In the RUMM2020 software (RUMM Laboratory, Perth, Western Australia, Australia) [11], for example, item thresholds exist where the probability of an item response category is equal to another for a particular level of person ability [5]. If the probability of each item response category is not in the expected order, this results in a disordered threshold. For example, the response options of “unable,” “assistance,” “supervision,” or “independent” are common for walking items. There may be, for example, no person ability for which it is most likely that a person required “supervision.” Therefore, the “supervision” category can be identified as redundant and may be removed or combined with another response option. Instrument and item misfit can be caused by the existence of disordered thresholds [5].

Rasch analysis provides a method for obtaining instrument construct validity. Measuring the ability of a person on a construct such as mobility is difficult as it can only be inferred by observing actions that are considered representative of the construct [12]. Although three-dimensional gait analysis is often considered the gold standard for measuring mobility [13], it has obvious limitations such as access to a gait laboratory and inability to use it in everyday clinical practice for assessment of mobility. Therefore, Rasch has a role in developing high-quality mobility outcome measures.

It has been suggested that mobility is gained and lost in a hierarchical fashion [14], and therefore, this construct was considered well suited to Rasch analysis. Mobility is also a fundamentally important construct for the clinical practice of physiotherapists and is an important health indicator.

Although there has previously been a systematic review of the application of Rasch in the rehabilitation outcome measures conducted [12], there has not been a review of the ways Rasch analysis has been applied in developing or refining health instruments. Therefore, the aims of this review were to identify the frequency that Rasch analysis had been used in health instrument development or refinement and to identify the characteristics of Rasch application in the development or refinement of mobility scales.

Section snippets

Inclusion and exclusion criteria

Freely obtainable published articles or unpublished papers that had applied Rasch analysis in the development or refinement of health instruments were included. Instruments were considered to have been developed with Rasch if Rasch was applied in the initial instrument development and were considered refined with Rasch if Rasch were applied to modify an existing scale. If an instrument had been examined with Rasch but not modified, it was not considered refined. Articles were excluded if not

Results

The total database search yield was 1,565. After deleting 1,331 articles based on title and abstract, 234 articles remained and were then categorized as either “mobility” or “other” instruments (Fig. 1). Hardcopies were obtained to facilitate the categorizing of articles if this was unclear based only on title and/or abstract.

Two hundred thirty-four articles met inclusion criteria. Fig. 2 shows that the prevalence of Rasch analysis in the application of health outcome measurement has increased

Discussion

This review has identified that the application of Rasch analysis in health outcome measures has increased markedly over the past decade. Although the Rasch model was initially designed for the field of education, Rasch analysis is now used broadly across many differing areas of health. This is likely to be attributable to the increasing demand for the use of outcome measures in health care and the many clinical useful features that Rasch analysis offers for health instrument development and

Conclusion

This review has demonstrated the variety of ways that Rasch analysis has been used in the development or refinement of mobility instruments. Uses of the Rasch model varied across instruments and depended on the objectives of the authors. This review has also confirmed the increasing use of Rasch analysis in the development and refinement of health instruments.

References (30)

  • B.D. Wright et al.

    Observations are always ordinal; measurements, however, must be interval

    Arch Phys Med Rehabil

    (1989)
  • S. Davenport et al.

    What instruments have been used to assess the mobility of community dwelling older adults?

    Phys Ther Rev

    (2008)
  • M. Bohlig et al.

    Content validity and misfitting items

    Rasch Meas Trans

    (1998)
  • D. Andrich et al.

    RUMM2020

    (2003)
  • L. Tesio et al.

    Rehabilitation and outcome measurement: where is Rasch analysis-going?

    Eura Medicophys

    (2007)
  • Cited by (0)

    View full text