Abstract
This study assessed the accuracy of a cardiorespiratory monitoring device versus polysomnography for the diagnosis of suspected sleep apnoea/hypoponea syndrome (SAS).
A total of 86 patients (89% male, mean age 52 yrs) that had been referred to a sleep laboratory with a clinical diagnosis of SAS underwent cardiorespiratory polygraphy in an unattended mode using an ambulatory device (MERLIN). Analysis was carried out both automatically and manually. Conventional overnight full-channel polysomnography was performed simultaneously.
Valid polygraphical recordings were obtained from 79 patients. The mean±sd apnoea/hypopnoea index (AHI) was 34.4±29.2. The results obtained with manual scoring were superior to automatic scoring for all AHI thresholds. For an AHI of ≥5, which is diagnostic SAS, the optimum cut-off value for the manual respiratory event index was 6.7 and the cardiorespiratory monitoring device had 97.1% sensitivity and 90.9% specificity. Correct classification according to the different cut-off points obtained via polysomnography and the corresponding cut-off points in the MERLIN manual index were confirmed in 90–96% of patients.
The MERLIN device is a useful diagnostic approach for the initial assessment of adult patients with clinical suspicion of sleep apnoea/hypopnoea syndrome. Manual scoring is clearly better than automatic scoring in terms of agreement with the apnoea/hypopnoea index and to discern patients with sleep apnoea/hypopnoea syndrome.
This study was supported by a grant from the Dept of Health, Basque Government.
The guidelines from most scientific societies 1–3 state that overnight full-channel polysomnography, which is an attended recording of sleep stage, respiratory and other parameters, is the gold standard diagnostic investigation of sleep apnoea/hypopnoea syndrome (SAS). However, this procedure is complicated, time-consuming, expensive and is not readily available in most hospitals. It is widely accepted that cardiorespiratory polygraphy, without recording neurophysiological variables, is a less costly and simplified alternative to full-night polysomnography for SAS diagnosis 2–3. This, and the high prevalence of sleep-related breathing disorders in the general population 4–6, has led to the development of portable recording devices and the generalised use of cardiopulmonary sleep studies among the specialists. However, validation of cardiorespiratory monitoring systems that can be used outside the sleep laboratory in an unattended setting, is essential 7. Despite the growing clinical use of cardiorespiratory polygraphy in the assessment of sleep apnoea, the evidence available concerning their diagnostic accuracy is inconclusive 8. Although reports have shown that these monitoring devices reliably detect abnormalities in the breathing pattern in selected patients, many currently available systems have not been sufficiently standardised. Further validatory data is therefore required before this approach can be recommended for widespread clinical use 1.
The aim of this study was to assess the overall sensitivity and specificity of such a cardiorespiratory polygraphy system (MERLIN; Healthdyne Technologies, Marietta, GA, USA) in patients with clinical suspicion of SAS using standard polysomnography as a gold standard.
Methods
Patients
In the period between July 1997 and March 1998, 86 patients with clinically suspected SAS from a sleep outpatient clinic were referred to the sleep laboratory of the Txagorritxu Hospital in Vitoria-Gasteiz, Spain, for overnight polysomnography. Clinical suspicion of SAS was established using a standardised questionnaire administered in the clinical interview. The questionnaire evaluated the presence of snoring, breathing pauses, daytime hypersomnolence and other clinical parameters. Conventional overnight full-channel polysomnography and cardiorespiratory polygraphical recordings were performed simultaneously on all patients. Analysis of recordings obtained by polysomnography was performed by a neurophysiologist with broad experience in the diagnosis of SAS, whereas a different experienced neurophysiologist carried out the readings of the polygraphy system. The observers were blind to the results of the other method. The institutional review board of the Txagorritxu Hospital approved the study and written informed consent was obtained from all participants.
Equipment
The polysomnography consisted of continuous polygraphical recordings for an entire night, from 22:30 to 07:00 h with standardised equipment (Alice 3; Healthdyne Technologies) or the Ultrasom system (Nicolet Biomedical Inc., Madison, WI, USA), from surface leads for electro-encephalography (C4/A1 and C3/A2 placements), electro-oculography, tibial and submental electromyograms, and electrocardiogram (modified V2 lead). For respiratory sensors, nasal and oral signals by thermistors were used, tracheal sounds (microphone) and the chest and abdominal effort was measured by two belt sensors (Healthdyne piezo-electric gauge; Healthdyne Technologies). Oxyhaemoglobin saturation was recorded by finger-pulse oximeter (model 340; Palco Laboratories, Santa Cruz, CA, USA) and the body position was monitored by the system. Each 30-s epoch of the recording was scored for sleep stage, breathing, oxygenation, and movement. Sleep data were staged according to the system described by Rechtschaffen and Kales 9. A complete cessation of the thermistor signal of ≥10 s was defined as apnoea. Hypopnoea was defined as a discernible reduction of ≥50% of the thermistor signal for ≥10 s, accompanied by a decrease of ≥3% in oxyhaemoglobin saturation and/or an electro-encephalographic arousal. An arousal was defined according to the American Sleep Disorders Association 10.The total number of scored apnoeas and hypopnoeas divided by the number of hours of sleep, referred to as the apnoea/hypopnoea index (AHI), was determined for each participant.
The MERLIN system is a cardiorespiratory polygraph for level III studies 2. The system can be used for both attended (on-line) and unattended studies. The unattended mode was selected in the present study. The system records oronasal airflow by thermistor, chest and abdominal respiratory movements, tracheal sounds, cardiac frequency, oxygen saturation, body position, and continuous positive airway pressure (CPAP) level. The beginning of the recording was programmed with a personal computer. Recordings were scored automatically by software included in the system or manually by visual evaluation of printouts. In all patients, sensors of the polysomnographical equipment were placed first. Cardiorespiratory polygraphical recordings were programmed to start some minutes later than initiation of polysomnographical testing. Respiratory events included apnoeas and hypopnoeas. The criteria of manual scoring were the same as that used with the polysomnography. The respiratory event index was calculated as the sum of the number of episodes of apnoea and hypopnoea per hour of polygraphical recording both in the automatic and manual analysis of data.
Statistical analysis
Agreement between AHI and automatic and manual respiratory event indexes obtained with the cardiorespiratory polygraphy was assessed using the method described by Bland and Altman 11. The mean value of measurements with the two methods was plotted on the abscissa, and the difference between the two values was plotted on the ordinate. The bias was estimated by the mean difference. The limits of agreement between the two methods were calculated as the mean bias±1.96×sd of the individual differences. The diagnostic accuracy of polygraphical findings for the AHI cut-off values of ≥5, ≥10, ≥15, ≥20, and ≥30 were evaluated. The discriminatory ability of the cardiorespiratory polygraphical scores for the diagnosis of SAS was evaluated using the receiver operating characteristics (ROC) curves 12. The best cut-off point for balancing sensitivity and specificity of the test over the range of AHI thresholds was determined. Positive and negative predictive values and 95% confidence intervals (CI) for each cut-off point were calculated.
Results
Sleep studies were invalid for analysis in seven (8%) of the 86 patients because of a sleep time of <240 min (n=4), lack of a thermistor signal (n=2), and incomplete recording due to technical problems for storing data in the computer (n=1). Preperation of the equipment and the time taken to attach the MERLIN polygraphical device to the patient was ∼20 min compared to ∼45 min for polysomnography. Conversely, the time needed for manual scoring using the MERLIN unit was∼30 min, whereas full polysomnography took ∼120 min. The mean±sd duration of polysomnography was 377.8±50.8 min as compared with 377.6±58.1 min of cardiorespiratory polygraphy. The mean difference±sd for the start time of these studies was 6.07±9.4 min.
Valid polygraphical recordings were obtained in 79 patients (89% male) with a mean±sd age of 52±11.1 yrs and a mean body mass index (BMI) of 30.1±4.4 kg·m−2. The mean AHI was 34.4±29.2. Eighty-one per cent of patients had an AHI of ≥10 (mean AHI 41.8±27.7). Results from MERLIN polygraphy and conventional polysomnography by AHI category are shown in table 1⇓. The mean value of the respiratory event index obtained by automatic analysis in the group of patients with an AHI of <10 was higher than that obtained by polysomnography, i.e. 5.3±5.1 versus 3.1±2.1, whereas the mean value obtained by manual analysis in the group of patients with an AHI of ≥10 was lower than that recorded by polysomnography, i.e. 36.4±23.4 versus 41.8±27.7. Results of cardiorespiratory polygraphy showed a tendency for overestimating mild SAS and underestimating severe SAS cases.
Results of cardiorespiratory polygraphy and polysomnogram by apnoea/hypopnoea index (AHI) category
The mean of the individual differences between polysomnographical AHI and the polygraphical respiratory event index is shown in figure 1⇓. The mean value was −23.98±30.2 for automatic scoring compared with −3.98±14 for manual scoring. As shown in figure 1a⇓, individual differences between AHI and polygraphical respiratory event index in the manual analysis showed a distribution of points towards a horizontal line, as opposed to the straight downward line of points in the automatic analysis (fig. 1b⇓).
The individual differences between polysomnographic apnoea/hypopnoea index (AHI) and polygraphical respiratory event for a) automatic scoring, b) manual scoring. The mean () and the mean±1.96 sd (═).
Table 2⇓ summarises the values of the area under the ROC curve for the automatic and manual MERLIN polygraphical scores when considering different AHI thresholds. The discriminatory ability of the manual scores was greater than that observed with automatic scores for all AHI cut-off points ranging from ≥5–≥30. The area under the ROC for an AHI of ≥5 was 0.976 in the manual scoring versus 0.818 in the automatic scoring. The corresponding figures for an AHI of ≥30 were 0.931 and 0.622, respectively. The best results of the area under the ROC were obtained for an AHI of ≥5. Table 3⇓ shows the validity of cardiorespiratory polygraphical scores corresponding to the optimal cut-off points selected from each ROC curve using different criteria for SAS. When considering the use of MERLIN recording as an exclusion test for an AHI of ≥5 as a diagnostic tool for SAS, the best cut-off point for the manual respiratory event index was 6.7, with a sensitivity of 97.1% (95% CI, 97–100%). Accordingly, a patient with clinical suspicion of SAS and a MERLIN score <6.7 would have a low probability for an AHI of >5, with a sensitivity of 97% and negative predictive value of 83.3%. When using the MERLIN system as a confirmation test, the best cut-off points were 27.6 for an AHI of ≥30, with a specificity of 90.9%. In this case, a patient with clinical suspicion of SAS and a MERLIN score of >27.6 would have a high probability for an AHI of ≥30, with a specifity of 90.9% and positive predictive value of 88.6%. As shown in table 4⇓, when patients were classified according to the identification or exclusion of patients with an AHI of <5 or ≥5 (MERLIN cut-off point 6.7), there were 76 patients correctly classified with the polygraphic device (96.2%). Moreover, distribution of patients by an AHI of <30 or ≥30 (MERLIN cut-off point 27.6) resulted in the correct classification of 71 patients (89.9%).
Area under the receiver operating characteristics (ROC) curve (95% confidence interval in parenthesis) of cardiorespiratory polygraphic scores in sleep apnoea/hypopnoea syndrome
Best cut-off points of cardiorespiratory polygraphic scores in sleep apnoea/hypopnoea syndrome (SAS) for different polysomnographic diagnostic criteria
Classification of patients according to two cut-off points in the polysomnography and their corresponding best cut-off points in the MERLIN manual index of respiratory events
Discussion
Polysomnography is the current gold standard for the diagnosis of sleep-disordered breathing 1–3 but the use of polysomnographical sleep studies as the only diagnostic method in all suspected cases of SAS may not be justified 13–16 and in this respect, various scientific societies have accepted the complementary role of cardiorespiratory polygraphy in the initial diagnostic approach of SAS 2, 3. The lower diagnostic yield of cardiorespiration compared with polysomnography infers that the diagnosis of SAS, based on any simplified method, should be assessed in terms of probability and subsequently validation studies of the method to be used in clinical practice are crucial.
Clinical outcome studies using polysomnography as the gold standard 2–3, that ultimately determine the usefulness and reliability of portable recordings, are infrequent compared with the substantial number of commercially available recording equipments. The devices Medilog 17, Vitalog PMS-8 18, SCSB 19, MESAM IV 20, MicroDigitrapper-S 21, PolyG 22, CID-102 23, and POLY-MESAM 24 are the most well standardised and validated. However, despite this monitoring systems have been used for measurement of sleep-related breathing disturbances in epidemiological studies 25 as well as for titrating CPAP for the treatment of SAS 26. Furthermore, because such recording devices are evolving rapidly, reports of efficacy often refer to devices that have been modified by the manufacturer and (at least in the form tested) are no longer available. In a systematic review of the literature prepared by the Agency for Health Care Policy and Research (AHCPR), which was pertinent to diagnostic testing and screening in sleep apnoea in adult patients 8, 25 studies of portable monitoring devices (1,631 patients) were evaluated. Sensitivity ranged from 32–100% and specificity from 33–100%. Because of this variability, comparison across studies is difficult and it is recommended that all sleep monitoring units that are proposed as prequalifiers or replacements for polysomnography must be validated in the setting in which they are intended to be used.
In this study, the MERLIN system, which is a level III device, has been evaluated in an unattended setting with polysomnography as the gold standard. It should be noted that the sample is relatively small and primarily consists of males with a high prevalence of sleep apnoea. This poses a limitation to the generalisability of the results, particularly to nonreferral populations. However, the simultaneous monitoring of full-channel polysomnography is an added strength to the findings since it removes the concern of biological variability (i.e. night-to-night) that could potentially impact the internal validity of the study.
In this study, as was the case in others 2, 20, visual analysis of data was superior to the automated analysis included in the system. This finding may have been related to difficulty in identifying apnoeas and especially hypopnoeas with this system 27. It may also have been affected by the fact that the thermistor of the MERLIN device was superimposed to the thermistor of the polysomnographical equipment and therefore was further away from the nasal and oral signals. By contrast, visual analysis of amplified recordings of each channel may allow a better identification of respiratory events. In the manual analysis, the correlation between MERLIN respiratory event index and AHI was close and for this reason, individual differences between AHI and the polygraphical respiratory event index showed a distribution of points towards a horizontal line, as opposed to the straight downward line of points in the automatic analysis, where respiratory events were not identified in many polygraphical studies giving rise to an index of respiratory events of almost zero and unrelated to AHI. However, visual evaluation of MERLIN recordings achieved a higher diagnostic accuracy in patients with clinical suspicion of SAS than automatic scoring, as shown by the largest area under the ROC curve and sensitivity and specificity values of the best cut-off points for the different AHI categories. The best results of the area under the ROC curves were obtained for an AHI of ≥5, with manual scoring clearly superior to automatic scoring (area under the ROC curve 0.976 versus 0.818).
The diagnostic value of this cardiorespiratory device in patients who are clinically suspected of having SAS is confirmed by 90–96% of patients being correctly classified according to the different cut-off points in the polysomnography and the corresponding best cut-off points in the MERLIN manual index of respiratory events. Alternatively, for patients incorrectly classified for low AHI cut-off points (<5 or ≥5), discrepancy with polysomnography was small and, probably, would not modify therapeutic decisions. Differences were greater with regard to high AHI cut-off points (<30 or ≥30), and in these patients, results of cardiorespiratory polygraphy showed a tendency for underestimating severe SAS cases.
In summary, while inpatient polysomnography remains the gold standard, in patients clinically suspected of sleep apnoea/hypopnoea syndrome and in the setting in which the system of cardiorespiratory polygraphy was tested, the present results support the use of the MERLIN unit as the initial diagnostic approach. Although polygraphy using the cardiorespiratory MERLIN device cannot be considered a complete replacement of polysomnography, the MERLIN unit may provide a valid alternative to avoid polysomnography in a considerable number of patients with a clinical diagnosis of sleep apnoea, reserving overnight full-channel polysomnography for doubtful cases or patients with persistent clinical symptoms and normal or slightly abnormal polygraphical recordings. These findings, however, cannot be extrapolated to other level III monitoring systems. Further studies should assess the diagnostic accuracy of the MERLIN system in an unattended setting in nonreferral populations.
Acknowledgments
The authors would like to thank M. Pulido for editing the manuscript and editorial assistance.
- Received November 22, 2001.
- Accepted July 12, 2002.
- © ERS Journals Ltd
References
