False impression of objectivity may deny patients affordable treatment

Editor—In their review of the clinical impact of obstructive sleep apnoea and the utility of treatment with nasal continuous positive airways pressure John Wright and colleagues make some important points but give a false impression of objectivity.1

Their criteria for excluding abstracts and letters are vague, and the predetermined validity criteria for papers were not well defined—disagreements between the two assessors had to be resolved by a third person. There is evidence of bias, particularly in the discussion on mortality. Two studies in which the design would be unlikely to show any effect are highlighted as showing no significant association between obstructive sleep apnoea and premature death2 3; another in which the apnoea index was a predictor of excess mortality4 is only briefly mentioned and is qualified by the negative statements that the duration of apnoea was not a predictor of mortality and that the excess deaths were not due to heart or lung causes. The important point that the apnoea index was a predictor of premature death was not discussed.

The concept that obstructive sleep apnoea varies from being normal to a life threatening condition is unacknowledged. Results of studies in which most of the patients had only mild disease were used to suggest that there is no link between obstructive sleep apnoea and medical problems. Similarly the authors seem unaware of current medical practice when they state that continuous positive airways pressure is the recommended initial treatment for obstructive sleep apnoea; simple measures such as weight loss are usually tried first.

We are also concerned by the accuracy of the review. There are three errors in the description of our study on nasal continuous positive airways pressure and obstructive sleep apnoea.5 In table 4 of John Wright and colleagues' paper1 the desaturation index for patients with mild obstructive sleep apnoea should be 8, not 38; disruptive daytime sleepiness was an indication for starting treatment with continuous positive airways pressure; and “no change in … symptoms” should read “no new unrelated symptomatic condition developed.” These errors completely change the interpretation of our results and raise questions about the accuracy of statements about the other papers quoted.

The impression of objectivity and accuracy and of a scientific approach in this review is illusory. The underlying assumption that clinicians are widely using nasal continuous positive airways pressure in patients without important symptoms is unjustified. It would be unfortunate if this review led to patients being denied a cheap and effective treatment that could prevent them from remaining excessively sleepy for the rest of their lives and running an increased risk of premature death.

Superficial analysis ignores evidence on efficacy of treatment

Editor—The conclusions of John Wright and colleagues have the potential to jeopardise public health as well as patients' care; their systematic review lacks the necessary depth and understanding of the issues.1 The authors note that sleep apnoea is currently viewed as a public health concern because it is prevalent and thought to be associated with morbidity and mortality. The collective evidence from the studies included in the review supports this view; the authors' conclusions that evidence for the health effects of sleep apnoea is “weak or contradictory” and that “the relevance of sleep apnoea to public health has been exaggerated” are disturbing.

The authors seem to have misinterpreted the findings of some studies. In their assessment of nine studies of hypertension judged to be methodologically adequate, the authors dismiss seven studies that show an independent association between sleep apnoea and hypertension. The authors choose to base their conclusion solely on the two studies that found no association. The studies that found an association were rejected because associations were found only for blood pressure measured in the morning, or because of failure to control for use of antihypertensive drugs, smoking, or use of alcohol. Acute spikes in blood pressure are known to follow episodes of apnoea and hypopnoea; therefore, it is not unreasonable to expect a stronger effect in the morning. If the effect on blood pressure later in the day is less, the small sample sizes of these studies may have precluded detection. Not controlling for antihypertensive drugs would only serve to reduce a true association. Smoking and alcohol use are unlikely to be strong enough risk factors to cause significant confounding.

Wright and colleagues' review of evidence on treatment with continuous positive airways pressure is also unbalanced. While we support the call for a large randomised clinical trial—the next logical step in this field—there is already evidence for the efficacy of continuous positive airways pressure as evaluated by short term neurobehavioural measures and quality of life assessments. We are concerned that the superficial analysis and conclusions of Wright and colleagues will lead to many patients being denied access to this treatment, which is associated with minimal risks.

Wright and colleagues should be alerted that well designed longitudinal studies on sleep apnoea are under way. We hope that the authors will conduct a more informed review of forthcoming reports.

Evidence for efficacy of continuous positive airways pressure is compelling

Editor—The review by John Wright and colleagues of the complications of obstructive sleep apnoea,1 a reincarnation of a previous publication,2 concluded that the relation between obstructive sleep apnoea and hypertension, coronary heart disease, stroke, and premature death are poorly established; this conclusion has been stated many times by other authors, including us.3 What was new was their inference that the treatment of obstructive sleep apnoea with continuous positive airways pressure is difficult to justify because much of the published evidence is poor or inconclusive. This is wrong since it fails to appreciate that patients are treated for disabling daytime somnolence and not for the risk of cardiovascular disease or premature death. Weak data on cardiovascular aspects are largely irrelevant to the assessment of treatment for this disorder.

How could Wright and colleagues have wandered so far from the mark? We suspect that their error resulted from the simple mistake of reviewing studies and assuming that because most revolve around the cardiovascular-mortality debate, this must be the most important area to consider. This would be a natural mistake for those with no experience of the disease and its management, who thus have no idea that it is mainly hypersomnolence which disturbs patients. There is considerable published work on the cardiovascular and mortality aspects of this disease precisely because this is a highly contentious subject. There are few randomised controlled data on the efficacy of continuous positive airways pressure for overwhelming somnolence because the therapeutic response is so compelling in case reports. Those of us who treat these patients have seen the predicted improvements when patients woken from sleep over 300 times a night for years finally regain normal sleep patterns with continuous positive airways pressure. Perhaps naively we had never expected anyone to question this; neither had we expected the opinions of people with no clinical experience of a complex disease to be taken seriously.

Until there is a fully informed view that includes the skill and experience of those who understand the disease, it would be wrong (and negligent) to base purchasing decisions on this incomplete assessment. An evidence based approach is crucial to clinical medicine and sensible purchasing decisions; it should not be brought into disrepute by being used in an unbalanced way.4 5

Review was misleading and may deny cost effective treatment to patients

Editor—John Wright and colleagues' review of obstructive sleep apnoea1 and the accompanying comments in Editor's choice2 and This week in the BMJ3 have serious adverse implications for many thousands of patients with a severely disabling condition. The review concentrates on the putative association with cardiovascular morbidity and mortality and on treatment with nasal continuous positive airways pressure. The unfortunate juxtaposition of these two topics will mislead readers into assuming that treatment with continuous positive airways pressure is aimed at preventing such complications.

The authors fail to explain that treatment with continuous positive airways pressure is given primarily for relief of disabling daytime sleepiness. We agree that the evidence for a causal relation with vascular disease is tenuous. Unfortunately, however, Wright and colleagues equate morbidity with cardiovascular morbidity and fail to acknowledge the profound morbidity associated with severe sleepiness. The resulting impression is unduly negative, and casual statements such as: “This may not be a disease after all” 2 or not “a separate disease entity”1 (whatever that means) are potentially highly damaging. Obstructive sleep apnoea cannot be dismissed as merely a manifestation of obesity, as many patients with serious symptoms are not overweight. Because of this mistaken assertion the authors suggest that greater emphasis should be put on weight reduction than on treatment with continuous positive airways pressure. However, there is much less evidence for the effectiveness of dietary treatment than there is for continuous positive airways pressure. Even in obese patients with obstructive sleep apnoea it is inappropriate to deny immediate, inexpensive treatment with continuous positive airways pressure while awaiting an outcome that is rarely achieved.4

Wright and colleagues emphasise the need for further placebo controlled studies of continuous positive airways pressure, but the choice of an appropriate placebo is less clear than they assume. Use of a subtherapeutic pressure shows only whether pressure itself has beneficial effects. This approach is not relevant to everyday practice, where the appropriate comparison is between the status quo (or an alternative intervention) and any beneficial effect of continuous positive airways pressure, offset by the inconvenience and discomfort of the associated paraphernalia. However many controlled studies are performed, this decision will inevitably remain a matter of trial and error in individual patients.

The investigation and management of patients with obstructive sleep apnoea in the United Kingdom lags noticeably behind several other countries, including North America, France, Germany, and Australia. We are extremely concerned that the negative attitudes expressed in the BMJ will be seized on by NHS purchasers as a reason not to fund effective and inexpensive treatment for a large number of disabled patients.

Treatment prevents road accidents, injury, and death caused by daytime sleepiness

Editor—In stating that the results of their meta-analysis “do not … provide sufficiently robust evidence for the effectiveness of continuous positive airways pressure,” John Wright and colleagues ignore their own conclusion that sleep apnoea causes sleepiness and possibly road accidents, and thereby injury and death.1 They also reach the opposite conclusion to that reached by other reports, including one from the Royal College of Physicians2 and one from the Australian National Health and Medical Research Council and the New Zealand Ministry of Health.3

Two randomised controlled trials, one published since Wright and colleagues' analysis was completed, found that patients benefited from continuous positive airways pressure.4 5 Neither the Royal College of Physicians nor the Australian National Health and Medical Research Council finds the authors' criticisms of the randomised controlled trial by Engleman et al convincing. Indeed, the Australian National Health and Medical Research Council, which included not only specialists in sleep medicine but other clinicians on their steering committee, found that this study produced “the most compelling evidence of the impact of [continuous positive airways pressure] and that there were no major methodological threats to its validity.”3

The danger of Wright and colleagues' study is that purchasers of health care within the NHS will give credence to it and cite it as a reason for not providing the appropriate services. To deny this treatment to patients would fly in the face of evidence presented in other reviews and lead to an increase in otherwise avoidable road accidents.

Some criticisms of studies are unfounded

Editor—At the Scottish National Sleep Laboratory we strongly agree with John Wright and colleagues that further research is required to clarify whether sleep apnoea causes vascular events,1 and we look forward to contributing to these studies. However, we are disappointed that they are not prepared to support the treatment of sleep apnoea, which, at least in Edinburgh, is aimed at improving symptoms, quality of life, objective sleepiness, and daytime performance. We showed that all of these improved with continuous positive airways pressure2 in the “one out of 45 studies of continuous positive airways pressure [that was] a truly randomised controlled trial.” 1

We do not believe that the criticism that our study “had important weaknesses” is justified. We are criticised for using an oral placebo instead of a sham continuous positive airways pressure machine. We did this for ethical and scientific reasons as continuous positive airways pressure at subtherapeutic pressure may worsen hypoxaemia during sleep; patients could differentiate between lower sham and higher effective pressure so unblinding the study; and wearing a placebo mask would impair sleep thus biasing in favour of active treatment. Furthermore, machine mystique—the idea that a physical device might have more of an effect than an oral placebo—would be unlikely to influence the objective variables measured more than our actively advocated placebo.

We are criticised for the lack of a washout period, but we did not take any measurements until 28 days after crossover and the benefits of continuous positive airways pressure wear off within one day.3 We tested for potential carryover by examining order effects using analysis of variance; we accept that between subject effects have lower power than within subject effects, but only the one variable indicated showed a significant (P<0.05) order effect.2 Critically, we found significant benefits from continuous positive airways pressure while any carryover effect would bias against such findings.

Also, Wright and colleagues omit to indicate that we found significant improvements compared with the placebo in four measures of cognitive function, including intelligence quotient (IQ).

The Australian National Health and Medical Research Council conducted a similarly detailed review and concluded that there was evidence from a randomised controlled trial (level II evidence) and from meta-analysis of other studies (level III evidence) that continuous positive airways pressure was effective and recommended treatment with it for sleepiness and impaired daytime function.4 This was also the conclusion of the Scottish needs assessment programme.5 All three reviews1 4 5 conclude that more evidence is needed about vascular events.

Wright and colleagues' review seems excessively negative about the merits of continuous positive airways pressure for sleepiness and cognitive impairment and this negative conclusion largely centres on their criticism of our study. In contrast, the Australian investigation, while also concerned about the use of an oral placebo, concluded that there were “no major methodological threats to the validity” of our study.4

Authors' reply

Editor—The variety of views expressed by the correspondents reflects the uncertainty about sleep apnoea. The common claim of associations between sleep apnoea and premature death and vascular morbidity1 2 3 is reiterated by John Shneerson and Ian Smith. Yet J R Stradling and R J O Davies and G J Gibson and K Prowse agree with us that the evidence is weak. We believe that most patients would be reassured by our conclusion that claims about associations with disability and death are unfounded or premature.

Another example of the uncertainty about sleep apnoea concerns the role of weight reduction. Shneerson and Smith suggest that weight loss is the first line of treatment, yet Gibson and Prowse believe that this is inappropriate and that patients should begin treatment with continuous positive airways pressure straight away. We agree that evidence of the effectiveness of weight reduction on sleep apnoea is lacking, but this probably reflects a lack of interest in this approach rather than a lack of efficacy. The strong association between sleep apnoea and obesity suggests that sleep apnoea may be a symptom of obesity. Surely it is logical to tackle the cause rather than the symptom, particularly in view of the other health benefits derived from weight loss. A recent review has indicated that weight loss interventions can be effective, but it highlights the need for a determined multidisciplinary approach.4

Shneerson and Smith correctly point out our typographical error, but we fail to see how the interpretation of their study changes. Our approach in undertaking a systematic review, far from being illusory, is to ensure an objective and scientific review of the evidence. All data from relevant studies are shown in tables so readers can derive their own conclusions. This is an advance on the sort of papers written by enthusiasts who selectively quote studies which support their view and ignore methodological quality.

The accusation that we are suggesting that there is no justification for treatment with continuous positive airways pressure is unfounded. We stated that there can be large benefits from treatment, but that these seem to occur predominantly in patients with very severe sleep apnoea. Clinicians, and their professional organisations, surely have the responsibility to pursue a research agenda to determine in which patients treatment is worth while and cost effective. To suggest, as do Stradling and Davies, that a health service is developed for 2-4% of the middle aged population on the basis of case reports is simply not acceptable.