External validation of a COPD diagnostic questionnaire

Development of the COPD diagnostic questionnaire in the original sample

The development of the COPD diagnostic questionnaire has been described in detail elsewhere 8, 9. In brief, 818 participants completed a list of questions and underwent spirometry. COPD was defined according to the Global Initiative for Chronic Obstructive Lung Disease (GOLD) guideline 14 by a post-bronchodilator (post-BD) forced expiratory volume in one second (FEV₁)/forced vital capacity (FVC) of <0.70. Multiple logistic regression models were constructed in order to identify eight items for discriminating between persons with and without COPD. The scoring system of the questionnaire permits calculation of an overall COPD risk based on the weighted scores for these items. The sum score ranges 0–38 points. According to the manual of the questionnaire, subjects can be classified as being at high (>19.5 points), moderate (16.5–19.5 points) or low (0–16.5 points) risk of COPD. The area under the receiver operating characteristic (ROC) curve (ROC_AUC) was 0.82. The diagnostic odds ratios (DORs) of the two cut-off points, 16.5 and 19.5, were 5.5 and 4.8; calculation based on that of Price et al. 9.

External validation sample

This external validation study was part of a study on the early detection of airflow limitation and subsequent smoking cessation treatment. The trial protocol was approved by the medical ethics committee of Maastricht University/Maastricht University Hospital (Maastricht, the Netherlands) and registered (International Standard Randomised Controlled Trial Number: 64481813) at the Netherlands Trial Register (Dutch Cochrane Centre, Academic Medical Centre, Amsterdam, the Netherlands).

Subjects were recruited from the general population (through advertisements in a local newspaper, flyers, posters and mailings to households) and from primary care practices (during consultations and through posters and personalised mailings) in the provinces of Dutch and Belgian Limburg (the region surrounding Maastricht, the Netherlands) during the period January 2005 to December 2006. Smokers aged 35–70 yrs who were current smokers but motivated to quit smoking were invited to take part in a study on individual counselling and medication for smoking cessation. The external validation of the COPD diagnostic questionnaire was performed in current smokers aged 40–70 yrs.

Eligibility was screened during an initial telephone interview. Inclusion criteria were: a smoking history of ≥10 pack-yrs; being motivated to stop smoking; being competent to read and speak Dutch; and reporting a respiratory symptom, defined as an affirmative answer to at least one of the following three questions. 1) “Do you cough regularly?” 2) “Do you cough up phlegm (sputum) when you don’t have a cold?” 3) “Have you been shorter of breath lately?” Exclusion criteria were: evidence of a prior respiratory diagnosis, defined by an affirmative answer to the following question: “Do you have COPD, chronic bronchitis, asthma or asthmatic bronchitis?” They were also not permitted to have undergone a lung function test (spirometry) during the preceding 12 months. Since candidates were being screened for taking part in a trial on smoking cessation, one or more contraindications to using the smoking cessation medication (nortriptyline) were also criteria for exclusion, among others the current use of antidepressants. Nortriptyline is a tricyclic antidepressant which should not be used for smoking cessation in conjunction with another antidepressant.

After the initial telephone interview, participant information, along with the informed consent form and the COPD diagnostic questionnaire, was sent to eligible subjects and a date was fixed for spirometry. Linguistic validation of the original English COPD diagnostic questionnaire for use in Dutch-speaking people had been performed by an experienced international research institute (Mapi Research Institute, Lyon, France). This process comprised conceptual definition of items, forward and backward translation, and testing. Subjects filled in the questionnaire at home and handed it in during the spirometry visit, during which they also signed the informed consent form.

Spirometry was performed by two qualified research assistants under the permanent supervision of a pulmonologist (G.J. Wesseling) according to the criteria of the American Thoracic Society/European Respiratory Society task force on standardisation of lung function testing 15, 16 using a Vitalograph® 2120 (Vitalograph, Buckingham, UK). After a minimum of three acceptable and reproducible FVC manoeuvres, a bronchodilator (500 μg terbutaline) was administered to the subject in preparation for the reversibility test. After 15 min, another series of three FVC manoeuvres was performed. All spirometric test results were independently validated by a pulmonologist (G.J. Wesseling) and a specialised lung function laboratory assistant who was not involved in the trial. Both were blinded to the scores on the COPD questionnaire. In the case of initial disagreement, consensus was obtained during re-examination. As in the study of Price and co-workers 8, 9, a diagnosis of COPD was confirmed by a post-BD FEV₁/FVC of <0.70, according to the international GOLD guideline 14.

Statistical analyses

The sum score for the COPD questionnaire was calculated based on the original scoring system 9. The ROC_AUC, sensitivity, specificity and DOR were then calculated for the two cut-off points presented in the questionnaire manual. A multiple logistic regression model was fitted, including the eight items from the COPD diagnostic questionnaire as independent variables and COPD as dependent variable in order to compare odds ratios, logistic regression coefficients and corresponding p-values between the original sample and the external validation sample.

Study population

A summary of the study enrolment protocol is shown in figure 1⇓. A total of 1,711 subjects were screened for eligibility by telephone. The majority, 1,209 (70%) subjects, had responded to an advertisement, 166 (10%) had been motivated by their general practitioner to take part in the study and 196 (11%) by a related study participant, and 140 (8%) had responded to flyers and posters or their motivation was not recorded. Of these, 1,052 subjects were found to be eligible and were invited for spirometry. Spirometry was performed and the COPD diagnostic questionnaire was collected from 826 subjects; the remaining 226 subjects cancelled their spirometry appointment beforehand or did not show up. Of the spirometric test results, 13% (110 out of 826) were found to be invalid and so had to be excluded from the analyses. This proportion of invalid test results was higher than in the original study (80 out of 898; 9%). The 110 subjects with invalid test results did not differ from those with valid test results with regard to sex, age, smoking history and COPD diagnostic questionnaire sum score (results of statistical tests not reported). In total, 40 subjects were excluded due to missing data for one or more of the items of the COPD diagnostic questionnaire. The characteristics of these subjects did not differ from those with complete data for the COPD diagnostic questionnaire except for the post-BD FVC, which was significantly lower (mean±sd 3.51±0.76 versus 3.98±0.95 L; p = 0.002). Complete valid data from 676 subjects were used in the analyses.

• Download figure
• Open in new tab
• Download powerpoint

Fig. 1—

A flow diagram to show study enrolment. COPD: chronic obstructive pulmonary disease.

Characteristics of the external validation sample

Of the 676 study subjects, 378 (59%) exhibited normal lung function (mean post-BD FEV₁/FVC of 77.9; mean post-BD FEV₁ of 94.1% of the predicted value) and 278 (41%) had a diagnosis of COPD (mean post-BD FEV₁/FVC of 61.9; mean post-BD FEV₁ of 79.7% pred). In the latter group, airflow limitation according to GOLD criteria 14 was mild in 142 (21%) subjects, moderate in 119 (18%) and severe-to-very severe in 17 (3%). All other characteristics are shown in table 1⇓. On average, subjects from the present external validation sample were younger and exhibited slightly worse lung function parameters than those from the original sample. The prevalence of COPD was much higher (41 versus 19%). All subjects were current smokers, whereas only 45% of the original sample had been current smokers. This difference in smoking status probably explains the large difference in mean smoking history (40 versus 26 pack-yrs in the original sample). The external validation sample was split into two subsets according to the study diagnosis in order to compare differences in characteristics. This is also shown in table 1⇓ (without calculation of statistical differences). Smokers with COPD were more likely to be older and male and to show greater cumulative cigarette consumption.

Discriminative ability of the COPD diagnostic questionnaire

When applying the two cut-off points from the questionnaire manual (16.5 points for a low risk and 19.5 points for a high risk), 127 (19%) out of 676 subjects were categorised as having a low risk of COPD, 183 (27%) as having a moderate risk and 366 (54%) as having a high risk. Within these three risk categories, the observed prevalence of COPD was 24% (30 out of 127 subjects) in the low-risk category, 36% (65 out of 183) in the moderate-risk category and 50% (183 out of 366) in the high-risk category. The ability of the COPD diagnostic questionnaire to discriminate between subjects with and without COPD is shown graphically by the ROC curve in figure 2⇓. The solid line represents levels of sensitivity and false positive rate for all cut-off points for the sum score of the questionnaire. The ROC_AUC was considerably lower (0.65) than in the original sample (0.82). Sensitivity, specificity and DOR for the cut-off point 16.5 were 89.2%, 24.4% and 2.67, respectively. For the cut-off point 19.5, these parameters were 65.8%, 54.0% and 2.26, respectively.

• Download figure
• Open in new tab
• Download powerpoint

Fig. 2—

Receiver operating characteristic curves for predictors of chronic obstructive pulmonary disease (COPD) in the external validation sample. –––––: sum score of the COPD diagnostic questionnaire (area under the curve (AUC) 0.65); -------: age alone (AUC 0.67); ·····: reference line.

Refit of the multiple logistic regression model

The results of the multiple logistic regression analysis that modelled the probability of having COPD versus not having COPD are shown in table 2⇓. Significant associations were observed for age, body mass index and the two items “phlegm in the morning” and “any wheeze”. The other items were associated with odds ratios that did not differ significantly from 1.

Since age seemed to be the most important predictor of the disease, another ROC curve was calculated, with age as the single factor for predicting COPD. This curve is shown in figure 2⇑. The ROC_AUC was 0.67, meaning that the variable age alone showed similar discriminative ability to the sum score from the COPD diagnostic questionnaire in the present sample.