Effect of peak expiratory flow data quantity on diagnostic sensitivity and specificity in occupational asthma

Serial self-measurement of peak expiratory flow (PEF) is recommended in the first-line investigation of workers with suspected occupational asthma 1. This is both sensitive and specific 2–6. It was originally suggested that PEF measurement should be performed every 2 h whilst awake for a minimum of 4 weeks, including periods at and away from work 3. Recording measurements this frequently and for this duration of time requires a significant degree of motivation and cooperation from the subject. Malo et al. 7 suggested that four evenly spaced daily readings were adequate for diagnosing occupational asthma, with little loss of sensitivity compared to two-hourly readings. There is little published data as to whether keeping PEF records for a shorter duration of time adversely affects diagnostic sensitivity and specificity. Nor is it known whether diagnostic sensitivity and specificity are influenced by the number of consecutive days at work or at rest, which would be of particular relevance to shift workers.

The aim of the present study was to determine how PEF data quantity (i.e. record duration, number of daily readings and number of consecutive days at work or rest) influences the sensitivity and specificity of PEF records for diagnosing occupational asthma.

Methods

Good-quality PEF records from workers with definite occupational asthma were sequentially reduced by removing: 1) data from the end of the record, to study the effects of duration; 2) daily readings, to study the effect of number of daily readings; and 3) data for whole days, to study the effects of number of consecutive days at work or rest. At each stage of the reduction process, the record was re-evaluated for evidence of occupational effect and diagnostic sensitivity was determined for that amount of data. The same process was performed on records from workers who did not have current occupational asthma in order to determine specificity.

Data reduction from good quality peak expiratory flow records

Duration of record

Good quality gold-standard positive PEF records were identified with: 1) ≥3 readings·day⁻¹ for ≥75% of days; 2) 4–6 consecutive days in each work period; 3) 2–3 consecutive rest days in each rest period; and 4) ≥5 work-rest-work or rest-work-rest complexes in duration. An example of a record containing 5 overlapping complexes of 5‐workday and 2‐rest-day format is shown in figure 1⇓. In cases in which a subject had more than one PEF record, only the first good quality PEF record was used for data reduction. The reduction process involved removing the last work or rest period, such that there was one less complex. The shortened record was then rescored using Oasys‐2. The number of complexes continued to be reduced until only one complex remained in each record. At each stage of the reduction process, a gold-standard positive record with a score of >2.5 was defined as a true positive; gold-standard positive records with an Oasys‐2 score of ≤2.5 were defined as false negatives. The sensitivity for diagnosing occupational asthma for records of a particular complex length was calculated by determining the proportion of records of that duration that had Oasys‐2 scores of >2.5.

• Download figure
• Open in new tab
• Download powerpoint

Fig. 1.—

a) Diurnal variability of peak expiratory flow (PEF) and b) corresponding gold-standard positive PEF record (–––: mean daily PEF; --‐: maximum/minimum daily PEF) showing 5 overlapping work-rest-work and rest-work-rest complexes (numbered horizontal bars), each of 5 workday (▓) and 2-rest-day (□) format. The whole-record Oasys‐2 score was 2.88 (true positive). During the reduction process, sections of the record were removed sequentially, starting with the last work period: the Oasys‐2 score for complexes 1–4 fell to 2.5 (false negative). The last rest period was removed next and the remainder of the record rescored. This was continued until only one work-rest-work complex remained. Var: variability.

The same process was applied to the gold-standard negative PEF records to see whether records became falsely positive. At each stage of the reduction process, a gold-standard negative record with an Oasys‐2 score of ≤2.5 was a true negative, and one with a score of >2.5 was a false positive. Specificity was calculated by determining the proportion of records of that duration that had Oasys‐2 scores of ≤2.5.

Number of consecutive days at work and rest

The same gold-standard records identified for the duration reduction process above were used for reducing the number of consecutive days at work or rest. The last day at work in each work period was removed and the record rescored using Oasys‐2. For example, the first step would be to remove a Friday from all work periods and then rescore the record. Thursdays would then be removed (fig. 2⇓) and so on until only the Monday workday remained. This was also performed for rest days.

• Download figure
• Open in new tab
• Download powerpoint

Fig. 2.—

a) Diurnal variability of peak expiratory flow (PEF) and b) corresponding gold-standard positive PEF record (–––: mean daily PEF; ----: maximum/minimum daily PEF) from figure 1⇑ but with the data for Thursdays and Fridays (the last 2 workdays of each 5-workday period) removed (0 readings·day⁻¹) such that there are 5 overlapping work-rest-work and rest-work-rest complexes (numbered horizontal bars), each of 3‐workday (▓) and 2-rest-day (□) format. The Oasys‐2 score remained positive at 2.8. Each stage of the reduction process, from 4 down to 1 workdays (each with 2 rest days), was scored by Oasys‐2 to determine the probability of occupational asthma for the relevant number of consecutive days at work. Var: variability.

Number of daily readings

Gold-standard records were identified for data reduction with: 1) ≥8 readings·day⁻¹ for ≥75% of days; 2) ≥3 complexes' duration; and 3) no night shift periods. PEF readings were removed from each day until there were 4 readings·day⁻¹, corresponding as closely as possible to timings of 06:00, 12:00, 17:00 and 24:00 h. The 4‐readings·day⁻¹ records were then rescored using Oasys‐2. Further readings were removed until there were 2 readings·day⁻¹, as close as possible to times of 06:00 and 17:00 h, and the records rescored. Records were also reduced to single daily readings (either morning or evening).

Results

Gold-standard positive records (137) were identified from 81 workers with occupational asthma. Sixty gold-standard negative records were identified from subjects not occupationally exposed. The demographics of all gold-standard positive and negative subjects are shown in table 1⇓. The characteristics of gold-standard positive subjects according to whether the PEF record was positive or negative are shown in table 2⇓.

Discussion

PEF data quantity appears to be an important determinant of diagnostic sensitivity and specificity in occupational asthma. Maximum sensitivity and specificity for diagnosing occupational asthma were obtained with records of 4 weeks' duration, 8 readings·day⁻¹ and ≥4 consecutive days in each work period. However, few records satisfy these criteria in practice and the gain in terms of sensitivity and specificity is slight. More inclusive criteria for defining whether a PEF record is of adequate data quantity are that they are of ≥3 complexes in duration (equivalent to 2.5 weeks), ≥4 readings·day⁻¹ and ≥3 consecutive days in each work period. The effect of defining adequacy of the PEF record according to these criteria was tested in PEF records of poorer quality than those used in the initial data reduction process; the sensitivity for records of adequate data quantity was 78.1%, with a specificity of 91.8%. Records that were of inadequate data quantity according to these criteria were less likely to show occupational asthma and were of lower specificity. It could even be suggested that <4 readings·day⁻¹ are required. However, 2 readings·day⁻¹ can cause underestimation of the true diurnal variation by up to 15% compared to 4% with 4 readings·day⁻¹ 10.

Consecutive subjects diagnosed as having occupational asthma by any of the generally accepted objective methods other than the PEF record itself were included as gold-standard positive subjects. In order to determine specificity, it was necessary to identify subjects who definitely did not have occupational asthma. Some studies have used subjects in whom a diagnosis of occupational asthma was excluded by specific bronchial challenge testing; however, the possibility of false-negative challenge test results remains (e.g. due to wrong agent, insufficient dose or inability to reproduce workplace conditions). Other studies have used asymptomatic individuals at low risk of occupational asthma as gold-standard negative subjects, e.g. Post Office workers 11. Unfortunately, most occupational groups are exposed to potential sensitisers in their workplace, e.g. office workers exposed to toning agents from photocopying machines and floor cleaning agents or healthcare workers to latex. It is possible that even asymptomatic subjects could have occupational asthma (analogous to poor perceivers of asthma); hence, if their PEF records were to show work-related deterioration, these would wrongly be classified as falsely positive. In order to avoid these potential difficulties, PEF records from asthmatic subjects who were not at work (and hence could not have current occupational asthma) were used as gold-standard negative records. Even though there was no occupational exposure, Mondays to Fridays of these records were deliberately designated as workdays. Any record that had an Oasys‐2 score of >2.5 would thus be a false positive. Despite no apparent occupational exposure, it is possible that these records could still show a genuine deterioration in PEF between Monday and Friday, e.g. due to undeclared work or a particular weekly pattern of activity such as heavy gardening work.

The method used to determine the effects of PEF data quantity involved sequentially reducing the data from good-quality PEF records from workers who definitely did or did not have occupational asthma. The Oasys‐2 computer program was used to determine whether a PEF record at any particular stage of data reduction was positive. It is possible that, as the Oasys‐2 program shows less diagnostic sensitivity than an expert, the adverse effect on sensitivity of reducing data quantity would be less marked if the record were being scored by an expert. This could mean that, if an expert were available to interpret the record, less data than that suggested by the present study could be required for adequate interpretation.

Oasys‐2 does have advantages over expert observers in that results are completely repeatable, and sensitivity and specificity are well described, whereas these would vary between experts. The results of a data reduction process depend on the nature of the PEF records selected. Workers with marked airflow obstruction occurring at work might require only a few readings at and away from work for their PEF record to suggest occupational asthma. Someone with less-marked airflow obstruction might require a greater quantity of data for the work effect to be evident, and, in such cases, removing data might diminish any notable work effect. In order to attempt to minimise selection bias, all available gold-standard positive records that were of high enough quality were used for data reduction. Indeed, several records were classified as positive but showing less-marked work-related deterioration (Oasys‐2 scores of ≥2.5 but <3). These records did not show an increased tendency to become negative when the data was reduced.

Data reduction appears to be an appropriate technique for examining data quantity effects when the removed data has no influence on the remaining data, e.g. removing data from the end of a record or reducing the number of readings in a day. However, in cases in which consecutive days in a work or rest period are removed, the removed data influences the rest of the record. Caution must, therefore, be exercised when interpreting the effect of the number of consecutive work or rest days in a week. With the data set used in the present study, most cases of occupational asthma would have been identified using Oasys‐2 even if only 1 rest day had been present in each complex. However, the present authors would still recommend that, if the PEF is still significantly below the predicted value by the end of a rest period, monitoring for a longer period away from work is required.

In conclusion, the minimal acceptable standard of peak expiratory flow data quality for diagnosing occupational asthma is a record that is ≥3 complexes in duration, with ≥4 readings·day⁻¹ and ≥3 consecutive days at work in each work period for the majority of the record. Diagnostic sensitivity and specificity are reduced if all of these criteria are not fulfilled, and such records should be interpreted with caution.