Bias in retrospective studies of trends in asthma incidence

A number of studies of various designs have documented a long-term increase in asthma prevalence in children 1–5 and to a certain extent in adults 6–16. It is uncertain whether this represents an increase in incidence or a longer duration of disease. Aetiological theories have been formed to explain the increase in asthma. Confirming an increase in asthma incidence as the cause of increasing asthma prevalence is important 17.

The evidence to suggest an increase in asthma incidence remains soft, based largely on healthcare usage statistics 18–22 or mortality 23–27. Population-based prospective studies of long-term trends in asthma incidence are scarce 28, 29. Several studies have used retrospective age-of-onset data to document changes in asthma incidence over the long term 30–32. Sunyer et al. 30 were able to show an increase in the incidence of asthma in Europe by year of birth, by asking subjects retrospectively about their age at first asthma attack. de Marco et al. 31 used the same data to show a sex reversal in asthma incidence at puberty. In a separate Italian study, de Marco et al. 32 also showed an increase in asthma incidence with year of birth. These two studies appear to be an important basis for the perception of an increase in asthma incidence. The validity of a retrospective analysis as used in these studies is unknown.

In the present paper, a prospective cohort study was used as a gold standard, and this was compared with two different retrospective estimates of asthma and bronchitis incidence. The studies covered the same population, the same geographical area and the same time period. Retrospective estimates came from an independent cross-sectional study in 1998, and from a retrospective analysis of the last follow-up of the cohort study.The objective of this paper was to assess the validity of a retrospective analysis of trends in asthma incidence, by using the prospective study as a gold standard.

Methods

Previous papers report the study design and data collection of the baseline 33, the follow-up 34 of the prospective study, and the cross-sectional study 35–37. For comparison between studies, only subjects born from 1927–1969, living in the city of Bergen and 11 surrounding municipalities, were eligible for the analyses. Both studies were based on random population samples of the adult population aged 15–70 yrs. The sampling for each study was done independently. Age was computed as of January 1, 1985.

Briefly, the prospective study was conducted between 1985–1996/1997. Of the 3,129 subjects who were eligible in 1985, 2,427 subjects responded to a postal questionnaire both at baseline and follow-up. In the retrospective study, 2,864 subjects were eligible. Data were collected through a postal questionnaire in 1998/1999, with a telephone follow-up for nonresponders. In previous studies, good comparability was found between postal and telephone survey modes 35, with little evidence of nonresponse bias 38. There was minimal overlap between the two samples. Sampling fractions were <2% in both studies, such that 47 subjects were sampled into both studies. Of these, 24 were responders and are included in the data analysed.

Results

In the prospective study, 77.6% of the original 3,129 subjects were responders in both the baseline and follow-up study. The response rate in the retrospective study was 88.3% (table 1⇑). The responders were comparable in age, sex and smoking habit.

A comparison of the cumulative 11‐yr incidence of asthma and bronchitis in the prospective study and retrospective study is given in table 2⇑. Overall, there was a 20–60% difference in the cumulative incidence of asthma. Taking the prospective study as the gold standard, the retrospective analyses gave an underestimate as expected. In the independent retrospective study, there was a large underestimation of asthma incidence for males, but little difference according tosmoking habit. Older birth cohorts had an underestimate of asthma incidence, while younger birth cohorts had an overestimate. For asthma, the subgroup analysis showed that there was a difference in the degree of bias among males and females (final p‐value 0.0053). In the retrospective analysis within the prospective study, all birth cohorts gave an underestimate of asthma incidence. The retrospective study underestimated bronchitis incidence overall. The subgroup analysis showed that this degree of bias was significantly different according to smoking habit (p=0.0019).

The smoothed OR of incident asthma in the period 1985–1997 by year of birth is shown in figure 1⇓. Those born in 1927 were the reference category (OR 1). There was a large discrepancy between the independent retrospective and the prospective estimates of the effect of year of birth. In fact, conclusions from either type of analysis would be exactly opposite. The OR comparing those born in 1969 with those in1927 was 2.89 in the retrospective study, and 0.23 in theprospective study. The bias was less severe for bronchitis, but in the same direction as for asthma. The same trends were evident when this figure was produced for subgroups of males, females, never-smokers, exsmokers and current smokers.

The ORs of asthma by year of birth in different subgroups are given in table 3⇑. The prospective study showed a decrease in asthma incidence with year of birth in all these groups, while the independent retrospective study gave the opposite conclusion. The subgroup analysis did not reveal any significantly different subgroups (p‐value for any interaction 0.68).

Missing answers to a question may indicate recall bias. In table 4⇓ the proportion of missing answers to questions on diagnosis and onset of disease by year of birth is shown. In the independent retrospective cross-sectional study in 1998, there was clear evidence of increasing uncertainty in questions on diagnosis by year of birth. For questions on asthma onset, there was a large proportion of missing answers in those born before 1940. For onset of bronchitis, the proportion of missing answers was large in all birth cohorts. There is much less evidence of such a trend in either of the two surveys in the prospective study.

Discussion

Asthma incidence has been shown to be underestimated by using age-of-onset data retrospectively. There was a severe bias when these retrospective data were used to study trends in asthma incidence by year of birth. This calls into question the previous retrospective studies that have shown increasing trends in asthma incidence.

Researchers interpreting a retrospective incidence study should always wonder “what if” a prospective study had been performed instead. The strength of the present study is that it provides that contrast, since the two studies are independent and cover the same age groups and geographic area. The studies are performed in the way that retrospective and prospective incidence studies are usually performed. It is an additional strength that the findings have been replicated by a retrospective analysis within the prospective cohort.

The data has been analysed and looked at for age and cohort effects. Analyses of trends in disease should ideally look for age, period and cohort effects simultaneously. This is not possible unless there are more than two time points 41. Since previous retrospective studies of trends in asthma incidence have looked for age-cohort effects, the same have been chosen here.

It is an open question whether the “start of the disease” can be defined accurately for asthma, and recalled correctly. The European Community Respiratory Health Survey asked for “age at first asthma attack” 30. This is perhaps more vividly remembered than the authors' term “age at start of disease”. Another Swedish retrospective study used “year of asthma diagnosis” 42. Little is known about the validity of any of these questions, or their merit relative to each other.

The major problem with using retrospective reports of disease onset to estimate incidence is the frailty of human memory. Dodge and Burrows 43 found somewhat similar estimates of asthma incidence when they compared prospective and retrospective estimates in the young, but retrospective estimates failed to show an increase in elderly females that was, however, found prospectively. Strachan et al. 44 found retrospective estimates of asthma incidence that were 60% lower than the true prospective estimates, and a substantial proportion of adult subjects had forgotten any childhood asthma.

Sunyer et al. 30 showed retrospectively an increase in the incidence of asthma by year of birth. Their report also included the region from which the present studies were drawn (Hordaland, Norway). They reported no increase in incidence of asthma in Hordaland. This is no doubt in part due to a smaller sample size. The most important factor is probably that their birth cohort started in 1945 and ended in 1971, whereas the present cohort started in 1927.

A comparable Swedish study 42 used the same retrospective method to investigate risk factors for adult-onset asthma, but found little apparent increase with year of birth. If there was a recall bias in this study as well, the neutral trend found in the study could in fact obscure a decreasing trend in asthma incidence.

While the overall asthma incidence was underestimated by only 20%, there was weak evidence of a stronger bias in subgroups. The present study had limited power for such an analysis. However, the degree of bias in males was such that retrospective estimates of asthma incidence must be considered suspect until further data can corroborate or disprove these findings.

The bias in the retrospective study was stronger in older birth cohorts. This may indicate poorer recall in general, but may also indicate different attitudes to healthcare in general and asthma specifically.

Some weaknesses must be acknowledged. The prospective study is used as the “gold standard” against which the retrospective study is compared. However, it is possible that there is some nonresponse bias in this study. The prevalence of an answer of “don't know” to questions on diagnosis at baseline in 1985 was nearly the same as the prevalence of asthma diagnosis (2.14%, n=52 versus 2.93%, n=71). These subjects may be at higher risk of incident asthma during follow-up. Of these 52 subjects, five (10%) answered that they had asthma in 1996/1997. This cumulative incidence is higher than among those who answered “no” in 1985. However, the numbers were not high enough to have any significant impact on the cumulative incidence.

There is also the possibility that there were “interval cases” in the prospective study, i.e. asthma or bronchitis cases who only had asthma for a short period between baseline and follow-up. These may have been “picked up” in the retrospective study, thus leading to a bias, but this is unlikely.

The follow-up in the prospective study lasted from September 1996 to May 1997. To ensure comparable follow-up in the retrospective study, only cases of incident asthma between 1985–1996 were included. Some retrospective cases that occurred in 1997 were thus excluded that should have been included; if they had occurred from January to May 1997. As there were only seven cases of incident asthma in all of 1997 this is unlikely to have affected results to any significant degree.

The present study cohort was followed for 11 yrs. At baseline, the study cohort was between 15–59 yrs of age, so the results are mostly relevant to adult-onset asthma. The increase in asthma in the study by Sunyer et al. 30 covers the period from birth to ∼30 yrs of age, thus the comparative study may not be directly comparable. However, the exact same mechanisms would operate to cause a recall bias.

It is likely that recall bias in the retrospective study can explain a significant part of the bias found in the analysis of trends by year of birth. There is increasing missing data in the questions on asthma diagnosis and asthma onset by earlier year of birth (table 4⇑), but much more so in the retrospective study than in the prospective study. This is not only likely to reflect that some of the older subjects were likely to have forgotten their age at onset, but also that the recall was poorer in general among the older subjects. This was probably because of the longer time span involved.

The high proportion of missing answers to the questions on bronchitis, and the undefined validity of this question, make the authors cautious of any conclusions regarding this outcome. There is likely to be some confounding of the “asthma diagnosis” with chronic obstructive pulmonary disease at >45 yrs. However, this should not influence the comparison between the retrospective and the prospective study.

Given the uncertainty of a retrospective asthma diagnosis, what about a retrospective analysis of symptom incidence? The authors did not ask for age at onset of any symptoms, and they are unaware of any asthma studies that have. In light of the recall bias and its mechanisms shown for asthma diagnosis, it is likely that the findings concerning symptoms would either be the same or worse.

To conclude, somewhat comparable asthma incidence in prospective and retrospective studies was found. However, retrospective analysis of trends in asthma incidence by year of birth was severely biased compared with the prospective analysis. Retrospective evidence for an increasing trend in asthma incidence should be treated with scepticism.