Go slow to go fast: A plea for sustained scientific rigor in air pollution research during the COVID-19 pandemic

Researchers in the air pollution field are well aware that indoor and outdoor air pollution is associated with multiple disorders, including an increase in respiratory infection-related morbidity and mortality [1, 2]. Investigating associations linking air pollution and COVID-19 spread and prognosis of respiratory disease in COVID-19 patients is therefore timely, relevant and plausible. Over a ten-day period, three papers involving original research associating COVID-19 mortality and air pollution were published [3–5]. These publications attracted considerable attention from international news outlets and on social media [6, 7]. The first study was performed in the USA and associated COVID-19 mortality in 1783 counties with reported long-term (2000–2016) average air pollution levels (fine particulate matter (PM_2.5) with ∼1 km² resolution) [3]. Adjustments on the county level were made for sixteen variables, including population size, ethnicity, hospital beds, number of individuals tested for COVID-19, weather, and socioeconomic and behavioral variables such as income, obesity and smoking. The study reported that an increase of 1 μg·m⁻³ in PM_2.5 was associated with a remarkably large 15% increase in the COVID-19 death rate. An updated analysis, adjusted for three more confounding factors; days since first COVID-19 case reported, population age distribution, and days since issuance of stay-at-home order for each state, saw the risk estimate halved to an 8% increase in death rate [8].

The second study used European data and, based on simple correlation analyses, associated long term (Jan-Feb 2020) exposure to nitrogen oxides (NOx) in the troposphere (resolution ∼7*3.5 km), assessed using satellite data, and absolute numbers of COVID-19-related deaths [4]. The study made use of data from 66 administrative regions in France, Germany, Italy and Spain. In this analysis no standardisation or adjustments to basic variables as population size and age distributions were made. The result, which was strongly driven by a few data points from Italy and Spain, was interpreted as an association between NOx levels and fatal COVID-19 case counts.

The third and smallest study associated levels of NO_x and ozone between 2018 and 2019 obtained from monitoring stations across England with COVID-19 mortality in a mere 7 different English regions [5]. Positive associations were seen between levels of nitrogen dioxide and nitrogen oxide and increased COVID-19 mortality and reported number of cases, without adjustment for population size, age distribution or other confounding variables. A negative association was observed with ozone levels. The authors concluded that their study provided a useful framework to guide health policy in countries affected by the pandemic.

All three so-called ecological studies rely on aggregate data, which can suffer from the well-known problem of ecological fallacy, where a misjudgment in interpretation occurs as inferences about individuals are reasoned from the group to which the individual belongs [9]. The study design is prone to potential confounding as on an aggregate level, many factors may vary and even co-vary between geographical entities. Given the number of groups (i.e. counties) involved in the first study, and the higher quality of exposure assessment, ecological biases are clearly less likely than in the second and third study. The first study is also methodologically stronger because it is adjusted for a range of confounders at the county level, while in the two other studies, no adjustment for confounding factors was considered.

However, a major issue, mostly ignored in these studies, resides in the complexity of a potential association between air pollution and COVID-19 morbidity and mortality. In particular, the effect of air pollution on the spread of SARS-CoV-2 infection and on COVID-19 prognosis should be distinguished. The dynamics of an epidemic are dependent on transmissibility, contact rates and duration of infectiousness summarised in the basic reproduction number or R₀. This factor may differ between geographical regions and over time. Changes in R₀ can occur over time as a result of the natural history of the epidemic and infection control measures being implemented, as is shown by the levelling off of epidemic curves in many countries where physical distancing is practiced. In particular, the reproduction number can be high in early stages of an epidemic when disease recognition and control measures are less adequate. Whether air pollution affects the spread of an infectious disease, depends on whether the factors that determine the reproduction number; transmissibility, contact rates and duration of infectiousness, are influenced by air pollution. Theoretically one can imagine that each of these variables can be influenced by air pollution, by changing host susceptibility, mobility and contact patterns and severity and duration of the infection.

To explore whether air pollution influences R₀, high resolution temporal and spatial data are required, preferably supported by virus sequencing data. This far exceeds the granularity of the data used in air pollution studies and most infectious disease outbreak studies. In particular the two ecological studies which crudely correlate reported numbers of COVID-19 cases or mortality to regional air pollution levels ignored the time of introduction of COVID-19 in the different areas, did not take into account disease dynamics in any way, and ignored basic epidemiologic principles by using inadequate measures of disease frequency [10]. To study the effect of air pollution on the spread of the pandemic in ongoing studies requires knowledge about the time of the virus introduction in the study area(s) and the dynamics of the pandemic in the study area(s). When comparing different study areas, the local outbreak size needs to be accounted for. The authors of the US study seem well aware of their study's limitations related to outbreak dynamics and acknowledged that using total population size in a county as a proxy for the local size of the outbreak in a given county, may result in unmeasured confounding through partial adjustment for outbreak size [3]. The updated study probably better adjusted for outbreak dynamics by including days since first COVID-19 case reported and days since issuance of stay-at-home order for each state, resulting in a considerably smaller regression coefficient for PM_2.5 and COVID-19 mortality. The fact that the epidemic had spread more evenly across the country, also to less urbanised areas, from almost 42% of counties with COVID-19 deaths to almost 78% might also have contributed to a weaker association for PM_2.5.

A separate US study, associating COVID-19 fatality rates and mortality to PM2.5 in a three pollutant model (PM_2.5 adjusted for 0₃ and NO_x), showed a null association for PM_2,5 with COVID-19 fatality rates and a marginally significant association with mortality [11]. The use of the case fatality rate instead of COVID-19 mortality, and adjustments for epidemic timing of introduction and spread of COVID-19 by adjusting for the time the 1st and 100th COVID-19 case occurred, probably explains the marked difference in results for PM_2.5 with the other US study. The authors also indicate that control for spatial trends in exposure and residual autocorrelation in exposure between counties may also have contributed to the observed difference in PM_2.5 study results. The same study found positive and statistically significant associations for NO_x. However, it was concluded that NO_x air pollution level may act as a surrogate for urbanicity, despite of adjustment for population density.

These studies together highlights that uncovering the effect of air pollution on the spread of SARS-CoV-2 represents an enormous challenge that will require multidisciplinary collaboration and high-quality data. Other approaches, among which air pollution time series studies need to be explored when the appropriate COVID-19 data to perform such analyses become available. Such approaches should also consider the effect of changes in air pollution levels. However, changes in air pollution also occurred after introduction of measures to mitigate the impact of the pandemic. Here confounding still plays a major role as many additional factors changed at the same time (e.g. physical activity, stress, social economics, medical care visits) introducing other challenges in the data analyses.

In the meantime, multiple other studies have been published, many of which can be found in a useful study repository [12]. However, many of these studies did not take into account COVID-19 outbreak dynamics. This leads to similar biases are described here.

The effect of air pollution on disease prognosis can be studied using more conventional approaches after COVID-19 infection. For instance by following up in time confirmed patients in different regions with different levels of air pollution. However, even this type of study might be complicated by confounding: for instance disease severity and testing policies can differ by treatment center, and the fact that quality of care delivered by health care systems may differently vary across regions depending on the size of the local outbreak. These scenarios are more classical examples of confounding for which a range of solutions may be considered. The use of test-negative designs has been proposed in which risk factors are compared between subjects that test positive or negative to SARS-CoV-2 [13]. Such designs can eliminate some of the aforementioned concerns (access to testing etc.), but has other limitations, mainly related to shared risk factors for being tested regardless of the outcome (e.g. presence of respiratory disease).

We certainly do not want to argue that air pollution does not play a role in the COVID-19 pandemic. There is previous evidence from the SARS coronavirus epidemic [14] (albeit that this study suffers from the same ecological problem as the current studies), similarities with a possible smoking contribution to the pandemic [15], mechanistic evidence involving the angiotensin-converting enzyme 2 (ACE-2) receptor in COVID-19 pathology that could be influenced by air pollution [16], and the fact that air pollution has previously been causally linked to risk of pneumonia and other co-morbidities that contribute to COVID-19 deaths [17]. However, determining if there is a causal effect and obtaining an accurate estimate of the effect requires rigorous and time consuming research.

A final major point of criticism that we wish to make is that one of the aforementioned papers was accepted for publication after a 1-day review period [4], while the other papers made headlines before the peer-review process based on preprint manuscripts [3, 5, 16]. It is important to ask why is this the way COVID-19 air pollution research is being reviewed and communicated? Research in this field may be of relevance for the spread of infectious disease and disease prognosis, but outcomes from air pollution research are unlikely to lead to changes in the implementation of pandemic control measures and in the clinical care of patients. Instead, air pollution and infectious disease epidemiology research will have an impact in the longer term, as to fully understand etiology and co-contributing factors. Thus, there seems little reason to abandon the rigorous peer review process and expedite manuscript publication for COVID-19 air pollution studies. Study quality is of great importance and all stakeholders, including the scientific community, policy makers and the general public, are served best by well-conducted and thoroughly appraised studies on air pollution and COVID-19. We need to avoid creating myths that are diverting political and scientific attention from other urgent and actionable questions. The sentence go slow to go fast is used in industry to combat the short-sighted fixation on speed while sacrificing product quality and hence utility. In the case where research does not have immediate impact on clinical and societal interventions related to the current pandemic, we may want to adopt this philosophy. Air pollution researchers and scientific journal editors have an important role in maintaining calm scientific rigor during this tumultuous time.