Abstract
The awake prone position (AP) strategy for patients with acute respiratory distress syndrome (ARDS) is a safe, simple, and cost-effective technique used to improve hypoxemia. We aimed to evaluate intubation and mortality risk in patients with coronavirus disease (COVID-19) who underwent AP during hospitalisation.
In this retrospective, multicentre observational study conducted between May 1 and June 12, 2020 in 27 hospitals in Mexico and Ecuador, non-intubated patients with COVID-19 managed with AP or supine positioning were included to evaluate intubation and mortality risk through logistic regression models; multivariable and centre adjustment, propensity score analyses, and E-values were calculated to limit confounding. This study was registered at https://clinicaltrials.gov/ct2/show/NCT04407468
827 non-intubated patients with COVID-19 in the AP (n=505) and supine (n=322) groups were included for analysis. Less patients in the AP group required endotracheal intubation (23.6% versus 40.4%) or died (20% versus 37.9%). AP was a protective factor for intubation even after multivariable adjustment (OR=0.39, 95%CI: 0.28–0.56, p<0.0001, E-value=2.01), which prevailed after propensity score analysis (OR=0.32, 95%CI: 0.21–0.49, p<0.0001, E-value=2.21), and mortality (adjusted OR=0.38, 95%CI: 0.25–0.57, p<0.0001, E-value=1.98). The main variables associated with intubation amongst AP patients were increasing age, lower baseline SpO2/FiO2, and management with a non-rebreather mask.
AP in hospitalised non-intubated patients with COVID-19 is associated with a lower risk of intubation and mortality.
Abstract
Awake prone positioning in non-intubated hospitalised patients with COVID-19 was associated with a lower risk of intubation and mortality in this multicentre observational study.
Introduction
The awake prone position (AP) in non-intubated patients with acute hypoxemic respiratory failure results in improved oxygenation, as demonstrated by an increase in arterial partial pressure of oxygen (PaO2), peripheral arterial oxygen saturation (SpO2), and PaO2/inspired oxygen fraction (PaO2/FiO2), without deleterious effects on the level of partial arterial pressure of carbon dioxide (PaCO2), pH, respiratory rate (RR), or haemodynamics [1, 2]. The physiological mechanism by which prone positioning is useful for ARDS is by increasing functional residual capacity, reducing dead space, reducing intrapulmonary shunts, increasing ventilation in areas dependent of gravity, and relieving the weight that the heart exerts over the lungs [3].
The coronavirus disease (COVID-19) pandemic has unleashed a high global demand for respiratory support, a reason why AP in non-intubated patients has become popular and clinical interest has rapidly increased. AP combined with non-invasive ventilation (NIV) or high-flow nasal cannula (HFNC) in patients with moderate to severe acute respiratory distress syndrome (ARDS) [4, 5] and COVID-19 [6–8] has been shown to be safe and may prevent intubation. One further advantage of AP is that it allows patients to interact with their family during hospitalisation, thereby favouring humanisation of healthcare [9]. Nonetheless, few observational studies have evaluated AP against control groups (i.e. awake supine patients managed with NIV or HFNC) with conflicting findings [10–12]. Thus, the utility of AP remains to be further elucidated in larger observational or randomised studies.
In this multicentre retrospective observational study, we sought to evaluate intubation and mortality risk in conscious patients with COVID-19 who underwent AP during hospitalisation.
Methods
Study design
A multicentre retrospective cohort study was conducted with patients diagnosed with COVID-19 admitted to 27 hospitals in Mexico and Ecuador (Appendix 2) from the emergency department. The study was approved by the Health Services Research Committee of the State of Querétaro (registration number 1178/SESEQ-HGSJR/08-05-20) and all other participating centres. This study was prospectively registered in ClinicalTrials.gov (NCT04407468); STROBE recommendations were followed during the reporting of this study.
Study population and data collection
In each participating hospital centre, data collection was carried out by medical specialists in emergency medicine, respiratory medicine, anaesthesiology, and intensive care medicine, who collected information from patients’ medical records. A separate group of physicians were appointed to review the data obtained and check for plausibility. In cases of doubt physicians in charge at each centre were contacted. All patients were followed-up during their entire in-hospital stay, until discharge or in-hospital death.
Patients were deidentified by assigning them a code. All patients admitted to the emergency department during the period between May 1 and June 12, 2020 who met the following criteria were considered for inclusion in the study: 1. Age ≥18 years; 2. Positive test for SARS-CoV-2 or imaging study compatible with COVID-19 (see section ahead); 3. clinical record available in accordance with the official Mexican standard NOM-004-SSA3-2012 (http://dof.gob.mx/nota_detalle_popup.php?codigo=5272787) or equivalent in Ecuador; and 4. Room-air peripheral arterial oxygen saturation (SpO2) <94% upon admission to the emergency department, and 5. two or more of the following symptoms: eye pain, cough, fever, dyspnoea, headache, myalgia, arthralgia, or odynophagia.
Due to the differences in funding and infrastructure between centres, two criteria were employed to standardise COVID-19 diagnosis: 1. A positive RT-PCR test for SARS-CoV-2 from a respiratory tract sample; or 2. Chest computed tomography (CT) scan with a COVID-19 Reporting and Data System (CO-RADS) score ≥3 (Appendix 3) [13]. The latter imaging criterion was applied only for patients in whom RT-PCR was not performed.
Exclusion criteria included: 1. Patients who were voluntarily discharged; 2. patients referred to another hospital prior to outcome ascertainment, and 3. those with incomplete clinical records (insufficient information to calculate SpO2/FiO2 ratio, or when unable to ascertain if the patient was managed in a prone or supine position).
Data recorded were demographic (age, sex) and clinical variables including comorbidities (diabetes, systemic arterial hypertension, obesity, heart disease, lung disease, cancer, liver disease, chronic kidney disease), pre-prone SpO2/FiO2 ratio (SpO2/FiO2 ratios of 235 and 315 correlate with SpO2/FiO2 ratios of 200 and 300) [14], post-prone SpO2/FiO2 (within one hour after proning), time-to-initiation of prone positioning (defined as the time elapsed from hospital admission to first successful attempt in prone lasting ≥2 h), total time in AP, type of care (emergency room, hospitalisation, or intensive care unit [ICU]), medications, supplemental oxygen delivery device used, need for orotracheal intubation, and lethal outcome. FiO2 was calculated based on the type of supplemental oxygen delivery device employed: low-flow nasal cannula, high-flow nasal cannula or non-rebreather mask (Appendix 4) [15].
Exposures and outcomes
Awake, spontaneously breathing patients managed with non-invasive oxygen devices who were able to remain in the prone position for at least 2 continuous hours were considered as patients in the AP group (main exposure); those not meeting this criterion or in whom prone positioning was not attempted at all, were considered as the comparison group (awake supine). The primary outcome was successful orotracheal intubation for invasive mechanical ventilation and the secondary outcome was death during in-hospital follow-up. Factors associated with intubation amongst patients in the AP group were also evaluated.
The decision to place patients in the prone position and perform orotracheal intubation were based on individualised medical criteria and were not priorly defined or standardised. Patients were managed with low-flow nasal cannula, non-rebreather mask, or high-flow nasal cannula; other non-invasive ventilation devices were either not used or unavailable across all centres.
Sample size
Sample size was calculated to observe a 10% difference of the incidence of intubation based on that reported by Argenziano et al. [16]. The calculated sample size was 309 subjects per group (Appendix 5). Convenience sampling for the original cohort was employed, with further propensity score-matched sampling performed to reduce bias.
Statistical analysis
The clinical and demographic characteristics of the patients were examined for all patients and for those in the AP or awake supine groups. Descriptive results for quantitative variables are presented as mean with standard deviation (sd) or median with interquartile range (IQR), and frequencies with percentage (%) for qualitative variables. Asymmetry and kurtosis were calculated for quantitative variables. Quantitative comparisons were performed with the independent-samples t-test; qualitative comparisons were done with chi-squared, chi-squared of trend, or Fisher's exact test. Baseline and post-AP SpO2/FiO2 ratios were compared with the dependent-samples t-test. The PH-Covid19 mortality score was calculated as described in the original model development and validation study [17].
To reduce the risk of bias due to unbalanced groups, propensity score analysis was performed through a logistic regression model adjusted for age, sex, the presence of 3 or more comorbidities, baseline SpO2/FiO2 ratio, supplemental oxygen device, ICU attention, and treatment with systemic steroids, enoxaparin, tocilizumab, or ceftriaxone. Patients were matched in a 1:1 ratio according to the nearest-neighbour matching algorithm; changes in density functions are shown in Appendix 6. All inferential analyses were performed for all patients in the original cohort and for the propensity score-matched cohorts.
Distinct multivariable logistic regression analyses were performed to determine the risk of orotracheal intubation and mortality associated with AP. Variables included in the models were selected by the Enter method; adjustment variables were those which had a p value <0.1 in univariate analyses which have been reported to be associated with higher (or lesser) risk for adverse events (age, sex [men], ICU attention, diabetes, systemic arterial hypertension, obesity, heart disease, cancer, chronic kidney disease), pre-prone SpO2/FiO2 ratio, supplemental oxygen delivery device, ceftriaxone, enoxaparin, tocilizumab, oseltamivir, and systemic steroids). A multivariable logistic regression model was subsequently created to determine the risk of intubation amongst patients who tolerated AP; the variables included in this model were selected with the Stepwise Forward method, including those with a p<0.1 in the final model. Odds ratios (OR) with their 95% confidence interval (95%CI) were calculated. The goodness of fit of the final models were evaluated with the Hosmer-Loemeshow statistic, and the discrimination of the model was determined by calculating the area under the curve (AUC). The risk of intubation amongst AP patients according to age and baseline SpO2/FiO2 ratio were graphed through the smoothing spline method.
Sub-analyses of intubation and mortality risk for patients who had a positive RT-PCR for SARS-CoV-2 (excluding patients in whom RT-PCR was not available but had a compatible CO-RADS study) were performed in the unmatched and propensity-score matched cohorts through logistic regression models; the size of effect was adjusted for the same variables as the main analyses.
E-values for the lower bound of the confidence intervals were calculated to determine the value at which an unmeasured confounding factor could potentially alter the observed effect of AP on the outcomes and drive them to a non-significant value [18]. Regression analyses were verified through residual analysis.
To determine the variability of the association between AP and intubation rates across different centres, multicentre adjustment was performed through generalised estimating equations (GEE); the centre with the lowest intubation rate throughout the entire study period was set as the reference. The main effect of every centre and AP were calculated in the same model, as well as their interaction within the model.
A systematic search of studies of AP was conducted; the search strategy and inclusion criteria for studies are provided in Appendix 7. Results of eligible studies were summarised alongside the propensity score-matched cohort of APRONOX through a random-effects model in a forest and funnel plot of the overall risk of intubation for patients in AP versus supine position.
Missing values were not imputed. A p-value <0.05 was used to define bilateral statistical significance. All analyses and graphs were created with the SPSS software v.21, R software v.3.4.2, and RevMan 5.3.
Results
Out of 932 patients identified across all 27 hospital centres, 827 patients were ultimately included for analysis (figure 1). Descriptive results for all patients are provided in table 1. Amongst all 927 patients, 227 (27.4%) were female and mean age was 54.3 (sd:14.2) years, with most patients being in the 50 to 59-year category (25.3%). The most prevalent comorbidities were diabetes (38.1%) and hypertension (34.5%). Most patients were managed with low-flow nasal cannulas (48.6%). Out of 249 patients who underwent orotracheal intubation, 69.9% (n=174) died during in-hospital follow-up. In comparison, out of 578 patients who were not intubated, 8.0% (n=46) died (p<0.0001).
Flow diagram of participants included in the APRONOX cohort.
Demographic and clinical characteristics at hospital admission and outcomes of patients in the APRONOX cohort
The characteristics of patients in the AP and supine groups, in both the unmatched and matched cohorts, are provided in table 2. Patients managed in AP had a median time-to-initiation of prone positioning of 15.5 (IQR: 8–48) hours. The median time spent in the prone position during the hospital stay (total time in prone) was 12 (IQR: 8–24) hours. A lesser proportion of patients in the AP group required endotracheal intubation (23.6% versus 40.4%) or had a lethal outcome (19.8% versus 37.3%). After propensity score matching, these differences prevailed. The SpO2/FiO2 ratio in the AP group was statistically significantly higher after prone (217.42, sd: 81.9) compared with baseline values (182.39, sd: 81.91), with a mean difference of 35.03 (95%CI: 29.99–40.06, p<0.0001) units.
Comparison of demographic and clinical characteristics at hospital admission and outcomes of patients in the awake prone and supine groups in both the unmatched and propensity score-matched cohorts.
The results of univariable logistic regression models for orotracheal intubation risk are provided in table 3, for both the unmatched and matched cohorts. The main risk factors identified were age, diabetes, arterial hypertension, obesity, heart disease, cancer, a baseline SpO2/FiO2 <100 or between 100 and 199, and management with a non-rebreather mask. AP was a protective factor for orotracheal intubation even after multivariable adjustment (table 4) for confounding variables (Adjusted OR=0.35, 95%CI: 0.24–0.52, p<0.0001, E-value=2.12), which prevailed after propensity score analysis (Adjusted OR=0.41, 95%CI: 0.27–0.62, p<0.0001, E-value=1.86). Similarly, AP was a protective factor for mortality (Adjusted OR=0.38, 95%CI: 0.26–0.55, p<0.0001, E-value=2.03, Goodness of fit: Hosmer-Lemeshow X2=10.2, p=0.3 AUC=0.78, 95%CI: 0.74–0.81, p<0.0001) even after multivariable adjustment in propensity score analyses (Adjusted OR=0.40, 95%CI: 0.27–0.61, p<0.0001, E-value=1.88, Goodness of fit: Hosmer-Lemeshow X2=7.81, p=0.4 AUC=0.78, 95%CI: 0.74–0.82, p<0.0001). Lower intubation and mortality risks for AP prevailed after sub-analyses of patients with a confirmatory SARS-CoV-2 RT-PCR (excluding those in whom molecular testing was not performed) (Appendix 8).
Results of univariable logistic regression analyses of orotracheal intubation risk in patients with awake prone positioning.
Results of multivariable logistic regression analyses of orotracheal intubation risk in patients with awake prone positioning, adjusted by confounding variables.
After adjusting for centre through GEE, 9 centres had an effect over the risk of intubation. Despite this, AP continued to be associated with lower intubation risk (OR: 0.22, 95%CI: 0.15–0.34, p<0.0001); the interaction between centre and AP was non-significant for all the centres.
The main variables associated with intubation amongst AP patients were increasing age (OR=1.02, 95%CI: 1.01–1.04, p=0.005), SpO2/FiO2 <100 (OR=2.78, 95%CI: 1.35–5.72, p=0.005), SpO2/FiO2 100–199 (OR=2.18, 95%CI: 1.31–3.64, p=0.003), and management with a non-rebreather mask (OR=2.17, 95%CI: 1.34–3.49, p=0.002), Goodness of fit: Hosmer-Lemeshow X2=10.52, p=0.2; AUC=0.70, 95%CI: 0.64–0.74, p<0.0001. The distribution of risk for increases in age and baseline SpO2/FiO2 are shown in figure 2.
Risk of intubation amongst patients in the awake prone positioning group, according to age (a) and baseline SpO2/FiO2 (b)*. *For this analysis, baseline SpO2/FiO2 was studied as continuous variable, therefore, the range of odds ratios are different from others in the manuscript which consider baseline SpO2/FiO2 as a categorical variable and use a category of reference to compare other categories. 95%CI: 95% confidence intervals; FiO2: Inspired oxygen fraction; SpO2: peripheral arterial oxygen saturation.
After the search of the literature, 99 records were retrieved, of which only 9 studies [10–12, 19–24] were observational comparison-group studies including both AP and supine patients, with sufficient information to calculate the overall risk of intubation, which are summarised alongside the APRONOX study in figure 3; the funnel plot is provided as Appendix 9.
Forest plot of overall risk of orotracheal intubation in studies retrieved by the search strategy and the APRONOX cohort. *Only patients in the propensity score-matched cohorts were included for the APRONOX study. 95%CI: 95% confidence intervals; M-H: Mantel-Haenszel [37].
Discussion
In this multicentre observational study, we aimed to evaluate the association between awake prone positioning and orotracheal intubation, as well as predictors of intubation amongst AP patients, and mortality in hospitalised patients with COVID-19. Even after multivariable adjustment and propensity score analyses, prone positioning in non-intubated patients was associated with lower intubation and mortality risk.
Patients in our cohort were younger (mean age 53.4 years) than those in other studies (56.0–65.8) [10–12]; hospitalised patients with COVID-19 in Mexico have been reported to be young [25]. The prevalence of comorbidities in our study is similar to that reported in a population-based sample of Mexican patients hospitalised with COVID-19, although diabetes was more common in our study (38.1% versus 29.2%), whereas obesity (14.4% versus 22.5%) and heart disease (2.1% versus 4.4%) were less frequent [25].
The total time spent in the prone position during in-hospital stay in our study was 12 (IQR: 8–24) hours, which is considerable compared to a recent pilot randomised study which reported that self-proning patients spent only 1.6 (95%CI: 0.2–3.1) hours in the prone position in a 72-h evaluation period [26]. Daily time spent in the prone position has been reported to be highly variable, with only 43% of patients achieving a daily dose of ≥6 h in AP [27].
The overall intubation rate in the APRONOX cohort was higher (30.1%) than that reported for hospitalised patients with COVID-19 in Mexico City (20.2%) [25]; however, limited access to beds with ventilators in Mexico has been reported [28]. Intubation rates for patients in the unmatched AP (23.6%) and supine (40.4%) cohorts fall within those reported in previous studies (10–58% and 27.7–49%, respectively) [10–12]. AP in our study was associated with decreased intubation risk even after multivariable adjustment in both the unmatched and propensity-score matched cohorts, with an E-value of 2.01 and 2.21, respectively, which reflects that in order to drive this association to be non-significant, an unmeasured risk factor should have a lower-limit confidence interval that at least doubles the risk of the outcome between both groups. Out of all comorbidities, only diabetes and heart disease were associated with increased intubation risk after multivariable adjustment, however, diabetes was no longer a risk factor after propensity score analysis. A higher baseline SpO2/FiO2 was associated with reduced intubation risk. The mortality rate reported in our study was 19.8%, comparable to 23.4% [12] and 27% [10] in other studies.
Regarding variables associated to intubation amongst AP patients, age, low SpO2/FiO2, and the use of a non-rebreather mask were the main variables associated. The distribution of risk for quantitative values of age show that the risk of intubation after AP is higher with increasing ages, whereas higher baseline SpO2/FiO2 have the lowest risks.
AP has been presented as one the most cost-effective strategies to treat patients with COVID-19. In countries with limited oxygen delivery devices, and a shortage of ventilators, AP could be used to avoid intubating patients with COVID-19 [29]. Nonetheless, conflicting evidence from observational studies for AP exists.
The supine position alters pulmonary function in patients with respiratory insufficiency due to the gravitational differences between dependent and non-dependent regions, resulting in a more negative pleural pressure (Ppl), increasing transpulmonary pressure (TPP) in non-dependent areas (more distension), and producing the opposite effect in dependent areas where Ppl is less negative and TPP is lower (less distension). Ventilation in the prone position causes even distribution of TPP, favouring uniform ventilation [30]. Approximately 45 years ago, prone positioning was shown to increase oxygenation in patients with respiratory insufficiency, primarily by improving the ventilation-perfusion ratio (V/Q) [31].
Prone positioning has been evaluated in hospitalised patients with respiratory failure due to COVID-19, having observed improvements in SpO2 and PaO2, decreased respiratory rate (RR), decreased need for intubation and possible reductions in mortality, in addition to being cost-free [8, 32–35]. As summarised in figure 4, only three other studies to date have evaluated intubation risk among AP compared with AS. While Ferando et al. and Padrão et al. found no differences in intubation risk, Jagan et al. found reduced intubation risk in AP patients [10–12]. The APRONOX study is the largest study to date evaluating the effect of AP on intubation risk.
Regarding oxygenation modality, the use of a non-rebreather mask was associated with greater risk of intubation amongst all patients and within AP patients, whereas other oxygenation devices were not. There is documented evidence of the correlation between the oxygen saturation/fraction of inspired oxygen (SpO2/FiO2) ratio and the partial pressure of oxygen/fraction of inspired oxygen (PaO2/FiO2) ratio, with the advantage that the SpO2/FiO2 ratio only relies on a pulse oximeter, with no need to perform a blood gas test, thereby highlighting the value of validated cost-effective strategies [14].
Our study has the following limitations: 1) O2 delivery devices were not standardised to a unique device, 2) the number of hours of AP varied between hospitals and patients, 3) no standardised criteria were established to consider intubation in patients requiring mechanical ventilation, 4) we were unable to asses which patients had do-not-intubate orders or other reasons for not performing intubation, 5) availability of laboratory studies was limited across centres and were thus not collected and analysed, 6) not all patients with a CO-RADS score ≥3 ultimately have a positive RT-PCR test [13]; this limitation was partially addressed by sub-analysing patients with a positive SARS-CoV-2 RT-PCR, 7) a measure of oxygenation comparable to post-prone SpO2/FiO2 in AP patients was not collected for patients in the supine group, and 8) the length of stay of patients was not collected.
The strengths of our research include: 1) this is the largest study evaluating AP to date; 2) the large number of hospitals included; and 3) the fact that various O2 delivery devices were employed may reflect that the benefits of AP are not necessarily unique to NIV or HFNC devices, which are costlier and not always available.
AP in spontaneously breathing patients with acute hypoxemic respiratory insufficiency may be a justifiable treatment modality, given the improvements in oxygenation and its physiological benefits, but the decision to intubate is based on the clinician's best judgement and intubation should not be delayed if under consideration. Close clinical evaluation of patients is key to avoid poor outcomes. Studies of AP are challenging and randomised controlled trials are warranted to fully elucidate its usefulness since this is an easy to administer, safe, and reproducible intervention [36].
Conclusion
Prone positioning in awake hospitalised patients with COVID-19 is associated with a lower risk of intubation and mortality.
Acknowledgements
Healthcare workers treating COVID-19 patients: Edgard Díaz Soto, Jaziel López Pérez, José Antonio Meade Aguilar, Rubén Rodríguez Blanco, José Luis Patiño Pérez, Janisia Rodríguez Solís, Maribel Santosbeña Lagunes, Alberto Calvo Zúñiga, Manuel de Jesús Santaella Sibaja, Luis Iván Contreras Ley, María Alejandra Sicsik Aragón, Yessica Bernal Luna, Carlos Baez Ambriz, Yanira Jiménez Blancas, Alejandro Ayala Mata, Tania Gabriela Ramírez Lira, Iván Avalos Flores, Edwing Díaz Rodríguez, Roberto Robles Godínez, Eduardo Espino López, Hugo Francisco Díaz Ramírez, Concepción Mendoza Fragoso, Oliver Garaz Trujillo, and Jesús Elías Paredes Flores. We thank the anonymous reviewers for their recommendations which allowed us to make significant improvements to our manuscript.
Footnotes
This article has supplementary material available from erj.ersjournals.com
Ethical statement: This study was approved by the Health Services Research Committee of the State of Querétaro (registration number 1178/SESEQ-HGSJR/08-05-20) and all other participating centres.
Conflict of interest: Dr. Perez-Nieto has nothing to disclose.
Conflict of interest: Dr. Escarraman-Martínez has nothing to disclose.
Conflict of interest: Dr. Guerrero-Gutierrez has nothing to disclose.
Conflict of interest: Dr. Zamarron-Lopez has nothing to disclose.
Conflict of interest: Dr. Mancilla-Galindo has nothing to disclose.
Conflict of interest: Dr. Kammar-García has nothing to disclose.
Conflict of interest: Dr. Martinez-Camacho has nothing to disclose.
Conflict of interest: Dr. Deloya-Tomás has nothing to disclose.
Conflict of interest: Dr. Sanchez-Diaz has nothing to disclose.
Conflict of interest: Dr. Macías-García has nothing to disclose.
Conflict of interest: Dr. Soriano-Orozco has nothing to disclose.
Conflict of interest: Dr. Cruz-Sánchez has nothing to disclose.
Conflict of interest: Dr. Salmeron-Gonzalez has nothing to disclose.
Conflict of interest: Dr. Toledo-Rivera has nothing to disclose.
Conflict of interest: Dr. Mata-Maqueda has nothing to disclose.
Conflict of interest: Dr. Morgado-Villaseñor has nothing to disclose.
Conflict of interest: Dr. Martinez-Mazariegos has nothing to disclose.
Conflict of interest: Dr. Flores-Ramirez has nothing to disclose.
Conflict of interest: Dr. Medina-Estrada has nothing to disclose.
Conflict of interest: Dr. Ñamendys-Silva has nothing to disclose.
- Received January 28, 2021.
- Accepted June 26, 2021.
- Copyright ©The authors 2021.
This version is distributed under the terms of the Creative Commons Attribution Non-Commercial Licence 4.0. For commercial reproduction rights and permissions contact permissions{at}ersnet.org
References
