Abstract
Background The effects of convalescent plasma (CP) therapy hospitalised patients with coronavirus disease 2019 (COVID-19) remain uncertain. This study investigates the effect CP on clinical improvement in these patients.
Methods This is an investigator-initiated, randomised, parallel arm, open-label, superiority clinical trial. Patients were randomly (1:1) assigned to two infusions of CP plus standard of care (SOC) or SOC alone. The primary outcome was the proportion of patients with clinical improvement 28 days after enrolment.
Results A total of 160 (80 in each arm) patients (66.3% were critically ill and 33.7%, severe) completed the trial. The median age was 60.5 years (interquartile range [IQR], 48–68), 58.1% were men and the median time from symptom onset to randomisation was 10 days (IQR, 8–12). Neutralising antibodies titres >1:80 were present in 133 (83.1%) patients at baseline. The proportion of patients with clinical improvement on day 28 was 61.3% in the CP+SOC and 65.0% in the SOC group (difference, −3.7%; 95% Confidence Interval [CI], −18.8%-11.3%). The results were similar in the subgroups of severe and critically ill. There was no significant difference between CP+SOC and SOC groups in prespecified secondary outcomes, including 28-day mortality, days alive and free of respiratory support and duration of invasive ventilatory support. Inflammatory and other laboratorial markers values on days 3, 7 and 14 were similar between groups.
Conclusions CP+SOC did not result in a higher proportion of clinical improvement on at day 28 in hospitalised patients with COVID-19 compared to SOC alone.
Introduction
Coronavirus disease 2019 (COVID-19), caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), can cause severe illness in a considerable proportion of infected patients leading to severe progressive pneumonia, multiple organ dysfunction and death [1, 2].
Passive immunotherapy using convalescent plasma (CP) collected from COVID-19 recovered patients has been advocated for the treatment of severe cases of this disease [3]. The US Food and Drug Administration issued an emergency use authorisation for CP for the treatment of hospitalised patients with COVID-19 based on results of observational studies showing that CP was safe and could be associated with better clinical outcomes [4, 5]. Nevertheless, the two existing randomised clinical trials at the time of authorisation [6, 7] and further multicentre randomised clinical trials [8, 9] have failed to demonstrate significant clinical benefit of CP in patients with severe COVID-19. The long duration of disease when intervention occurred and low neutralising antibody titres in administered plasma may, at least partially, explain the absence of significant improvement in clinical outcomes in intervention groups in two of these trials [6, 7]. Other two larger clinical trials also did not find any benefit of CP on clinical outcomes. However, these studies used anti-SARS-CoV-2 spike IgG as a surrogate for neutralising antibodies titres, impairing inferences that could be done on the baseline patient status regarding these antibodies and the investigated intervention [8, 9].
Given the heterogeneity regarding CP characteristics, including volume, number of doses and neutralising antibody titres, as well as distinct levels of pre-existing antibody titres at baseline in both intervention and control groups, further clinical trials with different administration strategies and distinct populations are necessary to better define the role of this therapy in hospitalised patients with severe COVID-19. In the present randomised clinical trial, we assessed the effect of two doses of 300 mL of CP therapy administered in the first 14 days of symptoms onset on clinical improvement in severe and critically ill COVID-19 patients.
Methods
Study design and oversight
PLACOVID was an investigator-initiated, unicentric, randomised, parallel arm, open-label, superiority clinical trial performed at a single COVID-19 reference hospital from Porto Alegre, Brazil.
This study was approved by the Brazilian National Commission for Research Ethics and the institutional review board of Hospital de Clínicas de Porto Alegre (approval number, 20-0158). Written informed consent was obtained from all study participants or their legal representatives. The trial was overseen by an external and independent data and safety monitoring board (DSMB). The trial protocol and statistical analysis plan are available in Supplementary Material 1. The trial was registered with the number NCT04547660 (https://clinicaltrials.gov/ct2/show/NCT04547660).
Participants
Patients admitted to the hospital were assessed for eligibility if they were 18 or older, had a positive reverse transcriptase polymerase chain reaction (RT-PCR) for SARS-CoV-2 (Supplementary Material 2), had less than 15 days of initial symptoms onset, and had severe respiratory disease, as defined by the presence of at least one of the following: respiratory rate >30 breaths per minute in room air; oxygen saturation (O2) ≤93% in room air; arterial partial pressure of oxygen (PaO2)/fraction of inspired oxygen (FiO2)≤300; need for supplemental O2 to maintain O2 saturation >95%; need for supplemental O2 by high flow nasal cannula, non-invasive ventilation, or invasive mechanical ventilation. Exclusion criteria were impossibility for any reason to perform the first plasma infusion within 14 days of the onset of symptoms; use of immunosuppressive drugs for other non-COVID-19 underlying diseases in the last 30 days before enrolment; pregnancy; history of serious adverse reactions such as transfusion anaphylaxis; disagreement of attending physician; and participation in other interventional randomised clinical trials.
Plasma donation procedures
A full description of plasma donation selection and procedures is shown in Supplementary Material 2.
Randomisation and interventions
Patients were randomly assigned in a 1:1 ratio to receive two infusions 48 h apart of 300mL of CP plus Standard of Care (SOC) or SOC alone. Randomisation was performed using computer-generated randomisation with random block sizes of 2 or 4 and stratified according to the unit of hospitalisation on enrolment (medical ward or intensive care unit [ICU]; unit of hospitalisation on enrolment was used as a proxy for disease severity). Patients and investigators were unmasked, except interviewers performing follow-up telephone calls, who was unaware of the assigned trial group.
The SOC for COVID-19 was at the discretion of the treating physicians. The use of glucocorticoids, other immunomodulators, antibiotic agents, and antiviral agents was allowed. Remdesivir was not available in Brazil during the trial period.
Clinical and laboratory data
Definitions of baseline variables assessed in the baseline are presented in the Supplementary Methods. Neutralising antibodies were determined in all donor plasma units and on patient serum collected on days 0 and 3 (after the second plasma infusion) after enrolment, following previously described protocol [10]. Nasal and oropharyngeal swabs were collected at day 7 after enrolment or at hospital discharge. Blood samples were collected on days 0 (pre-infusion), 3 (post second infusion), 7 and 14 after enrolment in hospitalised patients.
Outcomes and follow-up
The primary outcome was the proportion of patients with clinical improvement 28 days after enrolment. Clinical improvement was defined as hospital discharge or reduction of 2 points in a 6-level ordinal scale. Levels on the scale were defined as follows: a score of 1 indicated not hospitalised; 2, hospitalised and not receiving supplemental oxygen; 3, hospitalised and receiving supplemental oxygen; 4, hospitalised and receiving oxygen supplementation administered by a high-flow nasal cannula or noninvasive ventilation; 5, hospitalised and receiving mechanical ventilation or extracorporeal membrane oxygenation; and 6, death. Prespecified secondary outcomes included RT-PCR for SARS-CoV-2 from nasal and oropharyngeal swab at day 7 from enrolment or hospital discharge (if earlier than 7 days); clinical status assessed using the 6-level ordinal scale and all-cause mortality at days 14 and 28 after enrolment; time to hospital discharge and days alive and free of supplemental oxygen support (non-survivors and patients requiring oxygen support at day 28 were assigned as 0 supplemental oxygen support free-days) within 28 days from enrolment; Sequential Organ Failure Assessment (SOFA) score and National Early Warning Score 2 (NEWS) 2 on day 7 after enrolment; and length of invasive ventilatory support (for those who received mechanical ventilation). Adverse events were assessed using the Common Terminology Criteria for Adverse Events (CTCAE) grade 3 or 4 adverse up to day 28 after enrolment or hospital discharge. Other prespecified exploratory outcomes were levels of serum inflammatory markers and cytokines, measured on days 3, 7 and 14 after enrolment (Supplementary Material 2).
Patients were followed daily up to day 28 after enrolment or hospital discharge by researchers who were aware of the trial-group assignments. For patients who were transferred to another hospital before day 28, a structured telephone call to the patient or the patient's family was conducted by an interviewer to assess the level on the ordinal scale at day 28.
Sample size calculation and protocol changes
We had originally planned for the trial to include 160 patients considering all-cause mortality within 28 days as the primary outcome and an absolute difference between arms of 20% to achieve a power of 80%, using the formula for two binomial proportions and two-sided tests, as described by Rosner [11]. However, due to the evolving knowledge on COVID-19, the steering committee assumed that a reduction of 20% in mortality would be very unlikely to occur and that estimated proportions for survival and death within 28 days were better suited for clinical response. Therefore, it was decided to submit a protocol amendment on July 27, 2020 (when eight patients had been included in the trial) modifying the primary outcome to clinical improvement on day 28 after enrolment.
In the revised sample size calculation, assuming a proportion of clinical improvement of 60% in the SOC group (Supplementary Material 2), a sample of 160 patients (80 in each arm) was estimated to achieve a power of 80% to detect an absolute difference of 20% or greater in the proportion of patients with clinical improvement at day 28 with a 2-sided α level of 0.05. Other modifications are detailed in the study protocol in Supplementary Material 1.
Statistical analysis
Data were primarily analysed according to the intention to treat principle. The proportion of patients with clinical improvement on day 28 and relative risk were assessed using robust Poisson regression. Prespecified subgroups were defined according to the unit of hospitalisation (medical ward [considered severe patients] or ICU [considered critically ill patients]) and mechanical ventilation needed on enrolment. Consistency of intervention effects on the primary outcome across these subgroups was assessed by means of interaction tests.
A post-hoc sensitivity analysis was performed for the primary outcome considering a per-protocol population. Secondary outcomes were compared Generalized Linear Models, according to the probability distribution of the outcome, or with the Wilcoxon-Mann Whitney test as appropriate. The potential effect of variables with a p value ≤0.20 at the baseline on the primary outcome was addressed in Poisson regressions models. Also as an exploratory analysis considering the clinical improvement outcome as a reduction of 1 point in the ordinal scale.
One pre-planned interim analysis for efficacy and safety evaluation after 80 patients with complete follow-up was conducted (Supplementary Material 2). The stopping rule for efficacy and safety was a p value<0.05. There was no adjustment in the final threshold for statistical significance for sequential analysis.
All analyses were performed using the R software version 4.0.2 (R Core Team). No adjustments for multiplicity were performed. Thus, the results of secondary outcomes and subgroup analyses should be interpreted as exploratory. A 2-sided p value of less than 0.05 was considered statistically significant.
Results
Patients
From July 15 to December 10, 2020, 496 patients were assessed for eligibility. Of these, 160 were eventually enrolled: 80 in the CP+SOC group, and 80 in the SOC alone group (figure 1). The follow-up was completed on January 7, 2021. A total of 106 (66.3%) patients were located at the ICU and 54 (33.7%) at the medical ward at randomisation. Baseline characteristics are shown in table 1. The median age was 60.5 years (interquartile range [IQR], 48.0–68.0), 93 (58.1%) were men, and the median time from symptom onset to randomisation was 10 days (IQR, 8–12). A total of 133 (83.1%) patients presented neutralising antibody titres above 1:80 at randomisation (median, 1:1280; IQR, 1:320–1:2560). All but 2 (1.2%) patients were receiving glucocorticoids at the time of entry into the trial. The baseline characteristics of the participants enrolled in CP+SOC group and of those enrolled in SOC alone group were similar, except for median neutralising antibody titres, which were significantly higher in control than in intervention group, and interleukin-6 levels, which significantly higher in intervention thank in control group (table 1 and Supplementary Material 2).
Interventions
Sixty-two (77.5%) patients received the CP from the same donor, while 15 (18.8%) received the second infusion from a distinct donor. The median neutralising antibody titres from donors’ plasma administered to patients from the intervention group was 1:320 (IQR, 1:160–1:960), which was significantly lower than baseline neutralising antibody titres of patients previously to the infusion (p<0.001). Only five donors’ plasma had neutralising antibody titres lower than 1:80 (four 1:40 and one 1:20). Other characteristics of CP donors are shown in Supplementary Material 2.
Two patients allocated to CP (1.3%) did not receive any intervention (1 due to the lack of compatible plasma units and 1 patient that died before receiving transfusion) and other two patients (1.3%) did not receive the second plasma infusion. One patient allocated to CP received four additional plasma infusions pending on discretion of the attending physician. One patient allocated to the control group received one unit of CP, also on discretion of the ICU team.
On day 3, there was a significantly higher increase in neutralising antibody titres in the intervention than in control group (p =0.001) in relation to titres at randomisation (day 0) (figure 2). The median neutralising antibody neutralising titres on day 3 was not significantly different between CP and SOC groups (1:5120 [IQR, 1:2560–1:10 240] versus 1:2560 [IQR, 1:1920–5120]; p=0.19) (figure 2).
Primary outcome
On day 28, there was no significant difference between the CP+SOC group and the SOC alone group in the proportion of patients with clinical improvement (61.3% versus 65.0%; difference, −3.7% [95% Confidence Interval [CI], −18.8 to 11.3]; Relative Risk [RR], 0.94 [95% CI, 0.74–1.19]; p=0.623) (table 2). Results for the per-protocol population were similar to those of the main analysis (Supplementary Material 2). In subgroup analyses, tests for interaction were not statistically significant for subgroups defined by the unit of admission, need of mechanical ventilation, age and neutralising antibody titres at baseline (Supplementary Material 2).
Secondary outcomes
CP+SOC group effects were not significantly different from SOC alone group for 28-day mortality (22.5% versus 16.3%; difference, 6.2% [95%CI, −7.5%-20.7%; RR, 1.38 [95% CI, 0.73–2.63]; p=0.32), proportion of scores on the 6-level ordinal scale on day 28 (p=0.64) and median of days alive and free of respiratory support within 28 days (11.0 versus 7.5; difference, −0.6 days [95%CI, −3.9–2.6]; p=0.44). There was no significant difference between groups in other secondary outcomes (table 2; Supplementary Material 2).
The proportion of patients with a positive RT-PCR for SARS-CoV-2 from nasal and oropharyngeal swab on day 7 or on discharge if earlier was similar CP+SOC and SOC alone groups (76.3% of versus 74.1%; difference, 2.2% [95%CI, −13.6–17.9]; RR, 1.03 [95%CI, 0.84–1.27 p=0.79) (table 2). There was no statistically significant difference in inflammatory markers and other laboratorial parameters between groups on days 3, 7 and 14 both in all patient's population and in those who had completed the sequence of the three collections of laboratorial (figure 2; and Supplementary Material 2).
Adverse events
The safety population included 79 patients who received at least one infusion of CP and 81 patients who received only SOC. A total of 52 (65.8%) and 48 (59.3%) of patients presented an adverse effect in CP+SOC and SOC alone groups, respectively (absolute difference 5.0% [95%CI. −10.0% to 20.1%]; RR, 1.08 [95%CI, 0.85–1.38]; p=0.51). CTCAE grade 3 or 4 adverse effects were noted in 50 (63.3%) and 44 (54.3%) of patients in intervention and control groups, respectively (absolute difference, 7.5% [95%CI, −7.8%-22.8%]; relative risk, 1.14 [95%CI, 0.88–1.48]; p=0.34). A full description of adverse effects is shown in Supplementary Material 2.
Post-hoc analyses
There was no significant difference in no prespecified subgroup analysis by age and neutralising antibody titres at baseline (Supplementary Material 2). There was no significantly difference between intervention and SOC groups in Poisson regression models including variables with a p value ≤0.20 at the baseline (table 3). There was also no significantly difference in clinical improvement on day 28, considering this outcomes as an one point reduction in the ordinal scale (61.3% versus 68.8% in intervention and control groups, respectively; RR, 0.89 [95%CI, 0.71–1.12]; p=0.33).
Discussion
In this randomised clinical trial with severe and critically ill COVID-19 patients, CP therapy administered in the first 14 days of the onset of symptoms plus SOC did not significantly increase the proportion of clinical improvement on day 28 compared with SOC alone. Similar results were found in both critically ill and patients hospitalised at medical ward subgroups. These findings are consistent with previous randomised clinical trials that could not find significant benefit of CP in hospitalised patients with COVID-19 [6–9].
There were also no significant differences in clinical and laboratorial outcomes between intervention and control groups, including 14- and 28-day mortality, clinical status on days 14 and 28 assessed by an ordinal scale, days free of ventilation, days of hospitalisations and SOFA and NEWS2 scores. No difference was observed considering one point reduction in the ordinal scale as clinical improvement.
One strength of our study is that virtually all patients were treated with corticosteroids, mostly dexamethasone, as SOC and other drugs were not used. Additionally, this study was the first to evaluate some laboratory exams in patients’ follow-up. The demonstration of absence of difference in these markers are consistent with clinical findings and help to reduce the level of uncertainty on the potential benefit of CP in severe COVID-19. Notably, in contrast to Li et al. [6] we could not find any difference in SARS-CoV-2 RT-PCR positivity rate between groups.
As found in a previous trial [7], most of the patients included in the study have already presented high levels (above 1:80) of neutralising antibody titres at randomisation. These titres were even higher in the SOC group (more than 75% of patients with titres equal or greater than 1:640). Two infusions of 300mL of CP increased the levels of these antibodies on day 3 in the intervention group. The increase in neutralising antibody titres from randomisation to day 3 was significantly higher in intervention group, and although the levels were higher in intervention than in control on day 3, this difference was not statistically significant probably because baseline levels in the former group was lower than the later. Nonetheless, this increase seemed to have no impact on both clinical and laboratorial outcomes, as indicated by the absence of any significant difference of inflammatory markers between groups in any point of collection from day 0 to 14. It must be acknowledged that the presence of high levels of neutralising antibodies titres at randomisation favours the null hypothesis, even though the effect on primary outcome was not affected when adjusted for this variable in the Poisson regression model. Furthermore, it is highly relevant from a pragmatic perspective, i.e., increment in antibody response in patients through passive administrations does not seem to be worthy in patients with severe COVID-19.
Notably, patients in the intervention group presented significantly higher levels of interleukine-6 at randomisation. Although interleukin-6 levels, as other variables analysed in Poisson regression models, did not significantly modify the effect of convalescent plasma on the outcome, and we at first attributed is as a casual difference that may be observed even with the randomisation process, we can not fully rule out that patients in intervention arm might be more severely ill. However, as shown in sensitivity analysis, if present, this would not be a disbalance able to affect the main results of the study.
A recently published meta-analysis evaluated the effect of CP on mortality and other clinical outcomes, including preprint publications and a press release of one randomised trial, could not find any significant difference of this strategy from SOC or placebo [12]. Given the heterogeneity of doses, neutralising antibody titres and time of CP administration, along with the fact that most randomised trials have been prematurely interrupted, as pointed out by the authors, the certainty of the evidence was low to moderate for all-cause mortality and low for other clinical outcomes. We updated that meta-analysis using the same methodology, including data from a preprint publication [9] previously available only as a press release, and PLACOVID trial for mortality. The updated result remains non-significant, with low inconsistency and narrow confidence interval (RR 0.98; CI 95% 0.81–1.19; p=0.29; I2=16%) (Supplementary Material 2).
This study has some limitations First, it is an open-label study and data collectors were not blinded to the patients’ group assignment. Despite not finding a positive effect of intervention, potential biases associated with this design cannot be completely ruled out. Second, our clinical trial is a single-centre study in a COVID-19 reference tertiary-care university-affiliated hospital, which may impair the generalisability of the findings; however, the overall findings point towards the same direction of previous multicentre studies. Third, this study is composed mostly by critically ill patients, a group of patients whose potential benefit could be less expected. Nonetheless, similar results were found in both critically ill and patients hospitalised at medical ward. Finally, we were underpowered to evaluate the efficacy in patients with low neutralising antibody titres. Despite the low number of patients, the exploratory analysis of patients with titre less 1:160 indicates a change in the direction of the effect (Supplementary Material 2). Along with previous studies suggesting a potential benefit with CP [13] or monoclonal antibodies in early periods of mild to moderate COVID-19 [14, 15] patients with severe COVID-19 and low levels of neutralising antibodies might still be a group of interest for future studies with passive immunotherapy.
In conclusion, in severe or critically ill COVID-19 patients, almost all receiving corticosteroids as SOC, CP+SOC did not result in a higher proportion of clinical improvement on day 28 compared to SOC alone.
Acknowledgements
We are grateful to Dr Patrícia Prolla and the “Grupo de Pesquisa e Pós-Graduação (GPPG)” team from Hospital de Clínicas de Porto Alegre for their significant administrative support during the study implementation and execution.
Footnotes
This article has supplementary material available from erj.ersjournals.com
NCT04547660 https://clinicaltrials.gov/ct2/show/NCT04547660
Data availability: The authors encourage interested parties to contact the corresponding author with data sharing requests, including for access to additional unpublished data.
Study Group: The PLACOVID Study Group at Hospital de Clínicas de Porto Alegre are Alecsandra Formentin Bello, Alessandra Aparecida Paz, Alldren da Silva de Souza, Ana Claudia Tonelli de Oliveira, Andréia R. Malaquias, Ane Katiussa Siqueira Fröhlich, Anelise Bergmann Araújo, Antônia Cícera da Silva Araújo, Bruna Blos, Carolina Rodrigues Cohen, Cristiane Tavares Borges, Cristiano Rossa da Rocha, Daniela Michelim Rodrigues Speranza, Delany da Silva Oliveira, Dimitris Rucks Varvaki Rados, Francine Bonacina, Gabrielle Dias Salton, Giovana Zucchetti, Isabel Cristina Freitas, Julia Plentz Portich, Juliana Goncalves Constante, Juliana Monteiro Furlan, Karine Kleber, Laís Pelentier Vieira, Leonardo Martins Pires, Liane Marise Rohsig, Lucia Mariano da Rocha Silla, Luciana do Nascimento Vargas, Marize S. V. Leão, Melissa Helena Angeli, Patrícia Paim Ferreira Seltenreich, Patrícia Santos da Silva, Rafael Selbach Scheffel, Renata Eliane Boehm, Renato G. B. de Mello, and Thabyta Silva Franco de Souza.
Support statement: The study was funded by “Fundação de Amparo à Pesquisa do Estado do Rio Grande do Sul (FAPERGS)” (Grant 16/2551-0000242-8), “Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)” (Grants 2020/06409-1 and 2016/20045-7) and “Instituto Cultural Floresta”. Fundação de Amparo à Pesquisa do Estado de São Paulo; DOI: http://dx.doi.org/10.13039/501100001807; Grant: 2016/20045-7, 2020/06409-1 ; Instituto Cultural Floresta; Fundação de Amparo à Pesquisa do Estado do Rio Grande do Sul; DOI: http://dx.doi.org/10.13039/501100004263; Grant: 16/2551-0000242-8.
Conflict of interest: Leo Sekine has nothing to disclose.
Conflict of interest: Beatriz Arns has nothing to disclose.
Conflict of interest: Bruna R. Fabro has nothing to disclose.
Conflict of interest: Murillo M. Cipolatt has nothing to disclose.
Conflict of interest: Raphael R.G. Machado received support from “Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)” (2017/24769-2).
Conflict of interest: Edison L. Durigon has nothing to disclose.
Conflict of interest: Edino Parolo has nothing to disclose.
Conflict of interest: José Augusto S. Pellegrini has nothing to disclose.
Conflict of interest: Marina V. Viana has nothing to disclose.
Conflict of interest: Patrícia Schwarz has nothing to disclose.
Conflict of interest: Thiago C. Lisboa has nothing to disclose.
Conflict of interest: José Miguel S. Dora has nothing to disclose.
Conflict of interest: Almeri M. Balsan has nothing to disclose.
Conflict of interest: Felipe da-Silva Schirmer has nothing to disclose.
Conflict of interest: Juliana P. M. Franz has nothing to disclose.
Conflict of interest: Luciana M. da-Silveira has nothing to disclose.
Conflict of interest: Raquel C. Breunig has nothing to disclose.
Conflict of interest: Viviana Petersen has nothing to disclose.
Conflict of interest: Monalisa Sosnoski has nothing to disclose.
Conflict of interest: Nanci F. Mesquita has nothing to disclose.
Conflict of interest: Fabiana Caroline Z. Volpato has nothing to disclose.
Conflict of interest: Daniel Sganzerla has nothing to disclose.
Conflict of interest: Maicon Falavigna has nothing to disclose.
Conflict of interest: Regis G. Rosa received research grants from Brazilian Ministry of Health.
Conflict of interest: Alexandre P. Zavascki is a research fellow of the National Council for Scientific and Technological Development (CNPq), Ministry of Science and Technology, Brazil (304226/2018-1), and receives a research grant not related to this work from Pfizer (WI242215 2018).
- Received April 30, 2021.
- Accepted June 16, 2021.
- Copyright ©The authors 2021.
This version is distributed under the terms of the Creative Commons Attribution Non-Commercial Licence 4.0. For commercial reproduction rights and permissions contact permissions{at}ersnet.org