Systematic reviews and meta-analyses in animal model research: as necessary, and with similar pros and cons, as in patient research
- 1Unitat de Biofísica i Bioenginyeria, Facultat de Medicina i Ciències de la Salut, Universitat de Barcelona, Barcelona, Spain
- 2CIBER de Enfermedades Respiratorias, Madrid, Spain
- 3Institut Investigacions Biomediques August Pi Sunyer, Barcelona, Spain
- 4Pneumology Dept, University and Polytechnic la Fe Hospital, Valencia, Spain
- 5Dept of Child Health, The University of Missouri School of Medicine, Columbia, MO, USA
- Corresponding author: Ramon Farré (rfarre{at}ub.edu)
Abstract
Systematic reviews and meta-analyses from research data in animal models of disease are useful for translational medicine https://bit.ly/3kOvVUN
Systematic reviews and meta-analyses (SRMAs) are very useful tools for evaluating the status of research on a given topic in medical sciences [1, 2]. Although SRMAs are primarily used in clinical investigation, the methodologies underlying SRMAs can also be readily used to assess extant data focused on research in animal models of disease [3]. Indeed, the rationale for using SRMAs is the same, regardless of the field of application, since the aim of SRMAs is to consider all available publications on a topic of interest, and to derive conclusions from a comprehensive and integrated analysis of all existing studies deemed valid by consensus, rather than simply enumerating the conclusions of individual articles as is common in conventional narrative reviews. SRMAs can be particularly useful in determining whether clinical trials that have been performed on a diagnostic or treatment procedure have generated sufficiently valid evidence to transition that specific issue to clinical practice. Similarly, and assuming the potential limitations imposed by inter-species or inter-strain differences, SRMAs applied to animal research publications may be helpful in elucidating whether the data available from animal disease model publications are robust enough to validate a mechanistic hypothesis or to justify translation into clinical research [4, 5]. This analogy is clearly illustrated by an early classical example showing that if a SRMA of animal research data had been conducted on the effects of nimodipine for focal cerebral ischaemia (a post hoc SRMA revealed a lack of efficacy and unsupportive evidence), 22 clinical trials involving 6468 patients could have been pre-emptively avoided [6].
It is also worth noting that SRMAs of disease model data in animals may offer an interesting added value. Indeed, they can be particularly useful in providing an updated global perspective to clinicians who are not intensely invested or interested in the specific methodological details employed by each of the basic science publications on which the SRMA is based. However, notwithstanding their potential interest and value, SRMAs have been scarcely applied to critically evaluate the available evidence from research in animal models in the respiratory field [7–9]. The article by Harki et al. [10] published in this issue of the European Respiratory Journal is therefore a welcome addition, and the authors are commended for their first SRMA of rodent data on a topic revolving around sleep apnoea. Specifically, the authors evaluated the literature on intermittent hypoxia-related alterations in vascular structure and function, a timely and important issue, with the intent to provide valuable insights into the current scientific debate on the potential cardiovascular effects of sleep apnoea and whether they can be prevented by nasal pressure treatment [11–20]. Harki et al. [10] identified >5000 publications from three major databases and selected 125 papers for the meta-analysis, with most of them having been carried out in wild-type rodents (90%), mainly rats (79%). The most relevant findings of this SRMA were that intermittent hypoxia increased both systolic and diastolic arterial pressures, attenuated vasodilation, and promoted endothelin-1-induced vasoconstriction and vascular remodelling, confirming causative relationships that, given the multitude of confounding factors, are difficult to establish in patient studies. Interestingly, Harki et al. [10] clearly and briefly discuss their study limitations, most of which seem to be due to the spectrum of published results available to carry out the SRMA. Thus, it is important to further comment in more detail on two of the main limitations that may adversely affect SRMAs similarly in clinical and experimental research.
It is well known that the robustness of the conclusions derived from any given SRMA depends on the quantity and quality of the published data [21–23]. In this context, a recent debate has been held on the quality of clinical data and SRMAs in the field of sleep medicine [24–26]. Regarding patient studies, a limitation of SRMAs is that the available publications retained for analysis correspond to studies that were designed to verify a certain hypothesis specifically posited by the authors. Since such studies aim at reaching optimal precision when answering the question posed, the authors of individual clinical trials will usually apply very well-defined and restrictive inclusion and exclusion criteria. However, such precision in approach implies that the cohort under study usually excludes a considerable fraction of the real-life patient variability that is pervasively present in clinical practice, for instance the patients who are most fragile and difficult to treat because of their comorbidities. Therefore, while such neat inclusion/exclusion criteria are necessary to clearly answer the hypothesis in a clinical trial, they can limit the translation of SRMAs conclusions to the clinical arena [27]. Remarkably, the potential problem arising from inclusion/exclusion criteria is not exclusive of patient data, but is also relevant when SRMAs are applied to animal data.
Indeed, simple decisions on the animal experiment design (equivalent to inclusion/exclusion criteria in clinical trials) may have important consequences on whether the SRMA conclusions can be generalised, as mentioned by Harki et al. [10]. An important issue is that animal studies have been almost exclusively carried out in males, with data from females being usually absent or only occasional. In this regard, it is notable that Harki et al. [10] report that intermittent hypoxia alters vasodilation in males, but not in females, although there was a sex imbalance since only four of the cited studies focused on females, as compared with 103 that exclusively included males (of note, four were studies in both sexes and in four the sex was not reported). Such a finding is not surprising, since sex-related differential responses concerning various biological variables have been previously identified when intermittent hypoxia has been used as a correlate of obstructive sleep apnoea in animals [28–33]. Fortunately, this problem of sex balance in animal research is being progressively addressed following implementation of formal policies by funding agencies and journal editors [34–39]. Another concern is that most of the diseases for which animal models are needed usually have increased prevalence in the elderly (e.g. sleep breathing disorders) but the experiments are actually carried out in young animals, commonly with an age equivalent to that of human late adolescence. Hence, it should not come as a surprise that significantly different responses emerge in response to hypoxia depending on age [40–43]. In fact, despite the limited data currently available, Harki et al. [10] were able to detect in their SRMA that vascular remodelling induced by intermittent hypoxia was reduced in aged animals. In addition to the obvious sex, age and obesity, other “inclusion/exclusion” criteria, which may seem of minor relevance in animal models, can considerably modulate the responses to intermittent hypoxia or sleep fragmentation, the two major disruptors in sleep breathing disorders. For instance, environmental temperature [44–48], diet and activity [49], and presence or absence of social interactions among animals [50–53] can considerably modify the metabolic and immune responses, thereby modulating the consequences of the sleep breathing disorder challenges. Moreover, the fact of choosing one type of rodent, a specific strain within mice or rats, or even truly wild animals, modulates the immune system and the response to the experimental exposures [53–56]. As such, it is of note that the SRMA by Harki et al. [10] concluded that intermittent hypoxia-induced cardiovascular remodelling occurred in mice, but not in rats, and that increases in mean arterial pressure depend on the rat strain. These data question to what extent the narrow genetic variability of laboratory animals, which greatly differs from the naturally wild spectrum, may limit the validity of the conclusions derived from most animal models.
Risk of bias (e.g. selection, performance, detection, attrition and reporting biases) must be assessed when performing SRMAs, as Harki et al. [10] actually did using the SYRCLE approach, which is a tool specifically designed for animal intervention studies [57]. However, there is a type of bias, known as publication bias [58], that potentially challenges the conclusions derived from SRMAs, and is virtually impossible to contend with. The more typical manifestation is that studies with negative results tend to be underrepresented in the literature. For instance, when testing the effectiveness of a clinical treatment, trials with positive results are more attractive to scientific journals since they draw more attention, press releases and ultimately citations, which are the petrol directly or indirectly feeding most scientific publications either for profit or non-for-profit. Fortunately, compulsory registration of clinical trials over the past several years has reduced the possibility that trials with negative results are ignored. However, looking at clinical trial public registries can just inform on the trials initiated with a given aim (e.g. testing the effectiveness of a treatment) but registries are not always updated with the conclusions and results of such trials. Since SRMAs are carried out on the basis of the papers actually published, editorial decisions of journals can lead to publication bias regardless of clinical trial registration requirements. This problem is similar or even greater in the context of SRMAs focused on animal model-based research. Indeed, in this case there is no compulsory public registry of the experiments started by researchers, and the presence of such an obvious void suggests that registry initiatives for animal (and cell culture) research that are similar to those currently implemented for clinical trials may be of interest and enhance the value and significance of subsequent SRMAs. Moreover, similar to the clinical research potential publication bias, increased venues that allow or seek publication of negative results in animal-based experiments may ultimately reduce the current publication bias that is pervasively found in animal-based research.
We should point out that the difficulties that have been mentioned herein, namely inclusion/exclusion criteria and publication bias, are not intrinsic limitations of SRMAs per se, but rather stem from the quality of the available published research that is used as the basis to generate the SRMAs. Fortunately, the adverse impact imposed by these difficulties can be progressively reduced by improving research and publication practices. Therefore, conventional SRMAs, and also individual participant data meta-analysis [59], should be viewed as extremely useful tools that can be used as frequently as required to evaluate and guide research with animal models within the translational research framework. Indeed, based on the fundamental assumption that animal research should be guided by the 3-R principles (Replacement, Reduction and Refinement) [60], implementation of meta-analyses will not only reinforce and refine the findings and conclusions of each of the studies, but should permit the advance of science through formulation of additional research questions with greater certainty (figure 1).
Shareable PDF
Supplementary Material
This one-page PDF can be shared freely online.
Shareable PDF ERJ-02438-2021.Shareable
Footnotes
Conflict of interest: None declared.
- Received September 8, 2021.
- Accepted September 18, 2021.
- Copyright ©The authors 2022. For reproduction rights and permissions contact permissions{at}ersnet.org