Abstract
Host response biomarkers can accurately distinguish between influenza and bacterial infection. However, published biomarkers require the measurement of many genes, thereby making it difficult to implement them in clinical practice. This study aims to identify a single-gene biomarker with a high diagnostic accuracy equivalent to multi-gene biomarkers.
In this study, we combined an integrated genomic analysis of 1071 individuals with in vitro experiments using well-established infection models.
We identified a single-gene biomarker, IFI27, which had a high prediction accuracy (91%) equivalent to that obtained by multi-gene biomarkers. In vitro studies showed that IFI27 was upregulated by TLR7 in plasmacytoid dendritic cells, antigen-presenting cells that responded to influenza virus rather than bacteria. In vivo studies confirmed that IFI27 was expressed in influenza patients but not in bacterial infection, as demonstrated in multiple patient cohorts (n=521). In a large prospective study (n=439) of patients presented with undifferentiated respiratory illness (aetiologies included viral, bacterial and non-infectious conditions), IFI27 displayed 88% diagnostic accuracy (AUC) and 90% specificity in discriminating between influenza and bacterial infections.
IFI27 represents a significant step forward in overcoming a translational barrier in applying genomic assay in clinical setting; its implementation may improve the diagnosis and management of respiratory infection.
Abstract
IFI27 could discriminate between influenza and bacterial infection in suspected respiratory tract infection cases http://ow.ly/VEaY30bmlS3
Introduction
Acute respiratory tract infections are the most common causes of infectious disease morbidity and mortality in the world (WHO - 2014). Viruses and bacteria are the main causes of respiratory infections. Among the viruses, the influenza virus causes the highest levels of morbidity and mortality. Accurately differentiating between the influenza virus and bacteria is the first key step in managing respiratory infections, because initial treatment (antiviral versus antibiotic) is guided mainly by the aetiology. However, differentiating between influenza and bacterial infections on clinical grounds is often difficult, because both infections share similar clinical features (e.g. cough, dyspnoea and fever).
Pathogen testing (e.g. virus detection by PCR) is an important step towards clarifying the diagnosis. Although important, virus detection alone is insufficient to guide patient management. Like most respiratory tract viruses, the influenza virus produces a wide clinical spectrum. In some patients, the virus acts as a bystander and its presence is unrelated to the presenting illness [1]. In others, the virus is directly responsible for the patient's symptoms. Clinical decision-making requires consideration of whether the virus is a bystander or the cause of presenting symptoms, because anti-viral therapy (e.g. Oseltamivir) should be administered only to individuals in whom symptoms are severe and directly caused by the virus [2–4]. Currently, no laboratory test exists that can distinguish between virus detection and “active infection”.
Host response biomarkers represent an alternative approach to circumvent this problem. Host gene expression biomarkers could accurately identify the immune response underpinning influenza infection, and thus assist clinicians in distinguishing between influenza and bacterial infections [5–8]. These biomarkers represent a promising strategy to complement current diagnostic methods. However, published biomarker panels contain a large number of genes, making it difficult to implement them in clinical practice. To overcome this translational barrier, we conducted an integrated genomic analysis of 1071 individuals, to search for a single-gene biomarker with high diagnostic accuracy equivalent to that of multi-gene biomarkers in distinguishing between influenza and bacterial infections.
Materials and methods
Study design
This study consisted of a discovery phase and a validation phase (figure 1). First, a candidate biomarker was identified in discovery cohorts (n=111). Next, the identified biomarker underwent a two-phase validation. In phase 1 validation, we evaluated the biomarker in four independent, external data sets (n=521) (figure 1). In addition, we performed in vitro experiments to understand the mechanism by which immune cells produced the biomarker, and we performed experiments with mice to validate the biomarker in vivo (see supplementary material for full details of all experiments). In phase 2 validation, we undertook a large prospective study (n=439) to evaluate real-world performance of the biomarker in patients with suspected respiratory infection.
Participants
Participants were recruited in discovery cohorts and a prospective validation cohort. The discovery cohorts consisted of three case-control studies. Each study included patients with well-defined phenotypes (influenza infection, bacterial infection and non-infectious illnesses) and healthy controls. In phase 2 validation, we performed a large, multi-centre study (“prospective validation study”) in patients presenting with “flu-like” illness to emergency departments. The study protocol was approved by the institutional review board of each participating institution and all study participants gave informed consent. Demographic and clinical characteristics of all study participants are summarised in table 1. Detailed information on all cohorts (inclusion criteria, recruitment process, sample collection and follow-up) is presented in the supplementary material.
Statistical analysis
To evaluate the diagnostic performance of the biomarker, sensitivity, specificity, diagnostic threshold and area under the curve (AUC) of the receiver operating characteristic curve were calculated using established methods (see supplementary material for full details). Comparisons between two groups were calculated using the unpaired two-tailed t-test or the non-parametric Mann–Whitney U-test where appropriate. Comparisons between multiple groups were calculated using a one-way ANOVA or the Kruskal–Wallis test, where appropriate. Statistical analysis was performed using the NCSS (LLC, Kaysville, UT, USA) and PRISM (GraphPad Software, La Jolla, CA, USA) software packages.
Results
Discovery cohorts
To screen for influenza biomarkers, we analysed the whole blood of influenza patients in three discovery cohorts (table 1 and supplementary table S1). Influenza patients were defined as those with a positive virus result detected on viral nucleic acid testing, who presented with severe illness compatible with influenza that required admission to a hospital or intensive care unit. We first performed microarray analysis to identify differentially expressed genes (between influenza patients and healthy controls) (figure 1). This analysis revealed that among differentially expressed genes, interferon alpha inducible protein 27 (IFI27) performed significantly better than all other genes (figure 2). This finding was consistent in two separate discovery cohorts (figure 2) and on different microarray platforms (supplementary figure S1). We subsequently performed PCR validation in all three cohorts, which confirmed that IFI27 was upregulated in influenza-positive patients (supplementary table S1). Altogether, findings from all discovery cohorts were consistent with each other and collectively suggested that IFI27 is a potential biomarker of influenza infection.
Host response
To confirm that IFI27 reflected “active infection” in patients, we examined IFI27's correlation with immune response genes. This analysis showed that IFI27 was co-expressed with many anti-viral immune response genes. It also showed that IFI27 was highly correlated (coefficient of correlation >0.6) with 69% of the differentially expressed genes (24 of 35) in the immune system pathway, and 90% of the differentially expressed genes (19 of 21) in the interferon signalling pathway (figure 3). Many of these IFI27-correlated genes have anti-viral functions, such as MX1 (anti-influenza), ISG15 (inhibits influenza virus replication), OAS1, OAS2 and OAS3 (degrades viral RNA), EIF2AK2 (inhibits viral replication), HERC5 (positive regulator of the anti-viral response) and IFIT2 (inhibits expression of viral mRNA). These findings confirm that IFI27 was part of the host response against respiratory viruses.
External validation
In phase 1 validation, we validated our findings in both mouse and human datasets. Mouse data suggested that ifi27 was upregulated in influenza infections of varying severity (supplementary figure S2). In human datasets, we analysed microarray datasets of four independent external cohorts (GSE6269, GSE42026, GSE68310 and GSE60244) (table 1). Two cohorts consisted of paediatric patients: GSE6269 (n=131) and GSE42026 (n=92) [9, 10]. Two other cohorts consisted of adult patients: GSE60244 (n=158) and GSE68310 (n=133) [11, 12]. We found that IFI27 was among the signature genes for influenza infection in all four cohorts. In three cohorts (GSE42026, GSE60244 and GSE68310), IFI27 was the most highly upregulated gene in infected patients (figure 4). In one cohort (GSE6269), IFI27 was the second most highly upregulated gene in infected patients. These findings are consistent with those of the discovery cohorts, confirming that IFI27 is an important signature gene for influenza infection.
In vitro model
To confirm further whether IFI27 gene expression was directly activated by the influenza virus, we co-cultured peripheral blood samples obtained from healthy volunteers with the live influenza virus. These experiments revealed a dose-response relationship of IFI27 gene expression in response to viral load (figure 5a), confirming that IFI27 was activated by the influenza virus. Additional experiments using different influenza A virus strains (H3N2 and H1N1) and the influenza B virus strain showed similar findings (data not shown).
IFI27 is produced by plasmacytoid dendritic cells that recognise the influenza virus
To identify the immune cells that expressed IFI27 in peripheral blood, we studied IFI27 gene expression in eight immune cell subsets, including neutrophils, B cells, CD4, CD8, monocytes, natural killer (NK) cells, myeloid dendritic cells and plasmacytoid dendritic cells (pDCs). We found that in response to stimulation by the influenza antigen, pDCs displayed the highest level of IFI27 gene expression relative to other cell subsets (figure 5b). Notably, neutrophils did not upregulate IFI27, despite their known effects on the interferon pathway (figure 5b and d). We also found that the IFI27 response in peripheral blood was predominantly driven by TLR7 activation (figure 5c), and this TLR7-driven IFI27 response was mainly observed in pDCs and NK cells (figure 5d). We noted that the IFI27 response was particularly strong in pDCs (its expression levels were significantly higher in pDCs than in NK cells) (figure 5e). Based on these findings, we concluded that pDCs (and to a much lesser extent, NK cells) were most likely the predominant sources of IFI27 in peripheral blood. Because the pDC-TLR7 pathway specifically recognises respiratory viruses (e.g. the influenza virus) but does not recognise bacteria, these findings suggest that IFI27 could be a useful biomarker to discriminate between influenza and bacterial infection.
IFI27 discriminates between bacterial and influenza infection
Consistent with the aforementioned findings, we found that IFI27 expression in peripheral blood could discriminate influenza pneumonitis from bacterial pneumonia in the discovery cohorts (figure 6a). This was further confirmed by our in vitro experiments (figure 6b). In these experiments, influenza viruses (H1N1 and H3N2) and bacterial antigen (lipopolysaccharide) were co-cultured with peripheral blood or purified pDCs. In response to lipopolysaccharide stimulation, IFI27 upregulation was negligible (in both peripheral blood and pDCs). In contrast, IFI27 upregulation in response to influenza viruses (both H1N1and H3N2) was very high (up to 777 fold change in peripheral blood and 1475 fold change in pDCs) (figure 6b).⇓
Prospective validation cohort
In phase 2 validation, we prospectively recruited 402 patients with suspected respiratory tract infections and 37 healthy volunteers across five teaching hospitals in Australia from 2012 to 2016 (table 1, supplementary table S2–S3). In this prospective cohort, the diagnostic threshold of IFI27 was tested on a real-time PCR platform. We divided the cohort into a training set (n=200) and a test set (n=202). The diagnostic threshold was first optimised in the training set and was subsequently confirmed in the test set. The calculated AUC of the receiver operating characteristic curve, specificity and sensitivity were nearly identical in both train and test sets. Henceforth, we present the summary findings with both datasets combined.
We found that a threshold of 74 fold change (based on PCR) provided the most optimal level of IFI27 to differentiate between influenza and bacterial infections (for full details on how this threshold was determined, see supplementary material). At this cut-off value, IFI27 had a sensitivity of 0.80 and specificity of 0.90. Furthermore, IFI27 had an AUC of 91% (figure 7), which was within the same range of the multi-gene biomarkers (87%–95%) (table 3). In addition, IFI27 had an AUC of 83% (sensitivity 0.80 and specificity 0.75) in differentiating influenza virus from other respiratory viruses. This performance is modest compared to its ability to distinguish influenza from bacterial infections. However, we noted that none of the previously published multi-gene biomarkers could distinguish influenza from other respiratory viruses (table 3).
To evaluate further the clinical utility of IFI27, we sought to determine whether it could identify influenza infections among all patients in the cohort (figure 7). This additional evaluation represents a more vigorous evaluation of its performance, because IFI27 would need to identify influenza infection against multiple alternative aetiologies (bacterial infections, other viral infections and all non-infectious conditions), rather than just two aetiologies (influenza versus bacteria). Only one previous multi-gene biomarker study evaluated biomarkers in this manner (GSE63990, see table 3) [13]. In our evaluation, IFI27 showed a high AUC (88%) in identifying influenza infection against multiple aetiologies (sensitivity 0.80 and specificity 0.80) (figure 7). This performance was similar to a previous multi-gene biomarker study (87%) (GSE63990) [13]. Collectively, these findings show that IFI27 has high diagnostic accuracy, equivalent to those obtained by previously published multi-gene biomarkers.
Discussion
This study addressed a major challenge in translating genomic science into clinical practice. Over the last decade, considerable advances have been made in discovering the gene expression signatures of respiratory viral infection. These gene expression signatures identify the immune response associated with the infection, thereby significantly improving the clinician's ability to make an accurate diagnosis. However, all published gene expression signatures consist of multi-gene lists, ranging from dozens of genes in some studies to hundreds of genes in others. Measuring expression levels of a large number of genes in clinical practice is problematic. Current real-time PCR technology measures only a small number of genes. Furthermore, determining a diagnostic threshold for many genes is computationally challenging. Most studies have used very complex machine learning algorithms to determine such a threshold. This is impractical in clinical practice, because most institutions do not have access to specialist bioinformatics expertise. In this paper, we demonstrated that a carefully chosen single-gene biomarker might offer an alternative approach. Using evidence from in vitro experiments, a mouse model and multiple patient cohorts, we demonstrated that as a single-gene biomarker, IFI27 could consistently identify influenza infection in different patient populations, in individuals with varying extents of disease severity and in patients with multiple comorbidities. Furthermore, our findings suggest that this biomarker could achieve high prediction accuracy in distinguishing between influenza and bacterial infections, equivalent to those obtained by multi-gene biomarkers.
Our study is by far the largest biomarker study undertaken in influenza virus infection. It consisted of eight cohorts (n=1071), with patients drawn from heterogeneous populations in different geographical locations, settings (outpatients, emergency departments and intensive care units) and different levels of infection severity (mild, moderate to severe). Such a broad clinical case-mix makes it likely that our findings are generalisable to other patient populations. Our findings are also consistent with a recently published meta-analysis of respiratory viral infection. That meta-analysis included five influenza cohorts (n=292) (two cohorts originated from the present study) [14]. It identified 127 multi-gene signatures associated with the influenza infection. Of these 127 genes, IFI27 was the most highly upregulated gene.
Our in vitro data are important, because it provides a biologically plausible explanation for the ability of IFI27 to differentiate influenza from bacterial infections. Our findings revealed that IFI27 was produced by plasmacytoid dendritic cells (pDC), a pivotal immune cell subset that is known to recognise influenza virus, but not bacteria [15, 16]. Our data also revealed that the TLR7 pathway mediates IFI27 upregulation. This is consistent with established literature, which shows that pDCs use TLR7 to recognise single-stranded RNA viruses, such as the influenza virus [17, 18]. We also found that bacteria-recognizing TLRs (TLR1, TLR2, TLR4, TLR5 and TLR6) did not induce IFI27 upregulation, whereas TLR7 ligand did. These findings, together with previous studies [17–20], collectively support the notion that the TLR7-pDC pathway represents a virus-specific detection system (for single-stranded RNA viruses). This mechanistic insight helps explain the ability of the IFI27 biomarker to differentiate between influenza and bacterial infections.
The prospective validation cohort in the present study had two design features (large sample size and prospective patient recruitment), which made it possible to address an important limitation in previously published biomarker studies. In previous studies, the validation cohort typically used patient phenotypes that were clearly “infectious” or phenotypes that were clearly “non-infectious” (e.g. healthy volunteers). A recent study by Tsalik et al. [13] demonstrated that using such clearly defined phenotypes resulted in an over-estimation of biomarker performance. A methodologically more vigorous approach should include less distinctive phenotypes, such as patients with non-infectious “flu-like” illness [13]. Our prospective validation cohort had a significantly greater number of non-infectious patients compared to previous studies (table 1). Another strength of the present study was the recruitment of patients prospectively. Patients presented in an undifferentiated manner, whereby both infectious and non-infectious aetiologies were possible, thus simulating a realistic clinical scenario commonly faced by clinicians. Furthermore, many of our patients had respiratory comorbidities (asthma 17% and chronic lung diseases 22%), which could also mimic respiratory infections (e.g. with similar symptoms of cough and shortness of breath). Our cohort therefore recapitulated many of the real-world challenges and embodied the most stringent conditions under which the performance of IFI27 could be tested.
A single-gene biomarker, such as IFI27 circumvents several methodological problems faced by multi-gene biomarkers. Multi-gene biomarkers are vulnerable to over-fitting (using too many genes to develop the disease signature). This results in poor reproducibility across external patient populations. In the present study, we minimised over-fitting by focusing on a small number of biomarkers, from which we identified a single biomarker that was highly reproducible across multiple patient cohorts. Another common problem with multi-gene biomarkers is that there are many non-overlapping genes among published studies. This creates an immense translational problem, because clinicians cannot determine which gene list to use with their patients. The simplicity of using a single-gene biomarker such as IFI27 represents a major step forward in overcoming this translational roadblock. A third problem with multi-gene lists is that most studies have been derived from microarray studies. Here, we validated IFI27 on a real-time PCR platform, which is widely available in most hospitals. This opens the door for other clinicians to perform independent validation, an essential next step towards translation into clinical practice.
It is worth noting that existing multi-gene biomarkers do have an advantage over a single biomarker. Multi-gene biomarkers broadly reflect host response, by capturing a constellation of genomic signals from different parts of the immune signalling pathway. In comparison to multi-gene biomarkers, a single biomarker might underestimate both the range and depth of the host response. However, our findings indicate that IFI27 levels did provide a reasonable assessment of the immune signalling pathway. This was evidenced by a high level of correlation between IFI27 and many genes in the interferon pathway (90%) and immune system pathway (69%). Altogether, these findings suggest that IFI27, as a single-gene biomarker, is reflective of the immune response to a similar extent as multi-gene biomarkers.
IFI27 differs from previously established influenza biomarkers (e.g. OAS and PKR) in three important aspects. First, IFI27 performed significantly better than other biomarkers, as demonstrated in our microarray analysis. Second, IFI27 is produced predominantly by plasmacytoid dendritic cells (pDCs), immune cells that recognise respiratory viruses. In contrast, most biomarkers are produced by immune cells (e.g. neutrophils and macrophages) that can be activated by bacteria. Third, IFI27 has considerable discriminatory power in differentiating between influenza and bacterial infections. Our data show that as a single-gene biomarker, IFI27 alone provides equivalent diagnostic capability comparable to that of multi-gene biomarkers. Previous biomarkers do not have comparable capabilities.
Nevertheless, the present study has limitations. Firstly, the specificity of IFI27 in differentiating between influenza virus and other respiratory viruses is moderate (AUC 83%). Second, we had insufficient data on patients with DNA viral infections (e.g. cytomegalovirus) or virus-bacteria co-infection; our cohort had only thirteen such cases, which is too small a number for in-depth analysis. In our previously published study, we found that a longitudinal study design was needed to investigate patients with co-infection (in order to capture the transition from influenza infection to co-infection) [21]. Unfortunately, the current study is limited by the fact that patient samples were collected only at one time point. Third, our study did not assess important factors that might affect clinical decision-making, such as prior probability or the cost of missed diagnoses and its impact on patient/economic outcomes. These factors are best addressed in future cost-benefit studies or clinical trials.
In conclusion, our findings suggest that IFI27 is a promising immune biomarker; however, further study is needed to evaluate its clinical utility in patients with viral/bacterial co-infections.
Supplementary material
Supplementary Material
Please note: supplementary material is not edited by the Editorial Office, and is uploaded as it has been supplied by the author.
Supplementary methods ERJ-02098-2016_Supplementary_Methods
Figure S1. Gene-expression analysis using a different microarray platform ERJ-02098-2016_Figure_S1
Figure S2. IFI27 gene expression in mouse influenza infection model ERJ-02098-2016_Figure_S2
Table S1. Discovery cohorts - influenza positive patients ERJ-02098-2016_Table_S1
Table S2. Prospective validation cohort - patient characteristics ERJ-02098-2016_Table_S2
Table S3. Prospective validation cohort - pathogens identified ERJ-02098-2016_Table_S3
Disclosures
Supplementary Material
A. McLean ERJ-02098-2016_McLean
Grant P. Parnell ERJ-02098-2016_Parnell
M. Shojaei ERJ-02098-2016_Shojaei
B.M. Tang ERJ-02098-2016_Tang
Acknowledgements
We would like to thank Kenneth McPhie and Ian Barr (WHO Collaborating Centre for Reference and Research on Influenza, Melbourne, Australia) for supply of the influenza virus stock for our in vitro experiments, Ya Wang (Dept of Intensive Care Medicine, Nepean Hospital, Kingswood, Australia) and Nicole Fewings (Centre for Immunology and Allergy Research, Westmead Institute for Medical Research, Westmead, Australia) for assistance in analysing flow cytometry data and Ian Barr for helpful review on an earlier version of our manuscript. We would also like to thank several organisations for assisting our study, including: the Nepean Institute of Critical Care Education and Research (Kingswood, Australia) for providing funding to support our study in Australia, the Czech Republic and Hong Kong; Public Health Agency of Canada (PHAC) (Ottawa, Canada) for providing funding to support the Canadian arm of our study; Roche Pharmaceuticals (Ottawa, Canada) for funding H1N1 sample collection in Canada; and the Canadian Severe Influenza Study group (see below) for contributing patient samples to the study.
The collaborating members of the Canadian Severe Influenza Study group are as follows: Anand Kumar (Section of Critical Care Medicine and Section of Infectious Diseases, Dept of Medicine, Medical Microbiology and Pharmacology, University of Manitoba, Winnipeg); Gordon Wood (Royal Jubilee Hospital and Victoria General Hospital, Victoria); Steve Reynolds (Vancouver General Hospital, University of British Columbia, Vancouver); Brent Winston (Foothills Medical Centre and Peter Lougheed Hospital, University of Calgary, Calgary); Sean Bagshaw (University of Alberta Hospital, University of Alberta, Edmonton); Jim Kutsogiannis (Royal Alexandra Hospital, University of Alberta, Edmonton); Michael Silverman (Rouge Valley Hospital, Ajax, Canada and Lakeridge Hospital, Oshawa); Margaret Herridge (Toronto General Hospital and Toronto Western Hospital, University of Toronto, Toronto); Rob Fowler (Sunnybrook Hospital, University of Toronto, Toronto); Alison McGeer (Mount Sinai Hospital, University of Toronto, Toronto); Mary-Anne Aarts (St Michael's Hospital, University of Toronto, Toronto); Deborah Cook (St Joseph's Healthcare, McMaster University, Hamilton); Lauralyn McIntyre (Ottawa Hospital General Campus and Civic Campus, University of Ottawa, Ottawa); Kosar Khwaja (Montreal General Hospital and Royal Victoria Hospital, McGill University, Montreal); Natalie Bandrauk (Health Sciences Centre and St Claire's Mercy Hospital, Memorial University, St. John's); Richard Hall (Queen Elizabeth Health Sciences Centre – Halifax Infirmary and Victoria General Site, Dalhousie University, Halifax).
The authors contributed to the study in the following ways. Benjamin M. Tang conceived the study. Maryam Shojaei conducted the in vitro study. Klaus Schughart conducted the mouse study. Klaus Schughart and Grant P. Parnell conducted the microarray analysis. Maryam Shojaei, Grant P. Parnell, Anand Kumar, Aseem Kumar, John Ho, Yoav Keynan, Adrienne F.A. Meyers, Terry Ball, Amarnath Pisipati, Amy L. Phu, Stephen Huang, Marek Nalos, Kate O'Connor, Stephen Schibeci, Kevin Lai, Hao Luo, Fahad Gul, Mark Gillett, David Booth, Elizabeth Moore, Robert Geffers, Jens Schreiber, Damon Eisen and Benjamin M. Tang participated in the study and critical review of the manuscript; B. Tang drafted the manuscript.
Footnotes
This article has supplementary material available from erj.ersjournals.com
Support statement: This work was supported by Nepean Institute of Critical Care Education and Research (Kingswood, Australia). The mouse study was funded by an intra mural grant from the Helmholtz-Association (Program Infection and Immunity) and research grant FluResearchNet (no. 01KI07137) from the German Ministry of Education and Research to Klaus Schughart. Benjamin M. Tang, Grant P. Parnell, Maryam Shojaei and Anthony McLean had filed a patent application (PCT) on IFI27. The Nepean Institute of Critical Care Education and Research, and the public health agency of Canada, had no role in: the design and conduct of the study; the collection, management, analysis, and interpretation of the data; or the preparation, review, or approval of the manuscript. The corresponding author had full access to all the data and the final responsibility to submit for publication. Funding information for this article has been deposited with the Crossref Funder Registry.
Conflict of interest: Disclosures can be found alongside this article at erj.ersjournals.com
- Received June 21, 2015.
- Accepted March 15, 2017.
- Copyright ©ERS 2017