Risk stratification in pulmonary arterial hypertension using Bayesian analysis

Discussion

Risk stratification using a Bayesian network model approach (PHORA) provides improved discrimination to the existing Cox regression multivariate model (REVEAL 2.0), and effectively depicted risk in two large external registry cohorts, COMPERA and PHSANZ. This improvement probably stems from the ability of the Bayesian network model to understand both the dynamic influences of each risk factor on each other, as well as with the outcome itself.

The utility of the Bayesian network methodology was only recognised within the past 25 years, with the publication and application of Bayesian network-based decision support tools in a variety of medical disciplines [13–16]. In these clinical scenarios, Bayesian network-based tools were noted to have superior predictive performance over traditional statistical methods [8]. Bayesian networks do not require restrictive modelling assumptions outside of expressing independencies whenever these are justified. Descriptively, Bayesian networks provide the advantages of a rigorous probabilistic framework that uses inference of multiple variables and a visual representation that is interactive and easy to interpret. This also allows a user to input these various scenarios and calculate the changes in predicted mortality and other adverse events in a highly interactive fashion. When performing prediction, Bayesian networks allow for estimating the outcome probability based on partial observations, as often happens in a clinical setting. Indeed, just converting the methodology of evaluating the pertinent REVEAL 2.0 variables produced a tool with boosted the discriminatory power of the model (from an AUC of 0.76 to an AUC of 0.80) [17]. Whether this improvement translates to clinical significance remains to be seen. Lastly, Bayesian networks offer more flexibility and result in more intuitive models.

Appropriate risk-stratification tools are necessary to guide clinical treatment goals and monitor disease progression. Clinically, a good risk assessment tool should be evidence based, easy to administer, externally validated, have good discrimination (C-index >0.7), account for “missingness” in data, incorporate weighting of individual variables and reflect the dynamic interactions between variables as well the primary outcome [2]. In the development of contemporary risk stratification in PAH, investigators are limited in their ability to produce robust and highly discriminatory (i.e. C-index >0.8) predictive tools. This relates in part reliance on registry datasets, which are limited in data quality, quantity and comprehensiveness. Although real-world in nature, these registries provide limited yield of high-quality data in light of the differences in patient characteristics enrolled, number of patients observed, quality of data collected and failure to capture relevant variables (i.e. imaging or novel biomarkers) that could add substantially to the comprehensiveness and discriminatory power of equations and calculator. Another significant limitation to the predictive power of contemporary risk assessments is their reliance on traditional statistical methods (Cox proportional hazard) or expert opinion. Cox proportional hazard models allow for estimating the effect of multiple risk factors on survival, with the impact of each individual risk factor expressed by their hazard ratio. However, hazard ratio remains constant over time and is unaffected by concomitant risk factors [18]. In addition, clinically relevant variables such as rate of disease progression remain unaccounted for [19]. Lastly, traditional models are not capable of handling several missing clinical variables, which may not have been obtained at the time of evaluation. This results in a unidimensional and sometimes oversimplified risk prediction, which lacks in robustness with respect to predicting outcome in complex disease. Thus, at this point, until new datasets are made available, adapting our statistical methodology may improve our discrimination. The use of Bayesian networks could help with several of these shortcomings.

As per the 2015 ESC/ERS treatment guidelines, PAH should be risk-stratified as low (<5%), intermediate (5–10%) or high (>10%) risk of mortality at 12 months, to enable guidance on therapeutic decisions. However, in clinical practice, some patients may present with a combination of low-, intermediate- or high-risk features, which can then cloud clinical judgment and misguide subsequent medical therapy. PHORA can be deployed as a decision tool in the clinical arena to integrate the sometimes conflicting information. Another unique advantage of PHORA is that it allows for estimation of the outcome probability based on partial observations, without knowledge of presence or absence of remaining risk factors (figure 4).

Although PHORA was derived from a primarily prevalent patient registry (REVEAL), it was able to predict outcomes with equally good discrimination across two completely different real-world registries, regardless of whether patients were mostly incident (COMEPRA) or prevalent (PHSANZ). Lastly, longitudinal monitoring with PHORA could guide treatment strategies by providing a specific, quantitative metric for satisfactory clinical response (a relative reduction of baseline percentage risk as opposed to lowering a risk stratum). It is envisioned that PHORA outputs and clinical variable entry will be depicted in an easy-to-visualise format on a web-based application, along with comparative REVEAL 2.0, COMPERA and French scores [5, 7] (www.myphora.org; figure 5), allowing a side-by-side decision tool for clinicians to understand both the ranges in risk, the degree of influence of each variable on predicted outcome and likelihood scenario of each clinical case added.

• Download figure
• Open in new tab
• Download powerpoint

FIGURE 5

A screenshot of the webpage that will demonstrate the predicted clinical outcome (survival at 12 months). Outcomes as predicted by Pulmonary Hypertension Outcomes Risk Assessment (PHORA) are shown as a blue bar, as predicted by Registry to Evaluate Early and Long-Term PAH Disease Management (REVEAL) 2.0 as a red bar at 1 and 5 years; Comparative Prospective Registry of Newly Initiated Therapies for Pulmonary Hypertension (COMPERA) risk stratification is shown in yellow; and French noninvasive score in green. The clinical variables are shown at the bottom.

We acknowledge that this study has several important limitations of deriving this new tool from clinical registry data, including missing data pertaining to the independent variables. Although the REVEAL database is large and representative, like other registries it suffers from incomplete capture of many data elements. This could impact the analysis by allowing patients used in both the model training and validation whom have up to 40% of their data missing. This could be particularly pertinent, if the missing data are related to the health of the patient per se (e.g. patient was too sick, so tests could not be done), thus skewing the analysis toward healthy patients. However, the fact that the model is not built on ‘ideal/ complete’ datasets and can handle data missingness is also reflective of real-life clinical scenarios where all clinical data may not be available at each time-point. An additional limitation is the dependency on REVEAL-based cut-points and data used to derive PHORA only reflected prevalent patients who were alive and in the study at 12 months of follow-up. This was done to account for all-cause hospitalisation data in the previous 6 months, but raises concerns that the risk score is subject to survival bias. However, risk prognostication is typically not subject to survivor bias because risk is assessed only during the time the patient has participated in the registry. Whether a change in projected risk prediction scores in PAH reflects a true change in a patient's outcome remains a topic of debate. Lastly, interactions noted between the variables and survival are clinically likely to be even more complex than was captured by the TAN model.

In order to address these limitations, further derivation and validation studies using Bayesian networks that can appropriately handle mixed (categorical and continuous) data are already in progress in a harmonised, contemporary clinical trial dataset (n>3000) in conjunction with the United States Food and Drug Administration (FDA). A combination of both feature engineering (evidence-based, expert guided selection), feature learning (via information scoring) and dimensionality reduction (via unsupervised methods) will be incorporated in these newer iterations of PHORA with a key goal of maximising its discrimination (C-index >0.8), while keeping the tool easy to use. Newer versions of PHORA will not rely only on existing REVEAL variables, but will include other novel and significant variables determined by unsupervised modelling methods and further enhanced by expert opinion. Lastly, Bayesian network-based models at follow-up time-points will be evaluated to capture the impact of variables that may change over time allowing a more comprehensive prediction based on disease progression. We believe that such analyses will allow for a cumulative risk analysis, balancing therapy side-effects against improved outcomes in PAH patients. Moreover, we hope to be able to demonstrate a change in score in response to therapy as being reflective of improved survival in this analysis.

The FDA advocates the prospective use of patient characteristic(s) to select a study population in which detection of a drug effect (benefit, or lack thereof) is more likely than in an unselected population. The use of enhanced risk scores in PAH drug efficacy trials could accommodate enrolment of patients that are deemed to be at intermediate- or high-risk for clinical worsening, hence allowing for substantially smaller sample size and cost-saving.