Pulmonary embolism in COVID-19: D-dimer threshold selection should not be based on maximising Youden's index

To the Editor:

The severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) pandemic has raised new challenges in the diagnosis of pulmonary embolism (PE) [1]. Patients with coronavirus disease 2019 (COVID-19) are at increased risk of developing venous thromboembolism, but symptoms of COVID-19 and PE may overlap, which makes it difficult to identify those with a higher likelihood of PE. Simple and minimally invasive diagnostic algorithms that can safely rule-out PE in patients with COVID-19 are urgently needed. Therefore, we read with interest the recent paper by Mouhat et al. [2] in the European Respiratory Journal.

In their study, the authors retrospectively evaluated factors associated with PE among 162 hospitalised patients with severe COVID-19 who had undergone computed tomography pulmonary angiography (CTPA) as the reference standard for PE. They reported that two variables were significantly associated with PE: not receiving anticoagulant therapy and D-dimer testing. The authors report that they “identified a D-dimer cut-off value of 2590 ng·mL⁻¹ to best predict occurrence of PE”. They propose that “our data plead in favour of a wider screening strategy for PE by performing CTPA in COVID-19 patients who have signs of clinical severity and D-dimer levels >2590 ng·mL⁻¹”, and that “particular attention should be paid to search for potential PE in patients […] with a D-dimer level above 2590 ng·mL⁻¹”. However, we believe that the rationale for selecting this D-dimer threshold is not clinically relevant, and that it cannot be safely applied in COVID-19 patients.

D-dimer testing is insufficiently accurate to be used as a standalone test in the diagnosis of PE [3]. Therefore, diagnostic algorithms such as the Wells’ rule and the YEARS criteria have been developed, in which D-dimer is used as a triage test [4, 5]. In patients with a low D-dimer (usually, a threshold of 500 or 1000 ng·mL⁻¹ is applied), PE can safely be ruled-out without CTPA. In contrast, in those with a D-dimer above the threshold, subsequent CTPA needs to be performed. In these algorithms, D-dimer thresholds were selected based on the fact that they correspond to a negative predictive value (NPV) that is close to 100%, ensuring that PE can be safely ruled-out without further testing [6].

Whether similar D-dimer thresholds can be applied in COVID-19 patients suspected of PE is unknown, because COVID-19 triggers a hyperinflammatory state with endothelial activation and high D-dimer levels [7]. In their article, Mouhat et al. [2] propose a threshold of 2590 ng·mL⁻¹. This threshold was not selected to obtain an NPV close to 100%, but it was based on the highest “Youden's index”. This works as follows: within the study population, for every D-dimer threshold, a corresponding sensitivity and specificity for diagnosing PE is calculated. If the D-dimer threshold is set at 0 ng·mL⁻¹, every patient is considered “positive”, implying that sensitivity is 100% and specificity is 0%. When increasing the threshold, more patients will have a “negative” D-dimer, resulting in a decreasing sensitivity and an increasing specificity. Each possible D-dimer threshold corresponds to a pair of sensitivity and specificity, and these pairs can be plotted into a receiver operating characteristic (ROC) curve (figure 5 in the article of Mouhat et al. [2]). The threshold that provides the highest Youden's index is the one that maximises the sum of (sensitivity + specificity). It is also the threshold that maximises the odds ratio in logistic regression modelling.

In most situations, selecting the positivity threshold for a biomarker based on the highest Youden's index may be statistically meaningful, but is clinically irrelevant. Not only is such data-driven selection of an “optimal” threshold likely to result in biased accuracy estimates and poor reproducibility [8], but it generally also leads to sensitivity and specificity parameters that are too low to, respectively, rule-out or rule-in the target condition with a sufficient level of certainty. In the study by Mouhat et al. [2], the proposed D-dimer threshold of 2590 ng·mL⁻¹ leads to a sensitivity of 83.3% (95% CI 68.6–93.0) and a specificity of 83.8% (95% CI 73.8–91.1), corresponding to a positive predictive value of 72.9% (95% CI 61.7–81.8) and an NPV of 90.5% (95% CI 82.9–95.0). This implies that using this threshold in clinical practice would result in missing 17% of PE, which is unacceptable.

We believe that Mouhat et al. [2] should have provided an ROC table, showing sensitivity and specificity at multiple D-dimer thresholds, so that a threshold corresponding to an NPV close to 100% could have been identified, as this is clinically much more meaningful. We strongly urge physicians not to apply the threshold proposed by the authors in clinical practice, as this is likely to result in a considerable proportion of PE being missed. We also encourage researchers evaluating the diagnostic accuracy of markers such as D-dimer to select thresholds based on minimally acceptable targets of accuracy estimates rather than on maximising Youden's index [9]. There are considerable concerns regarding the quality of diagnostic accuracy and prediction model studies in the COVID-19 literature [10, 11]. It is crucial that researchers make efforts to improve this by applying the available methodological and reporting guidelines for such studies [12, 13].

Shareable PDF