Microscopic observation drug susceptibility assay for the diagnosis of TB and MDR-TB in HIV-infected patients: a systematic review and meta-analysis

Introduction

Tuberculosis (TB) is a global public health priority, being the single leading cause of death by bacterial infection and the second cause of death by an infectious disease [1]. In 2012, there were an estimated 8.6 million new cases of TB (range 8.3–9 million) globally, equivalent to an incidence of 122 cases per 100 000 population. TB is the cause of 1.3 million deaths worldwide each year [2]. About 320 000 of these deaths are in people living with HIV (PLWH) and it is the first cause of death among PLWH in low-income countries (LICs) [2].

HIV infection increases the risk of having active TB. It has been shown that after seroconversion, the risk of developing TB doubles and continues to increase as CD4 count declines [3–5]. Moreover, despite immunological recovery with CD4 counts above 500 cells·μL⁻¹, the risk continues to be two-fold higher than control individuals in the same community [6]. Therefore, the scale-up of antiretroviral treatment and subsequent increase in the prevalence of HIV due to reduced HIV mortality has led to an increase in TB cases in PLWH in some settings [7, 8].

The diagnosis of TB in HIV-infected patients is challenging. TB often has an atypical clinical presentation and a paucibacillary nature in PLWH, masking the traditional syndrome of chronic cough, fever and weight loss [9, 10]. Sputum smear alone fails to diagnose TB in more than 60% of HIV-infected patients in settings with a high prevalence of HIV infection and does not allow detection of drug resistance [11–13]. Indeed, the difficulties of diagnosing TB in PLWH are among the main factors contributing to the high mortality of HIV/TB co-infection in these settings [14–16]. TB drug resistance is also of special concern in PLWH, due to the extremely high mortality of multidrug-resistant (MDR)-TB and extensively drug resistant (XDR)-TB in these patients [17]. According to the World Health Organization (WHO), in 2012, only 19% of the estimated incident MDR-TB cases were notified [2].

Improvement and expansion of affordable TB diagnostic tests is urgently needed to increase the detection of TB, MDR-TB and XDR-TB worldwide [18–20]. New automatic molecular methods such as GeneXpert MTB/RIF (Cepheid, Sunnyvale, CA, USA) for the diagnosis of TB are currently available [21, 22], but the cost per test is too high for LICs without committed long-term, external funding [23]. In addition, the performance of these tests in HIV-infected patients has been shown to be below what would be desirable [24].

The microscopic observation drug susceptibility (MODS) assay is known as an inexpensive, low-complexity, culture-based technique for the diagnosis of TB infection and TB drug resistance [25–27]. It consists in culturing a previously decontaminated sample in liquid culture media and detecting the growth of Mycobacterium microcolonies with an inverted light microscope; direct drug susceptibility testing can be performed at the same time [25]. Previous meta-analyses have shown that MODS performs very well in the diagnosis of TB and MDR-TB in the general population, with a sensitivity of 96% (95% CI 94–98%) and specificity of 96% (95% CI 89–100%) for the diagnosis of TB, a sensitivity of 98% (95% CI 94.5–99.3%) and specificity of 99.4% (95% CI 95.7–99.9%) for rifampicin resistance, and a sensitivity of 97.7% (95% CI 94.4–99.1%) and specificity of 95.8% (95% CI 88.1–98.6%) for isoniazid resistance (0.1-μg·mL⁻¹ cut-off) [28, 29]. However, they did not specifically analyse PLWH. Although some diagnostic test accuracy studies have included HIV-infected patients, few studies have concluded directly on its accuracy in this target population [30–32]. The potential variation in the performance of MODS in PLWH, and the urgent need for affordable new diagnostic assays for TB and drug-resistant TB in HIV-infected patients led us to perform this study.

The objective of the study was therefore to perform a systematic review and possible meta-analysis of the literature to evaluate the test accuracy of MODS for the diagnosis of active TB and MDR-TB in PLWH. We also aimed to evaluate the diagnostic test accuracy for smear-negative pulmonary TB (SN-PTB) in HIV patients, and to evaluate the time to culture positivity and cost per sample.

Results

The initial screening yielded a total of 3259 citations. Of these, 29 were selected for full-text review and 10 studies were included in the analysis, including a total of 3075 biological samples evaluated for presence of TB bacilli from PLWH (fig. 1) [30–32, 35–41]. 19 out of the 29 initially selected publications were excluded on the basis of: not including HIV⁺ patients [15], insufficient data [2], double publication [1] and not being a diagnostic accuracy study [1]. The characteristics of the studies included are shown in table 1. The studies selected were performed on three continents (South America, sub-Saharan Africa and Asia) from 2002 to 2012. Seven studies followed the cross-sectional design [30, 32, 35–38, 40, 41] and two were case–control studies [31, 39]. All except one were performed in urban settings [40]. The patients included in the studies varied in race (Afro-American, Asiatic and Hispanic) and age; seven studies included adult patients (mean age 30–35 years) [30, 32, 35–38, 40] and two studies focused on children [36, 39]. A variety of samples were cultured: sputum, induced sputum, string, gastric aspirates, nasopharyngeal aspirates, cerebrospinal fluid (CSF), pleural fluid and stool specimens. One study focused on the diagnosis of TB meningitis [35]; the rest focused on the diagnosis of pulmonary TB. One study was designed to evaluate the performance of the string test versus induced sputum with MODS and Löwenstein-Jensen medium [31]. However, although the data were only partly extractable, the author kindly provided the data and the study was therefore retained in the analysis.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1–

Flow diagram of the search and study selection.

Risk of bias and applicability judgments

A summary of the risk of bias is shown in table 2 and figure 2. On evaluating the patient selection, in two studies [31, 39], it was considered that there was a potentially high risk of bias due to their case–control design. In the index test domain, no study was found to have a high risk of bias. Regarding the reference standard, a potential for a high risk of bias was found in three studies as it is known that the performance of TB cultures is not optimal for the diagnosis of TB in small children or for the diagnosis of TB meningitis [35, 36, 39]. Regarding flow and timing, two studies showed a high risk of bias as the sampling was not consecutive [31, 32]. With regard to applicability, we did not find a high degree of concern in any study with respect to patient selection, index test applicability or the reference standard.

• Download figure
• Open in new tab
• Download powerpoint

Figure 2–

Risk of bias and applicability concerns: review authors’ judgements about each domain across the 10 studies included.

View this table:

• View inline
• View popup

Table 2– Risk of bias and applicability concerns summary: review authors’ judgements about each domain for each study included

Sensitivity and specificity

The sensitivity varied across studies and the results of each study are summarised in a forest plot (fig. 3a). The lowest sensitivity reported was 77% (95% CI 60–90%) [35] and the highest was 100% (95% CI 63–100%) [31]. The specificity also varied from the lowest 92% (95% CI 73–99%) [37] to the highest 100% (95% CI 96–100%) [31]. The overall sensitivity and specificity found were 88.3% (95% CI 86.18–90.19%) and 98.2% (95% CI 97.75–98.55%), respectively (fig. 4a).

• Download figure
• Open in new tab
• Download powerpoint

Figure 3–

Forest plot of sensitivity (Se) and specificity (Sp) of microscopic observation drug susceptibility assays for the diagnosis of a) tuberculosis (TB), b) multidrug-resistant TB, c) pulmonary TB and d) smear-negative pulmonary TB, in people living with HIV. Vargas [31] a and b are results for induced sputum and string, respectively. TP: true positive, FP: false positive; FN: false negative; TN: true negative.

• Download figure
• Open in new tab
• Download powerpoint

Figure 4–

Hierarchichal summary receiving operating curves of microscopic observation drug susceptibility assays for the diagnosis of a) active tuberculosis (TB), b) multidrug-resistant TB, c) pulmonary TB and d) smear-negative pulmonary TB, in people living with HIV. Open circles represent individual studies, with the size of the circle proportionate to the study. The summary point is a closed circle, representing sensitivity and specificity estimates pooled with a bivariate random-effects model.

Only four studies reported data on MDR-TB (fig. 3b) [30, 37, 38, 41]. The summary sensitivity and specificity were 89% (95% CI 66.07–97%) and 100% (95% CI 94.81–100%), respectively (fig. 4b).

Data related to the diagnosis of SN-PTB were only retrieved from four studies (fig. 3d) [31, 32, 37, 41]. A summary sensitivity–specificity point was plotted, providing a sensitivity of 88.2% (95% CI 86.1–89.9%) and a specificity of 98.2% (95% CI 96.8–98.9%). On evaluation by visual inspection of the forest plots, the study of Makamure et al. [37] was found to be an outlier. A sensitivity analysis was performed excluding this study, and a sensitivity of 88.8% (95% CI 84.1–92.3%) and a specificity of 98.2% (95% CI 97.7–98.6%) were found.

Caws et al. [35] evaluated the sensitivity and specificity of MODS for the diagnosis of TB meningitis. They showed a sensitivity of 77% (95% CI 60–90%) and a specificity of 100% (95% CI 93–100%) when compared with other cultures (MGIT and Löwenstein-Jensen). On assessing the forest plots and the HSROC curves for heterogeneity, this study was found to be an outlier, as it showed a sensitivity somewhat below the remaining studies. A sensitivity analysis was performed excluding this study and the summary sensitivity increased to 95% (95% CI 95–96%).

Two studies were found to evaluate the usefulness of MODS in HIV-infected children [36, 39]. The samples used were heterogeneous (sputum, gastric fluid, nasopharyngeal aspirates, pleural fluid, CSF and stools) and were presented as a pooled analysis in the original studies. The sensitivity in children living with HIV was 80% (95% CI 28–99%) in one study and 100% (95% CI 3–100%) in the other [36, 39]. In one of these studies, sputum samples were used and sensitivity was 80% (95% CI 28–100%) [36]. The specificity was high in both studies, being 100% (95% CI 74–100%) and 100% (95% CI 92–100%). A sensitivity analysis was performed excluding these studies and the pooled estimate remained almost unaltered (sensitivity 88.1% (95% CI 83–91.8%) and specificity 98.2 (95% CI 98.1–98.3%)).

Data about the time to culture positivity was only retrieved in five studies [30, 32, 36, 37, 41], varying from study to study and ranging from 5.5 to 14.3 days. As individual-level patient data were not available, a mean weighted by the study sample size was calculated, yielding a result of 8.24 days till culture positivity.

Information about the cost per sample processed for TB diagnosis was only found in four studies [30, 32, 35, 38]. Caws et al. [35], Ha et al. [32] and Moore et al. [38] only reported costs for reagents and consumables, and did not account for labour, capital or overhead costs. A sample size weighted mean of the cost per sample was USD 0.72 per test (absolute range USD 0.53–2.00). Reddy et al. [30] took a more comprehensive approach, obtaining a total cost per sample of USD 7.31.

Discussion

To our knowledge this is the first systematic review and meta-analysis to evaluate the performance of MODS in PLWH. The results suggest that MODS could be a valuable technique for the diagnosis of TB in PLWH, with good sensitivity and specificity, allowing for faster results than conventional culture techniques and at a lower cost.

MODS was also found to have both a good sensitivity and specificity for the diagnosis of active MDR-TB. Although the number of studies included was small and results should be interpreted with caution, these results are in line with those published on the diagnosis of MDR-TB by MODS in the general population (sensitivity 97.7% and specificity 95.8% for isoniazid resistance, and sensitivity 98% and specificity 98.6% for rifampicin resistance) [29]. Being able to obtain the results of drug susceptibility testing within 8 days and the possibility of also performing second-line drug susceptibility testing make MODS a very interesting and attractive technique for LICs [42, 43]. Moreover, it has recently been validated for DST in patients failing first-line TB treatment [44].

Due to the paucibacillary nature of TB in HIV-infected patients, the performance of MODS in SN-PTB is especially interesting. A summary sensitivity of 88.2% (95% CI 86.1–89.9%) and a specificity of 98.2% (95% CI 96.8–98.9%) for sputum cultures in SN-PTB was found. These results are in line with another recently published study that evaluated the performance of MODS in SN-PTB [45]. Moreover, data from the study by Vargas et al. [31] suggest that if a string test is used in smear-negative HIV-infected patients, the sensitivity and specificity of MODS are higher than those of standard microbiological cultures.

Another important issue in PLWH is how MODS performs in the diagnosis of extrapulmonary TB and in samples other than sputum. Data are again scarce and the different samples used in the different studies did not allow stratified analysis. The only extrapulmonary form of TB that we could evaluate was TB meningitis, and the results showed that MODS appears to be a promising technique with a diagnostic yield in line with other culture techniques [46]. However, the sensitivity was lower than that for sputum cultures. The authors of the study suggested that the volume cultured played a role in this low sensitivity, as they used 100 μL of deposit for MODS and 250 μL for Löwenstein-Jensen and MGIT. In a previous study, the same group found that the volume of CSF had a considerable influence on the positivity rate of the cultures [47]. Moreover, our sensitivity analysis showed that this study had a clear impact on the pooled estimates, suggesting that the sample type influences the sensitivity and specificity of the technique. Further studies are needed to evaluate how to optimise MODS in the diagnosis of TB meningitis and other forms of extrapulmonary TB in PLWH.

All the studies were designed to assess the performance of MODS in symptomatic patients but that of Reddy et al. [30] addressed its performance as a screening tool for asymptomatic HIV-infected patients. This specific use of MODS is interesting as ruling out TB in PLWH before starting isoniazid preventive therapy (IPT) is challenging in clinical practice. There is an ongoing debate about the best screening method for pulmonary TB among PLWH in LICs [11]. MODS may have an important role as it delivers results faster than other mycobacterial cultures. Nevertheless, its complexity may be a drawback. In their study, Reddy et al. [30] proposed that liquid culture of two sputum samples alone can be used as an effective screening strategy for pulmonary TB before IPT in PLWH. More research on the use of MODS as a screening strategy, alone or in combination with other screening strategies, in LICs is needed as IPT has the potential to save millions of lives and provide an important contribution to the control of TB in regions with a high burden of HIV infection.

The diagnosis of TB in children is challenging. Two studies have analysed the performance of MODS in HIV-infected children, showing a sensitivity and specificity that were roughly the same as for other mycobacterial cultures. However, it is known that in the case of children, the diagnosis of TB is difficult due to the paucibacillary nature of this infection in this population, and cultures do not diagnose all cases of active TB [48]. A sensitivity analysis was performed excluding the studies evaluating children, as doubts arose whether to include them or not, as no age restriction had been pre-specified in the study protocol. Nonetheless, the pooled estimate remained unaltered.

A high degree of heterogeneity of how costs were counted was found. Only counting costs for reagents and consumables, the cost varied from USD 0.53 to USD 2.00 per test. When a more comprehensive approach was used, the costs rose to USD 7.31 per test. This is very inexpensive in comparison with other alternatives, such as automated liquid cultures (average cost USD 16–32 per test) [49], or even subventioned rapid molecular assays, such as GeneXpert MTB/RIF (cost per cartridge USD 9.98 ex works) [50–52]. Nevertheless, capital, labour, overhead and other costs also have to be accounted for. Pragmatic trials in LICs should be encouraged to choose the best diagnostic option for these patients in these settings, evaluating not only diagnostic test accuracy but also its cost-effectiveness and affordability.

The greatest limitation of this study is the small number of studies included, as the number of studies evaluating MODS in PLWH is low. Moreover, many studies assessing the performance of MODS performed in settings with a high prevalence of HIV infection do not report separate data for PLWH and, in some of them, HIV testing was not systematically performed in TB patients [53–55]. Only four of the 10 studies included were specifically designed to address the performance of MODS in PLWH [30–32, 39]. Nevertheless, the search for published studies was thorough: a broad search in electronic medical databases as well as grey literature databases, and conference books of abstracts were also searched for studies. In addition, the authors of studies that did not provide complete data in the published papers were contacted to provide missing data. Further studies to assess the performance of MODS in HIV-infected adults and children in LICs are needed and disaggregation of data of PLWH should be encouraged. Another limitation is that quite a high risk of bias was found on assessing the quality of the studies included, being mainly related to the reference standard used. Cultures were not considered to correctly classify the target condition in many studies, reflecting the fact that there is no true gold standard for the diagnosis of TB. However, TB cultures are believed to be the best proxy available.

MODS was specifically developed and designed to be set up in LICs. Nevertheless, there are concerns about its feasibility in rural areas, and WHO has not recommended the implementation of MODS in rural laboratories at district levels but has recommended the use of the GeneXpert MTB/RIF assay [56]. The development of the GeneXpert MTB/RIF assay is undoubtedly a landmark event. It is easy to use and fast; in a recent meta-analysis by the Cochrane Collaboration Group, it showed a pooled sensitivity in PLWH of 80% (95% CI 67–88%); and it has a low biohazard level [24]. In modelling studies, cost-effectiveness has been evaluated and it has been postulated as cost-effective [57, 58]. However, it has been deemed to be too expensive for a point-of-treatment setting and has been recommended to be installed only at laboratory facilities [59]. In addition, clinical and programmatic effects have recently been evaluated; results have not been so encouraging since the inclusion of more patients on same-day treatment and more culture-positive patients on treatment; and shorter time to treatment did not translate into lower TB morbidity [60]. This study also suggested that the cost-effectiveness may have been overestimated. However, MODS may perform better in PLWH and could be less costly. Studies directly comparing MODS and GeneXpert MTB/RIF, especially in high HIV burden settings, could help defining better diagnostic strategies for TB and MDR-TB in LICs.

In summary, we found MODS to be a good technique for the diagnosis of TB, MDR-TB and SN-PTB in PLWH. However, the data available are scarce, and more studies on its performance in HIV-infected patients and comparisons with molecular techniques such as GeneXpert MTB/RIF should be encouraged.