Skip to main content

Main menu

  • Home
  • Current issue
  • ERJ Early View
  • Past issues
  • ERS Guidelines
  • Authors/reviewers
    • Instructions for authors
    • Submit a manuscript
    • Open access
    • COVID-19 submission information
    • Peer reviewer login
  • Alerts
  • Subscriptions
  • ERS Publications
    • European Respiratory Journal
    • ERJ Open Research
    • European Respiratory Review
    • Breathe
    • ERS Books
    • ERS publications home

User menu

  • Log in
  • Subscribe
  • Contact Us
  • My Cart
  • Log out

Search

  • Advanced search
  • ERS Publications
    • European Respiratory Journal
    • ERJ Open Research
    • European Respiratory Review
    • Breathe
    • ERS Books
    • ERS publications home

Login

European Respiratory Society

Advanced Search

  • Home
  • Current issue
  • ERJ Early View
  • Past issues
  • ERS Guidelines
  • Authors/reviewers
    • Instructions for authors
    • Submit a manuscript
    • Open access
    • COVID-19 submission information
    • Peer reviewer login
  • Alerts
  • Subscriptions

Interferon release does not add discriminatory value to smear-negative HIV–tuberculosis algorithms

M.X. Rangaka, H.P. Gideon, K.A. Wilkinson, M. Pai, J. Mwansa-Kambafwile, G. Maartens, J.R. Glynn, A. Boulle, K. Fielding, R. Goliath, R. Titus, S. Mathee, R.J. Wilkinson
European Respiratory Journal 2012 39: 163-171; DOI: 10.1183/09031936.00058911
M.X. Rangaka
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • For correspondence: mxrangaka@yahoo.co.uk
H.P. Gideon
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
K.A. Wilkinson
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
M. Pai
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
J. Mwansa-Kambafwile
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
G. Maartens
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
J.R. Glynn
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
A. Boulle
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
K. Fielding
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
R. Goliath
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
R. Titus
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
S. Mathee
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
R.J. Wilkinson
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Article
  • Figures & Data
  • Info & Metrics
  • PDF
Loading

Abstract

Clinical algorithms for evaluating HIV-infected individuals for tuberculosis (TB) prior to isoniazid preventive therapy (IPT) perform poorly, and interferon-γ release assays (IGRAs) have moderate accuracy for active TB. It is unclear whether, when used as adjunct tests, IGRAs add any clinical discriminatory value for active TB diagnosis in the pre-IPT assessment.

779 sputum smear-negative HIV-infected persons, established on or about to commence combined antiretroviral therapy (ART), were screened for TB prior to IPT. Stepwise multivariable logistic regression was used to develop clinical prediction models. The discriminatory ability was assessed by receiver operator characteristic area under the curve (AUC). QuantiFERON®-TB Gold in-tube (QFT-GIT) was evaluated.

The prevalence of smear-negative TB by culture was 6.4% (95% CI 4.9–8.4%). Used alone, QFT-GIT and the tuberculin skin test (TST) had comparable performance; the post-test probability of disease based on single negative tests was 3–4%. In a multivariable model, the QFT-GIT test did not improve the ability of a clinical algorithm, which included not taking ART, weight <60 kg, no prior history of TB, any one positive TB symptom/sign (cough ≥2 weeks) and CD4+ count <250 cells per mm3, to discriminate smear-negative culture-positive and -negative TB (72% to 74%; AUC comparison p=0.33). The TST marginally improved the discriminatory ability of the clinical model (to 77%, AUC comparison p=0.04).

QFT-GIT does not improve the discriminatory ability of current TB screening clinical algorithms used to evaluate HIV-infected individuals for TB ahead of preventive therapy. Evaluation of new TB diagnostics for clinical relevance should follow a multivariable process that goes beyond test accuracy.

  • Diagnostic research
  • HIV
  • incremental value
  • interferon-γ release assay
  • Mycobacterium tuberculosis
  • QuantiFERON®-TB Gold in-tube

The HIV-1 epidemic has derailed tuberculosis (TB) control in sub-Saharan Africa. It is recognised that additional interventions, apart from the otherwise successful directly observed treatment short course (DOTS) strategy, are required to control TB [1]. The World Health Organization (WHO) therefore promotes isoniazid preventive therapy (IPT), intensified TB case finding and TB infection control to curb TB in HIV-infected individuals as part of the “Three Is” initiative [2]. Active TB should be excluded prior to starting IPT to avoid drug resistance from isoniazid monotherapy [3]. Where sputum culture for Mycobacterium tuberculosis has been employed as a gold standard method in various settings, the median prevalence of positivity has ranged from 0.7% in population-based surveys to 8.5% in HIV-1 voluntary counselling and testing services [4]. Asymptomatic and smear-negative forms of disease are also common and so the operational need to screen for prevalent TB has become very important [5, 6].

The gold standard for intensified TB case finding remains sputum culture. It is more sensitive than smear, but takes several weeks and is expensive. Thus, screening algorithms based on TB symptoms have been suggested but sensitivity and specificity vary between strategies, studies and location [5, 7–11]. A recently proposed algorithm to screen HIV-infected persons living in resource-poor areas for TB reported that any one symptom/sign out of cough of any duration, fever, night sweats or weight loss were 79% sensitive with a negative predictive value of 98% (in a 5% TB prevalence setting), but only 50% specific. The authors concluded that reliably ruling out active TB might have priority over generating a consistent group of subjects with positive screening and no TB. However, with such poor specificity, an implicit consequence of the proposed algorithm is that the majority of those with false positive results would still require further investigation to rule out active TB disease [12]. This highlights the need for alternative or additional techniques.

Infection by M. tuberculosis can be inferred from evidence of immune sensitisation either in vivo by the Mantoux tuberculin skin test (TST) or in vitro by antigen-specific release of interferon (IFN)-γ by T-cells (IFN-γ release assay; IGRA). These present additional TB screening strategies. However, the utility of IGRAs in high-TB incidence settings where HIV-1 co-infection is also common remains unclear because, like the TST, IGRAs cannot distinguish between latent infection and active disease. Used as single stand-alone tests, both IGRA and the TST are imperfect for diagnosing those who have prevalent active disease as they have moderate positive predictive value and can be falsely negative [13–17]. Similarly, a single positive result may not necessarily mean the presence of active disease. Based on existing data reporting poor performance of current commercial IGRAs in low- and middle-income countries, a recent WHO Strategic and Technical Advisory Group meeting on the use of commercial IGRAs for the diagnosis of latent and active TB planned to support an approach to develop a “negative” WHO policy recommendation to discourage the use of commercial IGRAs in those countries [18]. However, the utility of IGRAs may be better appreciated in multivariable clinical algorithms. It is thus important to understand the incremental or added value of IGRAs in active TB diagnosis, over and beyond conventional tests for active TB [19, 20]. Considering IGRAs with other predictors might modify both its accuracy estimates of sensitivity, specificity and likelihood ratios, and the overall posterior probability of active TB disease [21, 22]. This is the crux of a multivariable evaluation. Such an evaluation of IGRAs has not previously been explored and in the context of the pre-IPT evaluation. The WHO has identified these as priority research questions in the IGRA and TB prevention areas [23].

Our hypothesis was that when used as adjunct tests, even poorly to moderately performing TB biomarkers will improve the ability of current multivariable clinical algorithms to discriminate those with and without TB. Therefore, we assessed the incremental or additive discriminatory value of QuantiFERON®-TB Gold in-tube (QFT-GIT; Cellestis, Carnegie, Australia) or Mantoux TST when added as adjunct tools to detect active TB among sputum smear-negative persons who were being prescribed or about to commence antiretroviral therapy (ART) and being assessed for IPT.

MATERIAL AND METHODS

Participant recruitment and study setting

The University of Cape Town (Cape Town, South Africa) research ethics committee approved this study (REC 013/2007) and written informed consent was provided. The study setting was the Ubuntu Clinic in Khayelitsha Site B, an HIV-TB outpatient facility 30 km east of Cape Town, South Africa. Overall TB incidence during the study was ∼1,500 per 100,000 inhabitants. Consecutive participants undergoing screening for an ongoing pragmatic randomised controlled trial (RCT) were invited. The trial was established to determine the effectiveness of the combination of IPT and ART provided under routine clinical practice to reduce the risk of active TB in HIV-infected persons (clinicaltrials.gov identifier NCT00463086 and Lancet D-09-02885 approved protocol). Participants were eligible if they were willing to provide additional consent for further screening by TST and QFT-GIT. They were screened regardless of initial TB symptoms or signs and regardless of eligibility for the ART–IPT study. The selection criteria for the RCT were not applied when assessing patients' eligibility for evaluation by QFT-GIT. Screening occurred between November 2007 and September 2009. Adults aged ≥18 yrs, who were already prescribed ART or about to commence ART and whose TST response could be assessed by the study team in 48–72 h, were eligible to participate. There were no other specific exclusion criteria for the QFT-GIT substudy. To meet our study objectives, all analyses were undertaken on a restricted dataset of sputum smear-negative participants. This evaluation of standard QFT-GIT occurred under routine clinical practice and was not intended as an optimisation study for QFT-GIT.

Study procedures

All TB screening tools were administered on the day of screening. Participants were evaluated for the following TB symptoms or signs at the baseline screening visit: cough ≥2 weeks; night sweats in the last 2 weeks; self-reported weight loss for those newly prescribed ART or weight loss ≥1.5 kg in 1 month for patients already taking ART at screening; fever ≥2 weeks; and significant lymphadenopathy (>2 cm in diameter) on examination. The algorithm reflects South African guidelines including the TB Programme and the National IPT guidelines for HIV-infected persons. Chest radiography (CXR) did not form part of the screening strategy.

A single sputum sample for TB microscopy and culture was obtained. Patients who failed to produce sputum either spontaneously or following ultrasonic nebulisation with hypertonic saline were either asked to return the following day (with a sample) or nebulisation was repeated if necessary at their next clinic visit. A window period of up to 1 month for sputum collection was permitted. Specimens were sent to an academic reference laboratory at Groote Schuur Hospital (Cape Town) for processing. The laboratory is part of the South African National Health Laboratory Service. A single positive culture (BACTEC mycobacterial growth indicator (Becton Dickinson Microbiology Systems, Cockeysville, MD, USA) and Lowenstein–Jensen media) for M. tuberculosis was regarded as definite active disease and formed the reference standard criterion for diagnostic accuracy in this study. Given the high rate of active TB in the recruited population, all cases with positive cultures were considered as definite TB and thus referred for prompt DOTS. Upon referral to the TB programme, further assessments initiated included physical examination and/or CXR. During this study period, the reference laboratory reported low cross-contamination; of 500 dummy sputum samples, none were false positive for M. tuberculosis (M. Nicol, Division of Medical Microbiology, Faculty of Health Sciences, University of Cape Town and National Health Laboratory Service, Cape Town, South Africa; personal communication).

The TST (2 TU RT23 purified protein derivative (PPD); Statens Serum Institut, Copenhagen, Denmark) was administered on the volar aspect of the left forearm by personnel trained in its administration. The TST induration was recorded after 48–72 h by the ballpoint pen and ruler method. Phlebotomy for QFT-GIT was performed on the same day that the TSTs were administered and preceded placement of PPD. QFT-GIT assays were performed in a Cellestis-accredited laboratory and interpreted according to the manufacturer's guidelines. Results were also recorded quantitatively (antigen-stimulated IFN-γ corrected for background, expressed in IU·mL−1). Laboratory technicians were blinded to TST results, TB symptoms and signs and culture results, while clinicians were blinded to QFT-GIT results.

Diagnostic accuracy of single tests for prevalent sputum smear-negative, culture-positive TB

Using culture positivity as the gold standard, we computed standard test accuracy measures to detect prevalent disease. Post-test probabilities of disease based on negative test results were calculated to assess the clinical utility of a negative single stand-alone test result for ruling out active TB. Discriminatory ability was evaluated by receiver operating characteristic area or area under the curve of sensitivity versus 1 minus specificity (AUC). That is, based on the results of a stand-alone test, do patients with culture-positive TB have higher risk predictions than those without [24]?

Selection of potential predictors of TB

All analyses were performed with STATA 10 MP (StataCorp, College Station, TX, USA). Candidate clinical predictors were determined a priori based on clinical judgment and the published literature [10, 25] and they included: ART status (taking ART or not), sex, age, weight at screening, most recent CD4 count (no older than 6 months), prior TB and TB symptoms and clinical signs. TST at the clinically relevant cut-off of 5 mm and QFT-GIT at standard manufacturer's cut-offs were pre-specified for multivariable analysis. Further details are provided in the online supplementary material, S-1.

Prediction model development and assessment of added discriminatory value

Multivariable logistic regression analysis was performed to develop diagnostic models for culture-positive TB. First, a reduced, most parsimonious, clinical model without additional tests of TB infection (TST and QFT-GIT) was derived. TB tests of infection were added singly to the reduced clinical model and then simultaneously to explore the added predictive value of a single test and of combined tests, respectively. The ability of a multivariable model to discriminate M. tuberculosis culture-positive and -negative patients was assessed by AUC. That is, based on the results of the test when added to multiple clinical predictors, do patients with the outcome have higher risk predictions than those without? The AUC was chosen as a measure of model discrimination because it is an objective estimate where individual predicted probabilities, and hence all probability cut-off points, are used in the assessment of model performance. Full details on model selection, calibration and discrimination are provided in the online supplementary material, S-2.

RESULTS

Characteristics of the reference groups

1,686 eligible participants were consecutively screened between November 2007 and September 2009 (fig. 1). Of these, 1,405 had culture results available, with an overall prevalence of culture-positive TB of 8.2% (95% CI 6.8–9.7%); after exclusions (fig. 1), 779 smear-negative participants were available for analysis. The prevalence of smear-negative, culture-positive TB was 6.4% (95% CI 4.9–8.4%). 56% of TB cases had at least one TB symptom or sign upon referral for DOTS. 88% of cases had CXR performed and 57% of those had signs suggestive of TB. The median (interquartile range) duration between the baseline clinical assessment and referral was 6 (4–12) weeks. The baseline characteristics of participants with both TST and QFT-GIT results stratified by culture status are shown in table 1. Patients with prevalent TB had a lower CD4 count (median 169 versus 198 cells per mm3), were less likely to be taking ART at screening (54% versus 34%) and were more likely to report any of a range of symptoms and signs of TB, but the overall frequency of any one TB symptom or sign was low. At least 80% of those with culture-positive disease had either a positive TST (at 5 mm) or QFT-GIT result. The highest proportion of those with TB was represented in the highest QFT-GIT tertile compared with the lower strata (p=0.005, score test for trend).

Figure 1–
  • Download figure
  • Open in new tab
  • Download powerpoint
Figure 1–

Flow-chart detailing recruitment into the study. This shows participant flow and the numbers used in the final multivariable discriminatory value analysis. QFT-GIT: QuantiFERON®-TB Gold in-tube; TST: tuberculin skin test.

View this table:
  • View inline
  • View popup
Table 1– Characteristics of sputum smear-negative patients stratified by Mycobacterium tuberculosis culture status

Diagnostic accuracy of single tests for prevalent sputum smear-negative, culture-positive TB

The diagnostic accuracy measures of the TST and QFT-GIT as single stand-alone tests are moderate and comparable regardless of whether QFT-GIT results included indeterminate results or not (table 2). The combined variable of either TST or QFT-GIT positive was the most sensitive test, 80% (95% CI 66–90%) and 83% (95% CI 70–93%) when QFT results included indeterminate results as negatives or when indeterminate results are excluded, respectively. The clinical utility of single negative tests to rule out smear-negative, culture-positive TB was poor and the post-test probability of disease following application of negative tests remained close to the pre-test probability of 6%. The discriminatory ability of the individual predictors to identify those with disease and those without disease, based on AUC, was limited. Both the TST and QFT-GIT were comparable, and also comparable to the variable of any one positive TB symptom/sign (table 2). Details of other accuracy measures and those for TB symptoms are presented in table 2.

View this table:
  • View inline
  • View popup
Table 2– Diagnostic accuracy of single tests: tuberculosis (TB) symptoms and signs, tuberculin skin test (TST) and QuantiFERON®-TB Gold in-tube (QFT-GIT)

Predictors of prevalent active TB

The following clinical covariates emerged as significant independent predictors of TB during univariable logistic regression analyses (table 3): not being on ART at screening, weight <60 kg and CD4+ count <250 cells per mm3. No history of TB in the past was a strong predictor of prevalent TB; that is, those who had been treated for active TB were less likely to have prevalent TB. Patients with any one positive TB symptom/sign were 4.0 times as likely to have TB compared with those with no symptoms (OR 4.0 (95% CI 2.0–7.9)). The TST at the 5, 10 and 15-mm threshold (OR 3.0 (95% CI 1.6–5.6), OR 3.3 (95% CI 1.8–6.0) and OR 3.3 (95% CI, 1.9–5.9), respectively) and TST quantitative results also emerged as significant predictors. Patients with a positive QFT-GIT response (quantitative and at the manufacturer's thresholds) were 2.7 times more likely to have culture positive TB (OR 2.7 (95% CI 1.5–5.2)). Either TST or QFT-GIT positivity was also a significant predictor of TB. There was a tendency towards increasing positivity for those with indeterminate and positive results compared with those with negative responses; however, the score test for trend was not significant (p=0.88). When QFT-GIT responses were grouped into tertiles, it was seen that individuals in the highest then the middle tertiles were three- and two-fold as likely, respectively, to have culture-confirmed active TB compared with those with negative responses (score test for trend p=0.005).

View this table:
  • View inline
  • View popup
Table 3– Univariable predictors of active tuberculosis (TB)

Multivariable logistic regression models and the discriminatory value of TST or QFT-GIT when added to the final multivariable clinical model

Our final clinical prediction model included the following predictors: not being on ART at screening; weight <60 kg; no prior history of TB; any one positive TB symptom or sign; and CD4+ count <250 cells per mm3 (table 4). Compared with individual predictors of TB in table 2, this multivariate model had significantly improved discrimination for active TB (AUC of 0.72 for the model versus AUC range of 0.51–0.64 for predictors; individual AUC comparison p-values are not displayed). The clinical model was then extended by adding the TST (at 5 mm) and QFT-GIT (manufacturer's cut-off) as separate tests and with both included in the extended model. Models showed good calibration using the Hosmer–Lemeshow goodness of fit test (table 4). The odds ratios for the extended models with TST and QFT-GIT, added singly and together, were similar and the confidence intervals overlapped. However, there was no significant difference in the AUC between the QFT-GIT extended model and the clinical model (p=0.33 for AUC comparisons). By contrast, the discriminatory ability of the clinical model was statistically significantly improved by addition of the TST (p=0.04 for AUC comparisons; table 4). However, the confidence levels of the TST and QFT-GIT extended clinical models overlapped. Simultaneously adding both TST and QFT-GIT to the best clinical model also resulted in an AUC that was significantly different from the clinical model, suggesting further overall improved discriminatory ability for culture-positive and -negative TB.

View this table:
  • View inline
  • View popup
Table 4– Multivariable logistic regression estimates for culture-positive disease in clinical model with and without tests of tuberculosis (TB) infection

In secondary analyses, addition of either a positive TST (5 mm) or QFT-GIT result, and of quantitative QFT-GIT divided into tertiles of increasing reactivity did not improve the final clinical model (for details see the online supplementary material, S-3).

DISCUSSION

Undiagnosed TB is a major concern in HIV-infected persons. Existing clinical algorithms are not highly accurate in smear-negative patients and available biomarkers, such as IGRAs, perform moderately. Our main finding was that even when considered as an adjunct test, QFT-GIT did not improve the ability of our best multivariable clinical model to discriminate between culture positive and culture negative. This incremental discriminatory value of QFT-GIT when added to current clinical algorithms used in assessment of HIV-infected persons in whom TB must be ruled out prior to TB prevention had not previously been explored. The WHO has identified this as a gap in the IGRA body of knowledge [18, 23].

In a conventional analysis, QFT-GIT interpreted according to the manufacturer's guidelines, did not have high sensitivity (64–68%) and had poor specificity (56–59%) for active disease in this setting. Furthermore, single negative test results did not significantly reduce the post-test probability of disease. This is not new. Moderate sensitivity for active TB was also shown in a recently published systematic review and meta-analysis for the diagnosis of latent TB infection in HIV-infected individuals (61% (95% CI 41–75%) for QFT-GIT, using culture as a surrogate reference standard) [26]. Thus, neither QFT-GIT nor the TST should be used in isolation as a rule out or rule in test for active TB prior to IPT provision. This conclusion is in line with a similar body of evidence that suggested that stand-alone IGRA tests have limited application in excluding active TB in challenging clinical scenarios [14, 26–29]. Based on such data, WHO plans to issue a statement to discourage the use of commercial IGRAs in low- and middle-income countries [18]. However, simple test accuracy based on single stand-alone tests does not in itself demonstrate clinical relevance [20, 21, 30, 31]. It is important to demonstrate the extent to which IGRAs have true added value in the clinical context, when all identified predictors of TB are considered.

In our multivariable analyses, not being on ART at screening, weight <60 kg, no history of prior TB, any one positive TB symptom or clinical sign (includes cough ≥2 weeks) and CD4+ count <250 cells per mm3 all emerged as strong independent predictors of smear-negative, culture-positive TB (AUC 72%). Not having had TB in the past is strongly associated with prevalent active TB in HIV-infected individuals and has previously been cited as a predictor of risk in ART cohorts [32–34]. The value of this predictor is interesting and was highlighted in our recent immunological evaluation of T-cell responses of HIV-infected patients without active TB who were newly starting ART [35]. Patients with a prior history of TB who had previously completed TB treatment had a reduced response in an in house IGRA than those who had not previously received TB treatment, possibly suggesting the likelihood of having a lower bacterial load [35]. Clinical prediction models derived for similar settings should therefore continue to evaluate this predictor.

QFT-GIT did not improve the ability of the clinical model to discriminate between culture positive and culture negative. By contrast, addition of the TST (5-mm cut-off) and the combination of both the TST (5-mm cut-off) and QFT-GIT at the standard manufacturer's cut-off statistically significantly improved the discriminatory ability of the clinical tools (AUC ∼80% for both models). However, the AUC confidence intervals overlapped. The conclusion that, in its current format, QFT-GIT may add very little to current clinical tools is consistent with recent evidence. A paper by Metcalfe et al. [19] explored whether QFT-GIT improves the classification of HIV-uninfected, smear-negative pulmonary and extrapulmonary TB suspects into researcher-selected risk categories. They concluded that although QFT-GIT improved risk stratification of patients when added to clinical and demographic risk factors, there was no additional benefit when clinical judgment of experienced clinicians was also considered.

Despite the current poor performance of commercial IGRAs, the technology remains a significant operational and conceptual advance. It is logistically simpler than the TST, especially in the provider/patient environment and in scoring. The TST could not be determined for 193 patients as they failed to return for their readings. By contrast, fewer (55 subjects) QFT-GIT results were unobtainable, mainly as a result of blood samples not being processed. A further improved IGRA might therefore provide greater clinical utility. Two approaches being pursued include the addition of new antigens [36] or the evaluation of different cytokine biomarkers [37]. It is also evident from our finding of a prevalence of culture positivity of 6.4%, in the absence of symptoms in many cases, that the distinction between latent and active TB may be arbitrary. TB infection appears to exist as a spectrum and the immune responses that characterise active and latent disease therefore inevitably overlap [38, 39]. Insight into the distinction between active and latent TB, and the necessity to potentially factor multiple biomarkers to achieve consistent risk stratification comes from transcriptional profiling of the blood of TB patients [40]. 393 differentially elevated or reduced transcripts characterised active TB but ∼10% latent patients also had a transcriptomic pattern that resembled active disease.

Our finding that TST may offer some utility in pre-IPT assessment was unexpected. The TST is often falsely negative in moderate-to-severely immunocompromised HIV-1 infected persons, which often limits its utility [41]. However, evaluation hitherto has been as a stand-alone test and not as part of a multivariable clinical process. A major pitfall of adopting conventional test accuracy measures as the source of information for considering a test in clinical practice is that accuracy estimates are assumed to be constant [22, 42]. These can differ across populations, within groups of patients with different risk profiles (e.g. age, symptoms and signs, etc.) and can vary according to other test results [42]. Therefore, when those additional predictors of disease are considered in clinical diagnosis, as simulated in our multivariable models, the accuracy of the test is modified [42]. This possibly explains the improved discriminatory ability of TB symptoms when considered in multivariable models, despite poor individual sensitivity estimates. Similarly, the marginal difference in the added discriminatory ability between TST and QFT-GIT, despite comparable sensitivity estimates, is explained. As a tool for assessing the models' ability to discriminate M. tuberculosis culture-positive and -negative patients, the AUC may be insensitive to model changes [43, 44] and does not give an absolute indication of risk prediction. This work may be improved by an assessment in the independent absolute risk of culture-positive TB disease, of the model without TB tests of infection from addition of TST and QFT-GIT, by the use of new risk stratification methods [43, 45, 46]. A clinical scoring rule based on the best-extended clinical model would further provide an application of the model in practice; this would aid our appreciation of the models' absolute clinical relevance. The TST identifies HIV-infected persons who will benefit from short-term IPT [47] as well as those that may benefit from continuous therapy [48]. However, there is little enthusiasm for performing TST in programmes. If addition of TST to existing clinical screening algorithms could additionally help stratify those who should further be evaluated by culture, this might prove useful. However, the possibility that this utility may be location specific means the TST should be validated prospectively in multivariable clinical models in other similar settings.

Our study had limitations. CXR was not included in our multivariable assessment as radiography was not part of our pre-IPT screening protocol. The decision not to include radiography in our 2007 protocol was based on national IPT guidelines and supported by data from two IPT trials, one local [8] and the other a study that evaluated the value of CXR for Botswana's national IPT programme [7]. The local study found that CXR added very little to a clinical algorithm that included two of more standard TB symptoms and signs. The Botswana study similarly suggested that screening of TB symptoms alone was adequate to rule out TB and that CXR added very little (0.2% of 560 HIV asymptomatic patients screened in the Botswana programme had TB diagnosed solely on the basis of radiography). However, a number of studies have since been published on the utility of CXR in the pre-IPT evaluation of HIV-infected patients but, overall, with inconclusive results. Based on four such studies (n=2,805), a WHO-led patient meta-analysis suggested that the addition of CXR results to their best clinical rule increased sensitivity from about 79% to 90% but decreased specificity further from 50% to 40% [12]. This will unlikely be a useful rule-in strategy for a clinical setting in a high-burden country. This lack of consensus on the utility of the CXR in the pre-IPT evaluation of HIV infected patients is further reflected in the recently published WHO guidelines for intensified TB case-finding and IPT for people living with HIV in resource-constrained settings, a symptoms screen and not CXR is mandated [49].

A single sputum specimen, ultrasonically induced in some cases, was sent for microscopy and culture. It is possible that more culture-positive disease may have been diagnosed if more than one sample had been sent. All samples were processed in an accredited laboratory according to strict protocols to prevent cross-contamination. Our clinical model included absolute weight rather than body mass index. Although this potentially simplifies clinical algorithms, it may not be very useful at the level of the individual.

Our study also had strengths. First, this was a large, single-centre study undertaken in the context of high TB incidence and among HIV-infected patients, either already on ART or about to commence ART, evaluated for TB prevention. It focused on a difficult-to-diagnose group of smear-negatives. Secondly, our multivariable models were developed on a large dataset that included a small number of a priori determined predictors, and the ratio of TB cases to the number of parameters in our models was around the cited rule of thumb of 1:10, therefore limiting model over-fitting [50]. Furthermore, selection of predictors was based on clinical judgment and published literature and not just on statistical significance (p-values/Akaike information criterion). Thirdly, objective measures of incremental value and the tests' ability to discriminate TB patients from those without TB were used, thus going beyond conventional standard test accuracy.

In summary, QFT-GIT, in its current formulation, does not improve the discriminatory ability of current TB screening clinical algorithms and should not be used alone to rule out or rule in active TB prior to IPT provision. However, an improved IGRA might provide greater clinical utility. Further research is required to improve IGRAs either through the addition of new antigens or the evaluation of different cytokine biomarkers. Evaluation of new TB diagnostics for clinical relevance should follow a multivariable process that goes beyond test accuracy.

Acknowledgments

We thank patients and staff at Ubuntu Clinic in Khayelitsha, South Africa for inclusion and clinical evaluation, respectively. The ART plus IPT Study Team is thanked for its assistance and V. Azevedo (Khayelitsha Subdistrict, City of Cape Town Health Dept, Cape Town, South Africa) for valuable input.

Footnotes

  • This article has supplementary material available from www.erj.ersjournals.com

  • Support Statement

    This work was supported by the Foundation for Innovative New Diagnostics (FIND), the Wellcome Trust (grants 084323, 084670 and 088316), the Dept of Health of South Africa and the Medical Research Council of the UK and the European Union (SANTE/2005/105-061-102). M. Pai was supported by a New Investigator Award from the Canadian Institutes of Health Research. When this work was conducted, M. Pai was a consultant for FIND. M. Pai is a consultant for the Bill and Melinda Gates Foundation.

  • Statement of Interest

    None declared.

  • Received April 5, 2011.
  • Accepted June 4, 2011.
  • ©ERS 2012

REFERENCES

  1. ↵
    1. Maartens G,
    2. Wilkinson RJ
    . Tuberculosis. Lancet 2007; 370: 2030–2043.
    OpenUrlCrossRefPubMedWeb of Science
  2. ↵
    World Health Organization. WHO Three I’s Meeting. Intensified Case Finding (ICF), Isoniazid Preventive Therapy (IPT), and TB Infection Control (IC) with people living with HIV. Report of a joint World Health Organization HIV/AIDS and TB department meeting. Geneva, World health Organization, 2008
  3. ↵
    1. Balcells ME,
    2. Thomas SL,
    3. Godfrey-Faussett P,
    4. et al
    . Isoniazid preventive therapy and risk for resistant tuberculosis. Emerg Infect Dis 2006; 12: 744–751.
    OpenUrlCrossRefPubMedWeb of Science
  4. ↵
    1. Kranzer K,
    2. Houben RM,
    3. Glynn JR,
    4. et al
    . Yield of HIV-associated tuberculosis during intensified case finding in resource-limited settings: a systematic review and meta-analysis. Lancet Infect Dis 2010; 10: 93–102.
    OpenUrlCrossRefPubMedWeb of Science
  5. ↵
    1. Bassett IV,
    2. Wang B,
    3. Chetty S,
    4. et al
    . Intensive tuberculosis screening for HIV-infected patients starting antiretroviral therapy in Durban, South Africa. Clin Infect Dis 2010; 51: 823–829.
    OpenUrlAbstract/FREE Full Text
  6. ↵
    1. Meintjes G,
    2. Wilkinson RJ
    . Undiagnosed active tuberculosis in HIV-infected patients commencing antiretroviral therapy. Clin Infect Dis 2010; 51: 830–832.
    OpenUrlFREE Full Text
  7. ↵
    1. Mosimaneotsile B,
    2. Talbot EA,
    3. Moeti TL,
    4. et al
    . Value of chest radiography in a tuberculosis prevention programme for HIV-infected people, Botswana. Lancet 2003; 362: 1551–1552.
    OpenUrlCrossRefPubMedWeb of Science
  8. ↵
    1. Mohammed A,
    2. Ehrlich R,
    3. Wood R,
    4. et al
    . Screening for tuberculosis in adults with advanced HIV infection prior to preventive therapy. Int J Tuberc Lung Dis 2004; 8: 792–795.
    OpenUrlPubMedWeb of Science
    1. Day JH,
    2. Charalambous S,
    3. Fielding KL,
    4. et al
    . Screening for tuberculosis prior to isoniazid preventive therapy among HIV-infected gold miners in South Africa. Int J Tuberc Lung Dis 2006; 10: 523–529.
    OpenUrlPubMedWeb of Science
  9. ↵
    1. Cain KP,
    2. McCarthy KD,
    3. Heilig CM,
    4. et al
    . An algorithm for tuberculosis screening and diagnosis in people with HIV. N Engl J Med 2010; 362: 707–716.
    OpenUrlCrossRefPubMedWeb of Science
  10. ↵
    1. Corbett EL,
    2. Bandason T,
    3. Cheung YB,
    4. et al
    . Prevalent infectious tuberculosis in Harare, Zimbabwe: burden, risk factors and implications for control. Int J Tuberc Lung Dis 2009; 13: 1231–1237.
    OpenUrlPubMedWeb of Science
  11. ↵
    1. Getahun H,
    2. Kittikraisak W,
    3. Heilig CM,
    4. et al
    . Development of a standardized screening rule of tuberculosis in people living with HIV in resource-constrained settings: individual participant data meta-analysis of observational studies. PLoS Med 2011; 8: e1000391.
    OpenUrlCrossRefPubMed
  12. ↵
    1. Seshadri C,
    2. Uiso LO,
    3. Ostermann J,
    4. et al
    . Low sensitivity of T-cell based detection of tuberculosis among HIV co-infected Tanzanian in-patients. East Afr Med J 2008; 85: 442–449.
    OpenUrlPubMed
  13. ↵
    1. Nicol MP,
    2. Davies MA,
    3. Wood K,
    4. et al
    . Comparison of T-SPOT.TB assay and tuberculin skin test for the evaluation of young children at high risk for tuberculosis in a community setting. Paediatrics 2009; 123: 38–43.
    OpenUrlAbstract/FREE Full Text
    1. Liao CH,
    2. Lai CC,
    3. Tan CK,
    4. et al
    . False-negative results by enzyme-linked immunospot assay for interferon-γ among patients with culture-confirmed tuberculosis. J Infect 2009; 59: 421–423.
    OpenUrlCrossRefPubMedWeb of Science
    1. Bamford AR,
    2. Crook AM,
    3. Clark JE,
    4. et al
    . Comparison of interferon-γ release assays and tuberculin skin test in predicting active tuberculosis (TB) in children in the UK: a paediatric TB network study. Arch Dis Child 2010; 95: 180–186.
    OpenUrlAbstract/FREE Full Text
  14. ↵
    1. Ling DI,
    2. Pai M,
    3. Davids V,
    4. et al
    . Are interferon-γ release assays useful for diagnosing active tuberculosis in a high-burden setting? Eur Respir J 2011; 38: 649–656.
    OpenUrlAbstract/FREE Full Text
  15. ↵
    World Health Organization. Strategic and Technical Advising Group for Tuberculosis (STAG-TB). Report of the Tenth Meeting. Geneva, World Health Organization, 2010
  16. ↵
    1. Metcalfe JZ,
    2. Cattamanchi A,
    3. Vittinghoff E,
    4. et al
    . Evaluation of quantitative IFN-γ response for risk stratification of active tuberculosis suspects. Am J Respir Crit Care Med 2010; 181: 87–93.
    OpenUrlCrossRefPubMedWeb of Science
  17. ↵
    1. Moons KG,
    2. Biesheuvel CJ,
    3. Grobbee DE
    . Test research versus diagnostic research. Clin Chem 2004; 50: 473–476.
    OpenUrlFREE Full Text
  18. ↵
    1. Moons KG,
    2. Grobbee DE
    . Diagnostic studies as multivariable, prediction research. J Epidemiol Community Health 2002; 56: 337–338.
    OpenUrlFREE Full Text
  19. ↵
    1. Moons KG,
    2. van ES GA,
    3. Michel BC,
    4. et al
    . Redundancy of single diagnostic test evaluation. Epidemiology 1999; 10: 276–281.
    OpenUrlCrossRefPubMedWeb of Science
  20. ↵
    World Health Organization TB/HIV Working Group Stop TB Partnership. Priority research questions for TB/HIV in HIV-prevalent and resource-limited settings. Report No.: WHO/HTM/TB/2010. Geneva, World health Organization, 2010
  21. ↵
    1. Hanley JA,
    2. McNeil BJ
    . The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982; 143: 29–36.
    OpenUrlCrossRefPubMedWeb of Science
  22. ↵
    1. Corbett EL,
    2. Zezai A,
    3. Cheung YB,
    4. et al
    . Provider-initiated symptom screening for tuberculosis in Zimbabwe: diagnostic value and the effect of HIV status. Bull World Health Organ 2010; 88: 13–21.
    OpenUrlCrossRefPubMedWeb of Science
  23. ↵
    1. Cattamanchi A,
    2. Smith R,
    3. Steingart KR,
    4. et al
    . Interferon-γ release assays for the diagnosis of latent tuberculosis infection in HIV-infected individuals: a systematic review and meta-analysis. J Acquir Immune Defic Syndr 2011; 56: 230–238.
    OpenUrlCrossRefPubMedWeb of Science
    1. Cattamanchi A,
    2. Ssewenyana I,
    3. Davis JL,
    4. et al
    . Role of interferon-γ release assays in the diagnosis of pulmonary tuberculosis in patients with advanced HIV infection. BMC Infect Dis 2010; 10: 75.
    OpenUrlCrossRefPubMed
    1. Davies MA,
    2. Connell T,
    3. Johannisen C,
    4. et al
    . Detection of tuberculosis in HIV-infected children using an enzyme-linked immunospot assay. AIDS 2009; 23: 961–969.
    OpenUrlCrossRefPubMedWeb of Science
  24. ↵
    1. Mazurek M,
    2. Jereb J,
    3. Vernon A,
    4. et al
    . Updated guidelines for using interferon-γ release assays to detect Mycobacterium tuberculosis infection – United States, 2010. MMWR Recomm Rep 2010; 59: 1–25.
    OpenUrlPubMed
  25. ↵
    1. Greenland P,
    2. O'Malley PG
    . When is a new prediction marker useful? A consideration of lipoprotein-associated phospholipase A2 and C-reactive protein for stroke risk. Arch Intern Med 2005; 165: 2454–2456.
    OpenUrlCrossRefPubMedWeb of Science
  26. ↵
    1. Kattan MW
    . Judging new markers by their ability to improve predictive accuracy. J Natl Cancer Inst 2003; 95: 634–635.
    OpenUrlFREE Full Text
  27. ↵
    1. Lawn SD,
    2. Badri M,
    3. Wood R
    . Tuberculosis among HIV-infected patients receiving HAART: long term incidence and risk factors in a South African cohort. AIDS 2005; 19: 2109–2116.
    OpenUrlCrossRefPubMedWeb of Science
    1. Lawn SD,
    2. Badri M,
    3. Wood R
    . Risk factors for tuberculosis among HIV-infected patients receiving antiretroviral treatment. Am J Respir Crit Care Med 2005; 172: 1348.
    OpenUrlPubMed
  28. ↵
    1. Lawn SD,
    2. Myer L,
    3. Bekker LG,
    4. et al
    . Burden of tuberculosis in an antiretroviral treatment programme in sub-Saharan Africa: impact on treatment outcomes and implications for tuberculosis control. AIDS 2006; 20: 1605–1612.
    OpenUrlCrossRefPubMedWeb of Science
  29. ↵
    1. Wilkinson KA,
    2. Seldon R,
    3. Meintjes G,
    4. et al
    . Dissection of regenerating T-cell responses against tuberculosis in HIV-infected adults sensitized by Mycobacterium tuberculosis. Am J Respir Crit Care Med 2009; 180: 674–683.
    OpenUrlCrossRefPubMedWeb of Science
  30. ↵
    1. Dosanjh DP,
    2. Hinks TS,
    3. Innes JA,
    4. et al
    . Improved diagnostic evaluation of suspected tuberculosis. Ann Intern Med 2008; 148: 325–336.
    OpenUrlPubMedWeb of Science
  31. ↵
    1. Chegou NN,
    2. Black GF,
    3. Kidd M,
    4. et al
    . Host markers in QuantiFERON supernatants differentiate active TB from latent TB infection: preliminary report. BMC Pulm Med 2009; 9: 21.
    OpenUrlCrossRefPubMed
  32. ↵
    1. Barry CE 3rd.,
    2. Boshoff HI,
    3. Dartois V,
    4. et al
    . The spectrum of latent tuberculosis: rethinking the biology and intervention strategies. Nat Rev Microbiol 2009; 7: 845–855.
    OpenUrlCrossRefPubMedWeb of Science
  33. ↵
    1. Young DB,
    2. Gideon HP,
    3. Wilkinson RJ
    . Eliminating latent tuberculosis. Trends Microbiol 2009; 17: 183–188.
    OpenUrlCrossRefPubMedWeb of Science
  34. ↵
    1. Berry MP,
    2. Graham CM,
    3. McNab FW,
    4. et al
    . An interferon-inducible neutrophil-driven blood transcriptional signature in human tuberculosis. Nature 2010; 466: 973–977.
    OpenUrlCrossRefPubMedWeb of Science
  35. ↵
    1. Rangaka MX,
    2. Wilkinson KA,
    3. Seldon R,
    4. et al
    . Effect of HIV-1 infection on T-cell-based and skin test detection of tuberculosis infection. Am J Respir Crit Care Med 2007; 175: 514–520.
    OpenUrlCrossRefPubMedWeb of Science
  36. ↵
    1. Moons KG,
    2. van Es GA,
    3. Deckers JW,
    4. et al
    . Limitations of sensitivity, specificity, likelihood ratio, and Bayes' theorem in assessing diagnostic probabilities: a clinical example. Epidemiology 1997; 8: 12–17.
    OpenUrlCrossRefPubMedWeb of Science
  37. ↵
    1. Cook NR
    . Statistical evaluation of prognostic versus diagnostic models: beyond the ROC curve. Clin Chem 2008; 54: 17–23.
    OpenUrlAbstract/FREE Full Text
  38. ↵
    1. Greenland P
    . Comments on “Evaluating the added predictive ability of a new marker: From area under the ROC curve to reclassification and beyond” by M.J. Pencina, R.B. D'Agostino Sr, R.B. D'Agostino Jr, R.S. Vasan, Statistics in Medicine (DOI: 10.1002/sim.2929). Stat Med 2008; 27: 188–190.
    OpenUrlCrossRefPubMedWeb of Science
  39. ↵
    1. Steyerberg EW,
    2. Vickers AJ,
    3. Cook NR,
    4. et al
    . Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology 2010; 21: 128–138.
    OpenUrlCrossRefPubMedWeb of Science
  40. ↵
    1. Pencina MJ,
    2. D'Agostino RB Sr.,
    3. D'Agostino RB Jr.,
    4. et al
    . Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med 2008; 27: 157–172.
    OpenUrlCrossRefPubMedWeb of Science
  41. ↵
    1. Akolo C,
    2. Adetifa I,
    3. Shepperd S,
    4. et al
    . Treatment of latent tuberculosis infection in HIV infected persons. Cochrane Database Syst Rev 2010; CD000171.
  42. ↵
    1. Samandari T,
    2. Agizew TB,
    3. Nyirenda S,
    4. et al
    . 6-month versus 36-month isoniazid preventive treatment for tuberculosis in adults with HIV infection in Botswana: a randomised, double-blind, placebo-controlled trial. Lancet 2011; 377: 1588–1598.
    OpenUrlCrossRefPubMedWeb of Science
  43. ↵
    StopTB. Guidelines for intensified tuberculosis case-finding and isoniazid preventive therapy for people living with HIV in resource-constrained settings. Geneva, World Health Organization, 2011.
  44. ↵
    1. Peduzzi P,
    2. Concato J,
    3. Kemper E,
    4. et al
    . A simulation study of the number of events per variable in logistic regression analysis. J Clin Epidemiol 1996; 49: 1373–1379.
    OpenUrlCrossRefPubMedWeb of Science
View Abstract
PreviousNext
Back to top
View this article with LENS
Vol 39 Issue 1 Table of Contents
European Respiratory Journal: 39 (1)
  • Table of Contents
  • Table of Contents (PDF)
  • About the Cover
  • Index by author
Email

Thank you for your interest in spreading the word on European Respiratory Society .

NOTE: We only request your email address so that the person you are recommending the page to knows that you wanted them to see it, and that it is not junk mail. We do not capture any email address.

Enter multiple addresses on separate lines or separate them with commas.
Interferon release does not add discriminatory value to smear-negative HIV–tuberculosis algorithms
(Your Name) has sent you a message from European Respiratory Society
(Your Name) thought you would like to see the European Respiratory Society web site.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Print
Citation Tools
Interferon release does not add discriminatory value to smear-negative HIV–tuberculosis algorithms
M.X. Rangaka, H.P. Gideon, K.A. Wilkinson, M. Pai, J. Mwansa-Kambafwile, G. Maartens, J.R. Glynn, A. Boulle, K. Fielding, R. Goliath, R. Titus, S. Mathee, R.J. Wilkinson
European Respiratory Journal Jan 2012, 39 (1) 163-171; DOI: 10.1183/09031936.00058911

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero

Share
Interferon release does not add discriminatory value to smear-negative HIV–tuberculosis algorithms
M.X. Rangaka, H.P. Gideon, K.A. Wilkinson, M. Pai, J. Mwansa-Kambafwile, G. Maartens, J.R. Glynn, A. Boulle, K. Fielding, R. Goliath, R. Titus, S. Mathee, R.J. Wilkinson
European Respiratory Journal Jan 2012, 39 (1) 163-171; DOI: 10.1183/09031936.00058911
Reddit logo Technorati logo Twitter logo Connotea logo Facebook logo Mendeley logo
Full Text (PDF)

Jump To

  • Article
    • Abstract
    • MATERIAL AND METHODS
    • RESULTS
    • DISCUSSION
    • Acknowledgments
    • Footnotes
    • REFERENCES
  • Figures & Data
  • Info & Metrics
  • PDF

Subjects

  • Respiratory infections and tuberculosis
  • Tweet Widget
  • Facebook Like
  • Google Plus One

More in this TOC Section

Original Article

  • Lung volumes and survival in chronic lung allograft dysfunction
  • Social consequences of sleep disordered breathing
  • Diagnosing airflow obstruction in COPD
Show more Original Article

Tuberculosis

  • Gene expression signatures identify distinct TB endotypes
  • Biomarkers to identify M. tuberculosis infection among borderline QFT results
  • CD4 T-cell scoring discriminates TB infection from disease
Show more Tuberculosis

Related Articles

Navigate

  • Home
  • Current issue
  • Archive

About the ERJ

  • Journal information
  • Editorial board
  • Press
  • Permissions and reprints
  • Advertising

The European Respiratory Society

  • Society home
  • myERS
  • Privacy policy
  • Accessibility

ERS publications

  • European Respiratory Journal
  • ERJ Open Research
  • European Respiratory Review
  • Breathe
  • ERS books online
  • ERS Bookshop

Help

  • Feedback

For authors

  • Instructions for authors
  • Publication ethics and malpractice
  • Submit a manuscript

For readers

  • Alerts
  • Subjects
  • Podcasts
  • RSS

Subscriptions

  • Accessing the ERS publications

Contact us

European Respiratory Society
442 Glossop Road
Sheffield S10 2PX
United Kingdom
Tel: +44 114 2672860
Email: journals@ersnet.org

ISSN

Print ISSN:  0903-1936
Online ISSN: 1399-3003

Copyright © 2023 by the European Respiratory Society