Abstract
The clinical usefulness of ex vivo interferon-γ assays may largely depend on the assay format and epidemiological status of tuberculosis (TB) in the region studied.
From July 2004 to June 2005 a prospective comparison study was undertaken at a tertiary referral hospital in South Korea. The results of tuberculin skin tests (TST) and the commercially available QuantiFERON-TB Gold (QFT-G) and T SPOT-TB (SPOT) assays were compared in an intermediate TB-burden country.
Of the 224 participants studied, results from all three tests (TST, QFT-G, and SPOT) were available in 218; 87 with active TB and 131 at a low risk for TB. Using 10 mm as a cut-off for TST, SPOT sensitivity (96.6%) was significantly higher than that seen for TST (66.7%) and QFT-G (70.1%). QFT-G showed superior specificity over TST (91.6 versus 78.6%). Although the specificity of QFT-G was higher than that of SPOT (91.6 versus 84.7%), the difference was not statistically significant.
Whilst some differences were found in the performance of the two commercialised interferon-γ assays, they seemed to be superior in their detection of Mycobacterium tuberculosis infection compared with tuberculin skin tests. The most appropriate choice of interferon-γ assay to use may depend on the clinical setting.
One-third of the world's population is infected with Mycobacterium tuberculosis (MTB) 1, and tuberculosis (TB) remains a major worldwide cause of morbidity and mortality. Identifying patients with active disease and disease contacts at high risk for developing TB is particularly relevant in TB control programmes where both active disease and latent TB infection (LTBI) are prevalent. In South Korea, the incidence of TB is intermediate (92 per 100,000) and bacille Calmette–Guérin (BCG) vaccination is mandatory. In 2004, 65.4 per 100,000 of the general population were reported to have developed some form of TB, and 23.9 per 100,000 of the general population developed smear-positive pulmonary TB 2. Moreover, as the prevalence of active TB decreases and the number of TB-vulnerable subjects (e.g. elderly and immunocompromised patients) increases, physicians have become increasingly interested in chemoprophylaxis for LTBI. The first South Korean guideline for LTBI was recently introduced 3. The target for chemoprophylaxis is complicated by the fact that the tuberculin skin test (TST) does not reliably diagnose LTBI in the BCG-vaccinated population. TST has other drawbacks, including a lack of specificity due to the purified protein derivatives (PPD) used for TST (which share some antigens with BCG and several non tuberculous mycobacteria), and low sensitivity due to false negatives among subjects who most urgently need prophylaxis for LTBI if infected with MTB, such as immunocompromised individuals 4, 5.
The two commercialised ex vivo interferon (IFN)-γ assays, QuantiFERON-TB Gold (QFT-G; Cellestis Limited, Carnegie, Victoria, Australia) and T SPOT-TB (SPOT; Oxford Immunotec, Oxford, UK), use early secretory antigenic target 6 (ESAT-6) and culture filtrate protein 10 (CFP-10) as MTB-specific stimulants on ELISA and enzyme-linked immunospot assay (ELISPOT), respectively. Recently, both were approved as diagnostic tools for MTB infection 6. Although several studies have demonstrated their high sensitivity and specificity for MTB-infected and non exposed healthy subjects 7–9, these results may depend on the prevalence of MTB infection, immune status, ethnicity, and so on in the population being tested 10–12. Assay-related factors might also affect IFN-γ assay performance and interpretation. To date, no previous study has stringently compared the two commercially available ex vivo IFN-γ assays with TST in terms of their use in the diagnosis of MTB infection in a defined population. To identify the impact of these factors, the present study directly compared the two commercial IFN-γ assays and TST in their diagnosis of MTB infection in a defined epidemiological setting.
METHODS
Subjects
From July 2004 to June 2005, a prospective clinical study was undertaken at the Asan Medical Center (College of Medicine University of Ulsan, Seoul, South Korea), a tertiary referral medical centre at which ∼1,000 TB patients are treated annually. Due to the lack of a confirmatory test to diagnose MTB infection, it was assumed that MTB infection was present in active TB subjects and absent in subjects at low risk for TB 9. Patients with a high clinical suspicion of active TB or for whom relevant material (sputum or biopsy specimen) was available for microbiological or pathological diagnosis were screened. Healthy, BCG-vaccinated, first or second year high school students (15–16 yrs old) with normal chest radiographs, no known history of contact with TB patients and no symptoms of active TB were included as a group defined as at low risk for TB.
In subgroup analysis of active TB, “culture-confirmed TB” subjects were defined as culture positive for MTB; “clinical TB” included patients who were diagnosed histologically or who had symptoms and signs appropriate for diagnosing active TB and who responded clinically and radiologically to a full course of anti-TB treatment. “Immunocompromised” subjects were defined as having an immunosuppressive condition and included those on immunosuppressive drugs (post-transplantation, anti-cancer chemotherapy, receiving >15 mg·day-1 prednisone for >1 month), undergoing haemodialysis, who had a haematological malignancy (leukaemia, lymphoma) or diabetes mellitus 4, 9, 13.
All participating subjects were screened using chest radiography and clinical examination, and were questioned regarding their history of exposure to TB patients and BCG vaccination. The results of serological tests for HIV were evaluated. All subjects provided written informed consent and the Institutional Review Board of the Asan Medical Center approved the study.
Tuberculin skin test
TST was performed using 2TU PPD RT23 (Statens Serum Institut, Copenhagen, Denmark) and the Mantoux method 4. Induration size was measured using the ballpoint method after 48–72 h 14, 15. The TST results were interpreted according to the different induration sizes, 5 mm, 10 mm and 15 mm. An induration size of 10 mm was adopted as the cut-off value in this study. The reasons for this are as follows. 1) South Korea is still a TB-prevalent country and American Thoracic Society guidelines suggest 10 mm as a cut-off value in subjects from high-prevalence countries 4. 2) Other reports published in Asian countries also used 10 mm as a cut-off value for detecting MTB infection 9. 3) South Korean guidelines for LTBI also recommended 10 mm as a cut-off value 3.
QuantiFERON-TB Gold and T SPOT-TB
The two IFN-γ assays were performed according to the manufacturers' instructions 7, 9, 16, 17. Immediately before the TST, peripheral venous blood was collected and processed within 4 h. The samples were incubated for exactly 20 h.
For QFT-G, the IFN-γ value of the negative control for each subject was subtracted from the values obtained from the antigen-stimulated samples. The result was considered: “positive” if the response to the specific antigens was ≥0.35 IU·mL-1, regardless of the value of the positive control; “negative” if the response to the specific antigens was <0.35 IU·mL-1 and the IFN-γ level of the positive control was ≥0.5 IU·mL-1; and “indeterminate” if both antigen-stimulated samples were <0.35 IU·mL-1 and the value of the positive control was <0.5 IU·mL-1 9.
For SPOT, the spot count was performed using an automated ELISPOT Reader (AID systems, Strasberg, Germany). ESAT-6- or CFP-10-stimulated wells with at least five spots more than the negative control (where the negative control had fewer than or equal to five spots) were considered positive. In cases where the negative control well had six to 10 spots, the result was defined as positive when either the ESAT-6- or CFP-10-stimulated well contained at least twice as many spots as the negative control well. If the negative control well had >10 spots, the result was considered indeterminate 7, 16, 17.
New cut-off value determination for the study population
The optimal cut-off levels in QFT-G and SPOT were roughly calculated from a receiver operator characteristic (ROC) curve analysis using data from 87 subjects with active TB and 131 low-risk participants. For SPOT, the diagnostic criteria were simplified: the difference in spot count between antigen-stimulated samples and the negative control was used as a cut-off value, irrespective of the spot count of the negative control wells. Using these new cut-off values, the sensitivities and specificities of the two IFN-γ assays were recalculated.
Statistical methods
Differences in test performances were analysed using the Chi-squared test. Concordances between the tests were assessed using the kappa coefficient. Throughout the text, confidence intervals (CI) are given as 95%.
RESULTS
Baseline characteristics
The 224 screened subjects included 93 active TB patients and 131 subjects with low risk for TB. Six of the active TB subjects were excluded due to failure to measure TST induration size, leaving 218 subjects in the analysis. The mean age of the enrolled subjects was 28.6±19.0 yrs (range 15–83 yrs) and the male to female ratio was 6.5:1. One hundred and seventy-six (80.7%) subjects had received BCG vaccination (155 subjects had BCG scars) and 42 (19.3%) had either not received BCG vaccination or did not know their BCG vaccination status. Serological HIV tests were performed in 31 active TB subjects, all with negative results. No other subjects appeared to have clinical indications of HIV infection. The demographical characteristics of all subjects are presented in table 1⇓. The active TB group consisted of 87 subjects, including 55 (63.2%) with culture-confirmed TB and 32 (36.8%) with clinical TB. Of these 87 patients, 81 (93.1%) had pulmonary TB, three (3.4%) had TB pleurisy and another three (3.4%) had cervical TB lymphadenopathy. Twenty-nine (33.3%) active TB subjects had immunocompromised conditions, specifically, diabetes mellitus (37.9%), immunosuppressive drug use (48.3%; post-transplantation in three subjects, anti-cancer chemotherapy in nine, and prednisone use >15 mg·day-1 for >1 month in two), haematological malignancy (10.3%) and chronic renal failure on haemodialysis (3.4%). The 131 low-risk subjects were younger than those in the active TB group and all were male because they were selected from students (15–16 yrs of age) at a males' high school. None of the low-risk subjects were reported to have any systemic diseases affecting host immunity. The BCG vaccination rate of the low-risk subjects (100.0%) was higher than that of the active TB subjects (51.7%; p<0.001).
Baseline characteristics of subjects
Results of the PPD skin test and IFN-γ assays
The TST results based on three different cut-off values are presented in table 2⇓. Among the 218 subjects, eight (3.7%), all from the active TB group, showed indeterminate QFT-G results. By contrast, there were no indeterminate results in the SPOT analysis.
Results of the tuberculin skin test(TST) according to defined cut-offs
When 10 mm was used as the cut-off in TST, SPOT was more sensitive (96.6%) than TST (66.7%) or QFT-G (70.1%; p<0.05 for each; table 3⇓) and QFT-G showed superior specificity over TST (91.6 versus 78.6%; p = 0.005). Although the specificity of QFT-G was higher than that of SPOT (91.6 versus 84.7%), the difference was not statistically significant (p = 0.13; table 3⇓). There were no significant differences in the sensitivity of the three tests between culture-confirmed and clinical TB subjects and between BCG-vaccinated and unvaccinated subjects (data not shown). In immunocompromised subjects, the sensitivity of SPOT (100.0%) was significantly higher than that of TST (34.5%) or QFT-G (62.1%), and the sensitivity of QFT-G showed a tendency to be higher compared with TST (p = 0.065).
Comparison of the two interferon-γ assays and tuberculin skin test (TST) results in each group
Even though 5 mm was used as a cut-off in TST, SPOT was still more sensitive (96.6%) than TST (73.6%) and the sensitivities of TST and QFT-G (70.1%) were comparable to each other. When 15 mm was used as a cut-off in TST, the specificities of TST (95.4%) and QFT-G (91.6%) were comparable to each other even though the specificity of TST was higher than that of SPOT (84.7%). However, when 15 mm was used as a cut-off in TST, TST showed too low a level of sensitivity (43.7%).
Discordance among the test results
After excluding the eight subjects with indeterminate QFT-G results, there was good overall agreement among the test results. The kappa values between TST and QFT-G, TST and SPOT, and QFT-G and SPOT were 73.3% (kappa = 0.431), 75.2% (kappa = 0.496), and 79.0% (kappa = 0.569), respectively.
Among the 79 active TB subjects, 17 (21.5%) had discordant results between QFT-G and SPOT. Most (94.2%) were QFT-G-/SPOT+ and only one (5.8%; culture-confirmed TB) was QFT-G+/SPOT-, suggesting that SPOT was more sensitive than QFT-G for diagnosing MTB infection (fig. 1⇓). Among the 60 QFT-G+/SPOT+ subjects, 14 (23.3%) were TST-negative, and many of them (64.3%) were immunocompromised subjects, suggesting that IFN-γ assays are more sensitive than TST in diagnosing MTB infection, especially in immunocompromised subjects.
Venn diagram representing the number of positive results from tuberculin skin tests (TST), QuantiFERON-TB Gold tests (QFT-G) and T-SPOT-TB tests (SPOT) among the active tuberculosis subjects. TST, QFT-G, and SPOT were all negative in one subject.
In 131 low-risk subjects, two (1.5%) were TST+/QFT-G+/SPOT+, suggesting that they might in fact be infected with MTB or another ESAT-6- or CFP-10-containing non tuberculous mycobacteria (fig. 2⇓). Among the 102 subjects with QFT-G-/SPOT- results, 17 showed TST+ results, suggesting that TST+ results might be induced by previous BCG vaccination. Of the 131 subjects, 27 (20.6%) had discordant results between QFT-G and SPOT: nine were QFT-G+/SPOT- and 18 were QFT-G-/SPOT+. Eight of the nine QFT-G+/SPOT- subjects were also TST-, suggesting that the QFT-G results might be false positives. The 18 QFT-G-/SPOT+ subjects consisted of eight (44.4%) TST+ and 10 (55.6%) TST-. The reason for this discrepancy is unclear; possible explanations would be that the SPOT results were false positives or that the subjects were truly infected with MTB or another ESAT-6- or CFP-10-containing non tuberculous mycobacteria.
Venn diagram representing the number of positive results from tuberculin skin tests (TST), QuantiFERON-TB Gold tests (QFT-G) and T-SPOT-TB tests (SPOT) among the subjects at low risk for tuberculosis. TST, QFT-G, and SPOT were all negative in 85 subjects.
Best cut-off value determination
The above analyses were performed using the manufacturer-recommended cut-off values (fig. 3⇓). To examine whether the application of population-specific cut-off values could improve these analyses, ROC curve analysis was used to determine the best cut-off values that yielded both higher sensitivity and specificity (fig. 4⇓). For QFT-G, a cut-off value of >0.13 IU·mL-1 of IFN-γ level improved sensitivity (86.2%; 95% CI 77.1–92.7), with a minor decrease in specificity (87.0%; 95% CI 80.0–92.2). For SPOT, a cut-off value of more than seven spot counts improved specificity (91.6%; 95% CI 85.5–95.7), with a minor decrease in sensitivity (92.0%; 95% CI 84.1–96.7). When the cut-off levels calculated from a ROC analysis were adopted in this study population, the sensitivities of SPOT (92.0%) and QFT-G (86.2%) were comparable to each other and were higher than that of TST (66.7%). The specificities of SPOT (91.6%) and QFT-G (87.0%) were comparable to each other and the specificity of SPOT was higher than that of TST (78.6%; p = 0.005).
Dot plots of QuantiFERON-TB Gold and T SPOT-TB analysis. a) Interferon (IFN)-γ level and b) number of IFN-γ-producing cells responding to early secretory antigenic target 6 or culture filtrate protein 10 in active tuberculosis (TB) subjects (n = 87) and subjects at low risk for TB (n = 131). ··········: cut-off value recommended by the manufacturer. Each dot represents an individual subject.
Receiver operator characteristic curves. a) Interferon (IFN)-γ level (IU·mL-1) and b) number of IFN-γ-producing cells responding to early secretory antigenic target 6 or culture filtrate protein 10 in active tuberculosis (TB) subjects (n = 87) and subjects at low risk for TB (n = 131). a) Sensitivity 86.2%, specificity 87.0% and criteria IFN-γ >0.13 IU·mL-1; b) sensitivity 92.0%, specificity 91.6% and criteria spot-forming cells >7.
DISCUSSION
The present study reports the results of the first ever comparison of two commercial ex vivo ELISA- and ELISPOT-based IFN-γ assays with TST for the diagnosis of MTB infection in an intermediate-burden country. Although previous studies have compared ELISA- and ELISPOT-based IFN-γ assays 18, 19, no other work has stringently compared the two commercially available assays considered here. In the present study, when the manufacturer-recommended cut-offs were adopted, the sensitivity of SPOT was significantly higher than those of TST or QFT-G, especially in immunocompromised subjects, and the specificity of QFT-G was significantly higher than that of TST.
The two commercialised IFN-γ assays are similar in terms of the antigens used (ESAT-6 and CFP-10) and the incubation time (overnight or 16–24 h). The main differences between the two assays lie in the technique of IFN-γ detection (ELISA versus ELISPOT) and the specimens used (whole blood versus mononuclear cells). Of the two assays, QFT-G seems to be more convenient than SPOT, as it tests whole blood instead of mononuclear cells and because ELISA is more commonly used and simpler to perform than ELISPOT. In addition, the ELISPOT assay requires an expensive ELISPOT reader for accuracy. However, the ELISPOT assay is sensitive enough to detect single IFN-γ-producing cells.
The main theoretical advantage of IFN-γ assays using MTB-specific antigens is a high specificity in the diagnosis of MTB infection 9, 20–22. However, there is controversy regarding the degree to which the sensitivity of IFN-γ assays exceeds that of TST. In this study, when the manufacturer-recommended cut-off values were adopted, QFT-G was superior in terms of specificity (91.6%) and SPOT was superior in terms of sensitivity (96.6%). However, when new, population-specific cut-off values (>0.13 IU·mL-1 in QFT-G and more than seven spot counts in SPOT) were adopted, the sensitivity of QFT-G rose to 86.2% with minimal loss of specificity (91.6 to 87.0%), and the specificity of SPOT rose to 91.6% with minimal loss of sensitivity (96.6 to 92.0%). As a result, the QFT-G and SPOT assays showed similar sensitivities and specificities. In addition, the overall agreement between QFT-G and SPOT improved from 79.0% (kappa = 0.569) to 83.8% (kappa = 0.665). Recently, Harada et al. 23 also suggested that the cut-off value of QFT-G should be reset at a lower level when the test is applied to a “high prevalence” situation. These findings suggest that ex vivo IFN-γ assays are superior in the diagnosis of MTB infection compared with TST. However, the performance of IFN-γ assays may depend on epidemiological status, ethnicity, underlying diseases, and so on. Such new cut-off values should therefore be validated prospectively before being used in clinical practice. Further studies are needed to compare the usefulness of the two commercialised IFN-γ assays in clinical practice based on their applicability (equipment, labour time, etc.) as well as on the performance of the tests.
Comparison of the data obtained from the IFN-γ assays and TST is inherently limited by the lack of a gold standard method for diagnosing MTB infection. Even though the cause of discrepant QFT-G-/SPOT+ results in active TB subjects seems to be due to the different sensitivities of the two tests, it is not possible to clearly explain the reason for the discrepant results between both IFN-γ assays in the low-risk subjects. These issues can only be resolved by a long-term follow-up study.
One disadvantage of IFN-γ assays is that they can yield indeterminate results. Ferrara et al. 24 found that indeterminate results were common (21%) when using QFT-G assay. However, the majority of them were TST- or immunosuppressed 24. In the present study, only eight (3.7%) out of 218 subjects had indeterminate QFT-G results, and five of them were immunocompromised. In contrast to QFT-G, no indeterminate results were obtained from the SPOT assay in this study.
IFN-γ assays have the added disadvantages of higher costs and laboratory requirements compared with TST. These higher costs should, however, be offset by savings in the long term derived from avoiding unnecessary chemoprophylaxis and reducing the number of cases of active TB. This issue is not addressed in the current study. Further studies focusing on the costs and applicability of the commercialised IFN-γ assays are required.
The present study has several limitations. First, some of the low-risk subjects may actually have been infected with MTB and this might lead to an underestimation of the specificity of the IFN-γ assays. According to a South Korean national survey conducted in 1995, the MTB infection rate was ∼15% in 15 yr olds but rose dramatically to about 52–60% in 18–33 yr olds 25. Accordingly, 15–16-yr-old students were selected as a low-risk group because they have a lower chance of being infected with MTB. As a result, only two out of 131 subjects showed positive results for all three tests.
A second possible limitation of the present study was that while a positive culture of MTB is standard for the diagnosis of active TB, some cases of active TB were diagnosed based on clinical grounds rather than bacteriological confirmation. This might lead to an underestimation of the sensitivities of the IFN-γ assays. However, even when the analysis was restricted to culture-confirmed TB patients, there were no significant differences in the comparison results among the three tests.
In conclusion, whilst there were some differences in the performance of the two commercialised interferon-γ assays (high sensitivity of T-SPOT-TB and probable high specificity of QuantiFERON-TB Gold), they seemed to be superior in detecting Mycobacterium tuberculosis infection compared with tuberculin skin test. The most appropriate test to use may depend on the clinical setting. As with the tuberculin skin test, the results of each test should be interpreted with caution, taking into consideration the epidemiological status of tuberculosis in the region studied and individual immunological conditions, which might decrease the responsiveness of each assay.
Acknowledgments
The authors would like to thank Woongbe Meditech (Seoul, South Korea) for their QuantiFERON-TB Gold assay kits and ResearchNET Inc. (Seoul, South Korea) for their donations and technical support. Finally, the authors also extend their thanks to the volunteers who agreed to participate in the study.
- Received February 4, 2006.
- Accepted April 4, 2006.
- © ERS Journals Ltd
References
