To the Editors:
Population screening for tuberculosis was discontinued in most Western countries, largely due to the decreasing prevalence of the disease. The use of chest radiograph examination for population screening has, therefore, appropriately declined and is now limited to the screening of high-risk groups, such as immigrants 1, prisoners or homeless persons in certain countries 2 and for disease prevalence surveys 3. Chest radiography, however, remains a key tool for the clinical diagnosis of pulmonary tuberculosis. Previous studies suggest that chest radiograph abnormalities in tuberculosis are not specific and levels of intra- and inter-reader agreement are very variable 4–6. Modern digital tuberculosis screening has been noted to have high levels of sensitivity where chest radiograph examination is standardised, quality assured and a simple coding system is used 7–9. Levels of reader agreement are also reported to be higher among experienced readers 10.
In many low incidence countries, tuberculosis is concentrating in specific urban populations, such as homeless persons and drug users, with high prevalence rates reported 2, 11. In the UK, this may be contributing to the general rise in tuberculosis 12. Using a sample of digital chest radiographs from a hard to reach population at high risk of tuberculosis in London, we assessed some of the factors associated with greater diagnostic accuracy and determined levels of agreement between readers.
The study was carried out in London (UK), Amsterdam and Rotterdam (the Netherlands) between July and October 2008. Of 20 physicians invited, 13 participated: three radiologists, six respiratory physicians and one infectious disease physician from London and three public health tuberculosis physicians from Amsterdam and Rotterdam. 56 randomly selected chest radiographs were chosen: 18 confirmed cases of tuberculosis, 19 other abnormalities and 19 normal films from a screening programme which targeted homeless persons, problem drug users and prisoners. All cases had culture confirmed disease and eight were sputum smear positive. Study participants reviewed images using a DICOM (digital imaging and communications in medicine) viewer in a darkened room during a single viewing session, independently from each other. Participants were allowed 2 min to read each film and 1 min to complete a standard pro forma 13. Blind to the true diagnosis, readers categorised chest radiograph findings on each film as normal, abnormal or other. If abnormal, information was requested on whether the findings were suggestive of active tuberculosis or inactive tuberculosis 13. We also collected information from participants on the number of years of experience, speciality, annual number of tuberculosis cases notified and chest radioghraphs read in the previous year.
As culture confirmation was available on all chest radiographs included as having tuberculosis, we used this as the gold standard to calculate diagnostic odds ratios overall and by speciality. To assess the effect of the characteristics of readers, the logarithm of the diagnostic odds ratio for each participant was regressed against reader characteristics to determine its effect. The estimates were then anti-logged to give fold effects of the reader characteristics, including speciality type, number of years of experience, number of notifications and country of practice, on the diagnostic odds ratios.
To investigate the effect of patient characteristics on diagnostic accuracy with culture confirmation as a gold standard, we undertook sample level analysis where each film is a unit of observation. The number of readers who correctly diagnosed each film was determined and compared by the patient’s age group, sex and sputum smear positive status using the Kruskal–Wallis test.
We used the kappa statistic (two outcomes, multiple readers) with 95% confidence intervals to assess inter-reader agreement among the participants.
The 13 readers assessed a median (range) of 52 (10–450) notified cases per year and had a median speciality experience of 15 (8–29) yrs. The median (interquartile range) age of the population from whom chest radiographs were sampled was 41 (29–49) yrs and 82% (46) were male.
The mean diagnostic odds ratio for all participants for correctly identifying a patient with culture confirmed tuberculosis was 17.1 (95% CI 12.3–23.6). Based on an overall sensitivity of 77% and specificity of 95%, 497 individuals without tuberculosis will be referred compared to 61 correctly diagnosed cases of tuberculosis among 10,024 homeless persons in London. The mean diagnostic accuracy was high for physicians (16.8, 95% CI 10.7–26.2) and radiologists (17.7, 95% CI 8.0–39.3). Figure 1⇓ shows the receiver operating characteristic curve by speciality. However, the diagnostic accuracy did not differ significantly by speciality or country (p>0.05). More years in a speciality was associated with a 1.23 (95% CI 1.02–1.48) fold increase in diagnostic odds ratio per 10 yrs of identifying cases of tuberculosis (p = 0.033) (table 1⇓). Similarly, participants with greater number of notifications per year were more likely to correctly identify tuberculosis cases (OR 3.07, 95% CI 1.08–8.69 per 1000 notifications, p = 0.037).
Receiver operating characteristic curve comparing diagnostic accuracy for tuberculosis by speciality. ▪: radiologists; •: physicians.
Association of diagnostic accuracy with characteristics of readers(n = 13)
Participants were more likely to get the correct diagnosis when the individuals screened were younger (median number of correct diagnoses 13 versus 12; p = 0.024) or had sputum smear positive disease (median number of correct diagnoses 13 versus six; p = 0.028). There was no difference by sex (p = 0.677).
The overall level of agreement for diagnosing active tuberculosis was good, with a kappa value of 0.61 (95% CI 0.48–0.73) and slightly higher between physicians (0.64) compared to radiologists (0.54). In contrast, radiologists were significantly more likely to agree on a normal chest radiograph with a very good level of agreement of 0.84 (95% CI 0.68–0.95) when compared to physicians (kappa 0.46, 95% CI 0.32–0.55). The overall level of agreement was good for identifying the presence of infiltrates, consolidation or pleural effusion and moderate for the identification of cavities (table 2⇓). Radiologists were very good at identifying cavities and pleural effusions, with kappa values found to be 0.70 and 1 respectively.
Overall levels of agreement and levels of agreement for physicians and radiologists for the detection of abnormalities on chest radiograph
In contrast to previous reports, the number of chest radiographs examined per year leading to an improved diagnostic accuracy was lower than the 20,000 films previously reported 14. Physicians were particularly good at diagnosing tuberculosis, while radiologists were very good at detecting a normal chest radiograph.
Similarly high levels of inter-reader agreement have been reported from Switzerland among experienced readers 10 and from South Africa, where a standardised method for reading and reporting was used 8. In contrast, a Russian study 4 reported fair to moderate agreement between 101 clinicians using a non-diagnostic standard viewing platform, although levels of agreement were still higher among more experienced physicians.
Chest radiography will only be useful as a tool for tuberculosis screening if it has high diagnostic accuracy when applied to a high prevalence population. In contrast to older studies using conventional radiography for mass population screening 15, we found the diagnostic accuracy of digital radiography in this population to be high, which is consistent with more recent reports 7. Previous estimates, for example homeless persons (prevalence 788 per 100,000), problem drug users (354 per 100,000) and prisoners (208 per 100,000), suggest these groups may be suitable for screening 2. Using diagnostic accuracy data from this study and published prevalence of tuberculosis among homeless persons 2, we estimate the positive predictive value of chest radiograph screening to be 11% in this population.
Not surprisingly, previous studies suggest that chest radiograph screening of immigrants from high tuberculosis burden countries is not clinically or cost effective 16. The prevalence of tuberculosis in immigrants depends on their country of origin. It is also difficult to distinguish some cases of active from inactive tuberculosis based on a single radiological film. In contrast, the use of chest radiograph screening among groups with very high incidence of tuberculosis in metropolitan areas aids the identification of pulmonary disease, therefore contributing to halting transmission. A recent Dutch study using DNA fingerprinting demonstrated that mobile digital radiography contributed to a reduction in tuberculosis transmission among groups such as homeless persons and drug users 11. Furthermore, alternative interventions, such as preventive treatment of latent tuberculosis infection, can be challenging in groups such as prisoners and homeless persons 17.
Only 13 readers participated and 56 films were used, possibly limiting our ability to detect significant variation. The number of films is, however, comparable to the Russian study 4 and more readers participated than in other similar studies 8, 10. Our study did not include an intra-reader agreement component. Reports from previous studies suggest moderate 4 to high levels of intra-reader agreement 10.
An often cited concern regarding the feasibility of screening is the excess activity generated from evaluating a large number of individuals suspected of, but without, tuberculosis 4. Based on the diagnostic accuracy found in this study applied to a population of homeless persons in London, a modest number of false-positive cases will be referred and over 60 cases of pulmonary tuberculosis identified. Some of the false-positive cases may have old tuberculosis scars and will have the diagnosis excluded using bacterial culture, clinical and laboratory evaluation. Digital radiographs are easy to transmit electronically and use a lower dose of radiation.
Our findings suggest that highly trained readers may be able to improve the detection of tuberculosis in high-prevalence, hard-to-reach, population groups. Digital chest radiographic screening of these groups may, therefore, be an important component of a holistic approach to the control of tuberculosis, which includes rapid detection and confirmation of infectious cases, and supportive measures to ensure treatment completion.
Statement of interest
None declared.
Acknowledgments
We would like to acknowledge the 13 medical consultants who assessed the films. We are grateful to L. Bradshaw (TB Section, Health Protection Agency Centre for Infections, London, UK) for proofreading the manuscript. No external funding was received for this study.
- © ERS Journals Ltd
References
