Abstract
Radiological evaluation of incidentally detected lung nodules on computed tomography (CT) influences management. We assessed international radiological variation in 1) pulmonary nodule characterisation; 2) hypothetical guideline-derived management; and 3) radiologists' management recommendations.
107 radiologists from 25 countries evaluated 69 CT-detected nodules, recording: 1) first-choice composition (solid, part-solid or ground-glass, with percentage confidence); 2) morphological features; 3) dimensions; 4) recommended management; and 5) decision-influencing factors. We modelled hypothetical management decisions on the 2005 and updated 2017 Fleischner Society, and both liberal and parsimonious interpretations of the British Thoracic Society 2015 guidelines.
Overall agreement for first-choice nodule composition was good (Fleiss' κ=0.65), but poorest for part-solid nodules (weighted κ 0.62, interquartile range 0.50–0.71). Morphological variables, including spiculation (κ=0.35), showed poor-to-moderate agreement (κ=0.23–0.53). Variation in diameter was greatest at key thresholds (5 mm and 6 mm). Agreement for radiologists' recommendations was poor (κ=0.30); 21% disagreed with the majority. Although agreement within the four guideline-modelled management strategies was good (κ=0.63–0.73), 5–10% of radiologists would disagree with majority decisions if they applied guidelines strictly.
Agreement was lowest for part-solid nodules, while significant measurement variation exists at important size thresholds. These variations resulted in generally good agreement for guideline-modelled management, but poor agreement for radiologists' actual recommendations.
Abstract
Radiological variations in pulmonary nodule evaluation translate into good overall agreement for guideline-modelled management, but poor agreement for radiologists' actual recommendations: 21% of radiologists can disagree with majority decisions http://ow.ly/bPOu30mcRYA
Introduction
Pulmonary nodules detected on thoracic computed tomography (CT), either incidentally [1] or at screening [2, 3], are common and the majority are benign. However, subsolid nodules (encompassing part-solid (PSN) and pure ground-glass (GGN) lesions), are associated with a higher likelihood of malignancy [4, 5]. Published guidelines aim to standardise imaging follow-up and management [6–9] with initial decisions governed by classification of nodule morphology (i.e. solid versus part-solid versus pure ground-glass) and size (either the maximum diameter for a solid nodules or the solid component of part-solid lesions) [6–10].
Decisions about the management of nodules, especially larger nodules (typically those ≥8 mm in maximum diameter) usually follow multidisciplinary team discussion; this is especially true in tertiary referral or academic centres where such expertise is available. However, in nonspecialist centres, physicians do not adhere rigorously to existing guidelines [11], and the follow-up of nodules is likely, to a greater or lesser degree, to be influenced by the morphological description and recommendations in the radiological report. Previous studies indicate that, while the overall agreement for the characterisation of pulmonary nodules is fair to good, there is not-inconsiderable variation [12–14]. However, these studies have been limited either by the small numbers of radiologists or because of cohort enrichment, in which the original morphological categorisations (as provided by participating radiologists or from pre-annotated lung cancer screening cases), have been used to define a reference standard. Moreover, earlier studies have not formally investigated variation in radiologist's recommendation for further nodule management.
In recognising the importance of international collaboration for enhancing the relevance of research [15], the aim of the present study was to quantify interobserver agreement and the magnitude of variation among a large international cohort of thoracic radiologists for 1) nodule morphology and dimension; 2) hypothetical guideline-derived management decisions; and 3) the recommended management options chosen by radiologists.
Materials and methods
Case selection
The baseline CT studies of 64 consecutive patients from routine clinical practice (female n=36; median (range) age 66.5 (26–90) years), with 70 nodules reviewed by the pulmonary nodule management service at King's College Hospital (London, UK), were retrospectively identified for the study. All patients with incidentally detected lung nodules on CT are referred to the pulmonary nodule management service for decisions regarding discharge and further follow-up (based on risk factors, size and composition). Two patients had three nodules each, another two patients had two nodules each, and the remaining 60 patients each had a single nodule under follow-up. For the purposes of this study, each nodule was considered as a separate case; therefore, radiologists were instructed that there was only one nodule per case and that there were 70 nodules in 70 cases to be evaluated. While the upper maximum diameter for a pulmonary nodule is arbitrarily set at 3 cm [16], for the purposes of simplicity, there was no upper diameter threshold in this study. For the retrospective review of clinically indicated data in this fashion, the institutional review board waived the need for ethical approval and informed consent.
Image acquisition, reconstruction and presentation
For each case, anonymised targeted contiguous CT slices, extracted from the original study and displaying the whole nodule on both lung and soft tissue reconstructions and conforming to current practice were saved as Digital Imaging and Communication in Medicine (DICOM) datasets. To avoid confusion, each nodule was marked on image sections using a red arrow (figure 1).
a) Pulmonary nodule in the right lower lobe demarcated by a red arrow on the lung reconstruction, viewed on lung windows; b) soft tissue reconstructions on mediastinal windows. The first-choice classification for this nodule was ground-glass by 33 radiologists (all with ≥60% confidence), part-solid (PSN) by 71 radiologists (all with ≥55% confidence), and solid by three radiologists (all with ≥75% confidence). As 66 radiologists chose PSN as the first-choice categorisation with ≥70% confidence, this nodule was assigned a majority classification of PSN.
In each case, the clinical details comprising age, sex, pack-year smoking status, history of lung cancer (or other malignancy), and the presence/absence of emphysema on CT were provided (online supplementary material).
Observer invitation and registration
Members of the Fleischner Society, Society of Thoracic Radiology, European Society of Thoracic Imaging, British Society of Thoracic Imaging and the Korean Society of Thoracic Radiology were invited to participate via email. A bespoke secure website was created for registration (online supplementary material) and image evaluation. Following registration, radiologists securely downloaded a zipped file containing the DICOM datasets of all 70 cases.
Nodule characterisation: categorisation and measurement of dimensions
To mirror “real life” practice, radiologists were free to review cases on any viewer or picture archiving and communications system, alter window settings and orientation, and use electronic callipers/magnification tools. For each nodule, radiologists characterised nodule composition, size and morphology (table 1). Radiologists assigned a percentage level of confidence to each category of nodule composition (GGN, PSN and solid). For a given nodule, the category with the highest confidence level was taken as a radiologist's first-choice categorisation.
Nodule characterisation and management recommendations
Nodule management recommendations
Management based on extant guidelines
To assess the impact of nodule characterisation and measurement on interobserver agreement for initial and subsequent management of nodules, assuming adherence to existing pulmonary nodule management guidelines, we modelled the management decisions for each nodule and observer based on characterisation of that nodule in four scenarios (table 2), as follows. 1) The Fleischner Society 2005 [6] and 2013 [7] recommendations for the management of pulmonary nodules (FG1); 2) the updated Flesichner Society 2017 [9] recommendations for the management of pulmonary nodules (FG2); 3) a liberal interpretation of the British Thoracic Society (BTS) 2015 guidelines for the management of pulmonary nodules [8], where any suspicious morphological features warrant further investigation (BTS1); and 4) a parsimonious interpretation of the BTS 2015 guidelines where suspicious morphological features were not considered as warranting further investigation (BTS2) (table 2).
Modelling of guideline-based management decisions
We modelled both liberal and parsimonious interpretations of the BTS 2015 guidelines because the designation of morphological features that are suspicious (lobulation, a concave notch in the nodule margin, bubble-like lucencies, air bronchograms and pleural indentation or tags) can be subjective, and as such we wished to assess the effect of regarding any such feature as suspicious (in BTS1) or disregarding any of them entirely (in BTS2).
We assigned one of three management strategies to each nodule/observer combination as follows. 1) No further action; 2) CT surveillance; or 3) further investigation (including recommendations for positron emission tomography (PET)-CT and invasive workup).
Further details of our modelling are provided in the online supplementary material.
Management recommendations by radiologists
Following nodule characterisation, radiologists were asked to make recommendations on management (table 1) as they would in usual practice. Specifically, radiologists could select no further action, CT surveillance, proceeding directly to invasive management or a hybrid strategy comprising CT surveillance followed by invasive management (as indicated by free-text responses).
Radiologists were not specifically required to follow any published guideline in their decision-making.
Statistical analysis
We measured overall unweighted interobserver agreement for nodule composition and binary morphological variables with multirater Fleiss’ κ statistics, and pairwise weighted interobserver agreement for nodule composition with Cohen's weighted κ coefficient values (κw). Weighting was assigned by converting the percentage confidence level for nodule composition into a four-point (0–3) scale as follows. 0=not confident; 1=low confidence (5–25%); 2=intermediate confidence (30–65%); and 3=high confidence (≥70%). κw is expressed as median and interquartile range (IQR) values for all 5671 unique observer pair combinations per nodule. We categorised Fleiss' κ and Cohen's κw as poor (0<κ/κw≤0.20), fair (0.20<κ/κw≤0.40), moderate (0.40<κ/κw≤0.60), good (0.60<κ/κw≤0.80) and excellent (0.80<κ/κw≤1.00).
For each nodule, we calculated multiple classification and measurement metrics, and assessed relevant relationships between these measurements. Details are provided in the online supplementary material.
We analysed interobserver agreement on management strategies as chosen by the radiologists and within each modelled guideline using multirater Fleiss' κ. For the modelled guideline evaluation, we determined the majority decision for each nodule, and calculated the number and percentage of nodules where there would be disagreement on management even when radiologists agreed on the nodule classification.
Statistical analysis was performed using MedCalc for Windows (version 14.12.0; MedCalc Software, Ostend, Belgium) and Stata (StataCorp, College Station, TX, USA). Multirater Fleiss’ κ was generated using the Real Statistics Resource Pack (version 5.1; Charles Zaiontz, available at www.real-statistics.com). A p-value of <0.05 was considered statistically significant.
Results
Baseline data
Out of 70 nodules, one was excluded from analysis, as some radiologists wished to record a maximum diameter that exceeded the available dropdown options provided on the website. 107 radiologists (91 attending/consultants; median (range) experience 9 (1–50) years) from 25 countries (UK/Europe n=75; North and South America/Canada n=24; Asia and Africa n=8) completed the study.
Agreement and diagnostic confidence for nodule characteristics
There was good overall agreement for first-choice categorisation (κ=0.652, 95% CI 0.650–0.655) (table 3). However, the weighted agreement for first-choice categorisation was best for solid nodules, followed by GGNs and worst for PSNs. Interobserver agreement for nodule composition was neither affected by the duration of thoracic observer experience nor by attendance at a lung cancer or a dedicated pulmonary nodule multidisciplinary team meeting (online supplementary tables S2 and S3). The overall median percentage of radiologists disagreeing with the predominant classification was 12.1% (IQR 2.8–30.8%), and was highest among nodules predominantly classified as PSNs, followed by GGNs and lowest for solid nodules.
Characterisation of nodule features by radiologists
For all nodules, >50% of the 107 radiologists designated their diagnostic confidence for their first-choice categorisation as high (≥70%), with a median of 94 (87.9%) out of 107 radiologists. However, the number of radiologists with this high level of confidence varied widely, ranging from 55 (51.4%) to 107 (100%) radiologists. The variation in first-choice diagnostic confidence was narrow, with a median coefficient of variation (CoV) of 9.7% (IQR 5.5–13.2%).
The multirater Fleiss’ κ for the presence of the other morphological variables was poor to moderate (table 3). In particular, the agreement on spiculation was only fair (κ=0.365). In 13 (18.8%) out of 69 nodules, all 107 radiologists agreed that spiculation was absent, but there was no nodule in which all 107 radiologists agreed that spiculation was present.
Agreement for nodule size
The median (IQR) longest nodule diameter measurement was 7.8 (5.9–13.4) mm. There were 15 (21.7%), 30 (43.5%), 18 (26.1%) and six (8.7%) nodules with median longest diameters of ≤5 mm, >5 mm to ≤10 mm, >10 mm to ≤20 mm and >20 , respectively. The variation in diameter measurement for each nodule, in these categories, is shown in figure 2.
Forest plots depicting the variation in nodule diameters for nodules with mean diameters of a) ≤5 mm, b) >5 mm to ≤10 mm, c) >10 mm to ≤20 mm and d) >20 mm. Each nodule is represented by a horizontal line, with the central square along the line representing the mean diameter, and the two ends of the line representing the lower and upper limits of the 95% confidence interval for nodule mean, respectively.
The median (IQR) CoV for longest diameter was 15.7 (12.9–18.7)%. When analysed at clinically relevant median diameter thresholds of 5 mm, 6 mm, 8 mm, 10 mm and 20 mm, there were greater magnitudes of variation for smaller (i.e. below the threshold) compared to larger (i.e. above the threshold) nodules at the 5 mm and 6 mm thresholds, but not at higher thresholds (table 4). Paradoxically, the six nodules with median diameters of >20 mm demonstrated (statistically nonsignificant) greater CoVs than the 63 nodules ≤20 mm (18.1% versus 15.6%, respectively; p=0.45).
Magnitude of variation of longest nodule diameter at various thresholds.
Across 2091 nodule–observer combinations, where radiologists classified a nodule as PSN, the median (IQR) of the diameter and proportions of the solid component were 14.0 (9.0–19.0) mm and 40.0 (20.0–60.0)%, respectively, with wide variation (median (IQR) CoVs of 35.6 (21.7–44.3)% and 42.5 (27.8–59.9)%, respectively). In the 18 nodules predominantly classified as PSN, the median (IQR) of the diameter and proportions of the solid component were 6.5 (4.0–11.0) mm and 32.5 (20.0–50.0)%, respectively.
Relationships between nodule size, diagnostic confidence and magnitude of variation are presented in table 5. In brief, there was 1) a moderate negative correlation between the median and CoV of longest diameter (rs=−0.52, p=0.005) for the 28 majority-classified solid nodules; and 2) a strong negative correlation between the median and CoV of the solid component diameter (rs=−0.69, p=0.0015) and between the median and CoV of the solid proportion (rs=−0.88, p<0.0001) for the 18 majority-classified PSNs. In other words, interobserver variation increased with decreasing size for both solid nodules and the solid component of PSNs.
Relationships between nodule size, diagnostic confidence and magnitude of variation (measured by coefficient of variation (CoV))
Nodule management recommendations
Management based on extant guidelines
The overall agreement, majority decisions and number and proportions that would have hypothetically disagreed with the majority of nodule management decisions as defined by the four modelled guidelines are shown in table 6. All showed good agreement. Applying the latest Fleischner Society recommendations uniformly to this nodule cohort would have resulted in the lowest proportion of nodules requiring further investigation. Almost one in 20 radiologists would disagree with the majority management decision according to the more conservative model of the BTS guideline (BTS2), and this rises to almost one in 10 radiologists with the latest Fleischner Society recommendations.
Variation in management decisions based on hypothetical decision-making according to the Fleischner Society 2005 and 2013 (FG1) recommendations, the Fleischner Society 2017 (FG2) recommendations, the British Thoracic Society 2015 guidelines with suspicious morphological features warranting further investigation (BTS1) and without such features warranting further investigation (BTS2).
Even when radiologists agreed with the majority classification of the nodule, there would have been disagreement on the need for further action in 12 (17.4%) out of 69, 26 (37.7%) out of 69, 31 (44.9%) out of 69 and 31 (44.9%) out of 69 nodules using FG1, FG2, BTS1 and BTS2, respectively. Of those nodules where there was disagreement, 0 (0.0%) out of 12, eight (30.8%) out of 26, 15 (48.4%) out of 31 and eight (25.8%) out of 31 would have had variation in the type of management (CT surveillance versus further investigation) using FG1, FG2, BTS1 and BTS2, respectively.
Management recommendations by radiologists
The overall agreement for the choice between the four management strategies that radiologists could select was only fair (κ=0.296, 95% CI 0.294–0.298). In all cases, further investigation was recommended by at least one radiologist. The majority chose no further action in six (8.7%) nodules, CT surveillance in 57 (82.6%) nodules and proceeding directly to invasive management in six (8.7%) nodules. A median (IQR) 85 (72–102) radiologists (79.4%, 67.1–95.3%) agreed on the majority decision, and a median (IQR) of 22 (9–35) radiologists (20.6%; 8.4–32.9%) disagreed with the majority decision. Among radiologists disagreeing with the majority decision, the overall median (IQR) percentages of these observers recommending under- or over-management, relative to the majority decision, was 50 (0–100)% for both; therefore, there was no overall trend towards under- or over-management among disagreeing radiologists. However, there were 34 (49.2%) nodules in which the number of radiologists agreeing with the majority decision was below the median of 85; in other words, the majority decision in these nodules could be considered less compelling. In this subgroup of nodules, there was a tendency for the disagreeing radiologists to over-manage (median 77.6%, IQR 0–100%) rather than under-manage (median 22.4%, IQR 0–100%) such nodules, but this difference was not statistically significant (Wilcoxon test for paired samples p=0.86).
For 16 (23.2%) out of 69 nodules, all 107 radiologists agreed on the need for further action. For these nodules, the median (and percentage of total) observers choosing CT follow-up, proceeding directly to invasive management or a hybrid strategy comprising CT and invasive management (as indicated by free-text responses) were 77 (72.0%), 31 (29.0%) and three (2.8%) radiologists, respectively. In addition, overall agreement for management, and for PET-CT in these 16 nodules, was only fair (κ=0.33 and κ=0.32, respectively).
Nodule composition was considered the dominant factor in determining management by the majority of observers in the majority of nodules (65 (94.2%) out of 69). However, for a given nodule, there was a wide variation in the proportion of radiologists citing nodule composition as the most influential factor, with a median of 64 (59.8%) out of 107 observers (range 35 (32.7%) to 92 (86.0%) radiologists). In four (5.8%) out of 69 nodules, overall diameter was considered most important.
Discussion
The present study is the largest international evaluation of observer agreement for nodule characterisation and, to the best of our knowledge, the only study to have specifically evaluated variation for self-reported management recommendations. Our results show that agreement was lowest for PSNs, but that variation is not influenced by observer experience. We have also shown that around one in five radiologists disagreed with the part-solid classification assigned by the majority of radiologists. Moreover, there was an inverse relationship between the magnitude of variation and median diameters as well as proportions of the solid component in PSNs. With respect to radiologists' management recommendations, the overall agreement was only fair, and ∼20% of radiologists disagreed with the majority decision. These findings are important not only for highlighting difficulties in nodule characterisation, but also given current issues regarding the management of adenocarcinomas with a small solid component (measuring ≤5 mm) [21, 22], which may mandate more conservative management.
The overall variation for nodule classification in our study mirrors that in the study by Ridge et al. [14] for differentiating solid from subsolid nodules, but is greater than for screen-detected nodules reviewed by van Riel et al. [13]. The differences between studies may, in part, be attributable to variations in the relative prevalence of nodule types included in the studies. However, by requiring radiologists to assign a level of diagnostic confidence to their chosen categorisation, we were able to weight categorisations to emphasise differences in diagnostic confidence and, thus, arguably obtain a more accurate representation of interobserver agreement. Furthermore, we analysed variation at the important diameter thresholds which dictate either different nodule management recommendations (i.e. in the case of 5 mm, 6 mm and 8 mm) or tumour stage (in the case of 10 mm and 20 mm), should a nodule prove to be a primary lung cancer [23]. We found that variation in maximum nodule diameter was greatest for nodules at or below 5 mm and 6 mm, the key diameter thresholds (based on current guidelines [8, 9]), governing whether discharge or further CT surveillance should be recommended. The problem of interobserver variation for diameter measurement of small pulmonary nodules using electronic callipers is not new [24, 25]. Variation in measurement at the specified diameter thresholds for follow-up potentially translates into variation in recommendations for further follow-up, even though radiologists may agree on the nodule composition. Indeed, in our study over one-third of nodules would have had a variation in recommended management strategy, irrespective of the guideline used. Such variation in measurement and recommendations may be reduced by applying volumetric analysis, computer-aided detection tools [26, 27] and, potentially, artificial intelligence (AI) techniques. However, with respect to the latter, the interobserver variation in our study sounds an equally important note of caution for AI nodule characterisation tools developed using machine-learning techniques; such tools require datasets annotated by radiologists for their development, and may thus mimic the biases and interobserver variation of their contributing radiologists.
To assess how management recommendations might vary if guidelines were adhered to, we modelled hypothetical management decisions based on four guideline scenarios. Modelling of this type inherently foregoes interobserver variation in decision-making by assuming that guidelines will be applied consistently based on nodule composition and diameter, and is thus artificial. However, our findings illustrate that even in such artificial situations, agreement is at best good, and up to almost one in 10 radiologists would deviate from the majority management decision.
An important ancillary finding of the present study was the poor-to-moderate agreement for other morphological variables. Our results, derived from a much larger observer pool, contrast with the high agreement for assessing nodule margins, air bronchograms, bubble-like lucencies and pleural tags in the study by Lim et al. [17]. In part, this may be because different morphological features are more difficult to identify in smaller lesions: in our study, around two-thirds of nodules were ≤10 mm in diameter. This aside, a salutary finding was that observer variation for the presence/absence of spiculation (Fleiss’ κ=0.35) was below clinically-acceptable limits. Plainly, there are implications for future modifications to existing guidelines, not only because the presence of spiculation is regarded as a more indicative sign of cancer, but also because of its importance as a variable in risk calculators [5, 28]. In addition, it is noteworthy that pleural indentation (Fleiss’ κ=0.53) and “bubble-like” lucencies (Fleiss’ κ=0.49), features regarded as “suspicious” in the BTS guidelines [8], were associated, at best, with fair observer agreement. Subjectivity in the interpretation of these features will further add to the variation in management strategies. As we were interested in the effect that these other morphological variables could have on the decisions taken in the BTS guideline, we modelled two versions of this guideline, one mandating further investigation when these suspicious features were considered present, and the other with a conservative approach irrespective of their presence. The conservative approach resulted in only four fewer nodules requiring further investigation according to the majority of radiologists, suggesting that the effect of these morphological features on recommended nodule management was limited, at least in our cohort.
Since modelling of management strategies based on guidelines is artificial, it was essential that we also capture the real-life management decisions preferred by radiologists for each nodule. The interobserver agreement for these management strategies in practice was only fair, and much lower than that for any of the modelled guidelines, despite all radiologists listing nodule composition or nodule size as the most influential factors in their decision-making. This lower level of agreement reinforces the role of radiologists' subjective judgement in deciding nodule management, despite existing recommendations. A recent study demonstrated that such subjective upgrading of management recommendations using the American College of Radiology Lung CT Screening Reporting and Data System (Lung-RADS) assessment 4X category resulted in higher malignancy detection rates in lung cancer screening [29], underscoring this point. In addition, it is worth noting that, although our nodule dataset mirrored real-life clinical practice, the radiologists in our study still performed their evaluations under “test” conditions, and the majority were of higher grade (consultant/attending). As such, it is conceivable that real-world management variation may be considerably higher.
Our results also have potential ramifications for respiratory physicians interpreting radiological reports and recommendations. In particular, physicians practising in nonspecialist centres may wish to obtain a tertiary centre multidisciplinary opinion on the classification and management of nodules 1) that have been classified as PSNs (given the lower agreement for this classification); 2) that are at the potential diameter thresholds for discharge (i.e. 5 mm and 6 mm); 3) in which management has been influenced by morphological variables (given the poor-to-moderate agreement for some CT features (e.g. spiculation)); and 4) where the radiological recommendation is discordant with that suggested by guidelines.
The current study has limitations: firstly, for the purposes of management strategies, the nodules were assumed to be stable. In practice, growth of nodules often leads to further investigation; however, our prime intention was to model the management of nodules as first discovered. Also, both the prior and current versions of the Fleischner recommendations advocate the use of the average of short- and long-axis diameter for measurement [6, 30], whereas we have modelled management on the longest diameter only. In support of our approach, a recent survey suggests that, at least for subsolid nodules, only 9% of radiologists manage nodules based on bidimensional measurements [31]. While no such data are available for solid nodules, we suggest that our modelling reflects everyday practice. Finally, we did not instruct radiologists as to the type of reconstruction (lung or soft tissue) or viewing window (lung or mediastinal) on which measurements should be made. Although recommendations in this regard have been made [7, 30], variation between radiologists in this measurement practice could have influenced measurement variation.
In conclusion, in a large international cohort of radiologists, we found that the overall agreement for nodule composition was good, but was lowest for PSNs, and significant magnitudes of variation in diameter measurement exist at important diameter thresholds. As such, radiologists are not consistent in characterising or measuring pulmonary nodules. These variations resulted in generally good agreement for management modelled on current recommendations, but agreement on self-reported management decisions taken by radiologists is poor.
Supplementary material
Supplementary Material
Please note: supplementary material is not edited by the Editorial Office, and is uploaded as it has been supplied by the author.
Supplementary material ERJ-01359-2018_Supplement
Footnotes
Published online Dec 20, 2018; republished Dec 06, 2019 with a correction to the name of one collaborator.
This article has supplementary material available from erj.ersjournals.com
The Lung Nodule Evaluation Group collaborators: Fredrik Ahlfors, European Telemedicine Clinic, Sweden; Andréa Bastos, Universidade Federal de Minas Gerais – Faculdade de Medicina, Brazil; Julien Behr, Hospital Jean Minjoz Besançon, France; Jiri Benes Jr, General University Hospital, Charles University, Czech Republic; Gracijela Bozovic, Radiology Department at University Hospital Lund, Sweden; Maria Teodora Buzan, Papworth Hospital NHS Foundation Trust, UK; Eva Castañer, Institut Universitari del Parc Tauli, UDIAT – Centre Diagnòstic, Spain; Marcelo Chaves, DASA, Brazil; Yo Won Choi, Hanyang University Seoul Hospital, Republic of Korea; Giorgia Dalpiaz, Bellaria Hospital, Italy; Sarojini David, University Hospital Lewisham, UK; Pim A. De Jong, UMC Utrecht, the Netherlands; Gael Dournes, Centre Hospitalier Universitaire de Bordeaux, France; Matthias Eberhard, University Hospital Zurich, Switzerland; Dante Escuissato, Federal University of Paraná, Brazil; Alessandra Farchione, A. Gemelli University Hospital, Universita Cattolica del Sacro Cuore, Italy; Lucia Flors, University of Virginia, USA; Paola Franchi, AUSL Teramo, Italy; Thomas Frauenfelder, Radiology, University Hospital Zurich, Switzerland; Kiminori Fujimoto, Department of Radiology, Kurume University School of Medicine, Japan; Marco Gatti, University of Turin, Italy; Giampaolo Gavelli, Istituto Tumori Romagna, Italy; Aleksandar Grgic, Radiological Practice, University Hospital, Germany; Louise Haine, Royal Derby Hospital, UK; Linda Haramati, Albert Einstein College of Medicine, Montefiore Medical Center, USA; Thomas Hartman, Mayo Clinic, USA; Ieneke Hartmann, Maasstad Hospital Rotterdam, the Netherlands; Jan Philipp Hering, Institute of Clinical Radiology University Muenster, Germany; Vincent Herpels, AZ Groeninge, Belgium; Inmaculada Herráez, Hospital Universitario de León, Spain; Susan Hobbs, University of Rochester Medical Center, USA; Bruno Hochhegger, Pontificial Catholic University of Rio Grande do Sul, Brazil; Nur Hursoy, Department of Radiology, Ankara University School of Medicine, Turkey; Joana Ip, Fundação Champalimaud, Portugal; Srikanth Iyengar, Frimley Health NHSFT, UK; Juan Arenas Jiménez, Hospital General Universitario de Alicante, Spain; Katharine Johnson, Salisbury NHS Trust, UK; Jeffrey Kanne, University of Wisconsin – Madison, USA; Sangita Kapur, University of Cincinnati, USA; Can Zafer Karaman, Adnan Menderes University, Turkey; Robert Karl, LSU, USA; Galina Kirova, Tokuda Hospital Sofia, Bulgaria; Jane Ko, NYU Langone Medical Center, USA; Gerdien Kramer, CHRU, France; Melahat Kul, Department of Radiology, Ankara University School of Medicine, Turkey; Anna Rita Larici, Catholic University of the Sacred Hearth of Rome, Italy; Youkyung Lee, Hanyang University Guri Hospital, Republic of Korea; Leonard Leifels, Universitat Leipzig, Germany; Sebastian Ley, University Teaching Hospital, Germany; Stephen Machnicki, Lenox Hill Hospital, USA; Panagiotis Maidas, University Hospital of Aalborg, Denmark; Sze Mun Mak, Guy's and St Thomas' Hospital, UK; Francesca Mariani, Sant'Orsola Hospital, Italy; Gustavo Meirelles, Fleury Group, Brazil; Firdaus A. Mohamed Hoesein, University Medical Center Utrecht, the Netherlands; Xavier Montet, Geneva Universtiy Hospital, Switzerland; John Murchison, Edinburgh Royal Infirmary, UK; Hrudaya Nath, UAB, USA; Tatyana Nenkina, private hospital, Bulgaria; Edward Nganga, Aga Khan University Hospital, Kenya; Thi Dan Linh Nguyen-Kim, University Hospital Zurich, Switzerland; Tilo Niemann, Kantonsspital Baden, Switzerland; Carlos Francisco Muñoz Núñez, Hospital Universitari i Politècnic La Fe, Spain; Mariaelena Occhipinti, University of Florence, Italy; Anastasia Oikonomou, Sunny Brook Health Sciences Centre, Canada; Yoshiyuki Ozawa, Nagoya City University Graduate School of Medical Sciences, Japan; Tej Pandher, Plymouth Hospital NHS Trust, UK; Anagha P. Parkar, Haraldsplass, Deaconess Hospital, Norway; Sara Piciucchi, GB Morgagni–L Pierantoni Hospital, Forlì, Italy; Diogo Pinheiro, Hospital Vita, Brazil; Antonio Poerio, University of Bologna, Italy; Roberta Polverosi, Affidea-Padova, Italy; Kristen Pope, University of Kansas Medical Centre, USA; Helmut Prosch, Medical University of Vienna, Austria; Ilaria Pulzato, University of Genoa, Italy; Cristiano Rampinelli, European Institute of Oncology, Italy; Jonathan Rodrigues, Bristol Royal Infirmary, UK; Odyssefs Romanos, private practice, Greece; Chiara Romei, Pisa University Hospital, Italy; Justus Roos, Cantonal Hospital Lucerne, Department of Radiology and Nuclear Medicine, Switzerland; Santiago Rossi, ROSSI, Argentina; Roman Rubtsov, Heidelberg University Hospital, Germany; Margherita Ruggirello, University of Parma, Italy; Marcelo Sánchez, Hospital Clinic de Barcelona, Spain; Marcel Koenigkam Santos, University of Sao Paulo – School of Medicine of Ribeirao Preto, Brazil; Michael Schubert, University Hospital, Norway; Taiwo Senbanjo, Epsom and St Helier NHS Trust, UK; Claudio Silva, Clinica Alemana de Santiago, Chile; Mario Silva, University of Parma, Italy; Annemie Snoeckx, Antwerp University Hospital, Belgium; Sushilkumar Sonavane, University of Alabama at Birmingham, USA; Mona Sriharan, King's College Hospital NHS Foundation Trust, UK; Konstantinos Stefanidis, Kings College Hospital NHS Foundation Trust, UK; Onur Taydas, Hacettepe University, Turkey; Noriyuki Tomiyama, Diagnostic and Interventional Radiology, Graduate School Of Medicine, Osaka University, Japan; Sa Tran, King's College Hospital, UK; Alberto Villanueva, Hinchingbrooke Hospital and Peterborough City Hospital, UK; Michelle Williams, University of Edinburgh, UK.
Conflict of interest: A. Nair: this work was partly undertaken at University College London Hospitals (UCLH)/University College London (UCL), by A. Nair, who received a proportion of funding from the Department of Health National Institute for Health Research (NIHR) Biomedical Research Centres funding scheme. A. Nair is a member of the advisory board for Aidence Artificial Intelligence, but has not received any fees from this entity to date, and the work with this entity is not in any way related to the current submission.
Conflict of interest: E.C. Bartlett has nothing to disclose.
Conflict of interest: S.L.F. Walsh has nothing to disclose.
Conflict of interest: A.U. Wells has nothing to disclose.
Conflict of interest: N. Navani has nothing to disclose.
Conflict of interest: G. Hardavella has nothing to disclose.
Conflict of interest: S. Bhalla has nothing to disclose.
Conflict of interest: L. Calandriello has nothing to disclose.
Conflict of interest: A. Devaraj has nothing to disclose.
Conflict of interest: J.M. Goo reports grants from Lunit Inc., outside the submitted work.
Conflict of interest: J.S. Klein has nothing to disclose.
Conflict of interest: H. MacMahon reports personal fees for advisory board work from Riverain Technologies, personal fees for consultancy from GE Healthcare, grants from Philips Healthcare, honoraria from Konica-Minolta, stock options for research collaborations from Hologic, and payments for patents and licences from University of Chicago, outside the submitted work.
Conflict of interest: C.M. Schaefer-Prokop has nothing to disclose.
Conflict of interest: J-B. Seo has nothing to disclose.
Conflict of interest: N. Sverzellati reports personal fees from Roche and Boehringer Ingelheim, outside the submitted work.
Conflict of interest: S.R. Desai has nothing to disclose.
Support statement: This work was partly undertaken at University College London Hospitals (UCLH)/University College London (UCL), by A. Nair and N. Navani, who received a proportion of funding from the Department of Health National Institute for Health Research (NIHR) Biomedical Research Centres funding scheme. Funding information for this article has been deposited with the Crossref Funder Registry.
- Received July 18, 2018.
- Accepted October 7, 2018.
- Copyright ©ERS 2018
References
