Abstract
Rationale Few studies have investigated the collaborative potential between artificial intelligence (AI) and pulmonologists for diagnosing pulmonary disease. We hypothesized that the collaboration between pulmonologist and AI with explanations (explainable AI, XAI) is superior in diagnostic interpretation of pulmonary function tests (PFTs) than a pulmonologist without support.
Materials and methods The study was conducted in two phases, a mono-centre (P1) and a multi-centre intervention study (P2). Each phase utilized two different sets of 24 PFT reports of patients with a clinically validated gold-standard diagnosis. Each PFT was interpreted without (control) and with XAI's suggestions (intervention). Pulmonologists provided a differential diagnosis consisting of a preferential diagnosis and optionally up to three additional diagnoses. Primary endpoint compared accuracy of preferential and additional diagnoses between control and intervention. Secondary endpoints were number of diagnoses in differential diagnosis, diagnostic confidence and inter-rater agreement. We also analysed how XAI influenced pulmonologists’ decisions.
Results In P1 (N=16 pulmonologists), mean preferential and differential diagnostic accuracy significantly increased by 10.4% and 9.4%, respectively between control and intervention (p<0.001). Improvements were somewhat lower but highly significant (p<0.0001) in P2 (5.4% and 8.7% respectively, N=62 pulmonologists). In both phases, the number of diagnoses in differential diagnosis did not reduce, but diagnostic confidence and inter-rater agreement significantly increased during intervention. Pulmonologists updated their decisions with XAI's feedback and consistently improved their baseline performance if AI provided correct predictions.
Conclusion A collaboration between pulmonologist and XAI is better at interpreting PFTs than individual pulmonologists reading without XAI support or XAI alone.
Footnotes
This manuscript has recently been accepted for publication in the European Respiratory Journal. It is published here in its accepted form prior to copyediting and typesetting by our production team. After these production processes are complete and the authors have approved the resulting proofs, the article will move to the latest issue of the ERJ online. Please open or download the PDF to view this article.
Conflict of interest: ND holds a patent on automated quality control of spirometry.
Conflict of interest: SH has nothing to disclose.
Conflict of interest: IG has nothing to disclose.
Conflict of interest: MS has nothing to disclose.
Conflict of interest: ED reports consultancy fees from Chiesi, GSK, AstraZeneca, Boehringer-Ingelheim.
Conflict of interest: GB reports payment honoraria for lectures from Astra Zeneca, Boehringer-Ingelheim, Chiesi, GlaxoSmithKline, Novartis, Sanofi.
Conflict of interest: FB reports consultancy fees from Medical Graphics Coorporation Diagnostics.
Conflict of interest: MC reports grants from University of Ferrara, Chiesi and GlaxoSmithKline, consultancy fees and honoraria from Astra Zeneca, Boehringer-Ingelheim, Chiesi, GlaxoSmithKline, Novartis as well as support for attending meetings from Chiesi, Astra Zeneca, GSK and ALK-ABELLO.
Conflict of interest: ATD has nothing to disclose.
Conflict of interest: FF has nothing to disclose.
Conflict of interest: SG has nothing to disclose.
Conflict of interest: NG has nothing to disclose.
Conflict of interest: CH has nothing to disclose.
Conflict of interest: WM is part funded by a National Institute for Health Research (NIHR) Artificial Intelligence Award, and reports grants from the NIHR and British Lung Foundation, as well as honoraria from Mundipharma, Novartis, European Conference and Incentive Services DMC. WM is the Honorary President of the Association for Respiratory Technology and Physiology (ARTP, UK).
Conflict of interest: JM has nothing to disclose.
Conflict of interest: RP has nothing to disclose.
Conflict of interest: VP has nothing to disclose.
Conflict of interest: JQ reports grants from MRC, HDR UK, GlaxoSmithKline, Astra Zeneca, Chiesi and consultancy fees from Insmed, Evidera.
Conflict of interest: MS has nothing to disclose.
Conflict of interest: EV reports grants from Chiesi and consultancy fees and honoraria from Boehringer Ingelheim, Vertex, GlaxoSmithKline.
Conflict of interest: MA has nothing to disclose.
Conflict of interest: MT is part funded by a National Institute for Health Research (NIHR) Artificial Intelligence Award, he is co-founder and shareholder of ARTIQ.
Conflict of interest: WJ reports grants from Chiesi and AstraZeneca, consultancy and lecture fees from AstraZeneca, Chiesi and GSK, he is co-founder and shareholder of ARTIQ.
This is a PDF-only article. Please click on the PDF link above to read it.
- Copyright ©The authors 2023.
This version is distributed under the terms of the Creative Commons Attribution Non-Commercial Licence 4.0. For commercial reproduction rights and permissions contact permissions{at}ersnet.org
