University of Leicester
Browse

Collaboration between explainable artificial intelligence and pulmonologists improves the accuracy of pulmonary function test interpretation

Download (1.1 MB)
journal contribution
posted on 2023-12-21, 10:13 authored by N Das, S Happaerts, I Gyselinck, M Staes, E Derom, G Brusselle, F Burgos, M Contoli, AT Dinh-Xuan, FME Franssen, S Gonem, N Greening, C Haenebalcke, WDC Man, J Moisés, R Peché, V Poberezhets, JK Quint, MC Steiner, E Vanderhelst, M Abdo, M Topalovic, W Janssens
Background Few studies have investigated the collaborative potential between artificial intelligence (AI) and pulmonologists for diagnosing pulmonary disease. We hypothesised that the collaboration between a pulmonologist and AI with explanations (explainable AI (XAI)) is superior in diagnostic interpretation of pulmonary function tests (PFTs) than the pulmonologist without support. Methods The study was conducted in two phases, a monocentre study (phase 1) and a multicentre intervention study (phase 2). Each phase utilised two different sets of 24 PFT reports of patients with a clinically validated gold standard diagnosis. Each PFT was interpreted without (control) and with XAI’s suggestions (intervention). Pulmonologists provided a differential diagnosis consisting of a preferential diagnosis and optionally up to three additional diagnoses. The primary end-point compared accuracy of preferential and additional diagnoses between control and intervention. Secondary end-points were the number of diagnoses in differential diagnosis, diagnostic confidence and inter-rater agreement. We also analysed how XAI influenced pulmonologists’ decisions. Results In phase 1 (n=16 pulmonologists), mean preferential and differential diagnostic accuracy significantly increased by 10.4% and 9.4%, respectively, between control and intervention (p<0.001). Improvements were somewhat lower but highly significant (p<0.0001) in phase 2 (5.4% and 8.7%, respectively; n=62 pulmonologists). In both phases, the number of diagnoses in the differential diagnosis did not reduce, but diagnostic confidence and inter-rater agreement significantly increased during intervention. Pulmonologists updated their decisions with XAI’s feedback and consistently improved their baseline performance if AI provided correct predictions. Conclusion A collaboration between a pulmonologist and XAI is better at interpreting PFTs than individual pulmonologists reading without XAI support or XAI alone.

History

Citation

Das N, Happaerts S, Gyselinck I, et al. Collaboration between explainable artificial intelligence and pulmonologists improves the accuracy of pulmonary function test interpretation. Eur Respir J 2023; 61: 2201720

Author affiliation

Department of Respiratory Sciences, University of Leicester

Version

  • VoR (Version of Record)

Published in

European Respiratory Journal

Volume

61

Issue

5

Pagination

2201720

Publisher

European Respiratory Society (ERS)

issn

0903-1936

eissn

1399-3003

Acceptance date

2023-03-09

Copyright date

2023

Available date

2023-12-21

Spatial coverage

England

Language

eng

Usage metrics

    University of Leicester Publications

    Categories

    No categories selected

    Licence

    Exports

    RefWorks
    BibTeX
    Ref. manager
    Endnote
    DataCite
    NLM
    DC