Detection of Pneumonia in Children through Chest Radiographs using Artificial Intelligence in a Low-Resource Setting: A Pilot Study

BackgroundPneumonia is a leading cause of death among children under 5 years in low- and-middle-income-countries (LMICs), causing an estimated 700,000 deaths annually. This burden is compounded by limited diagnostic imaging expertise. Artificial intelligence (AI) has potential to improve pneumonia diagnosis from chest radiographs (CXRs) through enhanced accuracy and faster diagnostic time. However, most AI models lack validation on prospective clinical data from LMICs, limiting their real-world applicability. This study aims to develop and validate an AI model for childhood pneumonia detection using Nigerian CXR data. MethodsIn a multi-center cross-sectional study in Ibadan, Nigeria, CXRs were prospectively collected from University College Hospital (a tertiary hospital) and Rainbow-Scans (a private diagnostic center) radiology departments via cluster sampling (November 2023-August 2024). An AI model was developed on open-source paediatric CXR dataset from the USA, to classify the local prospective CXRs as either normal or pneumonia. Two blinded radiologists provided consensus classification as the reference standard. The models accuracy, precision, recall, F1-score, and area-under-the-curve (AUC) were evaluated. ResultsThe AI model was developed on 5,232 open-source paediatric CXRs, divided into training (1,349 normal, 3,883 pneumonia) and internal test (234 normal, 390 pneumonia) sets, and externally tested on 190 radiologist-labeled Nigerian CXRs (93 normal, 97 pneumonia). The model achieved 86% accuracy, 0.83 precision, 0.98 recall, 0.79 F1-score, and 0.93 AUC on the internal test, and 58% accuracy, 0.62 precision, 0.48 recall, 0.68 F1-score, and 0.65 AUC on the external test. ConclusionThis study illustrates AIs potential for childhood pneumonia diagnosis but reveals challenges when applied across diverse healthcare environments, as revealed by discrepancies between internal and external evaluations. This performance gap likely stems from differences in imaging protocols/equipment between LMICs and high-income settings. Hence, public health priority should be developing robust, locally relevant datasets in Africa to facilitate sustainable and independent AI development within African healthcare. Author SummaryPneumonia is a leading cause of death in children under five, especially in low-resource settings like Nigeria, where access to diagnostic tools and expertise is limited. Our study explores how artificial intelligence (AI) can help address this gap by detecting pneumonia from chest X-rays. We trained an AI model using a large dataset of childrens X-rays from the United States and tested it on images collected in Nigeria. While the AI model performed well on the U.S. data, its accuracy dropped significantly when tested on the Nigerian X-rays. This reveals how differences in imaging techniques and equipment between countries can affect the performance of such models. It highlights the need for AI systems to be adapted to local contexts to ensure they are reliable and effective in real-world settings. Our findings underline the importance of creating high-quality, locally relevant datasets in Africa to support the development of AI tools that address the unique challenges of the region. By investing in such efforts, we can improve access to life-saving technologies, particularly for vulnerable populations in resource-limited healthcare systems.