Replication: Deep Learning for ECG Analysis: Benchmarks and Insights from PTB-XL

Reproducibility SummaryO_ST_ABSScope of ReproducibilityC_ST_ABSThe authors of the original paper present six benchmark tasks on the previously published PTB-XL dataset, containing, 21837 12-lead ECGs from 18885 patients. They evaluate seven different neural network architectures on the six bench-mark tasks. The authors have published all code and claim full reproducibility. In addition, they published code for easy implementation of new models. To validate the claim of reproducibility we implemented a new model and tested it, and the seven models presented by the authors of the original paper, on the six benchmark tasks. MethodologyWe used the publicly available code, published by the authors of the original paper, as a starting point for our experiment. Furthermore, we modified the code slightly in order to make it compatible with a cloud-hosted Jupyter Notebook, Google Colab. We ran the experiments using Google Colab Pro, using 32 GB RAM and either 1 x NVIDIA P100 or 1 x NVIDIA T4 GPU. ResultsWe successfully managed to reproduce the original work and also verified the validity of the main claims of the original paper. In addition, we showed how robust the models were to noise and finally implemented a new model that showed comparable performance with the models proposed in the original paper. What was easyThe publicly available code published by the authors made it easy to reproduce and obtain the same results as reported in their paper. What was difficultWe faced two main issues in this work. (1) running the code in a cloudhosted jupyter notebook. This was done in order to get access to free or cheap GPUs. (2) Implement own models using the provided template. The description on how to use the base class and the configuration file could have been more detailed. Communication with original authorsCommunication with the authors of the original paper was established early in the project and helped us by clarifying some aspects of the work. In the final stage of this project the authors of the original paper were given this manuscript in order to read it and provide feedback.