An Effective Automated Algorithm to Isolate Patient Speech from Conversations with Clinicians

A growing number of algorithms are being developed to automatically identify disorders or disease biomarkers from digitally recorded audio of patient speech. An important step in these analyses is to identify and isolate the patients speech from that of other speakers or noise that are captured in a recording. However, current algorithms, such as diarization, only label the identified speech segments in terms of non-specific speakers, and do not identify the specific speaker of each segment, e.g., clinician and patient. In this paper, we present a novel algorithm that not only performs diarization on clinical audio, but also identifies the patient among the speakers in the recording and returns an audio file containing only the patients speech. Our algorithm first uses pretrained diarization algorithms to separate the input audio into different tracks according to nonspecific speaker labels. Next, in a novel step not conducted in other diarization tools, the algorithm uses the average loudness (quantified as power) of each audio track to identify the patient, and return the audio track containing only their speech. Using a practical expert-based evaluation methodology and a large dataset of clinical audio recordings, we found that the best implementation of our algorithm achieved near-perfect accuracy on two validation sets. Thus, our algorithm can be used for effectively identifying and isolating patient speech, which can be used in downstream expert and/or data-driven analyses.