Neural Decoding of Inferior Colliculus Multiunit Activity for Sound Category identification with temporal correlation and deep learning

Natural sounds are easily perceived and identified by humans and animals. Despite this, the neural transformations that enable sound perception remain largely unknown. Neuroscientists are drawing important conclusions about neural decoding that may eventually aid research into the design of brain-machine interfaces (BCIs). It is thought that the time-frequency correlation characteristics of sounds may be reflected in auditory assembly responses in the midbrain and that this may play an important role in identification of natural sounds. In our study, natural sounds will be predicted from multi-unit activity (MUA) signals collected in the inferior colliculus. The temporal correlation values of the MUA signals are converted into images. We used two different segment sizes and thus generated four subsets for the classification. Using pre-trained convolutional neural networks (CNNs), features of the images were extracted and the type of sound heard was classified. For this, we applied transfer learning from Alexnet, GoogleNet and Squeezenet CNNs. The classifiers support vector machines (SVM), k-nearest neighbour (KNN), Naive Bayes and Ensemble were used. The accuracy, sensitivity, specificity, precision and F1 score were measured as evaluation parameters. Considering the trials one by one in each, we obtained an accuracy of 85.69% with temporal correlation images over 1000 ms windows. Using all trials and removing noise, the accuracy increased to 100%.