Optimizing automated classification for zooplankton in coastal conditions: the impact of model selection, imaging instruments, and colour information

The advancement in deep learning techniques has made Convolutional Neural Networks (CNNs) a powerful tool for the fully automated classification of zooplankton images. In this study, we systematically investigate how network selection, colour information and differences in imaging instruments affect the classification of zooplankton images by comparing multiple state-of-the-art CNNs on images of zooplankton and marine snow from the in situ Continuous Particle Imaging and Classification Sensor (CPICS), Video Plankton Recorder (VPR), In Situ Ichtyoplankton Imaging System (ISIIS), and the on-board Plankton Imager (Pi-10). With differences between models of 7.8 to 19% in F1-score, we find that model selection strongly affects the classification performance, with EfficientNetV2S showing the most reliable overall performance. Moreover, differences between model architectures are largest for the least abundant classes (<100 labeled images), which implies that when these are present, careful model selection is most beneficial. The high image quality of the Pi-10 strongly increases the performance for the least abundant classes compared to the other instruments. In addition, we find a significant correlation (r = 0.597) between ImageNet the performance and F1-score on zooplankton images, which implies that more generally, a model that performs well on ImageNet will perform well for zooplankton classification. Colour information increases the F1-score of the best performing classifier with 2.8%, but provides a stronger benefit (25% F1-score) for classes with <100 images. The overall performance increase of colour information is less than expected and questions the advantage of recording colour information for zooplankton.