Millisecond-scale behaviours of plankton quantified in situ and in vitro using the Event-based Vision Sensor (EVS)

The Event-based Vision Sensor (EVS) is a bio-inspired sensor that captures detailed motions of objects, developed with the applicability to become the eyes of machines and especially self-driving cars. Compared to conventional frame-based image sensors as employed in video cameras, EVS has an extremely fast motion capture equivalent to 10,000-fps even with standard optical settings and additionally has high dynamic ranges for brightness and also lower consumption of memory and energy. These features make the EVS an ideal method to tackle questions in biology, such as the fine-scale behavioural ecology. Here, we developed 22 characteristic features for analysing the motions of aquatic particles from the raw data of the EVS, and deployed the EVS system in both natural environments and laboratory aquariums to test its applicability to filming and analysing plankton behaviour. Our EVS monitoring in turbid water at the bottom of Lake Biwa, Japan identified several particles exhibiting distinct cumulative trajectory with periodicities in their motion (up to 16 Hz), suggesting that they were living organisms with rhythmic behaviour. We also carried out EVS monitoring in the deep sea aided by infrared lighting to minimise influence on behaviour, and observed particles with active motion and periodicities over 40 Hz. Furthermore, we used the EVS to observe laboratory cultures of six species of zooplankton and phytoplankton, confirming that they have species-specific motion periodicities of up to 41 Hz. We applied machine learning to automatically classify particles into five categories (four categories of zooplankton plus passive particles), which achieved an accuracy up to 86%. Our attempts to use the EVS for biological observations, especially focusing on its millisecond-scale temporal resolution and wide dynamic range provide a new avenue to investigate rapid and periodical motion and behaviour in small organisms. Given its compact size with low consumption of battery and memory, the EVS will likely be applicable in the near future for the automated monitoring of the behaviour of plankton by edge computing on autonomous floats, as well as quantifying rapid cellular-level activities under microscopy.