A method for morphological feature extraction based on variational auto-encoder : an application to mandible shape

Shape analysis of biological data is crucial for investigating the morphological variations during development or evolution. However, conventional approaches for quantifying shapes are difficult as exemplified by the ambiguity in the landmark-based method in which anatomically prominent "landmarks" are manually annotated. In this study, a morphological regulated variational autoencoder (Morpho-VAE) is proposed that conducts image-based shape analysis using imaging processing through a deep-learning framework, thereby removing the need for defining landmarks. The proposed architecture comprises a VAE combined with a classifier module. This integration of unsupervised and supervised learning models (i.e., VAE and classifier modules) is designed to reduce dimensionality by focusing on the morphological features in which the differences between data with different labels are best distinguished. The proposed method is applied to the image dataset of the primate mandible to extract morphological features, which allow us to distinguish different families in a low dimensional latent space. Furthermore, the visualization analysis of decision-making of Morpho-VAE clarifies the area of the mandibular joint that is important for family-level classification. The generative nature of the proposed model is also demonstrated to complement a missing image segment based on the remaining structure. Therefore, the proposed method, which flexibly performs landmark-free feature extraction from complete and incomplete image data is a promising tool for analyzing morphological datasets in biology. AUTHOR SUMMARYShape is the most intuitive visual characteristic; however, shape is generally difficult to measure using a small number of variables. Specifically, for biological data, shape is sometimes highly diverse as it has been acquired through a long evolutionary process, adaptation to environmental factors, etc., which limits the straightforward approach to shape measurement. Therefore, a systematic method for quantifying such a variety of shapes using a low-dimensional quantity is needed. To this end, we propose a novel method that extracts low-dimensional features to describe shapes from image data using machine learning. The proposed method is applied to the primate mandible image data to extract morphological features that reflect the characteristics of the groups to which the organisms belong and then those features are visualized. This method also reconstructs a missing image segment from an incomplete image based on the remaining structure. To summarize, this method is applicable to the shape analysis of various organisms and is a useful tool for analyzing a wide variety of image data, even those with a missing segment.