Image-based patch selection for deep learning to improve automated Gleason grading in histopathological slides

Automated Gleason grading can be a valuable tool for physicians when assessing risk and planning treatment for prostate cancer patients. Semantic segmentation provides pixel-wise Gleason predictions across an entire slide, which can be more informative than classification of pre-selected homogeneous regions. Deep learning methods can automatically learn visual semantics to accomplish this task, but training models on whole slides is impractical due to large image sizes and scarcity of fully annotated data. Patch-based methods can alleviate these problems, and have been shown to produce significant results in histopathology segmentation. However, the irregular contours of biopsies on slides makes performance highly dependent on patch selection. In the traditional grid-based strategy, many patches lie on biopsy boundaries, reducing segmentation accuracy due to a reduction in contextual information. In this paper, we propose an automatic patch selection process based on image features. This algorithm segments the biopsy and aligns patches based on the tissue contour to maximize the amount of contextual information in each patch. This method was used to generate patches for a fully convolutional network to segment high grade, low grade, and benign tissue from a set of 59 histopathological slides, and results were compared against manual physician labels. We show that using our image-based patch selection algorithm results in a significant improvement in segmentation accuracy over the traditional grid-based approach. Our results suggest that informed patch selection can be a valuable addition to an automated histopathological analysis pipeline.