Deep Learning methods for 3D segmentation of neural tissue in EM images

Abstract

Current state-of-the-art methods in the 3D segmentation of EM stacks typically rely on a multi-stage processing of input data. Roughly, data processing consists of: acquistion, realignment, preprocessing, representation transformation, and post-processing. While most techniques strive to be fully automatic in each of these stages, errors inevitably occur. When they occur, they must be manually corrected; otherwise the errors will inevitably propogate through the rest of the pipeline to the detriment of the output segmentation. In this paper, we will explore various methods of improving accuracy at two of the stages: image transformation, and realignment. Specifically, we explore deep-learning based approaches that maximize image transformation performance on well-aligned data, and then explore various methods of learned realignment to make processing robust to misalignment. We find that while hand-crafted alignment methods currently outperform learned alignment methods, the training results suggest that further exploration of more sophisticated learned realignment schemes could potentially outperform hand-crafted methods. Additionally, we release a modular segmentation framework, DeepSeg, that allows for automatic segmentation of EM datasets and provides a flexible way to experiment with different techniques.