Deep Learning Approach for Cell Nuclear Pore Detection and Quantification over High Resolution 3D Data

Abstract

The intricate task of segmenting and quantifying cell nuclear pores in high-resolution 3D microscopy data is critical for cellular biology and disease research. This thesis introduces a deep learning pipeline crafted to automate the segmentation and quantification of nuclear pores from high-resolution 3D cell organelle images. Our aim is to refine computational methods capable of handling the data's complexity and size, thus improving accuracy and reducing manual labor in biological image analysis. The developed pipeline incorporates data preprocessing, augmentation strategies, random block sampling, and a three-stage post-processing algorithm. It utilizes a 3D U-Net with a VGG-16 backbone, optimized through cyclical data augmentation and random block sampling to tackle the challenges posed by limited labeled data and the processing of large-scale 3D images. The pipeline has demonstrated its capability to effectively learn and predict nuclear pore structures, achieving improvements in validation metrics compared to baseline models. Our experiments suggest that cyclical augmentation helps prevent overfitting, and random block sampling contributes to managing data imbalance. The post-processing phase successfully automates the quantification of nuclear pores without the need for manual intervention. The proposed pipeline offers an efficient and scalable approach to segmenting and quantifying nuclear pores in 3D microscopy images. Despite the ongoing challenges of computational intensity and data volume, the techniques developed in this study provide insights into the automation of complex biological image analysis tasks, with potential applications extending beyond the detection of nuclear pores.