Medical image segmentation is an important step in medical image analysis.With the rapid development of convolutional neural network in image processing,deep learning has been used for medical image segmentation, such as optic discsegmentation, blood vessel detection, lung segmentation, cell segmentation,etc. Previously, U-net based approaches have been proposed. However, theconsecutive pooling and strided convolutional operations lead to the loss ofsome spatial information. In this paper, we propose a context encoder network(referred to as CE-Net) to capture more high-level information and preservespatial information for 2D medical image segmentation. CE-Net mainly containsthree major components: a feature encoder module, a context extractor and afeature decoder module. We use pretrained ResNet block as the fixed featureextractor. The context extractor module is formed by a newly proposed denseatrous convolution (DAC) block and residual multi-kernel pooling (RMP) block.We applied the proposed CE-Net to different 2D medical image segmentationtasks. Comprehensive results show that the proposed method outperforms theoriginal U-Net method and other state-of-the-art methods for optic discsegmentation, vessel detection, lung segmentation, cell contour segmentationand retinal optical coherence tomography layer segmentation.