Abstract
This paper presents a unified training strategy that enables a novelmulti-scale deep neural network to be trained on multiple partially labeleddatasets for multi-organ segmentation. Multi-scale contextual information iseffective for pixel-level label prediction, i.e. image segmentation. However,such important information is only partially exploited by the existing methods.In this paper, we propose a new network architecture for multi-scale featureabstraction, which integrates pyramid feature analysis into an imagesegmentation model. To bridge the semantic gap caused by directly mergingfeatures from different scales, an equal convolutional depth mechanism isproposed. In addition, we develop a deep supervision mechanism for refiningoutputs in different scales. To fully leverage the segmentation features fromdifferent scales, we design an adaptive weighting layer to fuse the outputs inan automatic fashion. All these features together integrate into apyramid-input pyramid-output network for efficient feature extraction. Last butnot least, to alleviate the hunger for fully annotated data in training deepsegmentation models, a unified training strategy is proposed to train onesegmentation model on multiple partially labeled datasets for multi-organsegmentation with a novel target adaptive loss. Our proposed method wasevaluated on four publicly available datasets, including BTCV, LiTS, KiTS andSpleen, where very promising performance has been achieved. The source code ofthis work is publicly shared at https://github.com/DIAL-RPI/PIPO-FAN for othersto easily reproduce the work and build their own models with the introducedmechanisms.