Accurate multi-organ abdominal CT segmentation is essential to many clinicalapplications such as computer-aided intervention. As data annotation requiresmassive human labor from experienced radiologists, it is common that trainingdata are partially labeled, e.g., pancreas datasets only have the pancreaslabeled while leaving the rest marked as background. However, these backgroundlabels can be misleading in multi-organ segmentation since the "background"usually contains some other organs of interest. To address the backgroundambiguity in these partially-labeled datasets, we propose Prior-aware NeuralNetwork (PaNN) via explicitly incorporating anatomical priors on abdominalorgan sizes, guiding the training process with domain-specific knowledge. Morespecifically, PaNN assumes that the average organ size distributions in theabdomen should approximate their empirical distributions, a prior statisticsobtained from the fully-labeled dataset. As our training objective is difficultto be directly optimized using stochastic gradient descent , we propose toreformulate it in a min-max form and optimize it via the stochastic primal-dualgradient algorithm. PaNN achieves state-of-the-art performance on theMICCAI2015 challenge "Multi-Atlas Labeling Beyond the Cranial Vault", acompetition on organ segmentation in the abdomen. We report an average Dicescore of 84.97%, surpassing the prior art by a large margin of 3.27%.