A part-based object understanding facilitates efficient compositionallearning and knowledge transfer, robustness to occlusion, and has the potentialto increase the performance on general recognition and localization tasks.However, research on part-based models is hindered due to the lack of datasetswith part annotations, which is caused by the extreme difficulty and high costof annotating object parts in images. In this paper, we propose PartImageNet, alarge, high-quality dataset with part segmentation annotations. It consists of158 classes from ImageNet with approximately 24000 images. PartImageNet isunique because it offers part-level annotations on a general set of classeswith non-rigid, articulated objects, while having an order of magnitude largersize compared to existing datasets. It can be utilized in multiple vision tasksincluding but not limited to: Part Discovery, Semantic Segmentation, Few-shotLearning. Comprehensive experiments are conducted to set up a set of baselineson PartImageNet and we find that existing works on part discovery can notalways produce satisfactory results during complex variations. The exploit ofparts on downstream tasks also remains insufficient. We believe that ourPartImageNet will greatly facilitate the research on part-based models andtheir applications. The dataset and scripts will soon be released athttps://github.com/TACJu/PartImageNet.