We aim at constructing a high performance model for defect detection thatdetects unknown anomalous patterns of an image without anomalous data. To thisend, we propose a two-stage framework for building anomaly detectors usingnormal training data only. We first learn self-supervised deep representationsand then build a generative one-class classifier on learned representations. Welearn representations by classifying normal data from the CutPaste, a simpledata augmentation strategy that cuts an image patch and pastes at a randomlocation of a large image. Our empirical study on MVTec anomaly detectiondataset demonstrates the proposed algorithm is general to be able to detectvarious types of real-world defects. We bring the improvement upon previousarts by 3.1 AUCs when learning representations from scratch. By transferlearning on pretrained representations on ImageNet, we achieve a newstate-of-theart 96.6 AUC. Lastly, we extend the framework to learn and extractrepresentations from patches to allow localizing defective areas withoutannotations during training.