Recently there has been an interest in the potential of learning generativemodels from a single image, as opposed to from a large dataset. This task is ofpractical significance, as it means that generative models can be used indomains where collecting a large dataset is not feasible. However, training amodel capable of generating realistic images from only a single sample is adifficult problem. In this work, we conduct a number of experiments tounderstand the challenges of training these methods and propose some bestpractices that we found allowed us to generate improved results over previouswork in this space. One key piece is that unlike prior single image generationmethods, we concurrently train several stages in a sequential multi-stagemanner, allowing us to learn models with fewer stages of increasing imageresolution. Compared to a recent state of the art baseline, our model is up tosix times faster to train, has fewer parameters, and can better capture theglobal structure of images.