A central goal of unsupervised learning is to acquire representations fromunlabeled data or experience that can be used for more effective learning ofdownstream tasks from modest amounts of labeled data. Many prior unsupervisedlearning works aim to do so by developing proxy objectives based onreconstruction, disentanglement, prediction, and other metrics. Instead, wedevelop an unsupervised learning method that explicitly optimizes for theability to learn a variety of tasks from small amounts of data. To do so, weconstruct tasks from unlabeled data in an automatic way and run meta-learningover the constructed tasks. Surprisingly, we find that relatively simplemechanisms for task design, such as clustering unsupervised representations,lead to good performance on a variety of downstream tasks. Our experimentsacross four image datasets indicate that our unsupervised meta-learningapproach acquires a learning algorithm without any labeled data that isapplicable to a wide range of downstream classification tasks, improving uponthe representation learned by four prior unsupervised learning methods.