Score-based models generate samples by mapping noise to data (and vice versa)via a high-dimensional diffusion process. We question whether it is necessaryto run this entire process at high dimensionality and incur all theinconveniences thereof. Instead, we restrict the diffusion via projections ontosubspaces as the data distribution evolves toward noise. When applied tostate-of-the-art models, our framework simultaneously improves sample quality-- reaching an FID of 2.17 on unconditional CIFAR-10 -- and reduces thecomputational cost of inference for the same number of denoising steps. Ourframework is fully compatible with continuous-time diffusion and retains itsflexible capabilities, including exact log-likelihoods and controllablegeneration. Code is available athttps://github.com/bjing2016/subspace-diffusion.