Abstract
Given a factorization of an image into a sum of linear components, we presenta zero-shot method to control each individual component through diffusion modelsampling. For example, we can decompose an image into low and high spatialfrequencies and condition these components on different text prompts. Thisproduces hybrid images, which change appearance depending on viewing distance.By decomposing an image into three frequency subbands, we can generate hybridimages with three prompts. We also use a decomposition into grayscale and colorcomponents to produce images whose appearance changes when they are viewed ingrayscale, a phenomena that naturally occurs under dim lighting. And we explorea decomposition by a motion blur kernel, which produces images that changeappearance under motion blurring. Our method works by denoising with acomposite noise estimate, built from the components of noise estimatesconditioned on different prompts. We also show that for certain decompositions,our method recovers prior approaches to compositional generation and spatialcontrol. Finally, we show that we can extend our approach to generate hybridimages from real images. We do this by holding one component fixed andgenerating the remaining components, effectively solving an inverse problem.