Abstract
In the text-to-image generation field, recent remarkable progress in StableDiffusion makes it possible to generate rich kinds of novel photorealisticimages. However, current models still face misalignment issues (e.g.,problematic spatial relation understanding and numeration failure) in complexnatural scenes, which impedes the high-faithfulness text-to-image generation.Although recent efforts have been made to improve controllability by givingfine-grained guidance (e.g., sketch and scribbles), this issue has not beenfundamentally tackled since users have to provide such guidance informationmanually. In this work, we strive to synthesize high-fidelity images that aresemantically aligned with a given textual prompt without any guidance. Towardthis end, we propose a coarse-to-fine paradigm to achieve layout planning andimage generation. Concretely, we first generate the coarse-grained layoutconditioned on a given textual prompt via in-context learning based on LargeLanguage Models. Afterward, we propose a fine-grained object-interactiondiffusion method to synthesize high-faithfulness images conditioned on theprompt and the automatically generated layout. Extensive experimentsdemonstrate that our proposed method outperforms the state-of-the-art models interms of layout and image generation. Our code and settings are available at\url{https://layoutllm-t2i.github.io}.