CLIP-CLOP: CLIP-Guided Collage and Photomontage

  • 2022-05-06 12:33:49
  • Piotr Mirowski, Dylan Banarse, Mateusz Malinowski, Simon Osindero, Chrisantha Fernando
  • 31

Abstract

The unabated mystique of large-scale neural networks, such as the CLIP dualimage-and-text encoder, popularized automatically generated art. Increasinglymore sophisticated generators enhanced the artworks' realism and visualappearance, and creative prompt engineering enabled stylistic expression.Guided by an artist-in-the-loop ideal, we design a gradient-based generator toproduce collages. It requires the human artist to curate libraries of imagepatches and to describe (with prompts) the whole image composition, with theoption to manually adjust the patches' positions during generation, therebyallowing humans to reclaim some control of the process and achieve greatercreative freedom. We explore the aesthetic potentials of high-resolutioncollages, and provide an open-source Google Colab as an artistic tool.

 

Quick Read (beta)

loading the full paper ...