Abstract
This paper introduces a tuning-free method for both object insertion andsubject-driven generation. The task involves composing an object, givenmultiple views, into a scene specified by either an image or text. Existingmethods struggle to fully meet the task's challenging objectives: (i)seamlessly composing the object into the scene with photorealistic pose andlighting, and (ii) preserving the object's identity. We hypothesize thatachieving these goals requires large scale supervision, but manually collectingsufficient data is simply too expensive. The key observation in this paper isthat many mass-produced objects recur across multiple images of large unlabeleddatasets, in different scenes, poses, and lighting conditions. We use thisobservation to create massive supervision by retrieving sets of diverse viewsof the same object. This powerful paired dataset enables us to train astraightforward text-to-image diffusion architecture to map the object andscene descriptions to the composited image. We compare our method, ObjectMate,with state-of-the-art methods for object insertion and subject-drivengeneration, using a single or multiple references. Empirically, ObjectMateachieves superior identity preservation and more photorealistic composition.Differently from many other multi-reference methods, ObjectMate does notrequire slow test-time tuning.