Background Matting: The World is Your Green Screen

  • 2020-04-01 17:38:55
  • Soumyadip Sengupta, Vivek Jayaram, Brian Curless, Steve Seitz, Ira Kemelmacher-Shlizerman
  • 238

Abstract

We propose a method for creating a matte -- the per-pixel foreground colorand alpha -- of a person by taking photos or videos in an everyday setting witha handheld camera. Most existing matting methods require a green screenbackground or a manually created trimap to produce a good matte. Automatic,trimap-free methods are appearing, but are not of comparable quality. In ourtrimap free approach, we ask the user to take an additional photo of thebackground without the subject at the time of capture. This step requires asmall amount of foresight but is far less time-consuming than creating atrimap. We train a deep network with an adversarial loss to predict the matte.We first train a matting network with supervised loss on ground truth data withsynthetic composites. To bridge the domain gap to real imagery with nolabeling, we train another matting network guided by the first network and by adiscriminator that judges the quality of composites. We demonstrate results ona wide variety of photos and videos and show significant improvement over thestate of the art.

 

Quick Read (beta)

loading the full paper ...