Manifold Alignment for Semantically Aligned Style Transfer

  • 2020-05-21 16:52:37
  • Jing Huo, Shiyin Jin, Wenbin Li, Jing Wu, Yu-Kun Lai, Yinghuan Shi, Yang Gao
  • 36

Abstract

Given a content image and a style image, the goal of style transfer is tosynthesize an output image by transferring the target style to the contentimage. Currently, most of the methods address the problem with global styletransfer, assuming styles can be represented by global statistics, such as Grammatrices or covariance matrices. In this paper, we make a different assumptionthat local semantically aligned (or similar) regions between the content andstyle images should share similar style patterns. Based on this assumption,content features and style features are seen as two sets of manifolds and amanifold alignment based style transfer (MAST) method is proposed. MAST is asubspace learning method which learns a common subspace of the content andstyle features. In the common subspace, content and style features with largerfeature similarity or the same semantic meaning are forced to be close. Thelearned projection matrices are added with orthogonality constraints so thatthe mapping can be bidirectional, which allows us to project the contentfeatures into the common subspace, and then into the original style space. Byusing a pre-trained decoder, promising stylized images are obtained. The methodis further extended to allow users to specify corresponding semantic regionsbetween content and style images or using semantic segmentation maps asguidance. Extensive experiments show the proposed MAST achieves appealingresults in style transfer.

 

Quick Read (beta)

loading the full paper ...