NeuralMarker: A Framework for Learning General Marker Correspondence

  • 2022-09-19 11:04:38
  • Zhaoyang Huang, Xiaokun Pan, Weihong Pan, Weikang Bian, Yan Xu, Ka Chun Cheung, Guofeng Zhang, Hongsheng Li
  • 57


We tackle the problem of estimating correspondences from a general marker,such as a movie poster, to an image that captures such a marker.Conventionally, this problem is addressed by fitting a homography model basedon sparse feature matching. However, they are only able to handle plane-likemarkers and the sparse features do not sufficiently utilize appearanceinformation. In this paper, we propose a novel framework NeuralMarker, traininga neural network estimating dense marker correspondences under variouschallenging conditions, such as marker deformation, harsh lighting, etc.Besides, we also propose a novel marker correspondence evaluation methodcircumstancing annotations on real marker-image pairs and create a newbenchmark. We show that NeuralMarker significantly outperforms previous methodsand enables new interesting applications, including Augmented Reality (AR) andvideo editing.


Quick Read (beta)

loading the full paper ...