Discovering Visual Patterns in Art Collections with Spatially-consistent Feature Learning

  • 2019-03-07 01:12:16
  • Xi Shen, Alexei A. Efros, Aubry Mathieu
  • 51

Abstract

Our goal in this paper is to discover near duplicate patterns in largecollections of artworks. This is harder than standard instance mining due todifferences in the artistic media (oil, pastel, drawing, etc), andimperfections inherent in the copying process. The key technical insight is toadapt a standard deep feature to this task by fine-tuning it on the specificart collection using self-supervised learning. More specifically, spatialconsistency between neighbouring feature matches is used as supervisoryfine-tuning signal. The adapted feature leads to more accurate style-invariantmatching, and can be used with a standard discovery approach, based ongeometric verification, to identify duplicate patterns in the dataset. Theapproach is evaluated on several different datasets and shows surprisingly goodqualitative discovery results. For quantitative evaluation of the method, weannotated 273 near duplicate details in a dataset of 1587 artworks attributedto Jan Brueghel and his workshop. Beyond artwork, we also demonstrateimprovement on localization on the Oxford5K photo dataset as well as onhistorical photograph localization on the Large Time Lags Location (LTLL)dataset.

 

Introduction (beta)

None

 

Conclusion (beta)

None