Abstract
This work introduces Conditional Image Retrieval (CIR) systems: IR methodsthat can efficiently specialize to specific subsets of images on the fly. Thesesystems broaden the class of queries IR systems support, and eliminate the needfor expensive re-fitting to specific subsets of data. Specifically, we adapttree-based K-Nearest Neighbor (KNN) data-structures to the conditional settingby introducing additional inverted-index data-structures. This speedsconditional queries and does not slow queries without conditioning. We presenttwo new datasets for evaluating the performance of CIR systems and evaluate avariety of design choices. As a motivating application, we present an algorithmthat can explore shared semantic content between works of art of vastlydifferent media and cultural origin. Finally, we demonstrate that CIRdata-structures can identify Generative Adversarial Network (GAN) ``blindspots'': areas where GANs fail to properly model the true data distribution.