Abstract
Recent years have seen growing interest in developing and applying perceptualsimilarity metrics. Research has shown the superiority of perceptual metricsover pixel-wise metrics in aligning with human perception and serving as aproxy for the human visual system. On the other hand, as perceptual metricsrely on neural networks, there is a growing concern regarding their resilience,given the established vulnerability of neural networks to adversarial attacks.It is indeed logical to infer that perceptual metrics may inherit both thestrengths and shortcomings of neural networks. In this work, we demonstrate thevulnerability of state-of-the-art perceptual similarity metrics based on anensemble of ViT-based feature extractors to adversarial attacks. We thenpropose a framework to train a robust perceptual similarity metric calledLipSim (Lipschitz Similarity Metric) with provable guarantees. By leveraging1-Lipschitz neural networks as the backbone, LipSim provides guarded areasaround each data point and certificates for all perturbations within an$\ell_2$ ball. Finally, a comprehensive set of experiments shows theperformance of LipSim in terms of natural and certified scores and on the imageretrieval application. The code is available athttps://github.com/SaraGhazanfari/LipSim.