Abstract
Images are a powerful and immediate vehicle to carry misleading or outrightfalse messages, yet identifying image-based misinformation at scale posesunique challenges. In this paper, we present PIXELMOD, a system that leveragesperceptual hashes, vector databases, and optical character recognition (OCR) toefficiently identify images that are candidates to receive soft moderationlabels on Twitter. We show that PIXELMOD outperforms existing image similarityapproaches when applied to soft moderation, with negligible performanceoverhead. We then test PIXELMOD on a dataset of tweets surrounding the 2020 USPresidential Election, and find that it is able to identify visually misleadingimages that are candidates for soft moderation with 0.99% false detection and2.06% false negatives.