Abstract
The recent and increasing interest in video-language research has driven thedevelopment of large-scale datasets that enable data-intensive machine learningtechniques. In comparison, limited effort has been made at assessing thefitness of these datasets for the video-language grounding task. Recent workshave begun to discover significant limitations in these datasets, suggestingthat state-of-the-art techniques commonly overfit to hidden dataset biases. Inthis work, we present MAD (Movie Audio Descriptions), a novel benchmark thatdeparts from the paradigm of augmenting existing video datasets with textannotations and focuses on crawling and aligning available audio descriptionsof mainstream movies. MAD contains over 384,000 natural language sentencesgrounded in over 1,200 hours of video and exhibits a significant reduction inthe currently diagnosed biases for video-language grounding datasets. MAD'scollection strategy enables a novel and more challenging version ofvideo-language grounding, where short temporal moments (typically seconds long)must be accurately grounded in diverse long-form videos that can last up tothree hours.