Abstract
We introduce Affective Visual Dialog, an emotion explanation and reasoningtask as a testbed for research on understanding the formation of emotions invisually grounded conversations. The task involves three skills: (1)Dialog-based Question Answering (2) Dialog-based Emotion Prediction and (3)Affective emotion explanation generation based on the dialog. Our keycontribution is the collection of a large-scale dataset, dubbed AffectVisDial,consisting of 50K 10-turn visually grounded dialogs as well as concludingemotion attributions and dialog-informed textual emotion explanations,resulting in a total of 27,180 working hours. We explain our design decisionsin collecting the dataset and introduce the questioner and answerer tasks thatare associated with the participants in the conversation. We train anddemonstrate solid Affective Visual Dialog baselines adapted fromstate-of-the-art models. Remarkably, the responses generated by our models showpromising emotional reasoning abilities in response to visually groundedconversations. Our project page is available athttps://affective-visual-dialog.github.io.