Abstract
When a model attribution technique highlights a particular part of the input,a user might understand this highlight as making a statement aboutcounterfactuals (Miller, 2019): if that part of the input were to change, themodel's prediction might change as well. This paper investigates how welldifferent attribution techniques align with this assumption on realisticcounterfactuals in the case of reading comprehension (RC). RC is a particularlychallenging test case, as token-level attributions that have been extensivelystudied in other NLP tasks such as sentiment analysis are less suitable torepresent the reasoning that RC models perform. We construct counterfactualsets for three different RC settings, and through heuristics that can connectattribution methods' outputs to high-level model behavior, we can evaluate howuseful different attribution methods and even different formats are forunderstanding counterfactuals. We find that pairwise attributions are bettersuited to RC than token-level attributions across these different RC settings,with our best performance coming from a modification that we propose to anexisting pairwise attribution method.