Abstract
Adding explanations to recommender systems is said to have multiple benefits,such as increasing user trust or system transparency. Previous work from otherapplication areas suggests that specific user characteristics impact the users'perception of the explanation. However, we rarely find this type of evaluationfor recommender systems explanations. This paper addresses this gap bysurveying 124 papers in which recommender systems explanations were evaluatedin user studies. We analyzed their participant descriptions and study resultswhere the impact of user characteristics on the explanation effects wasmeasured. Our findings suggest that the results from the surveyed studiespredominantly cover specific users who do not necessarily represent the usersof recommender systems in the evaluation domain. This may seriously hamper thegeneralizability of any insights we may gain from current studies onexplanations in recommender systems. We further find inconsistencies in thedata reporting, which impacts the reproducibility of the reported results.Hence, we recommend actions to move toward a more inclusive and reproducibleevaluation.