Abstract
Explanations of AI systems rarely address the information needs of peopleaffected by algorithmic decision-making (ADM). This gap between conveyedinformation and information that matters to affected stakeholders can impedeunderstanding and adherence to regulatory frameworks such as the AI Act. Toaddress this gap, we present the "XAI Novice Question Bank": A catalog ofaffected stakeholders' information needs in two ADM use cases (employmentprediction and health monitoring), covering the categories data, systemcontext, system usage, and system specifications. Information needs weregathered in an interview study where participants received explanations inresponse to their inquiries. Participants further reported their understandingand decision confidence, showing that while confidence tended to increase afterreceiving explanations, participants also met understanding challenges, such asbeing unable to tell why their understanding felt incomplete. Explanationsfurther influenced participants' perceptions of the systems' risks andbenefits, which they confirmed or changed depending on the use case. When riskswere perceived as high, participants expressed particular interest inexplanations about intention, such as why and to what end a system was put inplace. With this work, we aim to support the inclusion of affected stakeholdersinto explainability by contributing an overview of information and challengesrelevant to them when deciding on the adoption of ADM systems. We close bysummarizing our findings in a list of six key implications that inform thedesign of future explanations for affected stakeholder audiences.