True to the Model or True to the Data?

  • 2020-06-29 17:54:39
  • Hugh Chen, Joseph D. Janizek, Scott Lundberg, Su-In Lee
  • 1


A variety of recent papers discuss the application of Shapley values, aconcept for explaining coalitional games, for feature attribution in machinelearning. However, the correct way to connect a machine learning model to acoalitional game has been a source of controversy. The two main approaches thathave been proposed differ in the way that they condition on known features,using either (1) an interventional or (2) an observational conditionalexpectation. While previous work has argued that one of the two approaches ispreferable in general, we argue that the choice is application dependent.Furthermore, we argue that the choice comes down to whether it is desirable tobe true to the model or true to the data. We use linear models to investigatethis choice. After deriving an efficient method for calculating observationalconditional expectation Shapley values for linear models, we investigate howcorrelation in simulated data impacts the convergence of observationalconditional expectation Shapley values. Finally, we present two real dataexamples that we consider to be representative of possible use cases forfeature attribution -- (1) credit risk modeling and (2) biological discovery.We show how a different choice of value function performs better in eachscenario, and how possible attributions are impacted by modeling choices.


Quick Read (beta)

loading the full paper ...