Abstract
In this paper, we propose to disentangle and interpret contextual effectsthat are encoded in a pre-trained deep neural network. We use our method toexplain the gaming strategy of the alphaGo Zero model. Unlike previous studiesthat visualized image appearances corresponding to the network output or aneural activation only from a global perspective, our research aims to clarifyhow a certain input unit (dimension) collaborates with other units (dimensions)to constitute inference patterns of the neural network and thus contribute tothe network output. The analysis of local contextual effects w.r.t. certaininput units is of special values in real applications. Explaining the logic ofthe alphaGo Zero model is a typical application. In experiments, our methodsuccessfully disentangled the rationale of each move during the Go game.