Abstract
Formal frameworks of causality have operated largely parallel to moderntrends in deep reinforcement learning (RL). However, there has been a revivalof interest in formally grounding the representations learned by neuralnetworks in causal concepts. Yet, most attempts at neural models of causalityassume static causal graphs and ignore the dynamic nature of causalinteractions. In this work, we introduce Causal Process framework as a noveltheory for representing dynamic hypotheses about causal structure. Furthermore,we present Causal Process Model as an implementation of this framework. Thisallows us to reformulate the attention mechanism popularized by Transformernetworks within an RL setting with the goal to infer interpretable causalprocesses from visual observations. Here, causal inference corresponds toconstructing a causal graph hypothesis which itself becomes an RL task nestedwithin the original RL problem. To create an instance of such hypothesis, weemploy RL agents. These agents establish links between units similar to theoriginal Transformer attention mechanism. We demonstrate the effectiveness ofour approach in an RL environment where we outperform current alternatives incausal representation learning and agent performance, and uniquely recovergraphs of dynamic causal processes.