Abstract
Reinforcement learning (RL) is applied in a wide variety of fields. RLenables agents to learn tasks autonomously by interacting with the environment.The more critical the tasks are, the higher the demand for the robustness ofthe RL systems. Causal RL combines RL and causal inference to make RL morerobust. Causal RL agents use a causal representation to capture the invariantcausal mechanisms that can be transferred from one task to another. Currently,there is limited research in Causal RL, and existing solutions are usually notcomplete or feasible for real-world applications. In this work, we proposeCausalCF, the first complete Causal RL solution incorporating ideas from CausalCuriosity and CoPhy. Causal Curiosity provides an approach for usinginterventions, and CoPhy is modified to enable the RL agent to performcounterfactuals. We apply CausalCF to complex robotic tasks and show that itimproves the RL agent's robustness using a realistic simulation environmentcalled CausalWorld.