Abstract
We present a Reverse Reinforcement Learning (Reverse RL) approach forrepresenting retrospective knowledge. General Value Functions (GVFs) haveenjoyed great success in representing predictive knowledge, i.e., answeringquestions about possible future outcomes such as "how much fuel will beconsumed in expectation if we drive from A to B?". GVFs, however, cannot answerquestions like "how much fuel do we expect a car to have given it is at B attime $t$?". To answer this question, we need to know when that car had a fulltank and how that car came to B. Since such questions emphasize the influenceof possible past events on the present, we refer to their answers asretrospective knowledge. In this paper, we show how to represent retrospectiveknowledge with Reverse GVFs, which are trained via Reverse RL. We demonstrateempirically the utility of Reverse GVFs in both representation learning andanomaly detection.