Abstract

In this paper we present Horizon, Facebook's open source appliedreinforcement learning (RL) platform. Horizon is an end-to-end platformdesigned to solve industry applied RL problems where datasets are large(millions to billions of observations), the feedback loop is slow (vs. asimulator), and experiments must be done with care because they don't run in asimulator. Unlike other RL platforms, which are often designed for fastprototyping and experimentation, Horizon is designed with production use casesas top of mind. The platform contains workflows to train popular deep RLalgorithms and includes data preprocessing, feature transformation, distributedtraining, counterfactual policy evaluation, and optimized serving. We alsoshowcase real examples of where models trained with Horizon significantlyoutperformed and replaced supervised learning systems at Facebook.

 

Introduction (beta)

None

 

Conclusion (beta)

None